Poor Data Quality and Lack of Website Functionality Hobble Recovery Act Recipient Reports

The release of the first round of Recovery Act contracts spending data marks the first time that recipients of federal funding have been required to report to the federal government on their use of the funds in a timely and transparent manner. This represents an important milestone in government transparency and accountability. However, the poor data quality and Recovery.gov's limited functionality hinder the promise of a new era of fiscal transparency – at least for this round of recipient reporting.

Since the Recovery Accountability and Transparency Board (Recovery Board) released the first round of Recovery Act recipient reporting on Oct. 15, everyone from federal officials, members of Congress, transparency advocates, and ordinary citizens have gone to the site to see the new data. These recipient report data provide a new level of detail on federal projects. Provisions in the Recovery Act require that recipients of Recovery Act funds report back to the federal government on the amount of funds received and expended on Recovery Act projects, including project status updates. The Recovery Act also requires that recipients indicate the number of jobs created or saved by the project, along with a narrative explaining why and what kind of jobs were created. Additional information is also being collected.

This level of information has never been reported before. However, this new dataset will deliver full transparency only when two dimensions of data publication are adequately implemented. First, the public should be able to access recipient reports on Recovery.gov in myriad ways that allow for an array of searches and analyses. Second, the data that are made available should accurately reflect how recipients used Recovery Act funds.

Recovery Act transparency requires that sufficient tools be available to access spending data. In this respect, the website built to disclose the recipient reports to the public, Recovery.gov [1], falls significantly short. Users have very limited options to search, sort, or sift through the recipient reports, limiting the connections that can be drawn between various data points or the ability to find out if a particular company has received Recovery Act funds. While the site does allow rudimentary searches by ZIP code, allowing users to find out how many Recovery Act contracts XYZ Corporation received in any given neighborhood, users cannot find out the total number of contracts and total dollar amount XYZ Corporation received in the state or throughout the nation. In other words, the user cannot search by recipient. This information is vital to developing a balanced understanding of how Recovery Act funds are being deployed. Without this type of searching and sorting that enables users to slice and dice Recovery Act spending data, Recovery.gov severely limits the usefulness of the data set produced by the transparency provisions in the Recovery Act.

In addition to online analytical resources provided by the federal government, Recovery Act data must be made available in machine-readable formats to allow outside stakeholders to create their own tools. When the Recovery Board first released the data, it also made recipient reports available in one machine-readable format, but the implementation of this feature was cumbersome. Initially, the data were only available in 180 separate files (organized by state), but after some loud complaints [2], the Recovery Board corrected this issue by re-releasing the recipient reports as one, nationwide file. When the Recovery Board received additional feedback that the file contained formatting errors, it released a corrected version in a very short timeframe. Although these issues have been fixed, it is still necessary to make additional data formats available on Recovery.gov, such as an ATOM feed, which makes it easier for machines to process and display the data without human intervention.

Beyond issues with information access, Recovery Act transparency is also hobbled along a second dimension: data quality. Specifically, the jobs information, a much-touted feature of the recipient data, is rife with errors. One recurring problem is that job creation narratives do not match up with job creation numbers. For example, the narrative description of the jobs created and saved might indicate that no jobs were created or saved, but the number field that contains a count of jobs shows that 10 jobs were created or saved. Another common problem is that similar projects have different job creation numbers (for instance, both Chrysler and General Motors were given projects to build cars for the government for similar amounts of money, but according to their respective recipient reports, Chrysler created no jobs at all [3], but General Motors created or saved more than 105 jobs [4]).

Furthermore, it is not always clear where jobs were created. A particular outlier in this regard is a report in which a recipient noted it created 4,685 jobs in Colorado, the most of any state in the nation. Yet a close reading of the report reveals that 3,852 of those jobs were actually created in other states.

From the large number of errors, it is clear many recipients have differing interpretations of the jobs reporting requirements. The upshot of these data quality problems is that the total number of jobs created or saved by Recovery Act contract recipients is simply an unreliable gauge of the impact the act is having on the economy.

Transparency in the Recovery Act will continue to be constrained unless Recovery.gov is substantially improved and unless recipient report data quality improves significantly. There have been improvements already to the website, and it is likely that subsequent rounds of recipient reports will contain improved data quality. The Recovery Board, which built and maintains Recovery.gov, has been responsive to outside feedback and criticism, giving good reason to be optimistic this groundbreaking transparency model will continue to improve.