You have reached a web page on our old web site.
To visit our new web site click here.




Comments on OMB's Data Quality


August 13, 2001

Ms. Brooke Dickson
Office of Information and Regulatory Affairs
Office of Management and Budget
Washington, D.C. 20503

Dear Ms. Dickson:

OMB Watch appreciates the opportunity to comment on the above-referenced proposed guidelines. OMB Watch is a nonprofit research and advocacy organization focused on strengthening government accountability and citizen participation. Since 1985 we have actively worked to improve public access to government information. We have worked on general policy matters, such as the Paperwork Reduction Act and OMB Circular A-130, and specific ways of promoting the public s right-to-know, such as providing public access to government data through RTK NET (www.rtk.net), working with EPA to improve its online access to data, and commenting on FirstGov, the government s web portal.

We believe that access to information is the lifeblood of our democracy and that government has an affirmative responsibility to ensure meaningful public access to its information. In this new Internet age, public access can be greatly improved. Information disseminated by the federal government should be timely, accurate, useful to and usable by the public, and the data used and disseminated by the federal government should be of the highest possible quality. However, pursuit of these goals should not become an excuse for delay or disruption of public access. In fact, it has been shown that public access to government information, particularly administrative and regulatory data, has the beneficial impact of improving the quality of the information that is submitted to government.

Section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (P.L. 106-554) requires OMB to  provide policy and procedural guidance to Federal agencies for ensuring and maximizing the quality, objectivity, utility, and integrity of information (including statistical information) disseminated by Federal agencies. Section 515 also requires agencies to issue guidelines within one year of the OMB guidance that includes mechanisms for correction of information  maintained and disseminated not in compliance with the guidelines. Agencies are also to report to OMB periodically on the number and nature of complaints received regarding  the accuracy of information disseminated and how the agency handled such complaints.

We are concerned that Section 515 may be inappropriately construed as a roadblock to public access, setting unclear, ambiguous standards to achieve  quality, objectivity, utility, and integrity before disseminating information to the public. We think the OMB guidance needs to be especially clear that Section 515 should not interpreted by agencies in that manner.

It appears OMB attempts to make this point in the supplementary information:  The purpose of Section 515 is not to stifle information dissemination... However, the current OMB proposal muddies the waters, particularly in the discussion of  Underlying Principles.

OMB suggests that if Section 515 standards are not met, then agencies should not disseminate information. In supplementary information OMB states,  Given the administrative mechanisms required by Section 515... it is clear that agencies should not disseminate information that does not meet some basic level of quality. A similar position is articulated in the guidance itself. Section III(2) Common sense dictates that a basic level of quality is necessary before collecting, maintaining, or disseminating information. But we are concerned that OMB reads too much into Section 515 by implying that the statutory language provides some type of standard that must be met before disseminating information. Section 515 does not address the issue of whether an agency disseminates information, only factors it should consider when doing so. Although Section 515 requires an  administrative mechanism to permit correction of information not in compliance with the guidance, nowhere does it contemplate that this mechanism has anything to do with withholding or delaying dissemination.

We strongly urge OMB to make this point very prominent in the guidance to agencies. We view this point  that Section 515 provides no determinative factors on dissemination  as a corollary to OMB s emphasis on developing guidelines  so that agencies can apply them in a common?sense and workable manner. It is critical, as you note, that these guidelines  not impose unnecessary administrative burdens that would inhibit agencies from continuing to take advantage of the Internet and other technologies to disseminate information that can be of great benefit and value to the public.

Of critical importance is OMB s comment that  encourages agencies to rely, to the extent possible, upon existing agency processes for evaluating information dissemination activities rather than require the creation of new and potentially duplicative or contradictory processes. Section 515 creates no new authority for the public to challenge the dissemination practices of agencies. To the extent that it requires an administrative mechanism for seeking input about corrections, this should be done in a manner consistent with agency procedures and practices.

In acknowledging that  agencies disseminate many types of information in many different ways, OMB mentions EPA s Toxics Release Inventory. The TRI provides an important lesson with regard to the quality of the data disseminated. Online public access to the data has demonstrated that the quality of the data  interpreted as reliable estimates of toxic releases and completeness of the form  greatly improved as public access increased. In other words, as Supreme Court Justice Louis Brandeis so memorably noted,  Sunlight is žthe best disinfectant. (Louis Brandeis, Other People's Money, 1933) We believe OMB should provide commentary in the guidance that public access to government information can improve quality.

OMB appropriately refers to the Paperwork Reduction Act (PRA) and OMB Circular A-130,  Management of Federal Information Resources in the supplementary information to provide some context to government dissemination policies. It may be useful to provide additional clarification on what these directives require. For example, in the discussion in the supplementary information, OMB notes that  agencies should weigh the costs... and the benefits of higher information quality in the development of such information, and the level of quality to which the information dissemination will be held. This commentary is consistent with Circular A-130's requirement to  disseminate information in a manner that achieves the best balance between the goals of maximizing the usefulness of the information and minimizing the cost to the government and the public. Sec. 8(a)(5)(d)

This balancing process can be a difficult one, but one in which the government should invariably err on the side of providing public access. This point of view recognizes that our First Amendment freedoms  freedom of speech, freedom of the press, freedom of assembly, freedom to petition the government for a redress of grievances  depend on the free flow of government information to the people.

In the context of weighing costs and benefits, we encourage OMB to modify the examples of costs that are used in the proposal s supplementary information. OMB s examples include  costs attributable to agency processing effort, respondent burden, maintenance of needed privacy, and assurances of suitable confidentiality. OMB should also include the cost to the public for not providing the information. For example, withholding information about the handling of hazardous chemicals in our communities may place families in life and death situations. Last year, a day care facility suffered significant damage when a nearby chemical plant had an explosion. Fortunately, it occurred at a time when no one was in the building. Had it been at a time when the children were in the facility, it could have been far more serious than simply the cost of reconstructing the building. There is human cost for not providing information about dangers that must be considered. In general, OMB should encourage agencies to assess the cost of lost opportunities when information is not disseminated. OMB should also note that in addition to  respondent burden, there is burden imposed on the public when information is difficult to find or obtain. These, too, are costs that agencies should consider.

We urge OMB to drop language that is subject to wide interpretation. For example, in its supplementary information, OMB states,  The more important the information, the higher the quality standards to which it should be held. How does OMB expect agencies to define what is  more important, or what would be the  higher quality standards to which it should be held, or who would be involved in the process of deciding? What one person considers less important information, another will consider more important. This language might be interpreted by agencies to mean that they should not release data or disseminate other information until it is  perfect. Before providing comments on the guidance itself and the two questions posed by OMB, we note the limited time OMB has to react to the comments it receives and still complete final rules by September 30, 2001. We urge OMB to proceed cautiously in developing final rules and take sufficient time to develop the final guidance. One approach we would encourage OMB to consider is to host a series of meetings involving the public interest community, the scientific community, and others who are concerned about the effect these guidelines might have on the free flow of information to discuss issues about data quality that may help OMB craft guidelines that meet the requirements of Section 515 without erecting barriers to the dissemination of information.


COMMENTS ON GUIDANCE

  1. The definition of  quality, utility, objectivity, and integrity should be left to the agencies. Consistent with OMB s recognition of the problems of one-size-fits-all guidelines, the definition of the key words used in Section 515 should be left to the agencies. For example, section V(1)(B) uses very subjective terms, such as  complete and  unbiased, when providing a definition. What does  complete mean when dealing with information submitted by a regulatory entity? Does OMB mean to suggest that if a form submitted by a regulated entity is not completely filled out that the information should not be disseminated? And what does  complete mean for information that is collected on a longitudinal basis? The issues raised by the term  unbiased are even broader.

    Section V(1)(B) raises other issues. OMB suggests that the Section 515 statutory language regarding quality  involves whether the information is presented within a proper context. While OMB may believe this to be true and needs to modify Circular A-130, this guidance is not the proper vehicle for such debate.

    Section V(1)(B)(ii)(a) also states  with respect to scientific information, the results must be substantially reproducible upon independent analysis of the underlying data. In its current form, we strongly object to this language. Does OMB suggest that someone must reproduce scientific work before it can be disseminated? Who would this be, someone insider or outside of government? Does OMB not consider the peer review process, a well-established procedure for reviewing scientific matters, to be adequate? Does this reach to  underlying data done by third parties, such as federal grantees? Who pays for reproducing the scientific work?

    If OMB decides not to drop the definitions for  quality and related concepts, we strongly recommend that Section V(1)(B) be expunged from the guidelines, as standards are already in place in the federal government for the information to which they appropriately apply. If OMB feels compelled to address the definitions for  quality and related concepts, we agree that it should include a dimension of the usefulness of the information to those to whom the agency disseminates the information. Section V(1)(A) This is current practice in most agencies. We also agree that information should be  protected from unauthorized access or revision, to ensure that the information is not compromised through corruption, or falsification. Section V(1)(C) This is a practice of good information resources management.

  2. The guidance needs to be clear that dissemination is not contingent solely on quality and related concepts. Section III(2) states  agencies should develop a process for reviewing and documenting for users the quality... of information before it is disseminated. We believe the  before it is disseminated should be dropped. OMB should add a sentence making clear that  quality and related concepts should be not used to stifle dissemination as it does in the supplementary information. Moreover, OMB should add language that indicates that in certain situations dissemination will actually help to strengthen the quality of the information.

  3. The  administrative mechanism for making corrections needs clarification and modification. We believe Section III(3) requiring  administrative mechanisms allowing affected persons to seek and obtain correction of information maintained and disseminated by the agency that does not comply with these OMB guidelines needs to be modified. We fear this provision has the potential for misuse in order to grind government to a halt and to prevent the dissemination or use of data. We urge that safeguards be put into place, not only to assure the free flow of information promoted by Circular A-130, but also to avoid unwarranted burdens on the federal agencies. Here are some suggestions:

    a) This  administrative mechanism should be prospective only. Neither OMB nor any agency should be required to retrospectively apply new standards to information previously disseminated. Moreover, the potential impact of retroactive application is enormous in terms of burden to federal agencies budget and staff. As was the case with recent Circular A-110 changes, OMB should make clear in the guidelines that application is prospective only.

    b) Agencies should be permitted to respond to complaints with a two-tier process. The first tier would simply require the agency to show that it followed its own established procedures before releasing the data. Only if it can be shown that the agency did not follow these procedures should the affected person be permitted to attack the substance.

    c) Data publication should not be delayed or data retracted every time someone who is opposed to the potential commercial impact or the potential regulatory use of the data decides to attack the underlying data or information. There should be a burden of proof on the challenger to show that there is serious scientific merit to the challenge before the agency and the scientist(s) are required to go through this procedure. The experts supporting the challenge and the challengers themselves should be required to disclose any financial or political interest (other than payment for time and expenses as is routine in the industry) they have in the subject matter to which the data pertain.

    d) For correction of information that has been submitted to the government, a log should be created and maintained to, at a minimum, identify what information was changed, by whom, when it was requested, and the source of the error (e.g., submitter or agency). The log should be publicly accessible along with the corrected information. Agencies should develop specific procedures for situations in which the agency does not concur with the correction.

    e) There should be no right of private action to enforce the guidelines created by OMB or by the agencies.

    We would also note that  affected persons should be understood to include both individuals and communities affected by the data, including the agency s failure to disseminate information that followed agency procedures.

  4. The agency reporting requirements should be modified to ensure that they do not impose unnecessary burdens on agencies. We are pleased that OMB has decided to put information policy oversight directly into the portfolios of agency CIOs and to make them responsible for resolving all complaints about the agency s compliance with Circular A-130.

    Given our concerns with the inappropriate application of statistical standards to all government information dissemination products, however, we do have concerns about some of the specifics of this section. Our concerns have to do with the burdens potentially imposed on the agencies by the requirement to respond in written form to the complaint. This has the potential to overwhelm agency staff.

    We note with approval, however, that OMB has not required the agencies to immediately resolve the complaint and we would refer back to the safeguards suggested above for a process that agencies should be directed to apply to the resolution of complaints (as opposed to specific correction of errors).

  5. The guidelines to be developed by the agencies should be subject to notice and comment in the Federal Register. OMB is to be lauded for making its guidance subject to notice and comment in the Federal Register. The same principle should be required of the agency guidance that is to be developed. Given OMB s emphasis on providing agency flexibility, the agency guidelines are likely to be even more central to future public access concerns. Accordingly, the public should be given an opportunity to comment on the agency proposals.

COMMENTS ON OMB S QUESTIONS

OMB specifically requested comments on two questions.

Should the OMB guidelines devote particular attention to specific types of information or information dissemination products? If so, please identify the areas where specific focus should be directed, explain why the focus is needed or is desirable, and describe any guidelines that you recommend for those areas.

We do not think OMB should devote particular attention to specific types of information. Furthermore, it would be inconsistent to do so in light of OMB s principle of providing agency flexibility.

More to the point, we believe that the statutory language in Section 515 is overly-broad, conflating different kinds of information disseminated by the federal government. For example, if information were divided into records, publications, regulatory or administrative data, statistical data, scientific data other than statistical, we would find that portions of Section 515 are irrelevant when applied to some of these categories. We do not think that these guidelines are appropriately applied to records, to publications (how do you determine the impartiality of a manual? Will no press releases ever be issued again because they are not impartial?), or to much scientific data (and would, indeed, cause havoc in the ability of the government to conduct scientific research or use the research of others). Nor do we think they can be appropriately applied to much administrative or regulatory data. Such data is collected by the government to implement statutory requirements.

Accordingly, we would encourage OMB to either expunge Section V(1)(B) entirely or to narrow its application to the only type of government information dissemination product to which it appropriately applies  statistical information.

Should OMB develop specific guidelines to address information that Federal agencies disseminate from a web page? Is there any need to adapt these guidelines to the agency use of a web page? If so, what guidelines are needed?

We do think that a need has long existed, unmet, for government-wide guidelines on agency web sites. We do not think, however, that these guidelines dealing with Section 515 are the appropriate vehicle. We think concerns with usability, permanent public access, the ability to even find information on government web sites all need to be addressed, but we do not think that web sites need any special application of the guidelines under consideration here.

Thank you for this opportunity to comment.

Respectfully submitted,



Gary D. Bass

Executive Director


[blue line]

[ HOME | BUDGET NEWS | INFORMATION POLICY | NONPROFIT ISSUES | GOVERNMENT PERFORMANCE ]

Back to the OMB Watcher Online .