Research | Mini-Monograph
Scientific and Legal Perspectives on Science Generated for Regulatory Activities
Carol J. Henry a...
Science generated for regulatory activities

“the best available, peer-reviewed science and           of the study in a c...
Henry and Conrad

that person had represented an interest group                 OMB’s peer review bulletin. Many fed-    ...
Science generated for regulatory activities

Federal Standards                                                           ...
Henry and Conrad

validity of information upon which they rely.                                                          ...
Science generated for regulatory activities

                              REFERENCES                                    ...
Adams   The Legalized Crime Of Banking And A Constitutional Remedy (1958)
Upcoming SlideShare
Loading in …5

Adams The Legalized Crime Of Banking And A Constitutional Remedy (1958)


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Adams The Legalized Crime Of Banking And A Constitutional Remedy (1958)

  1. 1. Research | Mini-Monograph Scientific and Legal Perspectives on Science Generated for Regulatory Activities Carol J. Henry and James W. Conrad Jr. American Chemistry Council, Arlington, Virginia, USA Information Quality Act. The Information Quality Act (IQA 2000) governs the quality of This article originated from a conference that asked “Should scientific work conducted for purposes information that federal agencies disseminate. of advocacy before regulatory agencies or courts be judged by the same standards as science con- The IQA required the Office of Management ducted for other purposes?” In the article, which focuses on the regulatory advocacy context, we and Budget (OMB) to issue an initial set of argue that it can be and should be. First, we describe a set of standards and practices currently being implementing guidelines (OMB 2002). Each used to judge the quality of scientific research and testing and explain how these standards and prac- federal agency then issued its guidelines apply- tices assist in judging the quality of research and testing regardless of why the work was conducted. ing the OMB guidelines to its particular cir- These standards and practices include the federal Information Quality Act, federal Good Laboratory c u m s t a n c e s [e.g., U.S. Environmental Practice standards, peer review, disclosure of funding sources, and transparency in research policies. Protection Agency (U.S. EPA) 2002a]. The more that scientific information meets these standards and practices, the more likely it is to be I t is important to note that the IQA of high quality, reliable, reproducible, and credible. We then explore legal issues that may be impli- applies not only to information that agencies cated in any effort to create special rules for science conducted specifically for a regulatory proceed- generate themselves but also to information i n g . Federal administrative law does not provide a basis for treating information in a given developed by nongovernment parties, to the proceeding differently depending on its source or the reason for which it was generated. To the con- e x t e n t the government “disseminates” it, trary, this law positively assures that interested persons have the right to offer their technical exper- either by adopting or endorsing it as its own tise toward the solution of regulatory problems. Any proposal to subject scientific information view or by relying on it to make a decision. generated for the purpose of a regulatory proceeding to more demanding standards than other scien- [ I n f o r m a t i o n prepared for administrative tific information considered in that proceeding would clash with this law and would face significant adjudications is exempt from IQA guidelines, administrative complexities. In a closely related example, the U.S. Environmental Protection Agency but agencies construe this exemption narrowly considered but abandoned a program to implement standards aimed at “external” information. Key (U.S. EPA 2002a).] Thus, to the extent that words: Administrative Procedure Act, agency proceedings, conflict of interest, financial disclosure, businesses, universities, or other private enti- industry science, Information Quality Act, interested persons, peer review, regulatory science, right ties conduct research or testing and the results to publish, scientific quality. Environ Health Perspect 116:136–141 (2008). doi:10.1289/ehp.9978 come into the possession of an agency such as available via [Online 7 November 2007] the U.S. EPA, the results cannot form a basis of the agency’s decision without becoming Standards and Practices agencies, that specify the content and charac- s u b j e c t to IQA requirements. And those teristics of studies to be conducted to meet requirements are most precise and demanding for Judging the Quality r e g u l a t o r y requirements, e.g., a standard in the area of scientific information. of Scientific Work bioassay for carcinogenesis) and “research” OMB’s guidelines (OMB 2002) prescribe T h e project on Scientific Knowledge and (studies that are hypothesis driven, addressing fairly detailed standards for “objectivity.” As a Public Policy (SKAPP) examines the nature b r o a d methodologic or mechanistic ques- general matter, information must be accurate, of science and the ways in which it is used tions, e.g., the biologic activity of chemicals reliable, and unbiased. Scientific information and misused in government decision making on the environment). m u s t be generated using sound research a n d legal proceedings. Last year, SKAPP Both government and the scientific com- m e t h o d s . The sources of the information commissioned papers to address the question munity outside of government have played must be disclosed and data should be docu- “Should scientific work conducted for pur- independent but reinforcing roles in develop- m e n t e d . Scientific information must be poses of advocacy before regulatory agencies ing and propagating these standards and prac- accompanied by supporting data and models. or courts be judged by the same standards as tices. The private sector is actually driving “Influential” scientific information must be s c i e n c e conducted for other purposes?” implementation of several relatively newer sufficiently transparent to be reproduced sub- (SKAPP 2006). This article is adapted from practices, in part to address concerns about ject to several caveats. (“Influential” informa- one of those papers. the credibility of industry-funded science. In tion is that which an agency “reasonably can Science is a social enterprise, and scientific general, the regulatory implementations of determine will have or does have a clear and work tends to be accepted by the community t h e s e concepts impose additional require- substantial impact on important public poli- w h e n it has been confirmed. Crucially, m e n t s well beyond those conventionally cies or private sector decisions.”) Influential experimental and theoretical results must be imposed outside of the regulatory context. information regarding risks to health, safety, reproduced by others within the science com- These government requirements promote a or the environment must also be based on munity and the validity of the work estab- high degree of reliability. As explained below, l i s h e d by replication (Wikipedia 2007a). the cumulative result of their applicability is This article is part of the mini-monograph “Science for However, such replication can take years, and that science conducted for regulatory purposes Regulation and Litigation.” what constitutes replication in a given case in many cases is actually likely to be more reli- Address correspondence to J.W. Conrad Jr., Conrad may also be disputable. Consequently, a vari- able than science conducted outside the regu- Law & Policy Counsel, 1615 L St. NW, Suite 1350, Washington, DC 20036 USA. Telephone: (202) 822- e t y of standards and practices have been l a t o r y arena. This is not to argue that the 1 9 7 0 . Fax: (202) 822-1971. E-mail: jamie@ established over the years for assessing the latter should be required to comply with these q u a l i t y of scientific work (Barrow and g o v e r n m e n t requirements to any greater At the time this article was accepted, C.J.H. and Conrad 2006). These standards and practices extent than it already is, but only to emphasize J . W . C . were both employed by the American apply to both “testing” (activities conducted that science conducted pursuant to them is Chemistry Council. pursuant to protocols, prescribed by regulatory relatively more reliable as a result. Received 12 December 2006; accepted 5 July 2007. 136 116 | NUMBER 1 | January 2008 • Environmental Health Perspectives VOLUME
  2. 2. Science generated for regulatory activities “the best available, peer-reviewed science and of the study in a clearly written protocol that The crucial role of reviewer independence supporting studies conducted in accordance must be approved by the agency. These pro- a n d expertise. T h e hallmark of any peer with sound and objective scientific practices; tocols have been validated and chosen, after review is the independence and expertise of and . . . data collected by accepted methods or extensive and careful review, to provide an the peer reviewer. Reviewer knowledge, expe- best available methods,” and must disclose acceptable degree of scientific certainty, in the rience, and expertise are central requirements significant uncertainties and relevant peer- agency’s view, regarding the reliability and for instructive peer review and ensure “techni- reviewed studies. relevance of test results, which in turn pro- c a l credibility” for the review. The peer A c c o r d i n g l y , if a business submitted a v i d e s the confidence necessary for making reviewer should have expertise at least equiva- paper to the U.S. EPA regarding research it safety determinations and other regulatory lent to that needed for the “original work” but had conducted on the risks posed by one of its determinations. m u s t be independent of the work being products, and the U.S. EPA was going to con- Conformance to GLPs does not, in itself, reviewed. These dual requirements for exper- sider this paper as part of making a decision ensure that scientific work is reproducible, or tise and independence offer the best chance of involving that family of products—a decision that the resulting data will be interpreted cor- obtaining objective, expert evaluation while that would have a substantial impact on pro- rectly. However, when research studies adhere maintaining scientific integrity (NRC 2003). ducers and customers of those products—that to GLPs, reviewers and those acting upon the C o n f l i c t of interest and bias in peer paper would be subject to the most demanding science may have a high degree of confidence review. Identifying and managing potential level of IQA objectivity standards. The under- t h a t the experimenters a) a d h e r e d to the conflicts of interest (COIs)—financial and non- lying data and methods would have to be pro- experimental protocol employed, b) took all financial—is critical to meeting the require- vided to the agency. Also, the agency (and the steps and measurements claimed to be ment for independent peer review. In turn, this hence the submitter) would have to be able to taken during conduct of the study itself, and requires distinguishing between COI and bias. document that the research reported in the c) accurately reported the test results. The National Academies (2003) defines paper was conducted in accordance with sound GLPs are neither required nor common in COI as any financial or other interest that and objective scientific practices. Finally, the research laboratories but have been imple- c o n f l i c t s with the service of an individual paper would need to be based on the best avail- mented at all U.S. EPA contract laboratories because it could impair the individual’s objec- able science, including any peer-reviewed work and at major universities that perform signifi- tivity or create an unfair competitive advan- in the literature. [Notably, by virtue of being c a n t medical and toxicology testing and tage for any individual or organization. provided to a federal agency, this information research for regulatory purposes. Sponsoring Conventionally speaking, some COIs are w o u l d become subject to the Freedom of organizations can require that research and “actual,” with an unambiguous potential for Information Act (FOIA 1966). Exemptions to testing studies comply with GLPs and can financial gain (e.g., the reviewer holds stock FOIA exist for privacy and commercial consid- incorporate that request into contractual vehi- in an entity likely to be affected by the rele- erations, although health effects data generally cles. As a result, GLPs have grown to be used vant regulatory action). Other actual COIs cannot be claimed as confidential business f a i r l y extensively outside the specific U.S. might be employment conflicts or close pro- information (Conrad 2006).] EPA and FDA regulatory contexts in which f e s s i o n a l financial relationships. Potential G o o d Laboratory Practice regulations. they are required. f i n a n c i a l COIs may exist on the part of B o t h t h e U.S. EPA and Food and Drug Peer review. While peer review has been an e x p e r t s remunerated by any individual or Administration (FDA) have adopted compara- integral part of medicine for centuries, it has o r g a n i z a t i o n (International Life Sciences ble sets of requirements specifying laboratory become a mainstay of the scientific process Institute 2005). Similarly, any scientist can practices and procedures that must be followed o n l y since the mid-twentieth century have a COI, depending on the topic. As the to ensure the quality and integrity of studies (Wikipedia 2007b). The National Research G o v e r n m e n t Accountability Office (GAO submitted to support agency decisions (FDA Council (NRC) has defined peer review as pro- 2001) has noted, association with industry 2006; U.S. EPA 2006a, 2006b). These Good viding an “in-depth critique of assumptions, does not by itself indicate a COI. Conversely, Laboratory Practice (GLP) standards prescribe calculations, extrapolations, alternate interpre- association with an environmental group does essential, routine features of sound laboratory tations, methodology, and acceptance criteria not inoculate a scientist against COIs. There science. All studies submitted to these agencies employed and conclusions drawn in the origi- well may be COIs among academics, because in connection with the relevant statutory pro- nal work” (NRC 1998). At a minimum, peer of intense competition for research funds and grams must be conducted in accordance with r e v i e w must exhibit the following features publications, that are difficult to identify. these standards. The GLP rules are more rigor- (International Life Sciences Institute 2005): An individual with a COI generally may ous than standards followed at university labo- • It must include multiple assessments. not participate in a peer review. Organizations ratories (Anderson et al. 2001). • It must be conducted by scientists with no that use peer-review systems require candi- GLPs have three basic elements: quality d i r e c t connection to the research, or its dates to disclose their organizational affilia- assurance, standard operating procedures, and sponsors. t i o n s , financial interests, personal and study protocols. • It must be conducted by scientists who have professional involvement, and other informa- Quality assurance. GLPs mandate docu- experience with or expertise in the research tion that may be pertinent to the topic, erring mentation of study conduct and results and in question. on the side of full disclosure, and to certify ensure that a full record of the study is pre- A rigorous peer review is a key part of the that the information is true and accurate to served for subsequent review, if necessary. foundation on which scientific excellence is the best of their knowledge. S t a n d a r d operating procedures. T h e r e achieved in all research programs. Science is “Bias” is a partiality or loss of objectivity must be written procedures for accurate and not “self-evident” and requires scientific judg- because of personal views or positions and may full data collection under the nonclinical lab- ment, and because judgments vary, a well- be perceived as arising from the close identifi- oratory study methods to be used, methods b a l a n c e d representation of intellectual cation or association of an individual with a determined by management to be adequate to perspectives is needed. Where that is obtained, particular point of view or with a particular ensure the quality and integrity of the data. the peer review is more likely to rebut any sci- group that may be affected by the research Study protocols. Each study must state entifically untoward or untenable hypothesis being reviewed. For example, questions about the objectives and all methods for the conduct and arrive at a successful review. t h e neutrality of a reviewer could arise if 137 Environmental Health Perspectives • VOLUME 116 | NUMBER 1 | January 2008
  3. 3. Henry and Conrad that person had represented an interest group OMB’s peer review bulletin. Many fed- Although journals have instituted disclo- at a hearing. eral agencies have written policies for when sure policies (sometimes termed “competing I t is generally recognized that bias is and how to conduct agency peer reviews (U.S. financial interests policies”) to increase trans- pervasive and not inherently undesirable. A EPA Science Policy Council 2000). To bring parency for their readers, granting agencies s u b c o m m i t t e e of the U.S. EPA Science greater consistency to and establish minimum and sponsors employ a variety of approaches A d v i s o r y Board (2000) has opined that, requirements for such reviews, the OMB and and requirements, and it is too early to con- “[a]lthough it is possible to avoid conflict of the Office of Science and Technology Policy clude that a broad consensus practice has been i n t e r e s t , avoidance of bias is probably not in early 2005 issued their “Final Information established. The Long-Range Initiative (LRI) p o s s i b l e . All scientists carry bias due, for Q u a l i t y Bulletin for Peer Review” (OMB of the American Chemistry Council (ACC) example, to discipline, affiliation and experi- 2005). The bulletin requires all federal agen- requires its contractors to acknowledge ACC ence.” Potential sources of bias in peer review- cies to conduct peer reviews of all “influential” as a sponsor of the research in all articles or ers are managed through disclosure. scientific information—defined as under the publications pertaining to the research con- Federal regulations generally implement IQA—that the agency intends to disseminate. ducted under agreement with the ACC. these concepts. Although these rules address [As with the IQA, information disseminated Transparent research policies. Research only federal employees, they include “special as part of an adjudication (e.g., a permit deci- p r o g r a m s are commonly regarded as more government employees,” such as participants sion) is exempt from rules outlined in the bul- credible, and their results less suspect, when i n panels organized under the Federal letin, unless it is novel or precedential and peer they employ transparent processes and proce- Advisory Committee Act (1972), and there- review is practical.] dures regarding ownership of data, release of f o r e they apply in the case of federal peer The bulletin sets especially high standards results, and publication of results. This prac- review panels such as the U.S. EPA Science for “highly influential” scientific assessments. tice is growing for assessing scientific quality, Advisory Board. These rules prohibit a federal For these documents, a) agency scientists gen- although it has not yet achieved broad accep- employee from participating directly and sub- erally may not participate; b) reviewers must be tance within the research community. stantially in a particular matter (as opposed to provided with background information suffi- When investigators own the data and the “broad policy options”) that will have a direct cient to understand the key findings or conclu- s c i e n t i f i c information that they generate and predictable effect on a) a financial interest sions of the draft assessment; c) where feasible, through their research efforts, they are in con- of the employee (generally, employment or public comment should be sought and pro- trol of how those data will be evaluated, used, stock ownership); b) the employee’s employer; v i d e d to the reviewers; and d) t h e agency and communicated. Sponsors who choose to or c) organizations for which the person has should respond to the reviewers’ report. employ this approach do so to lend strength, served in the last year in a paid capacity or as Under the guidelines of the OMB bul- objectivity, and credibility to the outcome of an active participant, where a reasonable per- letin, peer review should now be applied sys- the research. The ACC LRI program includes son would question the person’s impartiality t e m a t i c a l l y to all influential scientific this element of data ownership (ACC 2006). in the matter, unless covered by an exclusion information published by federal agencies or With ownership of the data—the right to o r issued a waiver (Office of Government used by them to make decisions—regardless of r e l e a s e data independently and to publish Ethics 1997). why the work was conducted—in a manner without prior sponsor approval—inappropri- P e e r review at federal agencies. that is more rigorous and revealing than that a t e sponsor interference can be avoided. H i s t o r i c a l l y , the primary venues for peer occurring outside the federal government. Interactions with sponsors can be beneficial in review have been in journal publication and Disclosure and acknowledgment of funding providing specialized knowledge and insight at in grant evaluation. Increasingly, federal agen- sources. In the mid-1990s, concern arose about a level of detail other scientific resources might cies have been conducting more demanding the integrity of scientific research because of not provide. However, the final decision on peer reviews of studies that will form the basis increasing commercial links and consequent whether to accept the sponsor’s information or of important agency decisions. For example, influences. In response, the Department of advice remains with the investigator. the U.S. EPA has several standing advisory Health and Human Services (DHHS) and the Congress has required the federal govern- bodies to conduct peer reviews, including its National Science Foundation (NSF) issued m e n t to take a slightly different tack in Science Advisory Board, its Science Advisory their “Investigator Financial Disclosure Policy” the area of research transparency. As a result Panel (for pesticide program decisions), and (DHHS/NSF 1995). This policy required dis- o f the “Shelby Amendment” (Omnibus its Clean Air Science Advisory Committee. closure of an investigator’s significant financial Appropriations Act 1998), the OMB revised its Some agencies request the National Research interest if it “would reasonably appear to be Circular A-110 governing federal grants, and Council of the National Academies to con- affected” by activities funded or proposed for now all federal agencies must make available to vene peer reviews for very prominent issues funding by NSF and DHHS. the public, pursuant to FOIA, final research and assessments. Although the principles for As the scientific community became more d a t a generated by agency grantees that an journal and agency peer review are the same, aware of the potential COIs for university agency cites in support of a rule or order (OMB they involve very different practices, proce- investigators with commercial ties receiving 1999). Thus, when the federal government dures, and decision pathways from the results federal grant support, questions also emerged funds scientific work and relies on it to support and conclusions of the peer review. about the potential implications for interpreta- agency action with the force and effect of law, Agency peer reviews are complex, time tion of research results and conclusions. By the data produced by that work will be placed consuming, and most often involve forming a 2001 some of the major scientific journals had in the public domain if someone in the public p a n e l of expert peer reviewers, where the established policies to encourage authors to asks for it. This way, federally funded research panel will meet in public for their delibera- declare any competing financial interests in data can be available so that the scientific com- tions. In some cases, peer reviews are con- relation to research papers. Financial disclo- munity can validate the work through replica- ducted by scientists within the agency, and in sure forms are now routinely required to be tion. Although Circular A-110 does not apply other cases by external scientists. Prior criti- submitted with manuscripts for use by the edi- to privately funded research submitted to fed- c i s m s of agency peer reviews (GAO 2001) tors, and journal articles generally acknowl- eral agencies, the same prospect of disclosure is have only heightened the scrutiny applied to edge research support, although the practice is a n inevitable effect of the IQA, discussed them both within and outside agencies. not universal (Nature 2006; Science 2006). above, if the agency relies on that information. 138 116 | NUMBER 1 | January 2008 • Environmental Health Perspectives VOLUME
  4. 4. Science generated for regulatory activities Federal Standards processes. In effect, Congress and the courts science is premised on the idea that the self- have determined that in an open democratic interest of regulated parties that conduct or for Considering s o c i e t y administered by a bureaucracy sponsor research creates a conflict with the “Regulatory” Science r e q u i r e d to act fairly and rationally, it is interest of the truth, whether conscious or The foregoing discussion outlines the standards important that agencies allow interested or unconscious, that renders the research invalid that one can use to judge the quality of scien- affected persons to provide information to or at least suspect (Krimsky 2005). tific research without regard to the purpose for them, and fairly consider that information. At the outset, we acknowledge the grow- which it was conducted. The following discus- The backbone of federal administrative ing literature purporting to find that industry- sion explains why regulatory agencies should law is the Administrative Procedure Act (APA funded research produces results that favor its not—and arguably cannot—treat science cre- 1 9 4 6 ) , which requires agencies to provide sponsors more often than other research on ated for purposes of an agency proceeding dif- notice and “give interested persons an oppor- the same topic (Krimsky 2005; vom Saal and ferently in that proceeding than science not t u n i t y to participate in [a] rulemaking Hughes 2005). As the authors of one of those created for purposes of that proceeding. t h r o u g h the submission of written data, s t u d i e s has fairly observed in comparing T h e breadth of research and testing views, or arguments. . . .” Courts interpreting industry and government-funded studies, one obligations. The data proffered by regulated Section 553 of the APA have made clear that o r both of two things could be going on: parties in agency proceedings often are not an agency must consider and respond to— either “industry-funded scientists [are] under solely the results of self-interested efforts to rather than discount—all significant matters real or perceived pressure to find and publish influence those proceedings. Rather, in many put before it. In particular, courts have specif- only data suggesting negative outcomes” or cases the parties have been obligated by regu- ically rejected the notion that the APA should “government-funded scientists [are] under lation or order to conduct particular studies, provide some basis for insulating agency offi- real or perceived pressure to publish only data according to particular protocols, and to pro- cials from the input of regulated parties. In suggesting adverse outcomes” (vom Saal and vide the results of that work to the agency t h e words of former D.C. Circuit Judge Hughes 2005). Indeed, the source of a scien- (Conrad 2006). Patricia Wald (Sierra Club v. Costle 1981): tist’s funding may be less a cause of bias than For example, under Section 4 of the Toxic a n effect of it. As an editor of The Lancet Under our system of government, the very legiti- Substances Control Act (TSCA 1976), the (Horton 1997) has argued, financial conflicts m a c y of general policymaking performed by unelected administrators depends in no small part U.S. EPA has broad power to issue rules order- “may not be [more] influential” than underly- upon the openness, accessibility, and amenability i n g persons manufacturing, processing, or ing biases, because “interpretations of scien- of these officials to the needs of the public from importing a chemical to conduct further tests tific data will always be refracted through the which their ultimate authority derives, and upon regarding the chemical’s health or environmen- e x p e r i e n c e s and biases of the authors.” whom their commands must fall. . . . Furthermore, tal effects. TSCA Section 8(d) authorizes the S i m i l a r l y , a student of the science/policy the importance to effective regulation of continu- U.S. EPA to compel, by rule, manufacturers interface (Sarewitz 2004) has argued that i n g contact with a regulated industry, other affected groups, and the public cannot be underes- and importers of a given chemical to submit stripping out conflicts of interest and ideological timated. Informal contacts can . . . spur the provi- lists and copies of existing, unpublished health commitments to look at ‘what the science is really sion of information which the agency needs. a n d safety studies for the chemical. TSCA telling us’ can be a meaningless exercise [because] even the most apparently apolitical, disinterested In addition to setting a single set of quality mandates have resulted in “more than 50,000 scientist may, by virtue of disciplinary orientation, standards for federally disseminated informa- studies covering a broad range of health and view the world in a way that is more amenable to tion regardless of provenance, the IQA also ecological endpoints” being filed with the U.S. some value systems than others. authorizes “affected persons to seek and obtain EPA since 1976 (U.S. EPA 2003a). As vom Saal and Hughes argued in their arti- correction of information . . . disseminated by S i m i l a r l y , the Federal Insecticide, cle (2005), the appropriate technical response [a federal] agency that does not comply with F u n g i c i d e , and Rodenticide Act (1972) when confronted with science conducted by the guidelines issued by [OMB].” Far from requires any potential pesticide chemical to interested parties is to use that fact as an alert having their own information judged weighted u n d e r g o more than 100 s c i e n t i f i c tests to look, perhaps more deeply than one other- l e s s under the IQA, affected persons are addressing chemistry, health effects, environ- wise might have, for “what specific factors, empowered to use their own information to mental effects, and residue chemistry to deter- other than source of funding,” may be associ- obtain correction of government information mine whether it can be used safely (U.S. EPA ated with the results of that science. t h a t does not meet the common set of 2 0 0 6 c ) . Only after the information has S u c h an approach is clearly the only standards that should apply to any information undergone a thorough and rigorous review by proper one in the case of interested research disseminated by the government. the U.S. EPA can the product be “registered” submitted to federal regulatory agencies, sim- Several other administrative law statutes by the U.S. EPA for use to protect crops or ply because the concept of “conflict of inter- embody the same orientation toward inter- public health. est” is not employed in federal laws governing ested persons and their rights to submit infor- Most federal processes for evaluating the the regulatory process (aside from the govern- m a t i o n to federal agencies and have it safety of chemicals, including the FDA review ment ethics rules noted above). Indeed, no c o n s i d e r e d . These include the Regulatory of drug applications (Federal Food, Drug, and federal laws, rules, or policies express a pre- F l e x i b i l i t y Act (1980), the Paperwork Cosmetic Act 1938), depend heavily on pri- sumption that agencies in a given proceeding R e d u c t i o n Act (1980), and the Federal vately generated data. Indeed, historically and should ignore or give less weight to scientific Advisory Committee Act (1972). The latter for the foreseeable future, the vast majority of w o r k on the basis of who conducted or also requires advisory committees to be “bal- chemical and product testing has been and will funded it or, more to the point, whether it anced,” which should equally prohibit exclu- be borne by industry, not the public sector. was prepared specifically for the relevant pro- sion of, as well as domination by, any interest. Policies regarding the treatment of the result- ceeding. To the contrary, federal administra- The upshot of this authority is not that ing data need to bear this reality in mind. t i v e law, as interpreted by the courts, r e g u l a t e d agencies are bound to accept T h e rights of interested persons under generally evinces a congressional mandate that unquestioningly any information that is gen- federal administrative law. The notion that a g e n c i e s give interested or affected parties e r a t e d specifically for that proceeding. s c i e n c e generated for regulatory purposes a c c e s s to and input into administrative A g e n c i e s can and, indeed, must assess the s h o u l d be evaluated differently than other 139 Environmental Health Perspectives • VOLUME 116 | NUMBER 1 | January 2008
  5. 5. Henry and Conrad validity of information upon which they rely. challenges in “establishing quality assurance also no question that many of the academics But they cannot adopt blanket approaches practices to improve the reliability, accuracy, working on these issues are highly invested that judge science generated for a regulatory and scientific basis of environmental data” both in their hypotheses and in the regulatory proceeding differently than other science con- ( U . S . EPA Inspector General 2002). The uses of their findings. Although these scien- sidered in that proceeding. Inspector General’s memo expressed similar tists may not have a financial or other tangible W h a t sorts of proceedings and entities concerns with respect to the accuracy and reli- stake in any particular regulatory proceeding, would be covered. The next hurdle in imagin- ability of information generated by U.S. EPA it seems artificial and formalistic to say that ing a system that imposed different standards c o n t r a c t o r s . Critics argued that there was their research is not being conducted, at least on science created specifically for a given reg- ample justification for the agency to apply its i n part, so that its results can be used in ulatory proceeding is to consider what sorts of proposed assessment factors to that informa- regulatory proceedings. agency “proceedings,” and what sorts of enti- tion, as well as to information submitted by Thus, any effort to establish special rules ties conducting or sponsoring science, would third parties. f o r consideration of science generated for be covered. These turn out not to be simple Critics of the U.S. EPA draft assessment regulatory proceedings will face difficult defin- determinations in all cases. factors also argued that assessment factors for itional challenges regarding what is a “pro- W h e r e testing is being conducted by a external information created the undesirable c e e d i n g , ” even more difficult evidentiary m a n u f a c t u r e r of a chemical or product appearance of a double standard and opened challenges determining whether and the extent (e.g., an exposure study conducted in support t h e door to arbitrary agency decisions to to which scientific work was being conducted of a pesticide’s reregistration), there is little exclude otherwise appropriate information for such proceedings, and politically loaded question that the research is being conducted received from external sources on the basis of challenges over when “unaffiliated” or “acade- f o r those proceedings. But manufacturers the selective application of assessment factors mic” work was in fact being conducted, at frequently conduct or sponsor research and to information products. Most important, least in part, for regulatory purposes. t e s t i n g for product stewardship and other they contended that the standards the agency Case study: the U.S. EPA assessment fac- business reasons. In some cases, that work may offered for judging the quality and reliability t o r s for external information. T h e issues also be useful in some other agency “proceed- o f third-party data were no different than raised above ultimately led the U.S. EPA to ing” (e.g., establishing a “reference concentra- those that should be applied to evaluate infor- abandon a related effort: to establish guide- tion” value in the U.S. EPA Integrated Risk mation generated by the agency itself, U.S. lines that treated “external” information dif- Information System database (http://www. EPA contractors, or U.S. EPA permittees, and ferently than information whose generation Such a proceeding may be ongo- hence a single set of assessment factors should the U.S. EPA controlled. ing, or the company may know the agency is apply to all. Early on, the U.S. EPA realized that the contemplating it, or the company may plan to When the U.S. EPA finalized the assess- IQA would apply to information generated p r o p o s e that the agency initiate it. What ment factors document, it clarified that “the by third parties that the agency relied upon or would the rules be in such “mixed-motive” document does not constitute a new standard otherwise disseminated. The U.S. EPA draft cases? Conversely, such an agency proceeding for information quality, nor does it describe a “ A s s e s s m e n t Factors for Evaluating the may arise later and be truly unanticipated by new process for evaluating third party infor- Q u a l i t y of Information from External the regulated entity. What sort of evidentiary m a t i o n . ” Second, and more important, it Sources” (U.S. EPA 2002b) noted that process would have to be established to deter- added that, “in general, we agree that consis- the Agency . . . receives information that is volun- mine what the research proponent intended or tent standards of quality should apply to both tarily submitted to EPA by external sources (‘third knew at the time research was initiated? All internally and externally generated informa- parties’) in hopes of influencing Agency actions. . . . The purpose of this document is to describe sets of t h e s e circumstances would need to be tion, when used for the same purposes” (U.S. ‘assessment factors’ that illustrate the types of con- addressed in a system that tried to treat “regu- EPA 2003b). siderations that EPA takes into account when evalu- latory proceeding” science differently. ating the quality and relevance of information that Conclusion Moreover, many regulatory settings are is voluntarily submitted or that we obtain from O n l y one set of standards and practices more than bilateral; that is, entities other than e x t e r n a l sources in support of various Agency should be used to judge the quality of scien- the agency and a single regulated party may actions. tific work in a given regulatory proceeding, be able to submit scientific information. Even The balance of the document consisted of an regardless of why the work was conducted. It if one accepted the premise that science pre- e l a b o r a t i o n on five “categories of general may be that, over time, more of these prac- pared for purposes of a proceeding should be assessment factors”: soundness, applicability tices and standards will apply to all scientific treated differently than other science, there is and utility, clarity and completeness, uncer- information. Many of these hallmarks of sci- no inherent reason that science prepared for a t a i n t y and variability, and evaluation and entific quality are incorporated into federal p r o c e e d i n g by opponents of the permit review. l a w , rules, and policy. These same federal should be treated differently than science pre- Critics argued that there is no basis, under authorities impose additional standards that pared by its proponents. Indeed, many regu- t h e IQA or any other legal authority, or further ensure the quality of scientific work l a t o r y proceedings have multiple parties i n d e e d , on any technical grounds, for the generated or submitted for regulatory pur- aligned with and against the agency and other U.S. EPA to assess “external” or “third-party” poses. Federal laws also ensure that interested parties on different issues in complex ways, information by different standards than first- parties have a right to submit information for making it difficult in many cases to determine or second-party information. On technical regulatory proceedings and to have that infor- who is on anyone’s “side.” grounds, critics noted that information gener- mation considered fairly and on its merits. Finally, it will often be difficult to demar- ated by the U.S. EPA or its contractors is not Any system of differential treatment for regu- cate “nonregulatory” science, given the extent immune from the same types of errors associ- latory science would face severe scrutiny in to which academic scientists are participants ated with information from external sources. light of that authority and would be difficult or are at least partisans in regulatory disputes. For example, the U.S. EPA Inspector General to administer. Most important, it would not Although highly controversial issues generally had just issued a memorandum noting that n e c e s s a r i l y lead to an improved scientific raise important intellectual questions (e.g., t h e agency faced a number of unresolved foundation for regulations. the effects of pollutants at low doses), there is 140 116 | NUMBER 1 | January 2008 • Environmental Health Perspectives VOLUME
  6. 6. Science generated for regulatory activities REFERENCES U . S . EPA (U.S. Environmental Protection Agency). 2002b. N a t i o n a l Research Council. 1998. Peer Review in the Developing assessment factors for evaluating the quality of E n v i r o n m e n t a l Technology Development Programs. information from external sources. Fed Reg 67:57225–57226. Washington, DC:National Academies. ACC (American Chemistry Council). 2006. Long-Range Research U . S . EPA (U.S. Environmental Protection Agency). 2003a. National Research Council. 2003. Peer Review in the Department Initiative—Funding Opportunities. Available: http://www. O v e r v i e w : Office of Pollution Prevention and Toxics of Energy—Office of Science and Technology—Interim = funding [accessed 16 April 2007]. Programs. Washington, DC: U.S. Environmental Protection Report. Washington, DC:National Academies. APA (Administrative Procedure Act). 1946. Public Law 79-404. A g e n c y . Available: Nature. 2006. Competing Financial Interests. Available: http:// Anderson W, Parsons B, Rennie D. 2001. Daubert’s backwash: oppt101c2.pdf [accessed 16 April 2007]. l i t i g a t i o n - g e n e r a t e d science. Univ Mich J Law Reform U . S . EPA (U.S. Environmental Protection Agency). 2003b. [accessed 16 April 2007]. 34:619–682. Clarification of Issues Identified Through Public Input on O f f i c e of Government Ethics. 1997. Standards of Ethical Barrow CB, Conrad JW. 2006. Assessing the reliability and credi- t h e Environmental Protection Agency (EPA) Draft Conduct for Employees of the Executive Branch. 5 CFR bility of industry science and scientists. Environ Health Assessment Factors Document. Copy on file with author. 2635. Washington, DC:U.S. Office of Government Ethics. Perspect 114:153–155. Available: http://www.ehponline. U.S. EPA (U.S. Environmental Protection Agency). 2006a. Good OMB (Office of Management and Budget). 1999. Circular A-110. org/mem/2005/8417/8417.pdf [accessed 16 April 2007]. Laboratory Practice Standards. 40 CFR 160. Washington, Uniform administrative requirements for grants and agree- Conrad, JW. 2006. Open secrets: the widespread availability of DC:U.S. Environmental Protection Agency. ments with institutions of higher education, hospitals, and information about the health and environmental effects of U.S. EPA. 2006b. Good Laboratory Practice Standards. 40 CFR other non-profit organizations. Fed Reg 64:54926. chemicals. Law Contemp Probs 69:141–165. 7 9 2 . Washington, DC:U.S. Environmental Protection OMB (Office of Management and Budget). 2002. Guidelines for DHHS/NSF (Department of Health and Human Services/National Agency. ensuring and maximizing the quality, objectivity, utility and Science Foundation). 1995. Investigator financial disclosure U.S. EPA. 2006c. Pesticides: Regulating Pesticides. Washington, integrity of information disseminated by federal agencies. policy. Fed Reg 60:35820. D C : U . S . Environmental Protection Agency. Available: Fed Reg 67:8452–8460. FDA (Food and Drug Administration). 2006. Good Laboratory OMB (Office of Management and Budget). 2005. Final informa- Practice Standards. 21 CFR 58. Washington, DC:U.S. Food [accessed 16 April 2007]. tion quality bulletin for peer review. Fed Reg 70:2664. and Drug Administration. U . S . EPA Inspector General. 2002. EPA’s Key Management Omnibus Appropriations Act. 1998. Public Law 105-277. Federal Advisory Committee Act. 1972. Public Law 74-220. Challenges. Washington, DC:U.S. Environmental Protection Paperwork Reduction Act. 1980. Public Law 96-511. Federal Food, Drug, and Cosmetic Act. 1938. Ch. 675. Agency. Available: Regulatory Flexibility Act. 1980. Public Law 96-354. F e d e r a l Insecticide, Fungicide, and Rodenticide Act. 1972. topchallenges2002.pdf [accessed 4 December 2007]. Sarewitz D. 2004. How science makes environmental contro- Public Law 92-516. U . S . EPA Science Advisory Board Environmental Health versies worse. Environ Science Policy 7:385–403. FOIA (Freedom of Information Act). 1966. Public Law 89-554. Committee. 2000. Review of the Draft Report to Congress S c i e n c e . 2006. Statement on Real or Perceived Conflicts of GAO (Government Accountability Office). 2001. EPA’s Science ‘Characterization of Date Uncertainty and Variability in IRIS Interest for Authors. Available: http://www.sciencemag. Advisory Board Panels: Improved Policies and Procedures Assessments, Pre-Pilot vs Post-Pilot.’ Washington, DC: U.S. org/about/authors/prep/coi.pdf [accessed 16 April 2007]. Needed to Ensure Independence and Balance. Washington, Sierra Club v. Costle. 1981. Case 79-1565, U.S. Court of Appeals Environmental Protection Agency Available: http://www. DC:Government Accountability Office. for the D.C. Circuit. [accessed 16 April 2007]. Horton R. 1997. Conflicts of interest in clinical research: oppro- SKAPP (Project on Scientific Knowledge and Public Policy). U.S. EPA Science Policy Council. 2000. Peer Review Handbook. brium or obsession? Lancet 349:1112–1113. 2 0 0 6 . Coronado Conference III. Available: http://www. Washington, DC:U.S. Environmental Protection Agency. Information Quality Act. 2000. Public Law 106-554. Vom Saal FS, Hughes C. 2005. An extensive new literature con- I n t e r n a t i o n a l Life Sciences Institute. 2005. Scientific Peer nado-Conference-Papers.cfm [accessed 16 April 2007]. cerning low-dose effects of bisphenol A shows the need for Review to Inform Regulatory Decisions. Washington, DC: TSCA (Toxic Substances Control Act). 1976. Public Law 94-469. a new risk assessment. Environ Health Perspect 113: International Life Sciences Institute. U . S . EPA (U.S. Environmental Protection Agency). 2002a. 926–933. Krimsky, S. 2005. The funding effect in science and its implica- G u i d e l i n e s for Ensuring and Maximizing the Quality, Wikipedia. 2007. Scientific Method; Evaluation and Iteration; tions for the judiciary. J Law Policy 13:43–68. Objectivity, Utility and Integrity of Information Disseminated C o n f i r m a t i o n . Available: National Academies. 2003. Policy on Committee Composition b y the Environmental Protection Agency. Available: Scientific_method [accessed 16 April 2007]. and Balance and Conflicts of Interest for Committees Used Wikipedia. 2007b. Peer review; Confirmation. Available: http:// in the Development of Reports. Washington, DC:National d o c u m e n t s / E P A _ I n f o Q u a l i t y G u i d e l i n e s . p d f [accessed (accessed 4 December A c a d e m i e s . Available: http://www.nationalacademies. 16 April 2007]. 2007). org/|coi/bi-coi_form-0.pdf [accessed 16 April 2007]. 141 Environmental Health Perspectives • VOLUME 116 | NUMBER 1 | January 2008