CMRN Home
RTK or Left-To-Wonder?
Recommended Reading


Water Pollution: Poor Quality Assurance and Limited
Pollutant Coverage Undermine EPA's Control of Toxic Substances

Source: Chapter Report, 02/17/94,
GAO/PEMD-94-9



The Environmental Protection Agency (EPA) controls toxic pollutant
discharges into waterways through seven "core" activities spread across
several programs.  GAO found that some of the information used to
analytically support these activities may be of doubtful quality,
raising questions about the effectiveness of these activities in
controlling toxic pollutants.  GAO also found that the current permit
process does not limit the vast majority of toxics being discharged from
the nation's factories and sewage treatment plants.  Although most of
these toxicants are "nonpriority" pollutants, they do pose risks to both
human health and aquatic life.  GAO tried to examine the implications of
uncontrolled pollution cases identified in the facility sample
population, but the majority of cases could not be evaluated because of
a lack of criteria for assessing the health risks posed by the
discharges.  This is so for most toxicants discharged across the nation.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  PEMD-94-9
     TITLE:  Water Pollution: Poor Quality Assurance and Limited 
             Pollutant Coverage Undermine EPA's Control of Toxic
             Substances
      DATE:  02/17/94
   SUBJECT:  Quality assurance
             Water pollution
             Water pollution control
             Wastewater management
             Toxic substances
             Environmental policies
             Monitoring
             Risk management
             Data bases
             Health hazards
IDENTIFIER:  Superfund Program
             EPA National Pollutant Discharge Elimination System
             EPA Permit Compliance System
             EPA Water Data Storage and Retrieval System
             EPA Toxics Release Inventory Program
             EPA Great Lakes Water Quality Initiative
             NPDES
             
**************************************************************************
* This file contains an ASCII representation of the text of a GAO        *
* report.  Delineations within the text indicating chapter titles,       *
* headings, and bullets are preserved.  Major divisions and subdivisions *
* of the text, such as Chapters, Sections, and Appendixes, are           *
* identified by double and single lines.  The numbers on the right end   *
* of these lines indicate the position of each of the subsections in the *
* document outline.  These numbers do NOT correspond with the page       *
* numbers of the printed product.                                        *
*                                                                        *
* No attempt has been made to display graphic images, although figure    *
* captions are reproduced. Tables are included, but may not resemble     *
* those in the printed version.                                          *
*                                                                        *
* A printed copy of this report may be obtained from the GAO Document    *
* Distribution Facility by calling (202) 512-6000, by faxing your        *
* request to (301) 258-4066, or by writing to P.O. Box 6015,             *
* Gaithersburg, MD 20884-6015. We are unable to accept electronic orders *
* for printed documents at this time.                                    *
**************************************************************************


Cover
================================================================ COVER


Report to the Chairman, Subcommittee on Superfund, Ocean, and Water
Protection, Committee on Environment and Public Works, U.S.  Senate

February 1994

WATER POLLUTION - POOR QUALITY
ASSURANCE AND LIMITED POLLUTANT
COVERAGE UNDERMINE EPA'S CONTROL
OF TOXIC SUBSTANCES

GAO/PEMD-94-9

Water Pollution


Abbreviations
=============================================================== ABBREV

  ASTM - American Society of Testing and Materials
  BAT - Best available technology
  BOD - Biochemical oxygen demand
  EPA - Environmental Protection Agency
  GAO - General Accounting Office
  ICS - Individual control strategy
  NPDES - National Pollutant Discharge Elimination System
  PCS - Permit Compliance System
  POTW - Publicly owned treatment works
  QA/QC - Quality assurance/quality control
  STORET - STOrage and RETrieval
  TMDL - Total maximum daily load
  TRI - Toxic Release Inventory
  USGS - U.S.  Geological Survey
  WET - Whole effluent toxicity

Letter
=============================================================== LETTER


B-249852

February 17, 1994

The Honorable Frank R.  Lautenberg
Chairman, Subcommittee on Superfund, Ocean,
 and Water Protection
Committee on the Environment and Public Works
United States Senate

Dear Mr.  Chairman: 

In a January 6, 1992, letter, you asked us to evaluate the quality of
the information EPA uses to control the discharge of toxic pollutants
to surface water and the extent to which EPA controls these
substances.  This report presents our findings on these issues. 

In our evaluation, we identified the EPA activities that have a
direct effect on controlling the release of toxics to surface water
and the information requirements for these activities.  We also
evaluated the extent to which EPA takes steps to ensure that the
information used to support these activities is of acceptable
quality.  In addition, we addressed the extent to which toxic
pollutants are controlled through the NPDES permit process and
examined the risk implications of discharged pollutants that are
uncontrolled. 

If you have any questions or would like additional information,
please call me at (202) 512-2900 or Kwai-Cheung Chan, Director of
Program Evaluation in Physical Systems Areas, at (202) 512-3092. 
Other major contributors to this report are listed in appendix III. 

Sincerely yours,

Eleanor Chelimsky
Assistant Comptroller General


EXECUTIVE SUMMARY
============================================================ Chapter 0


   PURPOSE
---------------------------------------------------------- Chapter 0:1

Toxic pollutants threaten the biological integrity of the nation's
water.  State officials say that toxic pollutants affect the quality
of more than 28,000 miles of the nation's rivers.  Aquatic life has
been affected and human health risk has increased.  Contamination by
toxic and conventional pollutants has resulted in fishing advisories
or bans on more than 1,000 bodies of water.  Pollutants discharged
from the nation's factories and sewage treatment plants are a major
cause of this water and fish contamination. 

To protect surface water from pollutants discharged from industrial
facilities and sewage treatment plants, the Congress passed
legislation in 1972 commonly referred to as the Clean Water Act.  The
Environmental Protection Agency (EPA) is responsible for setting
national standards on the types and amounts of pollutants that
industries and sewage treatment plants may discharge.  EPA and the
states enforce the standards by incorporating toxic limits on
facilities' discharge permits.  Many factors influence EPA's ability
to implement programs authorized under the Clean Water Act.  One
critical variable is the quality of available information.  EPA's
program activities are analytical efforts, resulting in decisions
about which toxic pollutants to control and at what levels.  If its
information is not of high quality, toxic pollutant control
activities are weakened in consequence. 

The Chairman of the Subcommittee on Superfund, Ocean, and Water
Protection of the Senate Committee on the Environment and Public
Works asked GAO to evaluate the quality of the information EPA uses
to control the discharge of toxics to surface water and the extent to
which the EPA process controls toxic substances. 

GAO (1) identified EPA activities that directly affect the control of
the toxic discharges and the information requirements of these
activities, (2) determined EPA's steps to ensure that the information
it uses to support these activities is of acceptable quality, (3)
addressed the extent to which permits control toxic pollutants, and
(4) examined the risk implications of uncontrolled discharged
pollutants. 


   BACKGROUND
---------------------------------------------------------- Chapter 0:2

For a number of years, EPA has been faced with reports by GAO and
others that individual activities it administers to control toxic
substances in surface water are producing information of suspect
quality.  Information about three specific types of entities--the
environment, the facilities that discharge toxic pollutants, and the
toxic substances themselves--is at the heart of any system for
controlling toxics.  Yet the quality of that information across EPA's
approach as a whole has not been evaluated. 

To identify the critical activities that control the release of toxic
substances into waterways, GAO submitted to EPA an extended list from
the literature and agency documents.  EPA identified activities that
are directly related to controlling discharges of toxic substances
into surface waters.  A panel of experts reviewed and validated EPA's
list.  GAO then interviewed managers for each of the principal
activities to identify information requirements as well as the
principal sources of that information.  Subsequently, GAO compiled
data quality assurance criteria from the scientific literature and
from EPA's quality assurance guidelines, assigning the relevant,
critical quality assurance criteria to each information requirement
for each EPA activity.  (A criterion was considered critical if its
absence, in and of itself, would compromise the quality of the
information required by an activity.) GAO then reinterviewed managers
and examined data to estimate the extent to which the relevant
quality assurance criteria were followed in the design, collection,
and analysis of the information.  Finally, GAO obtained discharge
permit files for 236 industrial facilities and compared a list of
their toxic limits with information from three sources about the
facilities' actual discharges. 


   RESULTS IN BRIEF
---------------------------------------------------------- Chapter 0:3

EPA implements the control of toxic pollutant discharges into
waterways through 7 "core" activities spread across a number of
programs.  GAO identified 13 types of information that are required
to analytically support these 7 activities and found that 5 of the 7
fell short in implementing the quality assurance steps that are
needed to produce accurate information.  This raises questions not
only about the quality of the information these 5 activities produced
but also about their effectiveness within a strategy for controlling
toxic pollutants. 

GAO also found that the current permit process does not limit the
vast majority of toxics being discharged from the nation's
facilities.  Although most of these toxicants are "nonpriority"
pollutants, they can and do pose human health and aquatic life risks
(for example, ammonia and xylene). 

GAO attempted to examine the risk implications of uncontrolled
pollution cases identified in the facility sample population, but the
majority of cases could not be evaluated because the criteria were
lacking by which to assess whether discharges posed a human health or
aquatic life risk.  This is so for most toxicants discharged across
the nation. 


   PRINCIPAL FINDINGS
---------------------------------------------------------- Chapter 0:4


      "CORE" PROGRAMS AND THEIR
      CRITICAL INFORMATION
      REQUIREMENTS
-------------------------------------------------------- Chapter 0:4.1

GAO initially identified 46 EPA water quality protection activities
and then refined this list to the 7 that are directly related to the
control of toxic pollutants.  Taken together, these activities
determine the toxic concentrations that are safe for human health and
aquatic life, establish allowable discharge limits, identify impaired
waterways, and monitor compliance infractions.  Thirteen types of
information are required to support these 7 core activities.  (See
figure 1.)

   Figure 1:  Thirteen Types of
   Information A-M Required Within
   Activities 1-7

   (See figure in printed
   edition.)


      ADHERENCE TO QUALITY
      ASSURANCE
-------------------------------------------------------- Chapter 0:4.2

Two of the 7 activities (effluent guidelines development and water
quality criteria development) have been exceptional in meeting
quality assurance requirements, and confidence in the integrity of
the information they have produced is thus relatively high. 
Collectively, these 2 activities have met approximately 93 percent of
their related quality assurance requirements.  However, the 5 other
activities fell short of the criteria GAO used, meeting only 58
percent of their requirements.  These activities (impaired water and
point source identification, total maximum daily load development,
water-quality-based permit issuance, local limits development, and
compliance problem identification) are critical to the success of a
standards-based toxics control program. 


      EXTENT OF CONTROL OF TOXIC
      POLLUTANTS
-------------------------------------------------------- Chapter 0:4.3

GAO reviewed the pollutants discharged from 236 facilities from three
industrial sectors (pulp and paper, pharmaceutical, and pesticides
manufacturing).  The vast majority of the toxic pollutant cases GAO
identified (77 percent) were not controlled through the permit
process.  Priority pollutants (for example, dichlorobenzene and
carbon tetrachloride) made up 33 percent of these cases while the
remainder were nonpriority pollutants.  GAO found that, for the
sample of facilities evaluated, the permit process rarely controlled
nonpriority toxic pollutants (with the exception of acids, ammonia,
and chlorine) and priority pollutants only partially.  It should not
be presumed that the former are innocuous:  nonpriority pollutants
can pose human health and aquatic life risks.  In calculating the
percentage of nonpriority pollutants reported in EPA's Toxic Release
Inventory for 1989, GAO found that that group makes up 98 percent of
the discharges by mass.  These pollutants are reported in the
inventory because they are used in commerce and because their release
to the environment could pose human health risks. 


      THE RISK IMPLICATIONS OF
      UNCONTROLLED TOXIC
      POLLUTANTS
-------------------------------------------------------- Chapter 0:4.4

GAO used standard modeling methods to estimate the water
concentration of uncontrolled toxic substances from the sample
population of facilities mentioned above to determine what proportion
presented human health or aquatic life risks.  For cases that GAO
could evaluate, only 8 percent did pose such risks.  However, GAO
could not evaluate many of the cases (62 percent) because the
criteria needed for determining whether their discharge posed a risk
had not been established. 


   MATTER FOR CONGRESSIONAL
   CONSIDERATION
---------------------------------------------------------- Chapter 0:5

The Clean Water Act emphasizes a standards approach for controlling
toxic pollutants.  In its reauthorization of the act, the Congress
should consider augmenting this approach with additional authority to
allow EPA to emphasize pollution prevention as a way of managing
toxic pollutant discharges. 


   RECOMMENDATIONS
---------------------------------------------------------- Chapter 0:6

To improve EPA's efforts to control discharges of toxic water
pollutants, GAO recommends that the Administrator of EPA

  initiate immediate efforts to address the information quality
     assurance problems GAO identified in the 5 toxic control
     activities where these problems occur and

  expand the use of the Toxic Release Inventory data base to identify
     nonpriority pollutants being discharged to water that should be
     considered for control through the permit process. 


   AGENCY COMMENTS
---------------------------------------------------------- Chapter 0:7

GAO discussed the results of its work, including tentative findings,
conclusions, and recommendations, with responsible EPA officials and
has incorporated their comments where appropriate.  Their major
comments and GAO's responses are included in chapter 6. 


INTRODUCTION
============================================================ Chapter 1

Toxic pollutants pose a substantial threat to the biological
integrity of the nation's water.  Reports prepared by the states
under section 304(l) of the Clean Water Act cite toxic discharges
from 879 factories and sewage treatment plants around the nation as
causing streams to violate water quality standards.\1 State
governments have also identified toxic pollutants as affecting the
quality of more than 28,000 miles of the nations' rivers.\2 Moreover,
state officials have reported that 15 percent of monitored river
miles and 39 percent of lake areas were contaminated by toxic
substances.\3 Contamination by toxic and conventional pollutants
resulted in more than 1,000 bodies of water around the country with
fishing advisories or bans.\4

In an effort to control discharges of toxic pollutants from factories
and sewage treatment plants, EPA, the agency with jurisdiction in
matters of toxic pollution of waterways, has developed an approach
that includes the collection of three different types of information. 
Under this approach, information about the environment, the
facilities that dispose of toxic pollutants, and the toxic substances
themselves provides the foundation for controlling toxics.  Ensuring
information of adequate quality about these elements, then, is
critical to the approach's success.  Adhering to information and data
quality assurance standards is a requisite for attaining such
quality.  In short, decisions made within the EPA water quality
programs can be only as good as the data on which they are based.  At
minimum, those data should be of high enough quality to appropriately
and accurately characterize the event or object of interest.  In the
recent past, concerns have been raised about the quality of
information EPA used in its water protection program.\5

The Chairman of the Subcommittee on Superfund, Ocean, and Water
Protection of the Senate Committee on Environment and Public Works
asked us to evaluate (1) how well EPA ensures information and data
quality in the water toxic control program and (2) the extent to
which the program controls toxic discharges.  This report responds to
that request. 


--------------------
\1 Bureau of National Affairs, Environment Reporter, June 23, 1989,
p.  466. 

\2 Environmental Protection Agency, National Water Quality Inventory: 
1990 Report to Congress (Washington, D.C.:  1992), p.  88. 

\3 Environmental Protection Agency, p.  86. 

\4 Environmental Protection Agency, p.  90. 

\5 U.S.  General Accounting Office, Environmental Enforcement:  EPA
Cannot Ensure the Accuracy of Self-Reported Compliance Monitoring
Data, GAO/RCED-93-21 (Washington, D.C.:  March 31, 1993), and Water
Pollution Monitoring:  EPA's Permit Compliance System Could Be Used
More Effectively, GAO/IMTEC-92-58BR (Washington, D.C.:  June 22,
1992). 


   BACKGROUND
---------------------------------------------------------- Chapter 1:1

As already noted, dealing with toxic discharges within the framework
of the Clean Water Act requires information on the aquatic
environment, the facilities that discharge the pollutants, and toxic
substances.  Collecting this information is not easy, because of both
the complexity and variability of the subject matter and the
technical and methodological challenges associated with measurement,
data collection, and analysis.  EPA has created a framework to
collect data on all these elements, as described below. 


      THE AQUATIC ENVIRONMENT
-------------------------------------------------------- Chapter 1:1.1

The nation's surface-water resources are truly immense.  They consist
of more than a million miles of rivers and streams (enough to extend
from the Earth to the Moon four times).  There are 61,000 square
miles of inland water bodies and 94,000 square miles of the Great
Lakes.  Taken together, this constitutes an area larger than the
combined size of 12 states. 

Besides being vast, the aquatic environment is highly complex.  It is
composed of different "compartments," including the "water column,"
sediment, and aquatic life.  Marshes, wetlands, lakes, rivers,
streams, estuaries, and open oceans constitute greatly diverse
habitats for aquatic life.  Within both the water column and the
sediment layers are varied collections of aquatic life, both animals
and plants, with interdependent life cycles.  Site-specific
characteristics of a body of water (such as temperature, pH, and
hardness) influence the areas in which different species reside and
their sensitivity to stresses from toxic substances and other
pollutants. 


      DISCHARGES INTO THE AQUATIC
      ENVIRONMENT
-------------------------------------------------------- Chapter 1:1.2

For as long as people have lived in villages and towns there has been
pollution of rivers and streams.  Many of the epidemics of the Middle
Ages in Europe were caused by exposure to polluted waters.  In the
United States, the industrial revolution of the 18th and 19th
centuries and the accompanying increases in manufacturing increased
discharges, sowing the seeds of a water quality crisis for the
nation's lakes, rivers, and streams.  The chemical revolution of the
20th century dramatically increased the complexity of these
discharges. 

These bodies of water serve as sources of drinking water for about
half the nation's population and have recreational uses as well. 
They are also the repository for wastes from about 64,000 factories
and sewage treatment plants.  Of these dischargers, about 7,000 are
considered "major dischargers" under federal legislation.  Industrial
processes have developed around the ability to dispose of industrial
wastes into adjacent bodies of water.  Disposing of wastes in this
way is integral to modern industry.  Billions of pounds of pollutants
are released into the nation's waters every year from these "point
sources" of pollution.\6


--------------------
\6 Discharges from specific locations are called point source
discharges.  Pollution that does not enter the water from a pipe is
"nonpoint" pollution.  Examples of sources of nonpoint pollution are
farms, fields, and highways. 


      TOXIC POLLUTANTS
-------------------------------------------------------- Chapter 1:1.3

More than 65,000 chemicals are manufactured or used in the United
States; over 1,000 new substances are introduced each year.  A
contrast is often drawn between "toxic" and "conventional"
pollutants.\7 The short list of conventional pollutants, so-named
because they were traditionally the objects of wastewater treatment
and control, consists of the following:  biochemical oxygen demand
(BOD), coliform, oil and grease, pH, and total suspended solids.  The
toxic effects produced by conventional pollutants are usually
immediately apparent.  Too high a concentration of nitrogen or
phosphorus in a lake can turn it green with algae.  Too high a level
of BOD in a lake or stream will deplete it of oxygen and suffocate
the aquatic life residing in it. 

Toxic pollutants usually have more subtle effects on the health of
the ecosystem and those who come into contact with it.  These effects
can range from harming the reproductive cycle of the wildlife in a
stream to causing cancer in people who drink that water.  In
addition, these substances often produce their toxic effect at very
low concentrations that may be beyond the capabilities of laboratory
instruments to detect. 

Examples of toxic pollutants are some metals such as mercury and
chromium; pesticides such as DDT and 2,4,5-TP; and organic chemicals
such as PCBs and dioxins.  Toxic substances can affect all organisms
that dwell in water.  When toxic substances settle, they can also
harm the organisms that dwell in sediments.  Further, there is
variation in the amount of time it takes for toxic substances to
produce harmful effects.  Some toxic substances have an immediate
action; others take a longer period of time.  A long-term effect of
some toxic substances is bioaccumulation, or the transfer of a
substance from the environment into living tissues through feeding
and bioconcentration (the passing of a substance from water into an
organism).  In fish, for example, bioconcentration occurs by
absorption through the skin or by uptake through the gills.  Living
tissues reach higher concentrations than the surrounding water when a
substance bioaccumulates.  Thus, contaminated fish and shellfish
present a health hazard to people who eat them.  Determining the
magnitude of risk associated with individual pollutants is a highly
complicated task involving complex technical studies. 


--------------------
\7 Within this report, we define a water pollutant as any constituent
or property of water that undermines the ability of the body of water
to support the uses to which it is put.  Thus, even fresh water can
be a pollutant in a salt water ecosystem, where the organisms that
reside there are adapted to highly saline water. 


      CONTROLLING POLLUTANTS:  THE
      CLEAN WATER ACT
-------------------------------------------------------- Chapter 1:1.4

The Federal Water Pollution Control Act of 1972, known as the Clean
Water Act, established the general framework for controlling these
pollutants.  Surface water can be characterized according to a number
of uses.  It is a source of drinking water for municipalities.  It
has recreational uses, such as boating, swimming, and fishing.  It is
the repository for wastes of numerous dischargers of pollutants such
as factories and municipal sewage treatment plants.  Much of the act
is designed to provide a way to limit discharges in order to minimize
the degree to which they interfere with the other uses of water
bodies.  In general, limits are placed on a pollutant if it is
thought to threaten human health or aquatic life. 

The general framework of the 1972 Clean Water Act consists of a set
of interrelated program activities based on provisions of the act and
subsequent amendments to it.  The process for controlling discharges
of pollutants is administered through the National Pollutant
Discharge Elimination System (NPDES).  NPDES was created through the
act as a way of controlling the pollutants discharged by the
thousands of facilities releasing their wastes into surface waters. 

The cornerstone of the NPDES framework is a process for issuing
permits to point source facilities:  That is, discharging any
substance into any body of water without a permit is forbidden.  When
a permit is issued, the pollutants requiring control are identified,
limits for their discharge are established, and then through
monitoring and enforcement EPA and the states are able to ensure
compliance with the permit.  So, placing a limit on a pollutant in a
permit constitutes the first step in the Clean Water Act framework
for controlling discharges from a particular point source. 

Permits are to be issued for every facility for up to 5 years. 
Prominent among the conditions contained within most permits are
requirements that the total amount of a discharge be monitored by a
discharging facility and that the amount of pollutants released be
limited to values established through the permit process.  For
example, chloroform might be limited through a permit by requiring
that it be monitored daily and that no more than 10 pounds be
discharged each day.  Other pollutants would have similar types of
discharge requirements defined on the permit.  Although issuing the
NPDES permit is the centerpiece of the current approach for limiting
discharges into surface waters, a related process uses information
provided by several other activities (described in chapter 2). 

Implementation of NPDES rests with 39 delegated states and EPA's 10
regional offices.  Each regional office retains implementation
authority for nondelegated states in that region.  Regions and
delegated states issue permits with effluent limits and monitoring
requirements, track compliance with the permit conditions, and carry
out enforcement actions. 

Controlling toxic substances has been one of the major challenges for
the Congress and EPA since the passage of the Clean Water Act.  The
number of toxic pollutants is large, the related risk is often
difficult to demonstrate, and measurement is expensive or
inconclusive at the low concentrations that pose a threat.  In 1977
and 1987, the Congress passed amendments to the act focusing renewed
attention on controlling toxics in surface water. 

In section 307 of the Clean Water Act as amended in 1977, the
Congress directed EPA to focus its regulatory activities on a list
(published elsewhere) containing 31 chemical groups (such as
halomethanes and polynuclear aromatic hydrocarbons) and 38 individual
substances (such as isophorone and thallium).  From this requirement,
the agency developed a list of 129 individual pollutants for priority
attention, but this list has been reduced to 126.  These pollutants
are termed the "priority pollutants," and EPA has directed most of
its attention with respect to toxics within the water protection
framework to these 126.  According to EPA,

     "The priority pollutant list identifies toxic pollutants of
     concern on a national basis.  It has served as a basis for
     numerous EPA actions, including the selection of pollutants for
     development of water quality criteria under section 304(a) of
     the Clean Water Act; the development of technology-based
     effluent guidelines under section 301 of the Clean Water Act;
     the listing of impaired waters under section 304(l) of the Clean
     Water Act."\8

EPA has recently been augmenting its approach for controlling toxic
pollutants under the Clean Water Act.  An important initiative in
this area is an approach called "whole effluent toxicity" (WET).\9
The agency has recognized that discharges from facilities often
consist of a "chemical stew" of many different pollutants, each with
its own "toxic signature," with a combined effect on a receiving
water body that is impossible to deduce from the individual
components.  EPA has found that the toxicity of the "whole effluent"
is what poses a human health risk and affects the environment and
has, thus, decided that the "stew" itself, and not just the
ingredients, is a proper subject for regulation and control through
the NPDES process. 

In the view of the agency, a WET limit (a limit based on the results
of testing the toxicity of the wastewater through a bioassay)
"provides a way to evaluate an effluent in the absence of detailed
information about the chemicals it contains."\10 However, because WET
test results do not specify which substances in the effluent are
toxic, they are not prescriptive:  That is, by themselves, they do
not indicate how to treat the effluent's toxicity to protect human
health and aquatic life.  Therefore, limits on individual pollutants
remain a critical component of the water quality approach in spite of
advances in WET testing.\11 Other initiatives EPA mentioned in this
area are toxicity identification evaluations and toxicity reduction
evaluations, which are intended to assist those holding permits to
achieve compliance with WET limits.\12


--------------------
\8 Environmental Protection Agency, "Water Quality Guidance for the
Great Lakes System and Correction; Proposed Rules," 58 Fed.  Reg. 
20801, at 20843 (April 16, 1993). 

\9 Environmental Protection Agency, Technical Support Document for
Water Quality Based Toxics (Washington, D.C.:  April 1991). 

\10 Environmental Protection Agency, Introduction to Water Quality
Based Toxics Control for the NPDES Program (Washington, D.C.:  March
1992), p.  6. 

\11 Environmental Protection Agency, Technical Support Document, p. 
1-4. 

\12 Environmental Protection Agency, Technical Support Document, p. 
114. 


   OBJECTIVES, SCOPE, AND
   METHODOLOGY
---------------------------------------------------------- Chapter 1:2


      OBJECTIVES
-------------------------------------------------------- Chapter 1:2.1

In the winter of 1992, the Chairman of the Subcommittee on Superfund,
Ocean, and Water Protection of the Senate Committee on Environment
and Public Works asked us to evaluate EPA's effort to ensure the
quality of the information used to control the discharge of toxics to
surface water and the extent to which toxic substances that are
discharged are listed on water quality permits.  In responding to the
request, we addressed four questions: 

1. What are the EPA water quality activities that control discharges
of toxic substances from point sources into surface waters and what
are their critical information requirements? 

2. To what extent do these activities meet critical information
quality assurance requirements? 

3. To what extent are limits for actual toxic substance discharges
included on NPDES permits? 

4. What are the effects on water quality of unlisted discharges of
toxic pollutants? 


      SCOPE
-------------------------------------------------------- Chapter 1:2.2

For questions 1 and 2, we examined activities across programs
administered by EPA's water quality office that are directly
responsible for controlling discharges of toxics from point sources. 
A number of these activities are overseen by EPA and implemented at
the state level. 

The scope for question 3 was comprised of facilities selected to
include three industrial categories from which EPA's industrial
technology division has recently gathered discharge monitoring data. 
These categories--pulp and paper, pharmaceutical manufacturing, and
pesticide manufacturing--are among the most significant contributors
of toxic substance discharges in the nation in terms of the amount of
toxic pollutants discharged.  We addressed question 4 using the same
set of facilities. 


      METHODOLOGY
-------------------------------------------------------- Chapter 1:2.3


         1.  PRINCIPAL ACTIVITIES
         AND THEIR INFORMATION
         REQUIREMENTS
------------------------------------------------------ Chapter 1:2.3.1

To identify and characterize the activities that EPA has established
to control toxic discharges to surface water and their critical
information needs, we used program documents and literature reviews
to develop a list of activities conducted across a number of water
quality programs.  We then submitted this list to EPA officials,
asking them to identify those that directly affect the control of
toxics. 

Our next step was to administer a structured interview to the
managers of the "core" activities to determine (1) the types of
information (for example, hydrology information about receiving
bodies of water and information about pollutants discharged) that are
required for conducting the activities and (2) the principal sources
of the information.  We validated our findings by asking an expert
panel to review the core programs and information requirements
identified (see appendix I for a list of the experts). 


         2.  THE EXTENT TO WHICH
         INFORMATION MEETS QUALITY
         ASSURANCE CRITERIA
------------------------------------------------------ Chapter 1:2.3.2

We could not evaluate the quality of the EPA data directly, because
of the diversity of data used and the amount of variation in the same
type of information from case to case.  Instead, we used commonly
accepted standards of quality assurance to assess the degree to which
the EPA process adhered to such standards in collecting, analyzing,
and reporting the required information (identified in question 1). 
This evaluation approach was feasible because EPA's official policy
for conducting studies requires that quality assurance criteria be
established and followed by every agency data collection effort. 
According to agency documents, quality assurance is indispensable for
producing valid data.\13 For these reasons, we were able to use an
evaluation of the EPA quality assurance effort as a surrogate for
directly examining data quality. 

We developed quality assurance criteria from a review of the
scientific literature and EPA's quality assurance/quality control
(QA/QC) guidelines.  Members of EPA's quality assurance management
staff then reviewed the list of criteria for relevance and
completeness.  We then determined which quality assurance criteria
were relevant and critical to each information requirement (from
evaluation question 1) for each of the core water quality protection
program activities, resulting in separate sets of criteria from our
overall set for each requirement.  We asked our panel of experts to
help us identify the quality assurance criteria relevant and critical
to each information requirement.  We then interviewed the managers of
the activities to determine the extent to which the criteria assigned
to each information type were met.  We interviewed other individuals
in EPA and the U.S.  Geological Survey (USGS) to obtain responses to
questions that program managers were unable to answer.\14 We obtained
detailed explanations for each response during the interviews and
independently reviewed supporting agency studies and data bases for
particular responses. 


--------------------
\13 Environmental Protection Agency, Minimal Requirements for a Water
Quality Assurance Program (Washington, D.C.:  1975), p.  1-3. 

\14 We obtained responses to a number of criteria from USGS staff. 
We also surveyed staff within EPA's 10 regional offices for responses
to several criteria. 


         3.  THE EXTENT TO WHICH
         THE NPDES PERMIT PROCESS
         CONTROLS TOXIC SUBSTANCES
------------------------------------------------------ Chapter 1:2.3.3

The third evaluation question addresses the extent to which toxic
substances receive even the first level of control provided by the
NPDES permit process.  We answered this question by comparing reports
of actual discharges against Permit Compliance System (PCS) permit
files for a sample of facilities and identifying the "unlisted"
pollutants--that is, toxic pollutants that were discharged but not
listed on the permit and were thus not directly controlled.\15 We
obtained the discharge data from (1) studies of discharges conducted
by the industrial technology division of EPA, (2) reports of
discharges by industrial facilities under section 313 of the
Emergency Planning and Community Right-to-Know Act (otherwise known
as the Toxic Release Inventory), and (3) NPDES permit applications
listing pollutants identified in facilities' effluent. 


--------------------
\15 The Permit Compliance System is EPA's data storage and retrieval
system for NPDES permit files. 


         4.  THE IMPACT OF
         UNLISTED DISCHARGES ON
         WATER QUALITY
------------------------------------------------------ Chapter 1:2.3.4

To address our last evaluation question, we analyzed the human health
and aquatic life risks associated with the uncontrolled toxic
pollutant discharges identified in question 3.  In order to determine
these risks, we used a method similar to that employed by EPA regions
and states when they review permit applications.\16 We developed
in-stream concentrations of unlisted pollutants using a water quality
model and used reported stream flow data.  We obtained these from
EPA.  The model is a "static" water quality model in which the
quantity discharged by the facility is diluted into the stream at
either low flow or mean flow conditions.\17 We then compared the
resultant concentrations against applicable water quality criteria to
determine which of the discharges posed a risk to either human health
or aquatic life. 


--------------------
\16 We reviewed our methodology with staff from EPA's office of water
to ensure its similarity to that employed by permit authorities. 

\17 Whether we estimated concentration at low flow or mean flow
conditions depended on whether we were comparing the value against an
"aquatic life" or "human health" criterion.  The general methodology
for conducting this evaluation is presented in EPA's Technical
Support Document and in Versar, Inc., ReachScan User's Manual
(Springfield, Va.:  September 1991). 


   STRENGTHS AND LIMITATIONS
---------------------------------------------------------- Chapter 1:3

One strength of the work performed for this evaluation is the
comprehensiveness of the effort to determine the principal activities
EPA has designed to control toxic discharges.  Our review of both
literature and documentation was designed to ensure that we would
identify every activity implemented in the overall EPA water quality
program.  Through indepth review of all activities identified,
interviews with EPA staff to determine which activities were directly
related to toxics control, and with the use of an expert panel to
validate our results, we were certain to identify the activities that
are absolutely essential to the toxics control program. 

To address EPA's steps to ensure data quality, our effort again was
comprehensive.  We used both EPA's quality assurance criteria and
other criteria identified through a second literature review, this
time in the specialized area of quality assurance.  We asked the EPA
staff who are responsible for the agency's quality assurance efforts
to review our initial list of criteria for appropriateness.  The use
of an expert panel to examine whether criteria were both relevant and
critical to ensure quality in the activities we identified served to
further validate our work in establishing the normative framework. 

A limitation with regard to the measurement of data quality is that
we could not assess that quality directly.  As noted earlier, we
looked at compliance with quality assurance criteria as a proxy for
direct measurement, because the complexity of direct measurement far
transcended the scope of this study.  Although compliance with
criteria for quality is an indirect measure, such compliance is a
necessary (if not sufficient) condition for quality, and it is EPA's
official policy concerning all agency data collection. 

Finally, our work in answering questions 3 and 4--determining the
extent to which the permit process controls toxic discharges and
determining the risk of uncontrolled discharges--has the strength of
being a national level assessment.  That is, the sample of facilities
whose permits we reviewed was geographically dispersed across the
country.  Further advantages were the comprehensiveness of our three
data bases (which defined the types of toxic pollutant discharges and
their amounts) and the importance of the industrial sectors we chose
to examine (pulp and paper, pharmaceutical, and pesticide
manufacturing).  These three sectors are among the most significant
across the nation for discharging toxic pollutants, in terms of both
the amount and types of pollutants discharged.  However, although our
sample was fairly large (236 facilities), our study was not designed
to allow its conclusions to be generalized beyond that sample to the
total population of the dischargers. 


EPA ACTIVITIES THAT LIMIT
DISCHARGES OF TOXICS INTO SURFACE
WATERS AND THEIR INFORMATION
REQUIREMENTS
============================================================ Chapter 2

In this chapter, we address our first evaluation question:  What are
the EPA water quality activities that control discharges of toxic
substances from point sources into surface waters and what are their
critical information requirements?  To answer this question, we
refined the larger number of activities EPA conducts across a number
of water quality programs to the 7 "core" activities that are
fundamental to EPA's basic approach to control the discharge of toxic
water pollutants from point sources.  We then identified 13 types of
information required by these activities.  In this chapter, we
describe each activity, the information it requires, and the roles
and functions of the information collected and list the data sources
for the various information types. 


   ACTIVITIES TO LIMIT DISCHARGES
   OF TOXICS
---------------------------------------------------------- Chapter 2:1

EPA is responsible for many activities that are designed to limit
discharges of pollutants into surface waters.  Through a literature
review, we identified 46 activities that EPA conducted within the
agency's office of water.  We first eliminated the 11 that deal with
drinking water or groundwater issues and, therefore, have no direct
relationship with controlling toxic discharges from point sources. 
We then administered a questionnaire and conducted a series of
interviews with EPA officials to identify the activities from the
remaining 35 that have a direct effect on controlling toxics.  In
this manner, we arrived at a set of 7 activities that form the
nucleus of EPA's approach for controlling toxic discharges from point
sources.  Our panel of experts then validated this list of 7
activities. 

The 46 activities are presented in table 2.1, which distinguishes the
7 that we retained because of their direct influence on controlling
discharges of toxic substances from the 39 others.  The principal
reasons for eliminating each of the 39 are also presented in the
table.  The 7 activities that are the focus of this chapter are then
described in table 2.2. 



                                    Table 2.1
                     
                     Activities Conducted Within EPA's Office
                                     of Water

                                                   Principal reason for
No.       Activity                       Status    elimination
--------  -----------------------------  --------  -----------------------------
1         Assess progress in             Eliminat  Administrative activity
          implementing the Coastal Zone  ed
          Management Act to control
          nonpoint source pollution in
          coastal waters

2         Assess toxicity control needs  Eliminat  Duplicates water-quality-
          for water-quality-based        ed        based permit activity number
          permits                                  28

3         Conduct water quality          Eliminat  Regulatory support activity
          standards triennial reviews    ed

4         Demonstrate accomplishments    Eliminat  Not related to controlling
          in maintaining active state    ed        toxic discharges
          and federal enforcement
          programs for drinking water

5         Develop ecological criteria    Eliminat  Developmental activity
          guidance                       ed

6         Develop stormwater permit      Eliminat  General regulatory activity
          strategies                     ed        with small need for data

7         Draft a strategy for           Eliminat  Developmental activity
          addressing contaminated        ed
          sediment

8         Draft plans for stormwater     Eliminat  Administrative activity
          permits                        ed

9         Enforce state testing for      Eliminat  Not related to controlling
          lead in drinking water         ed        toxic discharges

10        Establish drinking water       Eliminat  Not related to controlling
          laboratory certification       ed        toxic discharges
          regulations

11        Establish national primary     Eliminat  Not related to controlling
          drinking water regulations     ed        toxic discharges

12        Establish total maximum daily  Retained
          loads

13        Establish water quality        Retained
          criteria

14        Focus on reauthorization of    Eliminat  Nonspecific regulatory
          Clean Water Act                ed        activity

15        Identify and track water       Eliminat  Administrative activity
          quality improvements of        ed
          targeted water bodies

16        Identify compliance problems   Retained

17        Identify priority issues in    Eliminat  General regulatory activity
          stormwater application         ed        with small need for data
          process

18        Implement nonpoint source      Eliminat  Not related to controlling
          watershed control programs     ed        toxic discharges from point
                                                   sources

19        Improve effectiveness of       Eliminat  Regulatory developmental
          inspection activities          ed        activity

20        Improve quality and            Eliminat  Regulatory developmental
          timeliness of enforcement      ed        activity
          responses

21        Issue health advisories for    Eliminat  Not related to controlling
          drinking water                 ed        toxic discharges

22        Issue administrative           Eliminat  Not related to controlling
          compliance orders for          ed        toxic discharges
          drinking water

23        Issue drinking water           Eliminat  Not related to controlling
          regulations                    ed        toxic discharges

24        Develop effluent guidelines    Retained

25        Issue fines and negotiate      Eliminat  Not directly related to
          consent decrees for permit     ed        toxics control
          violations

26        Issue lists of impaired        Retained
          waters and related point
          sources

27        Issue regulations to control   Eliminat  Not related to point source
          pollutants in urban runoff     ed        control

28        Issue water quality permits    Retained

29        Make revisions to permits      Eliminat  Duplicates number 28
          codifying changes made by      ed
          1987 Clean Water Act
          amendments (antibacksliding
          provisions)

30        Manage state revolving funds   Eliminat  Administrative activity
                                         ed

31        NPDES regulatory revisions     Eliminat  Nonspecific regulatory
          contained in 1987 Water        ed        activity
          Quality Act

32        Prepare sewage sludge use and  Eliminat  Regulatory development
          disposal regulations           ed        activity

33        Propose revisions to           Eliminat  Not related to controlling
          monitoring requirements for    ed        toxic discharges
          drinking water

34        Reduce noncompliance with      Eliminat  Not related to controlling
          existing drinking water        ed        toxic discharges
          standards

35        Reduce exposure of population  Eliminat  Not directly related to
          to contaminants in drinking    ed        controlling toxic discharges
          water

36        Regulate sources of toxic      Retained
          discharges to publicly owned
          treatment works from indirect
          dischargers

37        Regulate substances not on     Eliminat  Not related to controlling
          drinking water priority list   ed        toxic discharges

38        Revise stormwater              Eliminat  Program in development
          implementation rules           ed

39        Revision of denial or          Eliminat  Not directly related to
          restriction of disposal sites  ed        controlling toxic discharges

40        Set federal NPDES fees in      Eliminat  Administrative activity
          states where EPA administers   ed
          NPDES program

41        Strengthen the scientific      Eliminat  Not directly related to
          basis of water quality         ed        controlling toxic discharges
          standards to protect critical
          aquatic resources

42        Track fish consumption         Eliminat  Administrative activity
          advisories                     ed

43        Track pretreatment programs    Eliminat  Administrative activity
                                         ed

44        Track sludge facilities        Eliminat  Administrative activity
                                         ed

45        Track stormwater permit        Eliminat  Not directly related to
          activity                       ed        controlling toxic discharges

46        Water quality standards        Eliminat  Not of national scope
          revisions for Indian tribes    ed
--------------------------------------------------------------------------------


                          Table 2.2
           
                Activities Designed to Control
                Discharges of Toxic Substances

No.       Activity\a          Objective
--------  ------------------  ------------------------------
1         Effluent            Develop national standards,
          guidelines          setting out baseline
          development (no.    requirements for pollution
          24)                 removal within individual
                              industries

2         Water quality       Develop statements of the
          criteria            maximum permissible
          development (no.    concentrations of individual
          13)                 toxic pollutants in water that
                              will not harm human health and
                              aquatic life

3         Impaired water and  Identify water threatened by
          point source        point source discharges of
          identification\b    toxic substances and the
          (no. 26)            facilities causing the
                              impairments

4         Total maximum       Estimate the amount of a
          daily load          pollutant that can be safely
          development (no.    discharged from all sources
          12)                 (including natural sources)
                              into a body of water

5         Water-quality-      Issue the legal documents
          based permit        containing the conditions and
          issuance (no. 28)   limits controlling a point
                              source discharger's release of
                              pollutants into a body of
                              water based on the expected
                              effect of the release on water
                              quality

6         "Local limits"      Restrict the amount of
          development (for    pollutants that "indirect
          reducing            dischargers" can release into
          discharges of       sewer systems
          pollutants into
          sewage treatment
          plants) (no. 36)

7         Compliance problem  Determine whether a discharger
          identification      is meeting its permit
          (no. 16)            conditions; used as a basis
                              for possible future
                              enforcement action
------------------------------------------------------------
\a Numbers in parentheses identify activities listed in table 2.1.\

\b This was a one-time exercise mandated by the Water Quality Act of
1987.  Section 304(l) called for three lists to be prepared by each
state.  Only one of them featured streams with impairment primarily
from toxic pollution from point sources (the others being broader
delineations of impaired water bodies).  We used the preparation of
this list, and not the two others, as the activity. 

Across these 7 activities, the essential data for implementing EPA's
approach are gathered and processed.  In many cases, information that
has been collected and decisions that have been reached within one
activity are then used within another.  For example, water quality
criteria (activity 2) are used to generate water-quality-based
permits (activity 5).  In terms of their information function, the
core program activities can be grouped into four categories by the
role each plays in the overall process:  (1) uniform guideline
development, (2) geographic targeting, (3) permit development, and
(4) compliance assessment. 

The overall process for controlling toxics from point sources begins
with uniform guideline development:  the development of general
practices for making decisions with respect to the thousands of
facilities subject to NPDES.  In this first category, EPA develops
national guidelines or criteria--specifically effluent guidelines and
water quality criteria (activities 1 and 2 in table 2.2).  They
provide the rationale for determining what substances should be
limited on permits and what those limits should be.  The second step
in the process is to identify specific facilities that require
expedited review of their permit.  The one time that geographic
targeting was employed by the Congress within the Clean Water Act
occurred in section 304(l) of the act as amended in 1987 (activity
3).  The third category, permit development, uses the national
criteria and standards (from the first category) along with
site-specific information to determine the conditions and limitations
needed on individual permits--specifically, total maximum daily load
(TMDL) development, NPDES permit issuance, and "local limits"
development (activities 4, 5, and 6).  The fourth category,
compliance assessment (that is, activity 7, compliance problem
identification), uses information about patterns of discharges to
determine whether facilities have complied with the conditions and
limits of their permits.  This activity compares actual monitoring
and test data against the limits and conditions imposed in permits. 
If problems are found, the results of this comparison can lead to
either an enforcement action against a facility for violating a
permit condition or to a revision of its permit. 


   INFORMATION REQUIRED WITHIN THE
   SEVEN CORE ACTIVITIES
---------------------------------------------------------- Chapter 2:2

We identified critical information needed to support the core
activities through interviews with EPA program managers and others. 
"Critical" information was defined as having primary importance to
conducting the activity with respect to meeting EPA's water quality
goals for controlling point source discharges of toxics into surface
water.  Weaknesses in critical information clearly undermine the
effectiveness of an activity.  Each of the 13 information types we
identified as required by the 7 activities is briefly described in
table 2.3. 



                          Table 2.3
           
              The Thirteen Types of Information
             Required Across the Seven Activities

No.       Information type    Description
--------  ------------------  ------------------------------
1         Hydrology           Characteristics of the
                              properties of lakes and rivers
                              such as stream flow

2         In-stream water     The chemical properties of the
          quality             receiving body of water
                              (typically the concentration
                              of the chemical constituents)

3         Concentration in    Concentrations of chemical
          sediments           constituents found within the
                              sediments of receiving water
                              bodies

4         Concentration in    Concentrations of toxics
          biota               within the aquatic creatures
                              inhabiting the water body

5         Aquatic life        The extent to which aquatic
          exposure            life is exposed to pollutants

6         Discharges from     Substances and amounts being
          individual sources  discharged from specific
                              plants and facilities

7         Removal             Characterizations of the
          technologies        technologies that are
                              effective within the industry
                              for removing pollutants from
                              wastewater

8         Water use           Characterizations of the
          reduction           techniques that can be used to
          technologies        reduce discharges of
                              pollutants through water reuse
                              in the industrial production
                              process

9         Aquatic life        The nature and extent of
          effects             effects upon aquatic organisms
                              that come into contact with
                              substances in the
                              environment\a

10        Human health        The nature and extent of human
          effects             effects associated with
                              exposure to the substance in
                              the environment

11        Fate and transport  The mechanisms and rate at
                              which a substance undergoes
                              change and moves through the
                              environment (or within a
                              publicly owned treatment
                              works)

12        Human health        The extent and way in which
          exposure            humans are exposed to water
                              pollutants through drinking
                              water and eating fish

13        Economic data       The effect on an industry's
                              profitability of changes in
                              categorical technologically
                              based effluent standards
------------------------------------------------------------
\a In connection with the section 304(l) exercise, this took on the
meaning of assigning "biocriteria," or using indicator species, to
measure environmental impairments. 


   THE SEVEN CORE ACTIVITIES AND
   THEIR INFORMATION NEEDS
---------------------------------------------------------- Chapter 2:3

We found, as expected, that all 7 activities depend heavily on
information.  Each requires a different set of specific data for its
implementation.  As already noted, 13 different types of information
in all are required across the 7 activities.  The types of
information listed as 1-13 in table 2.3 are labeled A-M in figure 1
and elsewhere in the report.  The types of information used to
support the 7 activities vary significantly. 


   THE ROLES AND FUNCTIONS OF THE
   INFORMATION
---------------------------------------------------------- Chapter 2:4

In the series of summary tables 2.4 through 2.10, we present the
reasons why the 13 types of information we identified in the
preceding section (types A-M) are required across the 7 activities. 
Each table characterizes the determinations, within each activity,
that the information is required to support (its purposes) and states
the source of the information and how it is developed.  In appendix
II, we present a more detailed description of each activity,
highlighting its statutory and regulatory basis in addition to the
role and principal information sources of each information type. 



                          Table 2.4
           
             The Purposes and Sources of Critical
           Information Within Activity 1: Effluent
                    Guidelines Development

                                          Typical principal
Information type      Purpose             source
--------------------  ------------------  ------------------
Discharges from       To identify the     Questionnaire to a
individual sources    pollutants that     sample of
(F)                   are typical of      facilities within
                      facilities within   industry; site
                      an industry         visits

Removal technologies  To identify the     Questionnaire to a
(G)                   most efficient      sample of
                      technologies        facilities within
                      available within    industry; site
                      an industry to      visits
                      remove pollutants

Water use reduction   To identify the     Questionnaire to a
technologies (H)      most efficient      sample of
                      technologies        facilities within
                      available within    industry; site
                      an industry to      visits
                      reduce discharges
                      through efficient
                      water use

Economic data (M)     To identify and     Questionnaire to a
                      characterize the    sample of
                      economic effect on  facilities within
                      an industry of      industry
                      various
                      alternatives
------------------------------------------------------------


                          Table 2.5
           
             The Purposes and Sources of Critical
             Information Within Activity 2: Water
                 Quality Criteria Development

                                          Typical principal
Information type      Purpose             source
--------------------  ------------------  ------------------
Concentration in      To determine how    Peer reviewed
biota (D)             pollutants          scientific
                      bioconcentrate in   literature; EPA's
                      the environment     field studies

Aquatic life          To provide a        Criteria from EPA
exposure (E)          measure allowing    (on magnitude,
                      the results of the  duration, and
                      effects studied to  frequency of
                      be related to       exposure)
                      "real world"
                      exposures

Aquatic life effects  To identify the     Aquatic toxicology
(I)                   nature and          studies,
                      magnitude of the    internally
                      effects of          developed or from
                      exposure of the     the scientific
                      pollutant to        literature; EPA
                      aquatic species     testing

Human health effects  To identify the     Peer reviewed
(J)                   nature and          scientific
                      magnitude of the    literature; EPA's
                      effects of          field studies
                      exposure of the
                      pollutant to
                      people

Human health          To provide a        Peer reviewed
exposure (L)          measure allowing    scientific
                      the results of the  literature; EPA's
                      effects studied to  field studies
                      be extrapolated to
                      the exposures of
                      people
------------------------------------------------------------


                          Table 2.6
           
             The Purposes and Sources of Critical
           Information Within Activity 3: Impaired
            Water and Point Source Identification

                                          Typical principal
Information type      Purpose             source
--------------------  ------------------  ------------------
In-stream water       To identify areas   State monitoring
quality (B)           that are            programs;
                      contaminated by     STORET\a; state-
                      toxics              specific data
                                          bases

Concentration in      To identify areas   State monitoring
sediments (C)         that have           programs; STORET
                      sediments           state-specific
                      contaminated by     data bases
                      toxics

Concentration in      To determine areas  State monitoring
biota (D)             that have aquatic   programs and
                      life contaminated   assessments
                      by toxics

Discharges from       To identify areas   Permit
individual sources    that may be         applications and
(F)                   contaminated by     discharge
                      toxics and the      monitoring reports
                      contributing
                      sources

Aquatic life effects  To determine        State monitoring
(I)                   whether aquatic     programs and
                      life is being       section 305(b)
                      harmed by           reports
                      discharges

Fate and transport    To provide precise  State section
(K)                   estimates of in-    305(b)
                      stream              assessments; state
                      concentrations      surveys; state and
                                          EPA compliance
                                          monitoring for
                                          point sources
------------------------------------------------------------
\a STOrage and RETrieval (STORET). 



                          Table 2.7
           
             The Purposes and Sources of Critical
             Information Within Activity 4: TMDL
                         Development

                                          Typical principal
Information type      Purpose             source
--------------------  ------------------  ------------------
Hydrology (A)         To estimate the     USGS gaging
                      "critical flow" of  stations and
                      the receiving       regression
                      water body          equations from
                                          gages on
                                          neighboring
                                          streams

In-stream water       To estimate the     State monitoring
quality (B)           background          programs; STORET
                      concentration of
                      the substance
                      being discharged

Concentration in      To determine        State monitoring
sediments (C)         whether the         programs; STORET
                      sediment is
                      contaminated

Concentration in      To determine        State monitoring
biota (D)             whether there is a  programs; STORET
                      contamination
                      problem

Discharges from       To estimate the     Permit
individual sources    amount discharged   applications and
(F)                   by the facility     discharge
                                          monitoring reports

Fate and transport    To provide precise  Textbooks, journal
(K)                   estimates of in-    articles, EPA and
                      stream              U.S. Fish and
                      concentrations      Wildlife Service
                                          publications; EPA-
                                          supported water
                                          quality models
------------------------------------------------------------


                          Table 2.8
           
             The Purposes and Sources of Critical
            Information Within Activity 5: Water-
                Quality-Based Permit Issuance

                                          Typical principal
Information type      Purpose             source
--------------------  ------------------  ------------------
Hydrology (A)         To determine the    USGS gaging
                      dilution limits     stations and
                      for aquatic life    regression
                      and human health    equations from
                      effects             gages on
                                          neighboring
                                          streams

In-stream water       To determine the    STORET and state
quality (B)           background          water quality data
                      concentrations for  bases
                      the substances
                      being considered

Discharges from       To determine the    Permit
individual sources    amount discharged   applications and
(F)                   from the facility   discharge
                                          monitoring reports

Fate and transport    To provide precise  Textbooks, journal
(K)                   estimates of in-    articles, EPA and
                      stream              U.S. Fish and
                      concentrations      Wildlife Service
                                          publications; EPA-
                                          supported water
                                          quality models
------------------------------------------------------------


                          Table 2.9
           
             The Purposes and Sources of Critical
            Information Within Activity 6: "Local
                     Limits" Development

                                          Typical principal
Information type      Purpose             source
--------------------  ------------------  ------------------
Discharges from       To determine the    Monitoring of the
individual sources    substances and      source's effluent,
(F)                   amounts being       by the indirect
                      discharged          discharger and the
                                          publicly owned
                                          treatment works

Removal technologies  To determine the    Self-monitoring of
(G)                   effectiveness of    the publicly owned
                      the publicly owned  treatment works'
                      treatment works at  effluent
                      removing the
                      substance

Fate and transport    To determine the    Self-monitoring of
(K)                   amount of each      the publicly owned
                      substance           treatment works'
                      remaining in        wastestream
                      wastewater,
                      volatilizing into
                      the atmosphere,
                      and precipitating
                      in sludge as a
                      result of
                      treatment at the
                      publicly owned
                      treatment works
------------------------------------------------------------


                          Table 2.10
           
             The Purposes and Sources of Critical
                Information Within Activity 7:
              Compliance Problem Identification

                                          Typical principal
Information type      Purpose             source
--------------------  ------------------  ------------------
In-stream water       To compare against  Self-monitoring by
quality (B)           permit limits in    facility; data for
                      order to evaluate   major facilities
                      compliance (in-     entered into PCS
                      stream monitoring
                      is rarely
                      included)

Aquatic life          To compare against  Self-monitoring by
exposure (E)          permit limits in    facility; data for
                      order to evaluate   major facilities
                      compliance (WET     entered into PCS
                      limits)

Discharges from       To compare against  Self-monitoring by
individual sources    permit limits in    facility; data for
(F)                   order to evaluate   major facilities
                      compliance          entered into PCS
------------------------------------------------------------

   SUMMARY
---------------------------------------------------------- Chapter 2:5

Seven activities across various EPA water quality programs form the
core of the agency's effort to control toxic discharges from point
sources into surface waters:  effluent guidelines development, water
quality criteria development, impaired water and point source
identification, total maximum daily load development,
water-quality-based permit issuance, "local limits" development, and
compliance problem identification. 

A wide variety of information is employed within the 7 core
activities.  All 7 use several different types of information for
making decisions.  In all, 13 different types of information are used
at 31 points across the 7 activities (tables 2.4 through 2.10). 

Information plays a critical role across these activities.  From
setting national standards to ensuring compliance with individual
NPDES permits, information forms the basis for these efforts.  Each
one of the 13 types of information is critical to the activities. 
For example, without reliable information about efficient and
feasible removal technologies, the effluent guidelines activity would
not produce technological requirements that meet congressional
objectives in the Clean Water Act.  Without reliable information
about the actual discharges of facilities with permits, the NPDES
compliance program would not be able to identify the facilities that
are not conforming to their permit requirements. 

Having established the core activities and their related information
requirements, we now address our second evaluation question.  In
chapter 3, we evaluate the extent to which EPA ensures that critical
quality assurance steps are taken in developing the required
information. 


THE EXTENT TO WHICH REQUIRED
INFORMATION MEETS QUALITY
ASSURANCE REQUIREMENTS
============================================================ Chapter 3

In chapter 2, we identified the EPA activities that control toxic
discharges to water and the types of information that support the
activities.  In this chapter, we address our second evaluation
question:  To what extent do these activities meet critical
information quality assurance requirements?  Here, we compare EPA's
official policy for quality assurance in data collection with
reported practice. 


   INTRODUCTION
---------------------------------------------------------- Chapter 3:1

As we discussed in chapter 2, information is a critical component of
the nation's approach for controlling discharges of toxics, and the
quality of that information is central to the effectiveness of the 7
core activities.  In general (that is, within each activity),
information quality is promoted through adopting and practicing
quality assurance standards.  Quality assurance practices and
standards control how data are collected, managed, and analyzed so as
to ensure their reliability, validity, and completeness.  These
standards also regulate whether the information base is sufficient
for drawing inferences and making decisions.  We evaluated
information quality according to two types of criteria.  The first
type concerns the methodological steps taken to ensure information
quality.\1 The second type concerns the completeness of the data
gathered vis-a-vis the envisaged purpose--that is, whether all the
information required to support the activity was in fact acquired. 

EPA's policy is to protect information quality by incorporating
quality assurance practices into individual studies.  EPA order
5360.1 requires a quality assurance program for all environmentally
related measurements performed by or for the agency.  The primary
goal of the program is to ensure that all such measurements yield
data of known quality that can be verified as being reliable and
appropriate to the program's objectives.  The quality assurance
management staff serves as the central management authority for this
program within EPA. 

The staff is charged with developing quality assurance policy and
training program managers in quality assurance practices.  Program
managers are responsible for specifying the quality of the data they
require and for providing sufficient resources to ensure that an
adequate level of quality assurance is performed.  All routine or
planned projects involving environmental measurement must begin with
a "quality assurance project plan" that specifies acceptable quality
goals. 


--------------------
\1 Concerning methodological adequacy, EPA and USGS have recently
determined that the analytical methods used to collect and analyze
the in-stream data for metals may be flawed.  As a result, concerns
have been raised about the soundness of the information on which the
states have been basing their water quality programs.  At a workshop
that EPA convened to study the issue, an agency official raised the
possibility that the numbers are problematic.  A professional
hydrologist suggested that much of the data collected in the past to
determine whether water bodies are exceeding standards is "almost
certainly useless" (Water Policy Report, February 3, 1993, p.  3). 
EPA and USGS, the two principal agencies involved in developing
protocols for sample collection, are currently studying how to deal
with these questions. 


   EVALUATING QUALITY ASSURANCE
   PRACTICES
---------------------------------------------------------- Chapter 3:2

Each of the 7 activities we identified in our evaluation has resulted
in a large number of program decisions and products.  For example,
dozens of effluent guidelines have been issued over the course of the
clean water program and thousands of NPDES permits are issued and
reissued each year.  Because of the large number of decisions and
products, which went far beyond the scope of our evaluation, we
focused our evaluation at the level of the 7 activities. 


      DEVELOPING CRITERIA
      STATEMENTS
-------------------------------------------------------- Chapter 3:2.1

Through a review of the scientific literature on quality assurance,
EPA's quality assurance guidelines, and interviews with experts in
the field, we identified a set of 40 criteria that define a basis for
ensuring both information quality and completeness for the toxic
pollutant control activities we are addressing.  Thirty-one of the 40
reflect consensus criteria developed by EPA itself or industrial
groups such as the American Society of Testing and Materials (ASTM)
relating to methodological adequacy.  That is, they deal with how
information collection activities should be planned and implemented
and how the information should be reviewed.  Most of them apply to
whether safeguards were taken to ensure the validity and reliability
of the information collected in the field and analyzed in the
laboratory.  The remaining 9 criteria relate to completeness--that
is, whether the information base of all studies is sufficient (aside
from data quality concerns of individual studies) to reach
conclusions about the subject of interest, such as the in-stream
concentration of dioxin (for setting a limit on an individual
permit). 

We converted the 40 quality assurance criteria into a set of
questions, using a response scale of frequency.  The questions,
presented in table 3.1, are divided into those dealing with
methodological adequacy and those that ensure a complete information
base. 



                          Table 3.1
           
                Quality Assurance Criteria for
               Evaluating the Seven Core Toxic
                 Pollutant Control Activities

              Criterio
Type          n         Question
------------  --------  ------------------------------------
Methodologic  1         How often are data quality
al adequacy             objectives instituted at the
                        initiation of a project?

              2         How often is the design for data
                        collection and analysis reviewed by
                        an individual not directly
                        associated with the program who is
                        familiar with the methodological
                        issues involved in data collection
                        and analysis?

              3         How often is a quality assurance
                        project plan employed within the
                        data collection process?

              4         How often are standard operating
                        procedures for field and laboratory
                        sampling and reporting followed?

              5         How often are QA/QC procedures for
                        data storage and maintenance
                        employed?

              6         How often are questionnaires
                        pretested?

              7         How often are chemical analyses and
                        bioassays conducted according to
                        EPA-approved methods or other
                        methods specified in the project
                        plan?

              8         How often are chemical analyses and
                        bioassays performed by laboratories
                        that follow good laboratory
                        practices?

              9         How often is an independent analysis
                        conducted to assess the quality of
                        the data that went into water
                        quality criteria?

              10        How often do assessments of
                        environmental quality that are based
                        on best professional judgment
                        undergo quality control checks and
                        independent verification?

              11        How often were subject area experts
                        consulted and drafts distributed for
                        public review before the data were
                        accepted?

              12        How often are multiple, independent
                        sets of data used to identify the
                        toxic substances that were included
                        within the data collection effort?

              13        How often are multiple, independent
                        sets of data used to identify the
                        facilities to gather data from
                        within the data collection effort?

              14        Where sampling data were
                        unavailable, how often were
                        assessments, based on observations
                        of water, biota, and sediment by an
                        experienced professional?

              15        How often do individual studies
                        present missing data exceeding 10
                        percent of planned sample size?

              16        How often is a probabilistic
                        sampling scheme (a random sample)
                        employed and documented?

              17        How often are all requirements of
                        the model, such as times and
                        locations for in-stream or discharge
                        monitoring, adhered to?

              18        How often are the data checked for
                        age and only those that are
                        appropriately current retained and
                        used in the analysis?

              19        How often are data from different
                        time periods gathered in such a way
                        as to ensure that they are
                        comparable?

              20        How often are independent studies
                        conducted to assess the degree to
                        which self-reported samples
                        represent discharges at different
                        production and discharge levels?

              21        How often is the critical flow based
                        on data from gaging stations
                        (whether on the reach itself or
                        estimated from gaging stations on
                        neighboring reaches)?

              22        When substituting one variable for
                        another within a model, how often
                        were steps taken to ensure that the
                        surrogate variable is correlated
                        with the original with respect to
                        the function it performs in the
                        model?

              23        For critical model parameters, how
                        often were laboratory findings
                        replicated with field studies?

              24        How often are estimates of in-
                        stream concentrations of discharges
                        based on water quality models of
                        proven validity with respect to
                        local stream conditions?

              25        How often do major externally
                        published reports of findings
                        discuss significant components of
                        bias and measurement error?

              26        How often was the laboratory "in
                        control" during the period in which
                        it performed the tests?

              27        How often are field-derived data
                        obtained under conditions that
                        adequately represent the range and
                        variability of biotic and abiotic
                        conditions?

              28        How often is the design for data
                        collection and analysis reviewed by
                        an individual within the
                        organization who is familiar with
                        the methodological issues involved
                        in data collection and analysis?

              29        How often is the survey design
                        reviewed by an individual familiar
                        with methodological issues involved
                        in survey methods?

              30        How often are manufacturers'
                        specifications adhered to in
                        calibrating gages?

              31        How often was the laboratory
                        instructed about detection and
                        quantitation levels and how to
                        report data below these levels?

Completeness  32        How often is a judgment made that
                        sampling and analysis included all
                        pollutants that were likely to be
                        present at measurable levels?

              33        How often are data gathered on all
                        relevant effects when developing
                        criteria?

              34        How often are there enough data of
                        proven quality concerning the most
                        likely effects to support the
                        contention that criteria are based
                        on the most sensitive effect?

              35        How often are enough data of proven
                        quality collected on a variety of
                        representative species or cell lines
                        to support the contention that
                        criteria are based on impacts to the
                        most sensitive species that will
                        encounter the toxic substance in the
                        environment?

              36        How often is an assessment made
                        downstream of each "major"
                        discharger?

              37        How often are sampling data
                        available for waters on which
                        "major" dischargers of toxics are
                        situated?

              38        How often are minimum data set size
                        requirements stipulated?

              39        How often are sampling frequency and
                        size based on an assessment of the
                        variability of discharge loads?

              40        How often is the data set of
                        adequate size to reflect variations
                        and differences for this regulatory
                        purpose?
------------------------------------------------------------

      DETERMINING THE RELEVANCE OF
      THE QUALITY ASSURANCE
      CRITERIA
-------------------------------------------------------- Chapter 3:2.2

The list of criteria for ensuring the methodological adequacy and
completeness of information is broad.  Whether a criterion is
relevant to an information requirement depends on the activity that
the information requirement supports.  Consequently, our first task
was to determine which criteria were relevant to and critical for
each information requirement as it supports each specific activity. 
We did this with the assistance of a panel of five experts familiar
with the information requirements of these activities.  That is, for
each of the 13 types of information used by the activities (figure
2.1), we identified the criteria that are both relevant and critical
for ensuring information completeness and quality.  We defined a
critical criterion as one that, if not routinely met, would
jeopardize the quality of the information and threaten EPA's ability
to draw conclusions from it.  The critical criteria are presented in
table 3.2.  Thirty-eight of the 40 criteria were applied to at least
one information type. 



                                    Table 3.2
                     
                     Quality Assurance Criteria Applicable to
                                Information Types

Activity                       Information type    Criteria
-----------------------------  ------------------  -----------------------------
Effluent guidelines            Discharges from     1-5, 7, 8, 12, 13, 19, 20,
development                    individual sources  28, 39, 40

                               Removal             1-5, 7, 8, 10, 13, 19, 25,
                               technologies        28, 39

                               Water use           1-5, 7, 8, 10, 13, 19, 25,
                               reduction           28, 39
                               technologies

                               Economic data       1, 3, 5, 6, 18, 28, 29

Water quality criteria         Concentration in    1-5, 9-11, 26-28, 34
development                    biota

                               Aquatic life        1-5, 9, 11, 26, 28
                               exposure

                               Aquatic life        1-5, 7-9, 11, 26, 28, 33-35,
                               effects             38

                               Human health        1-3, 5, 9-11, 18, 28, 33, 34,
                               effects             38

                               Human health        1-3, 5, 9, 11, 28
                               exposure

Impaired water and point       In-stream water     1, 3-5, 7, 8, 18, 26, 28, 32,
source identification          quality             40

                               Concentration in    1, 3-5, 7, 8, 18, 25, 26, 28,
                               sediments           40

                               Concentration in    1, 3-5, 7, 8, 18, 26, 28, 40
                               biota

                               Discharges from     1, 3-5, 7, 8, 18, 26, 28, 32,
                               individual sources  40

                               Aquatic life        1, 3-5, 7, 8, 28, 33, 40
                               effects

                               Fate and transport  1-3, 5, 18, 21, 22, 24, 28,
                                                   37, 39, 40

Total maximum daily load       Hydrology           21, 22, 24, 28, 38
development

                               In-stream water     1-5, 7, 8, 18, 28, 37, 39
                               quality

                               Concentration in    1-5, 7, 8, 18, 28, 32, 37, 39
                               sediments

                               Concentration in    1-5, 7, 8, 28, 37
                               biota

                               Discharges from     1-5, 7, 8, 18, 28, 39, 40
                               individual sources

                               Fate and transport  1-5, 7, 8, 18, 21, 22, 24,
                                                   26, 28, 32, 37, 39, 40

Water-quality-based permit     Hydrology           1-5, 19, 24, 25, 27, 28, 30,
issuance                                           36, 40

                               In-stream water     1-5, 7-12, 17-19, 25-28, 31,
                               quality             32, 36, 37, 40

                               Discharges from     1-5, 7-12, 15, 16, 18, 19,
                               individual sources  25, 26, 28, 31, 32, 39, 40

                               Fate and transport  1-5, 7-9, 17-19, 24-28, 31,
                                                   32, 36, 39, 40

"Local limits" development     Discharges from     1-5, 7, 8, 12, 15, 16, 18,
                               individual sources  25, 26, 28, 31, 32, 39, 40

                               Removal             1-5, 7, 8, 10, 19, 25, 26,
                               technologies        28, 39

                               Fate and transport  1-5, 7, 8, 10, 18, 19, 25,
                                                   26, 28, 39

Compliance problem             In-stream water     1, 3-5, 7, 8, 18, 26, 32, 40
identification                 quality

                               Aquatic life        1, 3-5, 7, 8, 18, 19, 26, 40
                               exposure

                               Discharges from     1, 3-5, 7, 8, 18, 26, 32, 38,
                               individual sources  40
--------------------------------------------------------------------------------


   QUALITY ASSURANCE PRACTICES
---------------------------------------------------------- Chapter 3:3

To determine the extent to which the completeness and quality
assurance criteria were employed within the 7 activities, we
interviewed the relevant EPA managers, using our 40 criteria as a
questionnaire with a five-point response scale rating the extent to
which the activity is conducted "always," "usually," "sometimes,"
"seldom," and "never." A criterion should be met at least "usually"
if quality assurance criteria are to be met.  Therefore, we
classified the responses by grouping them into two categories:  met
("always" and "usually") and not met ("sometimes," "seldom," and
"never").  We also asked for any supporting data and requested that
each response be explained as we conducted our interviews. 

We examined the extent to which the quality assurance criteria were
met from two different perspectives:  first, from the overall
perspective of the 7 activities, and, second, from the perspective of
the information types (that is, to what extent were they supported by
necessary quality assurance efforts?). 

A preliminary finding was that the officials we interviewed were
aware of EPA's position concerning quality assurance policies and
practices.  From our review, it is clear that these managers
recognize and have accepted the principle of quality assurance within
their activities. 

Table 3.3 presents the results of our interviews with EPA officials
concerning the degree of implementation of the quality assurance
criteria.  Table 3.4 summarizes our results for the activities and
information types.  As presented in table 3.4, 2 activities (those
that establish effluent guidelines and water quality criteria) were
exceptional in meeting quality assurance criteria that are critical
to the effort.  They met 93 percent of their quality assurance
requirements.  However, the 5 other activities fall short in taking
the necessary steps to ensure information quality.  That is, impaired
water and point source identification, TMDL development,
water-quality-based permit issuance, "local limits" development, and
compliance problem identification are less than optimal in ensuring
that the required quality assurance steps are taken in developing,
collecting, or maintaining their needed information.  Collectively,
about 58 percent of their quality assurance criteria were met. 



                                                                      Table 3.3
                                                       
                                                           Responses to Individual Quality
                                                                  Assurance Criteria



No.\
   b     F     G     H     M     D     E     I     J     L     B   C   D   F   I   K   A   B   C   D   F   K   A   B   F   K   F   G   K   B   E   F
----  ----  ----  ----  ----  ----  ----  ----  ----  ----  ----  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --
   1     5     5     5     5     5     5     5     5     5     4   4   4   2   3   2       4   4   4   4   4   5   4   5   4   4   4   4   3   4   4
   2     4     4     4           5     5     5     2     2                         2       3   2   2   2   3   5   3   4   4   4   3   3
   3     5     5     5     4     5     5     5     4     4     4   4   4   2   3   2       4   4   4   4   4   5   4   3   4   4   4   4   3   4   5
   4     5     5     5           5     5     5                 5   4   4   3   4           4   4   4   4   4   5   4   4   3   4  4,  4,   4   4   3
                                                                                                                                   5   5
   5     5     5     5     5     5     5     5     4     4     4   4   4   3   4   4       4   4   4   4   4   5   4   4   3   2   2   2   1   3   3
   6                       5
   7     5     5     5                       5                 4   3   4   5   4           4   4   3   4   4       4   4   4   4   4   4   2   4   4
   8     5     5     5                       5                 4   4   4   3   4           4   4   4   4   4       4   4   4   4   4   4   3   4   4
   9                             5     5     5     5     5                                                         3   3   3
  10           5     5           5                 5                                                               3   2           3   3
  11                             5     5     5     5     5                                                         4   4
  12     5                                                                                                         3   2       3
  13     4     4     4
  14
  15                                                                                                                   2       2
  16                                                                                                                   2      1,
                                                                                                                               2
  17                                                                                                               4       4
  18                       5                       5           3   4   4   4       4       4   4       4   4       4   4   4   5       3   2   5   4
  19     5     5     5                                                                                         4   3   4   4       4   4       3
  20     5
  21                                                                               3   3                   3
  22                                                                               3   4                   4
  23
  24                                                                               4   3                   3   2           4
  25           2     5                                             3                                           2   3   4   4   2   2   2
  26                             4     4     4                 4   4   4   3                               4       4   3   4   4   4   4   4   4   4
  27                             5                                                                             4   4       3
  28     5     5     5     5     5     5     5   4,5   4,5     3   2   3   2   3   3   3   3   2   2   3   3   4   4   4   4   3   4   4
  29                       5
  30                                                                                                           5
  31                                                                                                               3   3   3   4
  32                                                           2           2                   3           3       4   2   3   4           3      4,
                                                                                                                                                   5
  33                                         5     5                           3
  34                             3           3     3
  35                                         3
  36                                                                                                           3   4       5
  37                                                                               4       4   3   2       4       3
  38                                         5     5                                   2                                                           5
  39     5     5     5                                                             3       3   2       3   3           3   3   3   4   4
  40     5                                                     2   2   3   2   2   2                   3   2   4   2   3   4   3           2   3   3
----------------------------------------------------------------------------------------------------------------------------------------------------
\a Letters in column headings under each activity refer to
information types displayed in figure 1.  Numbers in cells are 1,
never; 2, seldom; 3, sometimes; 4, usually; 5, always. 

\b Criteria follow numbers in table 3.1. 



                          Table 3.4
           
            The Number of Criteria Met Within Each
                       Information Type


                                      Assign             Not
Activity          Information type        ed     Met     met
----------------  ------------------  ------  ------  ------
Effluent          Discharges from         14      14       0
 guidelines        individual
 development       sources
                  Removal                 13      12       1
                   technologies
                  Water use               13      13       0
                   reduction
                   technologies
                  Economic data            7       7       0
Water quality     Concentration in        12      11       1
 criteria          biota
 development
                  Aquatic life             9       9       0
                   exposure
                  Aquatic life            15      13       2
                   effects
                  Human health            12      10       2
                   effects
                  Human health             7       6       1
                   exposure
Impaired water    In-stream water         11       7       4
 and point         quality
 source
 identification
                  Concentration in        11       7       4
                   sediments
                  Concentration in        10       8       2
                   biota
                  Discharges from         11       2       9
                   individual
                   sources
                  Aquatic life             9       4       5
                   effects
                  Fate and transport      12       4       8
TMDL development  Hydrology                5       1       4
                  In-stream water         11       8       3
                   quality
                  Concentration in        12       7       5
                   sediments
                  Concentration in         9       5       4
                   biota
                  Discharges from         11       7       4
                   individual
                   sources
                  Fate and transport      17       9       8
Water-quality-    Hydrology               13      10       3
 based permit
 issuance
                  In-stream water         23      14       9
                   quality
                  Discharges from         22      12      10
                   individual
                   sources
                  Fate and transport      21      14       7
"Local limits"    Discharges from         18      11       7
 development       individual
                   sources
                  Removal                 13       9       4
                   technologies
                  Fate and transport      14       9       5
Compliance        In-stream water         10       2       8
 problem           quality
 identification
                  Aquatic life            10       7       3
                   exposure
                  Discharges from         11       8       3
                   individual
                   sources
============================================================
Total                                    386     260     126
Percent                                          66%     34%
------------------------------------------------------------
\a Criteria numbers are in table 3.1. 


   MEANINGFULNESS OF THE RESULTS
---------------------------------------------------------- Chapter 3:4

The work presented above shows that practice within 5 of the 7 core
activities does not measure up to EPA's enunciated information
quality assurance policies and principles.  This raises questions
about the quality of information generated and used within these 5
activities. 

We initiated a separate qualitative review of agency documents and
information bases and were able to confirm at least some of these
findings.  For example, many of our EPA interviewees had noted that
data on in-stream water quality for toxic pollutants are generally
lacking (criterion 37).  We were able to confirm the accuracy of
these responses through a review of the data in STORET, EPA's
principal repository of water quality data.  For a group of streams
around the nation on which large manufacturing facilities are
situated, we found very few with any data for toxic pollutants.\2 In
fact, we found that only 9 percent of these streams had had any
in-stream data for organic toxic pollutants or pesticides entered in
STORET since 1988, despite the presence of major industrial
facilities discharging a variety of toxic pollutants into all these
waterways. 

A lack of in-stream data can result in pollutants failing to be
identified or limited on NPDES permits.  For example, when there is
no information about in-stream concentrations of pollutants being
considered for a water-quality-based limit (activity 5), the
in-stream concentration parameter is set to zero within the modeling
routine used to derive permit limits.  For streams in which the
pollutant is present but is not reported, the absence of data
signifies that a greater amount of pollutant discharge from
facilities will be allowable than if valid in-stream data had been
available.\3 It may also lead to facilities not receiving limits
(restrictions) covering toxic pollutants although there is a water
quality concern in the receiving body of water.  EPA acknowledges
some of these problems and has embarked on a program to modernize
STORET to address them. 


--------------------
\2 These streams (river reaches) were selected because dischargers
within three significant industrial categories (pesticide
manufacturing, pharmaceutical manufacturing, and pulp and paper
manufacturing) are situated on them. 

\3 Environmental Protection Agency, Technical Support Document for
Water Quality Based Toxics (Washington, D.C.:  March 1991), p.  130. 


   SUMMARY
---------------------------------------------------------- Chapter 3:5

Our findings raise concerns about the quality of the information
supporting EPA's core activities controlling toxic discharges into
surface waters.  Although 2 of the activities had strong quality
assurance practices associated with them, the 5 others were much
weaker, meeting about 58 percent of their critical quality assurance
criteria. 


THE EXTENT TO WHICH LIMITS FOR
TOXIC SUBSTANCES ARE INCLUDED ON
NPDES PERMITS
============================================================ Chapter 4

In this chapter, we address our third evaluation question:  To what
extent are limits for toxic substance discharges included on NPDES
permits?  We answered this question by comparing information on
actual toxic pollutant discharges from three data sources with a list
of toxic pollutants controlled through NPDES permits for a sample of
facilities. 


   IDENTIFYING TOXICS TO BE
   LIMITED ON NPDES PERMITS
---------------------------------------------------------- Chapter 4:1

We based the development of our sample of facilities to examine on
three industrial categories:  pesticide manufacturing, pharmaceutical
manufacturing, and pulp and paper manufacturing.\1 We reviewed the
information on actual toxic effluent discharges for 236 facilities
within these categories and compared them to the facilities' permits. 
Identifying which pollutants to control through the NPDES permit
process is the critical step in issuing permits.  As stated earlier,
without an explicit permit limit on a pollutant, a facility is free
to discharge any amount.  Hence, identifying the range of pollutants
that should be included on NPDES permits is essential for the toxic
control program to succeed. 

It has been generally conceded that a discrepancy exists between the
number of pollutants that are regulated through permits and the
number that should be regulated.\2 However, this gap has not been
systematically examined.  We do so here and report our findings in
this chapter for a sample of dischargers.  It should be noted that
the existence of such a gap would not in itself suggest problems with
the permit process.  That would be the case only if the discharge of
uncontrolled toxicants resulted in unacceptable risks to human health
and aquatic life.  We address this issue in chapter 5. 


--------------------
\1 We selected these three because EPA's effluent guidelines program
had recently conducted monitoring studies for facilities within these
categories. 

\2 U.S.  General Accounting Office, Water Pollution:  Stronger
Efforts Needed by EPA to Control Toxic Water Pollution,
GAO/RCED-91-154 (Washington, D.C.:  July 1991), p.  4. 


   INFORMATION SOURCES ABOUT
   DISCHARGES
---------------------------------------------------------- Chapter 4:2

Our sources of information about toxic pollutant discharges for our
sample of facilities were (1) each facility's permit application (EPA
Form 2c); (2) EPA's Toxic Release Inventory (TRI), an annual
inventory of discharges from the largest manufacturing facilities in
the nation; and (3) discharge data collected under effluent
guidelines.\3 We pooled these three sources of information to
identify the toxic pollutants being discharged at the facilities in
our sample.\4 Each of these sources employs a different data
collection method.  Consequently, using all three sources provides us
a more comprehensive characterization of the toxic pollutant
discharges from our sample of facilities.  TRI in particular has a
different data collection method than the permit applications.  Much
of the data it contains were derived from mass balance calculations
or published emissions factors.  These data collection methods
complement the monitoring information contained within the other data
sources.  Table 4.1 presents the major attributes of the three data
sources. 



                                    Table 4.1
                     
                       Major Attributes of Three Sources of
                              Discharge Information


Attribute    Permit applications  Effluent guidelines     TRI
-----------  -------------------  ----------------------  ----------------------
Primary      To identify          To identify pollutants  To identify the
purpose      pollutants being     and concentrations      amounts of "hazardous
             discharged for       discharged by the most  chemicals" that are
             purposes of permit   efficient facilities    annually and routinely
             issuance             within an industry      discharged into the
                                                          environment

Data source  Approximately 7,000  Small sample within     Approximately 2,300
             major and 57,000     selected industries     facilities discharging
             minor facilities                             into surface waters

Frequency    Every 5 years        No schedule             Annually
of
reporting

Time period  Daily                Daily and annually      Annually
covered by
estimate

Number of    Varies               Typically, a series     No requirements
samples for                       over 3 days
each
estimate

Sampling     Not applicable       Individually selected   All facilities meeting
frame                             industries              size requirements
                                                          discharging toxics

Pollutants   Priority pollutants  More than 450           Approximately 300
covered      and                  pollutants              pollutants and
             80 others "if                                chemical groups
             expected to
             be present"

Principal    Monitoring for       Monitoring              Mass balance,
basis of     priority                                     engineering
estimate     pollutants;                                  calculations, or
             identification of                            monitoring
             80 others "if
             expected to be
             present"\a

Who          Facility             EPA                     Facility
collects
sample
--------------------------------------------------------------------------------

\a Details of monitoring requirements are contained in section 122.21
of title 40 of the Code of Federal Regulations. 


--------------------
\3 We used the TRI responses from 1989, the most recent available
when we conducted our analyses. 

\4 We included 234 facilities from TRI.  Because of the difficulty of
obtaining and reviewing permit applications, we drew a random sample
of 72 facilities for data from that source.  The third source of
data, effluent guidelines studies, contributed data for approximately
32 facilities, which was the number of direct dischargers available
within that data set. 


   TOXICS DISCHARGED WITHOUT
   PERMIT LIMITS
---------------------------------------------------------- Chapter 4:3

We found a large number of different pollutants being discharged by
one or more of the 236 facilities.  One hundred and eighty-five
different toxic pollutants were reported to have been discharged from
the 236 facilities.  Whereas some pollutants were discharged from a
single facility, others were reported from numerous facilities.  For
example, chloroform, a common byproduct of industrial processes that
EPA views as a "probable" carcinogen, was reported to be discharged
by 91 of the 236 facilities.  Table 4.2 presents the toxic pollutant
discharges we identified, indicates whether they are priority
pollutants or nonpriority pollutants, and shows the number of
facilities they were discharged from and the number that were
uncontrolled through the permit process across the 236 facilities in
our sample.  Because of the proprietary nature of some of the
discharge data, we cannot present a facility-by-facility breakdown of
our results. 



                          Table 4.2
           
            The Extent of Discharge and Control of
            Individual Toxic Pollutants, Priority
                       and Nonpriority

                                                    Faciliti
                                          Faciliti  es where
                                Priority        es    listed
                                pollutan  releasin        on
Pollutant                       t                g    permit
------------------------------  --------  --------  --------
1,1-dichloroethane              No               1         1
1,1-dichloroethylene            Yes              3         0
 (vinylidene chloride)
1,1,1-trichloroethane           Yes              7         3
1,1,2-trichloroethane           Yes              1         1
1,1,2,2-tetrachloroethane       Yes              1         1
1,2-dichloroethane              Yes             12         6
1,2-butylene oxide              No               2         0
1,2-dichlorobenzene             Yes              1         1
1,2-dichloroethylene            No               1         0
1,2-dichloropropane             Yes              1         1
1,2,4-trimethylbenzene          No               2         0
1,2,4-trichlorobenzene          No               1         0
1,3-butadiene                   No               1         0
1,3-dichlorobenzene             Yes              1         1
1,3-dichloropropylene           Yes              1         0
1,4-dichlorobenzene             Yes              1         1
2-hexanone                      No               1         0
2-phenylphenol                  No               1         0
2,3,7,8-TCDD (dioxin)           Yes              5         1
2,4-D                           No               3         2
2,4-diaminotoluene              No               1         0
2,4-dichlorophenol              Yes              2         1
2,4-dimethylphenol              Yes              1         0
2,4-dinitrophenol               Yes              1         0
2,4,5-TP                        No               1         0
2,4,6-trichlorophenol           Yes              1         0
3,3-dichlorobenzidine           Yes              1         1
3,4,5-trichloroguaiacol         No               1         0
3,4,5-trichlorocatechol         No               2         0
4,4'-isopropylidene diphenol    No               1         0
4,5-dichlorocatechol            No               2         0
4,5-dichloroguaiacol            No               2         0
4,5,6-trichloroguaiacol         No               1         0
5,6-dichlorovanillin            No               1         0
6-chlorovanillin                No               1         0
Acetaldehyde                    No               5         0
Acetone                         No             108         1
Acetonitrile                    No               4         0
Acrolein                        Yes              1         0
Acrylamide                      No               2         0
Acrylic acid                    No               3         0
Acrylonitrile                   Yes              3         2
Alachlor                        No               1         1
Aluminum                        No               7         1
Ametryn                         No               1         0
Ammonia                         No             147        56
Ammonium nitrate                No               7         0
Ammonium sulfate                No               9       1\a
Aniline                         No               4         0
Antimony                        Yes              3         1
Arsenic                         Yes              9         1
Asbestos                        Yes              1         0
Atrazine                        Yes              1         0
Barium                          No               6         0
Benomyl                         No               1         0
Benzene                         Yes              5         2
Biphenyl                        No               6         0
Bis(2-chloroethyl) ether        Yes              1         1
Bis(2 ethylhexyl) phthalate     Yes              9         1
Boron                           No               3         0
Bromacil                        No               1         1
Bromoform                       Yes              2         1
Bromomethane                    Yes              1         1
Butachlor                       No               1         1
Butylamine                      No               1         0
Cadmium                         Yes              1         0
Calcium                         No               3         0
Captan                          No               2         0
Carbaryl                        No               3         1
Carbon disulfide                No               2         1
Carbon tetrachloride            Yes              6         4
Catechol                        No              72         0
Chlorinated phenols             No               3       1\b
Chlorine                        No              44        19
Chlorine dioxide                No               2       2\c
Chloroacetic acid               No               2         0
Chlorobenzene                   Yes\d            4         3
Chloroethane                    No               2         0
Chlorofluoromethane             No               1         0
Chloroform                      Yes             91        28
Chloromethane                   Yes              5         0
Chlorothalonil                  No               1         0
Chromium                        Yes             22         4
Cobalt                          No               1         0
Copper                          Yes             17         4
Cresol                          No               1         0
Cyanide                         Yes             12         8
Cyclohexane                     Yes              2         0
Dacthal                         No               1         0
DDT                             Yes              1         1
DEET                            No               1         1
Di-n-butyl phthalate            Yes              1         0
Dicamba