John Felleman

S.U.N.Y. College of Environmental Science and Forestry

felleman@esf.edu

Adaptive Management is emerging as a dominant approach for sustainable stewardship of the environment. To be successful, it requires effective integration of continuing systems modeling and participatory democracy. The Internet holds great promise for providing the information and knowledge interface necessary to support effective adaptive management discourse. Open Modeling (OM) is seen as the keystone of this enterprise. A preliminary OM classification scheme, and the components of a robust OM System are introduced. Initial applied research needs are identified.

CONTENTS:
    INTRODUCTION
    DIMENSIONS OF OPEN MODELING
    A COMPREHENSIVE OPEN MODELING SYSTEM
    MODELING CORE
    PUBLICS' INTERFACE
    DECISION SUPPORT
    STEWARDSHIP
    WHAT'S NEXT?
    REFERENCES
 
 
 

INTRODUCTION
    The modern environmental era can be understood as the locus of two co-evolving movements. Environmental systems science analyzes bio-physical processes from the perspective of flows and transformations of matter and energy through space and time. This approach has yielded significant basic insights for issues ranging from animal population dynamics, to Superfund pollution plumes, to global warming. Participatory decision-making has drawn on our deep heritage of pluralism and "strong democracy" to generate a partial shift in power away from private firms and closed, centralized public bureaucracies enabling a broad spectrum of stakeholders to become actors (Barber, 1984). This movement has been facilitated by policies that require information dissemination, and provide the legal standing for adversarial judicial proceedings and/or negotiations. Together these movements provide the foundation for an emerging ecologically based normative world view generally known as sustainable development.

    The National Environmental Policy Act of 1980 (NEPA 1970, 42 USC 4321 et seq.) was a primary catalyst in this co-evolution, constituting a broad brush declaration of the primacy and universality of environmental health in public decisions. The teeth of this potentially symbolic law are found in the requirement for Environmental Impact Assessments (EIA). Federal decisions must be based on predictions from interdisciplinary science. A range of viable alternatives had to be analyzed, and prior to a final decision drafts of the study, (body written for general audience with technical appendices), were to be widely disseminated for review and comment.

    These two arenas of systems science and participatory/strong democracy have a historically weak interface. This is due in large part to the highly structured form of rational decision making, and its associated discourse, which has dominated most forms of modern public decisions including facility siting, command and control pollution regulations, and the master planning of public lands (Williams, 1995). As depicted in Figure 1, critical stages in the process include: selection of objectives, formulation of alternatives, prediction of expected effects, evaluation of tradeoffs, and monitoring and feedback. This idealized process that is semi-explicit in NEPA, and many other environmental laws. Typically these have been implemented in a project by project disjointed manner ignored cumulative, system wide effects. Additional critiques are well known, and include: narrowly constructed criteria often used with poorly grounded predictive models which do not incorporate either local uniqueness or larger biophysical systems; token "public participation" in the form of EIA’s and public hearings which come after the analyses are completed and preliminary internal decisions have been formulated; and the lack of monitoring,(optional in NEPA), and associated knowledge building.
 

     Over the past decade, the field of "adaptive management" has emerged with the express intent of advancing sustainability by answering these critiques through a comprehensive restructuring of the science-public discourse interface. Kai Lee, in his seminal study of the Northwest Power Planning Council (NWPPC), Compass and Gyroscope, postulates that the "compass" of science can best provide meaningful decision direction by establishing a broad, robust, continuing systems modeling framework (Lee, 1993). Model-based predictions can progress from general to more specific as development actions are monitored over extended time periods. The "gyroscope" of democratic processes provides both a check and balance against excesses, and a crucible of social learning where new societal values, which reflect our deepening knowledge regarding the linkages between sustainability and development, can be nurtured.

    Whereas the traditional rationalism shown in Figure 1 often took place within a closed public or private bureaucracy, adaptive management necessarily occurs in an open arena. Figure 2 illustrates the diverse context in which current environmental issues are resolved. The enfolding of large scale applied science, which frames management decisions as on-going experiments, and public discourse has traditionally faced the insurmountable barrier of achieving a prolonged, technically-based, dialog among spatially dispersed parties with widely differing values and scientific literacies. From NEPA in 1970 to Lee’s analysis of the Northwest Power Planning Council’s (NWPPC) innovative management of the Columbia Basin in early 1990’s, meaningful, effective multi-way communication has often remained a distant goal of the modern environmental movement.

    A central barrier has been the inability to provide physical and cognitive access to the systems science models and managerial models for all actors and stakeholders. This critical lynchpin can be defined as "Open Modeling" (OM).Now,the physical components of this communication barrier are being demolished by the explosive transformations underway in information technologies (IT). The Internet, coupled with increased desktop power, object programming, visualization software, and public dissemination policies has generated an entirely new context for adaptive management discourse. How this revolution will take place, and whether or not it will empower adaptive management is the subject of growing debate with opinions ranging from anarchy to a new plateau of environmental stewardship and sustainability.

    Many important experiments are underway in government, private firms, Non-Governmental Organizations (NGOs), and educational institutions utilizing IT for distributed process modeling, and promoting environmental discourse such as public relations, education, and negotiation. Most of these initiatives are grounded in a specific issue and reflect local needs and resources. Many appear to be strong in pushing content while lacking in analysis of end-user effectiveness. All are occurring in a policy vacuum.

Studies of of electronic democracy have been underway for a generation. An important first foray into understanding shared environmental (spatial) modeling was the National Center for Geographic Information Analysis’ (NCGIA) Initiative 17 Collaborative Spatial Decision-Making (NCGIA, 1996). which focused on Geographic Information Systems (GIS). More recently the National Academy of Sciences has published a broad overview of the technical and social science research that will be necessary for all citizens to have effective access to electronic information via the Internet (National Academy, 1997). While still somewhat inchoate, it is now possible to stand back and ascertain the basic general dimensions and components of a comprehensive "open modeling" (OM) system. Hopefully, an articulation of this framework will stimulate shared learning, help prioritize R and D needs, and focus attention on the policy gaps which need bridging.

DIMENSIONSIONS OF OPEN MODELING

Progress in developing scientific knowledge is the result of iterative cycles of conceptualization, data generation and analysis, sharing, independent replication, peer critique and revision. This process is inherently "open", particularly in its latter stages, within the scientific community (note: the filters of national security, trade secrets, and originality of authorship frequently constrain fully open science).

Similarly, policy pluralism as exemplified by a negotiation, is "open". In theory there needs to be sharing of descriptive information and causal knowledge related to expected impacts for a successful "win-win" outcome. Although certainly not a panacea, the impetus to avoid the gridlock and associated time and resource investments that have characterized a generation of environmental litigation has led to a negotiation-oriented framework for most current issues (Amy, 1987, Bingham, 1986).

Over the past two decades major investments in the public and private sectors have converted vast quantities of environmental data to digital form. Most new data, whether derived from satellite remote sensing, collected in the field with monitors or Global Positioning Systems, or generated in the lab now goes directly to digital format. These databases are the foundation for digital modeling which has become a dominant methodological approach both for scientists in developing new knowledge, and decision-makers in analyzing courses of action. The "openness" of the modeling has thus emerged as a critical determinant of the ultimate success of both enterprises.

In order to examine this issue it is useful to have a preliminary classification scheme. Figure 3 depicts an "Openness" classification space with two dimensions. "What If ?" Interactivity is a continuum of how open the model is to end-user queries, particularly alternative scenarios. What if the highway is located here? What if releases from the dam are increased during spawning season? The Logic Transparency axis reflects how clearly the underlying model construction and mathematics are presented. This is critical both for scientific credibility and policy legitimization. The usefulness of this scheme has been pilot tested in the spring of 1998 by a graduate seminar class who examined
a self-selected sample over 200 OM related web sites.

                                                                                                                                         WHAT IF?
 
 

    Categories on the "What-If?" axis begin with Closed. Many modeling sites are similar to EIA’s in that modeling has been done and results are presented but end-users cannot interact at all with the model. The Canned Scenarios set allows end-users to replicate model runs of pre-analyzed alternatives. Rather than interacting directly with a model, one accesses a database of stored model runs. Reduced Form modeling involves the utilization of a simplified version of the actual science/management model and associated databases. In the Expert Facilitator category queries are submitted to a remote analyst who subsequently runs the model. Finally, in a fully Open model the end user interacts directly with the systems model.

    A number of factors appear to be active in generating this wide range. Many of the original models are quite large, include massive data sets, and were developed by specialists for use by specialists. Issues include security as well as efficiency. Will the model be accessed remotely, or downloaded to run "stand alone" on the desktop, or some combination involving "applets"? For example, the Everglades Landscape Model runs on high end work stations and is being ported to a parallel supercomputer (Everglades, 1998). Visitors to the web site can learn much about the modeling effort but cannot access the model. In contrast, the NWPCC has developed "Operate the River" to illustrate the complex connectivity of the numerous dams in the Columbia River system. By the Internet the public can directly examine canned dam operation scenarios, or download this reduced form black box spreadsheet to run in stand alone mode (Northwest, 1997).

    The Logic Transparency axis begins with Black Box. Here the end user has no understanding as to what computations transform input data into output environmental predictions. Schematic Flow is a diagram, (typically box and arrow), of the internal processing including serial and parallel stages, and positive and negative feedbacks. Parameterization includes the establishment of values for the variables, while Forcing Functions are the equations which control the directions and rates of energy and matter flows and transformations through the system being modeled.

    Sensitivity Analysis involves modifying a single variable to see its effect on the system. Traditionally models have been deterministic, the same single value inputs always generate the same single value outputs. This approach assumes a simple mechanistic process. Error Analysis is a stochastic approach to modeling where statistical distributions of inputs propagate distributions of outputs. In a fully transparent Open Model the underlying assumptions are explicit and code logic is available.

    Black box process models are common in EIA’s, and in land planning which has a century long history of active citizen participation. In the innovative work of CIESIN’s AR/GIS highly visual colored landuse maps embedded in a Geographic Information System (GIS) are manipulated by end-users to create alternative development scenarios. The land use changes are subsequently inputted to linked black box predictive models, such as traffic and wastewater to yield tables and graphs of expected effects (Faber, 1997),(CIESIN, 1997).

    Essentially all computer models of environmental systems are based on flow diagrams. Traditionally, these diagrams were used to guide the initial computer coding (such as FORTRAN), but their logic became opaque as the code was modified and compiled. In recent years schematic flow visualizations are increasingly available due to the popularity of graphical object programming. Many environmental models, both educational and applied are now being developed in a visual object language. STELLA and Model-It are examples (High Performance, 1998),(COGITO, 1998). In completed models, access to model parameters and equations is determined solely by the modeler.

    Many agencies such as the EPA and Department of Agriculture have a long history of "technology transfer" where models and data are developed and distributed to technical end-users (EPA, 1998). More recent models have included stochastic error analysis. This capability may be written directly into the system model source code. Alternatives for doing error analysis include "add-ins" for spreadsheet based models (Palisade, 1998), and subroutines in GIS (Clark U., 1998).

    Process models that once originated with government sponsored research have been extended by value-added private firms to add features such as Graphical User Interfaces (GUIs), visualization, and linkages to GIS and associated data sets. Some, such as PCSWMM, (Computational, 1998),have increased transparency with parameterization and sensitivity support. Other firms have created new products with decreased scientific transparency designed to extend the market to non-technical end-users such as community emergency spill staff through the use of visualization (Radian, 1998).

    Stochastic modeling has only recently become available on the desktop. To run such analyses probabilistic data inputs are required. These typically do not exist. For example the millions of existing paper maps include little or no information regarding the precision of lines or the homogeneity of categories. The NCGIA explored in depth the topic of error propagation in spatial models, but the work is primarily theoretical (NCGIA Research, 1998). No major commercial vector GIS currently addresses this issue. In general, there is currently a dearth of models which are stochastic and generate the probabilistic estimates of expected outcomes which informed decision makers would expect. While some of the newer EPA sponsored efforts are of this form, the widespread reliance on determinism reflects an increasingly obsolete era of closed decision making.

    Although the openness dimensions of environmental modeling provide unique insights into the variety of existing efforts, OM is best understood from the perspective of adaptive management support. In the next section a comprehensive system frame will be introduced.

    While environmental process modeling is at the core of such a system, a number of other subsystems are necessary to provide a robust context. Figure 4 depicts the overview. The first is the Publics’ Interface, through which all end-users will engage the system. Most current environmental policies, such as NEPA, incorporate the naïve policy from the hard copy text era that there are two publics: technical, and general public (8^th grade education). This oversimplification has long been at the heart of many controversial EIA debates (O’Hare, 1983). In a digital world it is possible to recognize a rich spectrum of multiple publics, with differing needs, levels of knowledge, and values.

    Another subsystem focuses on Decision Support. These modules assist all stages of decision making as previously depicted in Figure 1. Particular emphasis is placed on supporting collaboration including negotiation. The final subsystem, Stewardship, addresses the long term (bio-geophysical time scales) systems learning upon which adaptive management is founded.

    At the center of the modeling core are tightly coupled process models of environmental systems and GIS. This reflects the inherently spatial nature of environmental phenomena. The degree of coupling is rapidly advancing. Early models typically used the GIS to create "lumped parameters", such as an average surface coefficient for a subwatershed, which was stored in a database to be subsequently read by the pollution model. Similarly it was common for the GIS to be used as a tool to assess and display model outputs. For example, a pollution plume from an atmospheric model can be exported to a GIS and overlayed on a population census to estimate exposure (note: this standard practice assumes everyone happens to be in their place of residence when the plume occurs).

    Process models, GIS and associated data sets are increasingly interwoven. An important example is the integration of landuse change, hydrological modeling of runoff, hydraulic modeling of flood routing, and estimation of flood damages (ASCE, 1997). The Evergalades project mentioned above models the flows and transforamtions of energy and matter between finely disggregated GIS grid cells (Spatial, 1998). Fueling this change are the industry movements to both open the elements of historically proprietary GIS (OpenGIS, 1998), the new ability to incorporate spatial attributes in major enterprises relational database systems, and the explosive growth of Internet-based GIS.

    Other components of the Modeling Core subsystem include Statistics and Error Analysis, and Scientific Visualization. Statistical analysis are used within many models to estimate parameters. In remote sensing, one of our major sources of environmental information, virtually all data has passed through a complex series of cleaning, smoothing, and interpretive algorithms to transform massive sets of spectral reflections and emissions to useful information such as a land cover map. Error Analysis has been discussed above under Logic Transparency.

    The field of Scientific Visualization has played a critical role in our ability to understand complex systems. Processes such underground pollution plumes, global warming, and spatial population dynamics require maps, graphs, 3-D depictions, and more recently, animations, and virtual "walk/fly throughs" for us to comprehend the numerical data sets which typify the raw output of environmental models. Watching the weather forecast on the evening news is a good barometer of the advancing sophistication of this field. NASA provides a useful listing of visualization sites (NASA, 1998). A fundamental challenge is to develop visualizations that are meaningful to nonscientists. Considerable progress in this arena is being made in education, mass media, and entertainment.
 
 

    The Segmented GUI Gateway provides customized end-user entrance to all OM subsystems, reflecting the wide variations in environmental understanding, differences in scientific literacies, and personalized long term longitudinal interactions with the OM system. The latter is of particular import to complement our need to adaptively steward the environment in perpetuity. The basic concept of a Segmented GUI is not new. It has been standard management practice for a generation to design a hierarchy of enterprise database read/write access to reflect different levels of organizational use and authority. In the field of computer games, opening screens have typically asked for selection of the "level of difficulty", and/or if the player wishes to begin where they last ended.

    The European Community is actively developing environmental information systems to be used by publics with differing languages and differing local regulations under the umbrella of Euro community standards (Burgard, 1997). Educators have expanded this approach in the development of web based instructional modules suitable for multiple age groups (SimScience, 1998). The advent of personalized "agents" has greatly expanded the potentials for segmentation (Patty Maes, 1997).

    The Legitimacy component includes three elements: Credibility, Disclaimers, and Privacy. The Internet is increasingly filled with mis- and dis-information. The former is incorrect due to error and obsolescence; the latter is intentionally wrong. How are end-users to know what they are presented is credible? In the traditional print fields, government documents and scholarly, peer reviewed journals, were readily differentiated from the popular press, and tabloid or underground sources. For web-related research a whole new generation of legitimization is needed. Libraries are educators are leading a credibility initiative (Syracuse, 1997).

    In the GIS field, the Federal Geographic Data Committee has set meta-data standards for spatial data generated by federal agencies,(guidelines for participating states and businesses), which include data quality and lineage (Federal Geographic, 1998). Unfortunately for much legacy data the metadata is simply "NA". The use of predictive models remains a controversial topic in scientific and non-scientific arenas (Water, 1990).

    Models are by definition simplifications of the real world. Some models are designed for illustrative communication, some are for heuristic learning, while others are intended for decision support. Legally it is important that end-users understand the purpose of the OM as well as its limitations. Disclaimers are becoming standard practice in distributed models. These range from copyright interests (NCAR, 1996), to analytical capabilities (Office of Radiation, 1998), to scope of analysis (Northwest, 1997).

    If OM is to be truly interactive, end-users will not only be downloading information, they will be generating model outputs, querying technical assumptions, and participating in issue forums and decision making processes. This openness raises serious questions of privacy. What does the system initially know about the end-users?, what does it learn over time?, how does the presence of "agents" affect privacy?, who can access this information? To date most decision support systems have been based on closed local networks. Federal web sites are just now addressing some of these issues (Government, 1998). One pioneering non-governmental effort has been RTK Net, which in addition to providing one stop access for toxic release and emergency planning information, provides free services to registered users which include topical discussion groups (RTK Net, 1998).

    It has long been known in commercial software that is designed for multiple end-users that context sensitive help is essential for productivity. Traditionally this help has been mechanistic, how to use the system. For environmental OM to be successful we must develop a robust Hypermedia Encyclopedia which assists in the understanding of terms, concepts, and processes. To date what one finds on the web are a spectrum of project based support "stuff". One the one hand is a plethora of traditional text based glossaries that are scanned and posted. An example is EPA’s "Terms of Environment" (EPA, 1997). Newer initiatives are educational efforts such as GIS (U. Cal., 1997), and the Nile simulation project which integrates systems modeling with on-line coursework covering the underlying natural and science and social sciences (U. Md., 1998).
 
 

    In the emerging arena of Internet-based OM we can identify four primary components of the Decision Support subsystem include: Workgroup Tools, Decision Tools, Policy Contexts, and Risk Analysis. Workgroup Tools are becoming standard in business and government as organizations move away from traditionally systematized bureaucratic work flows and toward collaborative team approaches to task management. Tools range from E-Mail, and shared data sets and calendars, to threaded discussions and compound multi-author document development. A fundamental challenge exists in effectively managing the very large number of discussants that the Internet may generate (Berge Collins, 1997).

    Decision Tools include methods of generating and evaluating alternatives, and negotiating decisions. Traditionally, many major environmental decisions were based on single objective sets such as maximization of benefit-cost. These approaches are highly controversial in both theory and practice and are increasingly being supplemented by multi-objective approaches that facilitate negotiation. A number of computer assisted approaches have been developed commercially. Typically they are designed to handle small to medium size groups (Expert Choice, 1998). CIESIN’s pioneering AR/GIS discussed above, integrates GIS, process models with multi-objective analysis and negotiation support software (CIESIN, 1997).

    Two additional components of Decision Support are Policy Context and Risk Analysis. The former includes the laws, regulations and administrative procedures which apply to environmental decisions. For OM to be effective, participants must have a working understanding of what constitutes substantive and procedural due process. An interesting pilot was done last fall in a graduate Great Lakes course specifically designed to bridge science and law (U. Buffalo (1997). There is a wealth of legal material available on the Internet (Cornell U., 1998). As in the fields of environmental science, much of the existing material in this subject is highly technical. The Hypermedia Encyclopedia will need to include a broad set of policy concepts.

    Risk Analysis is often the missing link in current environmental decisions. It is not required in either NEPA, and most environmental permits. What publics often want most to know is how a proposed action is expected to affect specific human and/or ecosystem receiver health, not that a deterministic model prediction number falls below a federal standard (McAllister, 1980). Regulatory agencies have traditionally set uniform workable standards which meet the criteria of equality and efficiency while begging the questions of genius loci, and scientific uncertainty. Most Risk Analysis materials and tools on the Internet are solely designed by and for experts. New Jersey, a pioneer in public risk involvement has a more open approach (New Jersey, 1997). OM should incorporate emerging capabilities and explicitly deal with these issues in a manner that promotes social learning.

    The function of the Stewardship subsystem of OM is to support the long term learning objective of adaptive management. As shown in Figure 4D, it’s basic components include Monitoring, Data Mining, and Accountability. The data sets which are depicted in the Modeling Core are quite massive and exist at different locations under different management. In recent years, considerable progress has been made in the public sector, and in public/private partnerships to share existing environmental data. Examples include: FGDC (Federal Geographic, 1998),Census Access Tools (U.S. Census, 1998), CIESEN (1997), and EPA’s Environmental Data Register (1998).

    The major issues related to Stewardship are longitudinal changes in the biophysical environment; changes in socio-economic processes ranging from land ownership, to regulatory conformance; and the accuracy of EIA and decision model predictions. Each of these has significant current deficiencies. The failure of EPA’s EMAP program, the long standing inability to successfully link wetlands data with real property data (Felleman, 1997), and the inherent difficulties in pollution monitoring (Russell, 1986), highlight the analytical and political difficulties in adequate monitoring.

    Assuming continued advances in Monitoring and Internet accessible data sets, two additional needs can be identified. Data Mining is an emerging set of analytical techniques which periodically are applied to warehoused data sets to reveal patterns of relationships which were not anticipated in the original data collection design (Facet, 1998). Since environmental systems are inherently complex, and models by definition are highly simplified, we need the ability learn both inductively and deductively. Accountability closes the loop between decisions and environmental change. It represents a robust set of policy and program evaluations. Are wetland permits effective in protecting habitats? Does the flood plain delineation model correctly delineate the hazard zone? Are EIS predictions accurate? These effectiveness questions are central to adaptive management as exemplified by the NWPPC’s Independent Advisory Boards (Northwest, 1998).

    All of the above systems are currently designed only to support small groups, either standalone or on closed networks. None use the Internet to engage open publics. Each uses a selection of the full OM subsystem components and incorporate models which are only partially open. All the U.S. examples are essentially agency or public/private grass roots experiments occurring without an explicit OM policy framework.

    The transformation to a fully Internet based OM environment is no longer an IT problem. A number of Internet-focused efforts are underway (Virtual, 1998; GIMDA,1998; Generalized, 1998). A broad based Rather, the issues involve both applied social science research and policy. The Publics’ Interface, particularly the Hypermedia Encyclopedia is ripe for development by a formal consortia of government, academic, and private sector participants. In the Modeling Core, the visualization of processes and error by nontechnical end-users is an applied research priority. Success in developing the Decision Support subsystem hinges on how we scale up from small closed groups to large open configurations of participants. This is the most challenging R and D arena.

    Progress in the Stewardship subsystem will be highly dependent upon new policy. Current information laws are founded on a centuries old construct of shared paper records. It assumes that descriptive information of past and current states is an adequate basis for decision making. Except for agencies that have an explicit education or technology transfer mandate, these policies say virtually nothing about sharing knowledge as represented in predictive systems models which are at the core of effective environmental stewardship.

    Clearly we need both more prototype projects, and a higher degree of shared learning between efforts. Hopefully the OM dimensions and subsystems highlighted above can help in this important undertaking.
 
 
 
 
 
 

Amy, D. (1987). The Politics of Environmental Mediation. N.Y.: Columbia U. Press.

Barber, B. (1984). Strong Democracy: Participatory Politics for a New Age. Berkeley: U. California Press.

Berge Collins Associates (1997). Resources for Moderators and Facilitators of Online Discussion. http://star.ucc.nau.edu/~mauri/moderators.html

Bingham, G. (1986). Resolving Environmental Disputes: A Decade of Experience. Wash. D.C.: The Conservation Foundation.

Burgard, J. and R. Guttler (1997). Environmental Problems- A Transnational Solution Supporting Cooperation of All Relevant Social Forces, in Denzer, R. et al. Eds. Environmental Software Systems v.2. N.Y.: Chapman and Hall.

CIESIN (1997). http://www.ciesin.colostate.edu

Clark U. (1998). IDRISI Home Page. http://www.idrisi.clarku.edu/

COGITO Learning Media (1998). Model-It Home Page. http://www.cogitomedia.com/model_it/mi_home.html

Committee For the National Institute for the Environment Home Page(1998). http://www.cnie.org/

Computational Hydraulics Int. (1998). PCSWMM Home Page. http://www.chi.on.ca/pcswmm.html

Cornell U. (1997). Legal Information Institute Home Page. http://www.law.cornell.edu/

EPA (1997). Terms of Environment http://www.epa.gov/OCEPAterms/

EPA (1998).BASINS. http://www.epa.gov/OST/BASINS/

EPA (1998). Environmental Data Register. http://www.epa.gov/edr/

EPA (1998). Technical Tools. http://www.epa.gov/epahome/datatool.htm

Everglades Landscape Model (Feb. 1998). http://kabir.cbl.cees.edu/Glades/ELM.html

Expert Choice Inc. (1997). Expert Choice Home Page. http://www.expertchoice.com/

Faber, B. and W.Wallace (1997). "Active Response GIS: An Architecture for Interactive Resource Modeling", Proceedings of the Annual Symposium on GIS ‘97. Vancouver: GIS World, Inc.

Facetps (1997). Forestry Applications. http://www.facet.com/html/forestr1.htm

Facetps (1998). Decision Warehouse: Canadian Department of Fisheries and Oceans. http://www.sun.com/products-n-solutions/solutions/nba/warehouse/documents/customer.profiles/CanadianFish.html

Federal Geographic Data Committee Home Page (1998). http://www.fgdc.gov/

Felleman, J. (1997). Deep Information: The Role of Information Policy in Environmental Sustainability. (Greenwich, Ct.: Ablex. Pub.

Generalized Researchable Sets Home Page, 1998. http://www.ccas.ru/mmes/mmeda/

GIMDA Geographic Information and Multicriteria Decision Analaysis Home Page(1998). http://www.geog.uwo.ca/gimda/index.htm

Government System Disclaimer (1998). http://www.fsl.orst.edu/home/usfs/emds/system/disclaim.htm

High Performance Systems (1998). STELLA Home Page. http://www.hps-inc.com/products/STELLA/STELLA.html

Lee,K. (1993) Compass and Gyroscope: Integrating Science and Politics for the Environment. Wash. D.C.: Island Press.

McAllister, D. (1980). Evaluation in Environmental Planning: Assessing Environmental, Social, Economic and Political Trade-Offs. Cambridge: MIT Press.

NASA (1998). Scientific Visualization Sites Homepage. http://science.nas.nasa.gov/Groups/VisTech/visWeblets.html

NCAR Land Surface Model Copyright Notice (1996). http://www.cgd.ucar.edu/cms/lsm/copyright.html

NCGIA (1998). Research Initiatives Home Page. http://ncgia.ucsb.edu/research/research.html

NCGIA (1996) Collaborative Spatial Decision-Making Home Page. http://ncgia.ucsb.edu/research/i17/I-17_home.html

National Academy of Sciences (1997). More than Screen Deep: Toward Every-Citizen Interfaces to the National Information Infrastructure. Wash. D.C.: National Academy Press. Available: http://www.nap.edu/readingroom/enter2.cgi?CO.html

National Water Resources Institute (1997). RAISON. http://www.cciw.ca/nwri-e/software/raison.html

New Jersey Risk Policy Forum Home Page (1997). http://www.newjerseyrisk.org/

Northwest Power Planning Council (1998). http://www.nwppc.org/welcome1.htm

Northwest Power Planning Council (1997). Energy Issues: Operate the River. http://www.nwppc.org/welcome1.htm

Office of Radiation and Indoor Air Radiation Protection Division, U.S.E.P.A. (1998). MurkeyBall Download. http://www.epa.gov/radiation/modeling/murkeyball/index.html

O’Hare. M. et al. (1983). Facility Siting and Public Opposition. N.Y.: Van Nostrand Reinhold.

OpenGIS Articles From GIS World Home Page (1998). http://www.opengis.org/gisworld/index.html

Palisade Corp. (1998). @Risk Home Page. http://www.palisade.com

Patty Maes on Software Agents: Humanizing the Global Computer (July/Aug. 1997). IEEE Internet Computing. 1089-7801/97/s10.00.

Russel, C. et al. (1986). Enforcing Pollution Control Laws. Wash. D.C.: Resources For the Future.

SimScience Home Page (1998). http://www.simscience.org/

Spatial Modeling Environment Home Page (1998). http://kabir.cbl.umces.edu/SME3/index.html

Syracuse U. Libraries (1998). Evaluating Web Materials Home Page. http://libwww.syr.edu/isd/evaluat.htm

RTK Net Homepage (1998). http://www.rtk.net/

Radian international (1998). CHARM Homepage. http://charm.radian.com/

U. Buffalo (1997). Water Quality Seminar Home Page. http://www.ublaw.buffalo.edu/fas/boyer/859/index.htm

U. California- Berkeley (1997). GIS Terminology. http://library.berkeley.edu:80/UCBGIS/gisterm.html

U. Maryland (1998). To Stem The Flow- The Nile: Technology, Politics, and Environment Home Page. http://www.inform.umd.edu/EdRes/Colleges/ResCtr/nile/library.d/simnile.html

U.S. Census Bureau (1998). Access Tools. http://www.cnie.org/.
 
 

Virtual Decision Making in Spatial Planning Home Page (1998). http://www.ccg.leeds.ac.uk/vdmisp/index.html

Water Science and Technology Board, National Research Council (1990). Ground Water Models. Wash. D.C.: National Academy Press.

Williams, B. and A. Matheny (1995). Democracy, Dialog, and Environmental Disputes: The Contested Languages of Social Regulations. New Haven: Yale. U. Press.