S.U.N.Y. College of Environmental Science and Forestry
Syracuse, NY 13210 firstname.lastname@example.org
A very similar version of this paper has been published as
Felleman, J. (Fall/Winter 1999). "Internet Facilitated Open Modeling: A Critical Policy Framework", Policy Studies Review,16:3/4, 193-219.
Adaptive Management is emerging as a
dominant approach for sustainable stewardship of the environment. To be
successful, it requires effective integration of ongoing systems modeling
and participatory democracy. The Internet holds great promise for providing
the information and knowledge interface necessary to support effective
adaptive management discourse. Open Modeling (OM) is seen as the keystone
of this enterprise. There is currently no federal or state policy covering
this arena. A preliminary OM classification scheme, and the major components
of a robust OM System are identified. These can provide the framework for
future policy analysis.
DIMENSIONS OF OPEN MODELING
A COMPREHENSIVE OPEN MODELING SYSTEM
The modern environmental era can be understood as the locus of two co-evolving movements. Environmental systems science analyzes bio-physical processes from the perspective of flows and transformations of matter and energy through space and time. This approach has yielded significant basic insights for issues ranging from animal population dynamics, to Superfund pollution plumes, to global warming. Participatory decision-making has drawn on our deep heritage of pluralism and "strong democracy" to generate a partial shift in power away from private firms and closed, centralized public bureaucracies enabling a broad spectrum of stakeholders to become actors (Barber, 1984:117-138). This movement has been facilitated by policies that require information dissemination, and provide the legal standing for adversarial judicial proceedings and/or negotiations. Together these movements provide the foundation for an emerging ecologically based normative world view generally known as sustainable development.
The National Environmental Policy Act of 1980 (NEPA 1970, 42 USC 4321 et seq.) was a primary catalyst in this co-evolution, constituting a broad brush declaration of the primacy and universality of environmental health in public decisions. The teeth of this potentially symbolic law are found in the requirement for Environmental Impact Assessments (EIA). Federal decisions must be based on predictions from interdisciplinary science. A range of viable alternatives had to be analyzed, and prior to a final decision drafts of the study, (body written for general audience with technical appendices), were to be widely disseminated for review and comment.
These two arenas of systems science and
participatory/strong democracy have a historically weak interface. This
is due in large part to the highly structured form of rational decision
making, and its associated discourse, which has dominated most modern public
decisions including facility siting, command and control pollution regulations,
and the master planning of public lands (Williams, 1995:11-35). As depicted
in Figure 1, Rational Decision Making, major stages in the process include:
selection of objectives, formulation of alternatives, prediction of expected
effects, evaluation of tradeoffs, and monitoring and feedback. This generic
process is legitimized in NEPA, and most other environmental laws.
The critical lynchpin in Figure 1 is the Prediction of Environmental Effects. Throughout the modern environmental movement, the use of computerized predictive science and economic models has been the sole domain of an elite of specialists.
Traditionally, environmental decisions have been implemented in a project by project disjointed manner ignoring cumulative, system wide effects. Additional critiques are well known, and include: narrowly constructed criteria often used with poorly grounded predictive models which do not incorporate either local uniqueness or larger biophysical systems; token "public participation" in the form of EIA’s and public hearings which come after the analyses are completed and preliminary internal decisions have been formulated; and the lack of monitoring,(optional in NEPA), and associated knowledge building.
Over the past decade, the field of "adaptive management" has emerged with the express intent of advancing sustainability by answering these critiques through a comprehensive restructuring of the science-public discourse interface. Kai Lee, in his seminal study of the Northwest Power Planning Council (NWPPC), Compass and Gyroscope, postulates that the "compass" of science can best provide meaningful decision direction by establishing a broad, robust, continuing systems modeling framework (Lee,1993:7-17). Model-based predictions can progress from general to more specific as development actions are monitored over extended time periods. The "gyroscope" of democratic processes provides both a check and balance against excesses, and a crucible of social learning where new societal values, which reflect our deepening knowledge regarding the linkages between sustainability and development, can be nurtured.
Whereas the traditional rationalism, with
its associated modeling, shown in Figure 1 often took place within a closed
public or private bureaucracy, adaptive management necessarily occurs in
an open arena. Figure 2, Actors and Stakeholders, illustrates the diverse
context in which current environmental issues are resolved. The enfolding
of large scale applied science, which frames management decisions as on-going
experiments, and public discourse has traditionally faced the insurmountable
barrier of achieving a prolonged, technically-based, dialog among spatially
dispersed parties with widely differing values and scientific literacies.
From NEPA in 1970 to Lee’s analysis of the Northwest Power Planning Council’s
(NWPPC) innovative management of the Columbia Basin in the 1990’s, meaningful,
effective multi-way communication has often remained a distant goal of
the modern environmental movement.
A central barrier to effective collaboration has been the inability to provide physical and cognitive access to the systems science and managerial models for all actors and stakeholders. This important new policy arena can be defined as "Open Modeling" (OM). Currently, the physical components of this communication barrier are being demolished by the explosive transformations underway in information technologies (IT). The Internet, coupled with increased desktop power, object programming, visualization software, and public dissemination policies have generated an entirely new context for adaptive management discourse. How this revolution will take place, and whether or not it will empower adaptive management is the subject of growing debate with opinions ranging from anarchy to a new plateau of environmental stewardship and sustainability.
A fundamental consensus on the primacy of the OM arena is evolving. The recent Report of Enterprise for the Environment (1998:19-24,44-50), calls for "place based", "collaborative", "information rich" decision processes. The National Center for Environment Decision-making Research (NCEDR) sees the Internet as the key to both dissemination and communication for diverse distributed actors and stakeholders,(Dobson,1997:4). The Center has prioritized information and communication technology, and improving stakeholder contributions as applied research needs (NCEDR, 1998).
A number of important experiments are underway in government, private firms, Non-Governmental Organizations (NGOs), and educational institutions utilizing IT for distributed process modeling, and promoting environmental discourse such as public relations, education, and negotiation. Most of these initiatives are grounded in a specific issue and reflect local needs and resources. Many appear to be strong in pushing data and information while lacking in analysis of end-user effectiveness.
All the ad-hoc initiatives are occurring in a policy vacuum. NEPA, the Freedom of Information Act, and essentially all other federal and state information, and public participation policies are focused on processing and accessing "records", and voicing opinions. They are mute on the subject of computer models. This distinction between information (record-based), and knowledge (model-based) policy is at the heart of developing effective environmental collaboration.
Studies of electronic democracy have been underway for a generation. An important first foray into understanding shared environmental (spatial) modeling was the National Center for Geographic Information Analysis’ (NCGIA) Initiative 17 Collaborative Spatial Decision-Making (NCGIA, 1996), which focused on Geographic Information Systems (GIS). More recently the National Academy of Sciences has published a broad overview of the technical and social science research that will be necessary for all citizens to have effective access to electronic information via the Internet (National Academy, 1997).
While still somewhat inchoate, it is now possible to stand back and ascertain the basic general dimensions and components of a comprehensive "open modeling" (OM) system. Hopefully, an articulation of this framework will stimulate shared learning, help prioritize R and D needs, and focus attention on the policy gaps which need bridging.
Similarly, policy pluralism as exemplified by a negotiation, is "open". In theory there needs to be sharing of descriptive information and causal knowledge related to expected impacts for a successful "win-win-win", (opponents and the environment), outcome. Although certainly not a panacea, the impetus to avoid the gridlock and associated time and resource investments that have characterized a generation of environmental litigation has led to a negotiation-oriented framework for most current issues (Amy, 1987, Bingham,1986:65-89).
Over the past two decades major investments in the public and private sectors have converted vast quantities of environmental data to digital form. Most new data, whether derived from satellite remote sensing, collected in the field with monitors or Global Positioning Systems, or generated in the lab now goes directly to digital format. These databases are the foundation for digital modeling which has become a dominant methodological approach both for scientists in developing new knowledge, and decision-makers in analyzing courses of action. The "openness" of the modeling has thus emerged as a critical determinant of the ultimate success of both enterprises.
In order to examine this issue it is useful
to have a preliminary classification scheme. Figure 3 depicts an Model
Openness Classification Space with two dimensions. "What If?" Interactivity
is a continuum of how open the model is to end-user queries, particularly
alternative scenarios. What if the highway is located here? What if releases
from the dam are increased during spawning season? The Logic Transparency
axis reflects how clearly the underlying model construction and mathematics
are presented. This is critical both for scientific credibility and policy
legitimization. The usefulness of this scheme has been pilot tested in
the spring of 1998 by a graduate seminar class who examined a self-selected
sample over 200 OM related web sites.
Categories on the "What-If?" axis begin with Closed. Many modeling sites are similar to EIA’s in that modeling has been done and results are presented but end-users cannot interact at all with the model. The Canned Scenarios set allows end-users to replicate model runs of pre-analyzed alternatives. Rather than interacting directly with a model, one accesses a database of stored model runs. Reduced Form modeling involves the utilization of a simplified version of the actual science/management model and associated databases. In the Expert Facilitator category queries are submitted to a remote analyst who subsequently runs the model. Finally, in a fully Open model the end user interacts directly with the systems model.
A number of factors appear to be active in generating this wide range. Many of the original models are quite large, include massive data sets, and were developed by specialists for use by specialists. Issues include security as well as efficiency. Will the model be accessed remotely, or downloaded to run "stand alone" on the desktop, or some combination involving "applets" (small downloadable Internet programs that run on the desktop)? For example, the Everglades Landscape Model runs on high end work stations and is being ported to a supercomputer (Everglades, 1998). Visitors to the web site can learn much about the modeling effort, see GIS maps, schematic flow diagrams, and graphs of preliminary results, but cannot access the model. In contrast, the NWPCC has developed "Operate the River" to illustrate the complex connectivity of the numerous dams in the Columbia River system. Via the Internet the public can directly examine canned dam operation scenarios, or download this reduced form black box spreadsheet to run in stand alone mode (Northwest, 1997).
The Logic Transparency axis begins with Black Box. Here the end user has no understanding as to what computations transform input data into output environmental predictions. Schematic Flow is a diagram, (typically box and arrow), of the internal processing including serial and parallel stages, and positive and negative feedbacks. Parameterization includes the establishment of values for the variables, while Forcing Functions are the equations which control the directions and rates of energy and matter flows and transformations through the system being modeled.
Sensitivity Analysis involves modifying a single variable to see its effect on the system. Traditionally models have been deterministic, the same single value inputs always generate the same single value outputs. This approach assumes a simple mechanistic process. Error Analysis is a statistical approach to modeling where distributions of inputs propagate ranges of performance predictions. In a fully transparent Open Model the underlying assumptions are explicit and code logic is available.
Black box process models are common in EIA’s, and in land planning which has a century long history of active citizen participation. In the innovative work of CIESIN’s AR/GIS highly visual colored landuse maps embedded in a Geographic Information System (GIS) are manipulated by end-users to create alternative development scenarios. The land use changes are subsequently inputted to linked black box predictive models, such as traffic and wastewater to yield tables and graphs of expected effects (Faber, 1997;CIESIN, 1997).
Essentially all computer models of environmental systems are based on flow diagrams. Traditionally, these diagrams were used to guide the initial computer coding in traditional languages such as (such as FORTRAN). Their logic subsequently became opaque as the code was subsequently modified and compiled to run. In recent years schematic flow visualizations are increasingly available due to the popularity of graphical object programming. Many environmental models, both educational and applied, are now being developed directly in a visual object language. STELLA and Model-It are examples (High Performance, 1998),COGITO, 1998). In developing environmental models, provision of end-user access to model parameters and equations is a design decision often made by the project manager or the delegated to the programmer outside a formal policy context.
Many agencies such as the EPA and Department of Agriculture have a long history of "technology transfer" where models and data are developed and distributed to technical end-users (EPA, 1998). More recent models have included stochastic error analysis. This capability may be written directly into the system model source code. Alternatives for doing error analysis include "add-ins" for spreadsheet based models (Palisade, 1998), and subroutines in GIS (Clark U.,1998).
Process models that once originated with government sponsored research have been extended by value-added private firms to include features such as Graphical User Interfaces (GUIs), visualization, and linkages to GIS and associated data sets. Some, such as PCSWMM, (Computational, 1998),have increased transparency with parameterization and sensitivity support. Other firms have created new products with decreased scientific transparency designed to extend the market to non-technical end-users such as community emergency spill staff through the use of visualization (Radian, 1998).
Stochastic modeling has only recently become available on the desktop. To run such analyses probabilistic data inputs are required. These typically do not exist. For example the millions of existing paper maps include little or no information regarding the precision of lines or the homogeneity of categories. The NCGIA explored in depth the topic of error propagation in spatial models, but the work is primarily theoretical (NCGIA Research, 1998). No major commercial vector GIS currently addresses this issue. In general, there is currently a dearth of models which are stochastic and generate the probabilistic estimates of expected outcomes which pluralistic decision makers would expect. While some of the newer EPA sponsored efforts are of this form, the widespread reliance on determinism reflects an increasingly obsolete era of closed decision making.
The openness dimensions of environmental modeling provide unique insights into the variety of existing efforts. New policy will have to engage difficult decisions regarding what level(s) of openness to legitimize. Although modeling is the core of OM, its ultimate effectiveness is best understood from the perspective of adaptive management support. In the next section a comprehensive system frame for policy analysis will be introduced.
While environmental process modeling is
at the core of such a system, a number of other subsystems are necessary
to provide a robust context. Figure 4 depicts the Basic set of Open Modeling
Subsystems. The context is provided by the Publics’ Interface, through
which all end-users will engage the system. Most current environmental
policies, such as NEPA, incorporate the naïve construct from the hard
copy text era that there are two publics: technical, and general public
(8th grade education). This oversimplification has long been
at the heart of many controversial EIA debates (O’Hare, 1983:99-117). In
a digital world it is possible to recognize a rich spectrum of multiple
publics, with differing needs, levels of knowledge, and values. Another
subsystem focuses on Decision Support. These modules assist all stages
of decision making as previously depicted in Figure 1. Particular emphasis
is placed on supporting collaboration including negotiation. The final
subsystem, Stewardship, addresses the long term (bio-geophysical time scales)
systems learning upon which adaptive management is founded.
At the center are tightly coupled process models of environmental systems and GIS. This reflects the inherently spatial nature of environmental phenomena. The degree of coupling is rapidly advancing. In a loose coupling example,a pollution plume from an atmospheric model can be exported to a GIS and overlayed on a population census map to estimate exposure (note: this standard practice assumes everyone happens to be in their place of residence when the plume occurs). A tight coupled model example is the integration of landuse change, hydrological modeling of runoff, hydraulic modeling of flood routing, and estimation of flood damages (ASCE, 1997). The Everglades project mentioned above models the flows and transformations of energy and matter between small GIS grid cells (Spatial, 1998). Fueling this change are the industry movements to both open the elements of historically proprietary GIS (OpenGIS, 1998), the new ability to incorporate spatial attributes in major enterprises relational database systems, and the explosive growth of Internet-based GIS.
Other components of the Modeling Core subsystem include Statistics and Error Analysis, and Scientific Visualization. Statistical analysis are used within many models to estimate parameters. In remote sensing, one of our major sources of environmental information, virtually all data has passed through a complex series of cleaning, smoothing, and interpretive algorithms to transform massive sets of spectral reflections and emissions to useful information such as a land cover map. Error Analysis has been discussed above under Logic Transparency.
The field of Scientific Visualization has played a critical role in our ability to understand complex systems. Processes such underground pollution plumes, global warming, and spatial population dynamics require maps, graphs, 3-D depictions, and more recently, animations, and virtual "walk/fly through" for us to comprehend the numerical data sets which typify the raw output of environmental models. Watching the weather forecast on the evening news is a good barometer of the advancing sophistication of this field. NASA provides a useful listing of visualization sites (NASA, 1998). A fundamental challenge is to develop visualizations that are meaningful to nonscientists. Considerable progress in this arena is being made in education, mass media, and entertainment.
The Segmented GUI Gateway provides customized end-user entrance to all OM subsystems, reflecting the wide variations in environmental understanding, differences in scientific literacies, and personalized long term longitudinal interactions with the OM system. The latter is of particular import to complement our need to adaptively steward the environment in perpetuity. The basic concept of a Segmented GUI is not new. It has been standard management practice for a generation to design a hierarchy of enterprise database read/write access to reflect different levels of organizational use and authority. In the field of computer games, opening screens have typically asked for selection of the "level of difficulty", and/or if the player wishes to begin where they last ended.
The European Community is actively developing environmental information systems to be used by publics with differing languages and differing local regulations under the umbrella of Euro community standards (Burgard, 1997:175-181). Educators have expanded this approach in the development of web based instructional modules suitable for multiple age groups (SimScience, 1998). The advent of personalized "agents" has greatly expanded the potentials for segmentation (Patty Maes, 1997).
The Legitimacy component includes three elements: Credibility, Disclaimers, and Privacy. The Internet is increasingly filled with mis- and dis-information. The former is incorrect due to error and obsolescence; the latter is intentionally wrong. How are end-users to know what they are presented is credible? In the traditional print fields, government documents and scholarly, peer reviewed journals, were readily differentiated from the popular press, and tabloid or underground sources. For web-related research a whole new generation of legitimization is needed. Libraries and educators are leading a credibility initiative (Syracuse, 1997).
In the GIS field, the Federal Geographic Data Committee has set meta-data standards for spatial data generated by federal agencies,(guidelines for participating states and businesses), which include data quality and lineage (Federal Geographic, 1998). Unfortunately for much legacy data the metadata is simply "NA" (not available). The use of predictive models which incorporate poor or incomplete data and knowledge remains a controversial topic in scientific and non-scientific arenas (Water, 1990).
Models are by definition simplifications of the real world. Some models are designed for illustrative communication, some are for heuristic learning, while others are intended for decision support. Legally it is important that end-users understand the purpose of the OM as well as its limitations. Disclaimers are becoming standard practice in distributed models. These range from copyright interests (NCAR, 1996), to analytical capabilities (Office of Radiation, 1998), to scope of analysis (Northwest, 1997).
If OM is to be truly interactive, end-users will not only be downloading information, they will be generating model outputs, querying technical assumptions, and participating in issue forums and decision making processes. This openness raises serious questions of privacy. What does the system initially know about the end-users?, what does it learn over time?, how does the presence of "agents", (personalized individual Internet assistants) affect privacy?, who can access this information? To date most decision support systems have been based on closed local networks. Federal web sites are just now addressing some of these issues (Government, 1998). One pioneering non-governmental effort has been RTK Net, which in addition to providing one stop access for toxic release and emergency planning information, provides free services to registered users which include topical discussion groups (RTK Net, 1998).
It has long been known in commercial software that is designed for multiple end-users that context sensitive help is essential for productivity. Traditionally this help has been mechanistic, how to use the system. For environmental OM to be successful we must develop a robust Hypermedia Encyclopedia which assists in the understanding of terms, concepts, and processes. To date what one finds on the web are a spectrum of project based support "stuff". One the one hand is a plethora of traditional text based glossaries that are scanned and posted. An example is EPA’s "Terms of Environment" (EPA, 1997). Newer initiatives are educational efforts such as GIS (U. Cal., 1997), and the Nile simulation project which integrates systems modeling with on-line coursework covering the underlying natural and science and social sciences (U. Md., 1998).
In the emerging arena of Internet-based OM we can identify four primary components of the Decision Support subsystem include: Workgroup Tools, Decision Tools, Policy Contexts, and Risk Analysis. Workgroup Tools are becoming standard in business and government as organizations move away from traditionally systematized bureaucratic work flows and toward collaborative team approaches to task management. Tools range from E-Mail, and shared data sets and calendars, to threaded discussions and compound multi-author document development. A fundamental challenge exists in effectively managing the very large number of discussants that the Internet may generate (Berge Collins, 1997).
Decision Tools include methods of generating and evaluating alternatives, and negotiating decisions. Traditionally, many major environmental decisions were based on single objective sets such as maximization of benefit-cost. These approaches are highly controversial in both theory and practice and are increasingly being supplemented by multi-objective approaches that facilitate negotiation.
A number of computer assisted approaches have been developed commercially. Typically they are designed to handle small to medium size groups (Expert Choice, 1998). The U. Washington/U. Idaho Collaborative Spatial Decision Making project closely couples a GIS with custom meeting room hard and software that allows alternative means of for weighting objectives and assessing outcomes in real time (U.Washington, 1997). CIESIN’s pioneering AR/GIS discussed above, integrates GIS, process models with multi-objective analysis and negotiation support software (CIESIN, 1997). It is important to note that closed, small group processes developed in the private sector are probably incompatible with public open access and participation policies.
Two additional components of Decision Support are Policy Context and Risk Analysis. The former includes the laws, regulations and administrative procedures which apply to environmental decisions. For OM to be effective, participants must have a working understanding of what constitutes substantive and procedural due process. An interesting pilot was done last fall in a graduate Great Lakes course specifically designed to bridge science and law (U. Buffalo, 1997). There is a wealth of legal material available on the Internet (Cornell U., 1998). As in the fields of environmental science, much of the existing material in this subject is highly technical. The Hypermedia Encyclopedia will need to include a broad set of policy concepts.
Risk Analysis is often the missing link in current environmental decisions. It is not required in NEPA, and most environmental permitting procedures. What publics often want most to know is how a proposed action is expected to affect specific human and/or ecosystem receiver health, not that a deterministic model prediction number falls below a federal standard (McAllister, 1980:22-25). Regulatory agencies have traditionally set uniform workable standards which meet the criteria of equality and efficiency while begging the questions of "genius loci" (local uniqueness), and scientific uncertainty. Most Risk Analysis materials and tools on the Internet are solely designed by and for experts. New Jersey, a pioneer in public risk involvement has a more open approach (New Jersey, 1997). OM should incorporate emerging capabilities and explicitly deal with these issues in a manner that promotes social learning.
The function of the Stewardship subsystem of OM is to support the long term learning objective of adaptive management. As shown in Figure 4D, it’s basic components include Monitoring, Data Mining, and Accountability. The data sets which are depicted in the Modeling Core are quite massive and exist at different locations under different management. In recent years, considerable progress has been made in the public sector, and in public/private partnerships to share existing environmental data. Examples include: FGDC (Federal Geographic, 1998),Census Access Tools (U.S. Census, 1998), CIESEN (1997), and EPA’s Environmental Data Register (1998).
The major issues related to Stewardship are longitudinal changes in the biophysical environment; changes in socio-economic processes ranging from land ownership, to regulatory conformance; and the accuracy of EIA and decision model predictions. Each of these has significant current deficiencies. The failure of EPA’s Environmental Monitoring and Assessment Program, (EMAP), the long standing inability to successfully link wetlands data with real property data (Felleman, 1997:122-126,159-172), and the inherent difficulties in pollution monitoring (Russell, 1986:64-89), highlight the analytical and political difficulties in adequate monitoring.
Assuming continued advances in Monitoring and Internet accessible data sets, two additional needs can be identified. Data Mining is an emerging set of analytical techniques which periodically are applied to warehoused data sets to reveal patterns of relationships which were not anticipated in the original data collection design (Facet, 1998). Since environmental systems are inherently complex, and models by definition are highly simplified, we need the ability learn both inductively and deductively. Accountability closes the loop between decisions and environmental change. It represents a robust set of policy and program evaluations. Are wetland permits effective in protecting habitats? Does the flood plain delineation model correctly delineate the hazard zone? Are EIS predictions accurate? These effectiveness questions are central to adaptive management as exemplified by the NWPPC’s Independent Advisory Boards (Northwest, 1998).
All of the above systems are currently designed only to support small groups, either standalone or on closed networks. None use the Internet to engage open publics. Each uses a selection of the full OM subsystem components and incorporate models which are only partially open. All the U.S. examples are essentially agency or public/private grass roots experiments occurring without an explicit OM policy framework.
The transformation to a fully Internet based OM environment is no longer an IT problem. A number of Internet-focused efforts are underway (Virtual, 1998; GIMDA,1998; Generalized, 1998). Rather, the central issues involve both applied social science research and policy. The Publics’ Interface, particularly the Hypermedia Encyclopedia is ripe for development by a formal consortia of government, academic, and private sector participants. In the Modeling Core, the visualization of processes and error by nontechnical end-users is an applied research priority. Success in developing the Decision Support subsystem hinges on how we scale up from small closed groups to large open configurations of participants. This is the most challenging R and D arena. Progress in the Stewardship subsystem will be highly dependent upon new information generation and sharing policy as envisioned by the Enterprise for the Environment (1998), Dobson, 1997, and the Committee for the National Institute for the Environment (1998).
There is an old saying, that in a meal
of bacon and eggs the chicken is "concerned" but the pig is "committed".
The fundamental challenge to bridging the gap between collaboration and
adaptive management remains for federal and state governments to move from
a state of being concerned about effective participation to one of being
committed. New policy which will provide physical and cognitive access
to environmental science and management models for diverse, distributed
publics is the litmus test for commitment to sustainable development in
the twenty first century.
Douglas Amy , The Politics of Environmental Mediation, Columbia U. Press,1987.
Benjamin Barber, Strong Democracy: Participatory Politics for a New Age, U. California Press,Berkeley,1984.
Berge Collins Associates, Resources for Moderators and Facilitators of Online Discussion,1997. http://star.ucc.nau.edu/~mauri/moderators.html
Gail Bingham, Resolving Environmental Disputes: A Decade of Experience, The Conservation Foundation, Wash. D.C.,1986.
Josef Burgard and Rainner Guttler, Environmental Problems- A Transnational Solution Supporting Cooperation of All Relevant Social Forces, in Ralf Denzer, et al. Eds. Environmental Software Systems v.2., Chapman and Hall,N.Y.,1997.
CIESIN, Home Page, 1997. http://www.ciesin.colostate.edu
Clark U.,IDRISI Home Page, 1998.http://www.idrisi.clarku.edu/
COGITO Learning Media, Model-It Home Page,1998. http://www.cogitomedia.com/model_it/mi_home.html
Committee For the National Institute for the Environment Home Page, 1998. http://www.cnie.org/
Computational Hydraulics Int., PCSWMM Home Page,1998. http://www.chi.on.ca/pcswmm.html
Cornell U.,Legal Information Institute Home Page,1997. http://www.law.cornell.edu/
Jerome Dobson, Proposed Conceptual Design for a National Environmental Decision-Making Information Infrastructure, National Center for Environmental Decision-making Research, Knoxville, 1997.
EPA, Terms of Environment,1997. http://www.epa.gov/OCEPAterms/
EPA, BASINS, 1998. http://www.epa.gov/OST/BASINS/
EPA, Environmental Data Register,1998. http://www.epa.gov/edr/
EPA, Technical Tools, 1998. http://www.epa.gov/epahome/datatool.htm
Enterprise for the Environment, The Environmental Protection System in Transition- Final Report,1998. www.csis.org/e4e
Everglades Landscape Model ,1998. http://kabir.cbl.cees.edu/Glades/ELM.html
Expert Choice Inc., Expert Choice Home Page,1997. http://www.expertchoice.com/
Brenda Faber and William Wallace, "Active Response GIS: An Architecture for Interactive Resource Modeling", Proceedings of the Annual Symposium on GIS ‘97 Vancouver, GIS World, Inc.,1997 (preprint).
Facetps, Forestry Applications,1997. http://www.facet.com/html/forestr1.htm
Facetps, Decision Warehouse: Canadian Department of Fisheries and Oceans,1998. http://www.sun.com/products-n-solutions/solutions/nba/warehouse/documents/customer.profiles/CanadianFish.html
Federal Geographic Data Committee Home Page,1998. http://www.fgdc.gov/
John Felleman, Deep Information: The Role of Information Policy in Environmental Sustainability, Ablex Publishing, Greenwich, CT, 1997.
Generalized Researchable Sets Home Page, 1998. http://www.ccas.ru/mmes/mmeda/
IMDA Geographic Information and Multicriteria Decision Analaysis Home Page,1998. http://www.geog.uwo.ca/gimda/index.htm
Government System Disclaimer,1998. http://www.fsl.orst.edu/home/usfs/emds/system/disclaim.htm
High Performance Systems,STELLA Home Page,1998. http://www.hps-inc.com/products/STELLA/STELLA.html
Kai Lee, Compass and Gyroscope: Integrating Science and Politics for the Environment, Island Press, Wash. D.C.,1993
Donald McAllister, Evaluation in Environmental Planning: Assessing Environmental, Social, Economic and Political Trade-Offs, MIT Press,Cambridge,1980.
NASA, Scientific Visualization Sites Homepage,1998. http://science.nas.nasa.gov/Groups/VisTech/visWeblets.html
NCAR Land Surface Model Copyright Notice,1996. http://www.cgd.ucar.edu/cms/lsm/copyright.html
NCGIA, Research Initiatives Home Page,1998. http://ncgia.ucsb.edu/research/research.html
NCGIA,Collaborative Spatial Decision-Making Home Page,1996. http://ncgia.ucsb.edu/research/i17/I-17_home.html
National Academy of Sciences, More than Screen Deep: Toward Every-Citizen Interfaces to the National Information Infrastructure, National Academy Press,Wash.D.C.,1997. Available: http://www.nap.edu/readingroom/enter2.cgi?CO.html
National Center for Environmental Decision-making Research, "New Directions Based on Decision-making Needs", Newsletter/Update 7, 1998. www.ncedr.org
National Water Resources Institute, RAISON,1997. http://www.cciw.ca/nwri-e/software/raison.html
New Jersey Risk Policy Forum Home Page,1997. http://www.newjerseyrisk.org/
Northwest Power Planning Council,1998. http://www.nwppc.org/welcome1.htm
Northwest Power Planning Council, Energy Issues: Operate the River,1997. http://www.nwppc.org/welcome1.htm
Office of Radiation and Indoor Air Radiation Protection Division, U.S.E.P.A., MurkeyBall Download, 1998. http://www.epa.gov/radiation/modeling/murkeyball/index.html
Michael O’Hare, et al., Facility Siting and Public Opposition, Van Nostrand Reinhold, New York,1983
OpenGIS Articles From GIS World Home Page,1998. http://www.opengis.org/gisworld/index.html
Palisade Corp., @Risk Home Page,1998.http://www.palisade.com
"Patty Maes on Software Agents: Humanizing the Global Computer", IEEE Internet Computing. 1089-7801/97/s10.00.
Clifford Russel, et al. Enforcing Pollution Control Laws,Resources For the Future, Wash. D.C.,1986.
SimScience Home Page,1998. http://www.simscience.org/
Spatial Modeling Environment Home Page, 1998. http://kabir.cbl.umces.edu/SME3/index.html
Syracuse U. Libraries, Evaluating Web Materials Home Page,1998. http://libwww.syr.edu/isd/evaluat.htm
RTK Net Homepage,1998. http://www.rtk.net/
Radian international, CHARM Homepage,1998. http://charm.radian.com/
U. Buffalo, Water Quality Seminar Home Page,1997. http://www.ublaw.buffalo.edu/fas/boyer/859/index.htm
U. California-Berkeley, GIS Terminology,1997. http://library.berkeley.edu:80/UCBGIS/gisterm.html
U. Maryland,To Stem The Flow- The Nile: Technology, Politics, and Environment Home Page,1998. http://www.inform.umd.edu/EdRes/Colleges/ResCtr/nile/library.d/simnile.html
U.S. Census Bureau, Access Tools,1998. http://www.cnie.org/.
U.Washington/U. Idaho Collaborative Spatial Decision Making, 1997. http://weber.u.washington.edu/~tjmoore/csdm.html
Virtual Decision Making in Spatial Planning Home Page,1998. http://www.ccg.leeds.ac.uk/vdmisp/index.html
Water Science and Technology Board, National Research Council, Ground Water Models, National Academy Press, Wash. D.C.,1990.
Bruce Williams and Albert Matheny (1995), Democracy, Dialog, and Environmental Disputes: The Contested Languages of Social Regulations,Yale U. Press, New Haven, 1995.