ENS 702  Environmental and Natural Resource Program Evaluation  
  Mon/Wed. 5:00- 6:20 PM   105  Marshall  Hall                               (updated 4/11/05)
       Prof. John Felleman  108B Marshall Hall 
     felleman@esf.edu   470-6550     http://www.esf.edu/es/felleman/
      Office Hours M/T/Th 11-12 and by appointment

 Overview
     Since the 1970's, governments at all levels have passed numerous laws and regulations intended to protect and enhance environmental quality. Many (most?) of the resultant implementation programs involve targeted interventions into often poorly understood, dynamic  biophysical/socio-economic systems. Both advocates for and opponents of these programs have raised fundamental questions regarding the resultant effectiveness and efficiency of these efforts.

               

    Environmental Program Evaluation (EPE) is the field of policy analysis that wrestles with the "who, what, when, where, how, and why"  we go about analyzing program performance issues. The intended uses of EPE in the policy cycle range from early policy and program development, to midcourse constructive program modification,  to sunsetting and termination. EPE is an evolving analytical arena that is central to the constructs of Adaptive Management, and Sustainable Development.  
   Methodologically, EPE is a multi dimensional hybrid, typically involving the integration of quantitative and qualitative research methods in both the physical and social domains. Although not formal course pre-requisites, students are expected to have and/or develop conceptual level understanding of these topics. The course design explores EPE through the review and discussion of related literature including theory, methods, and case studies. Students will play an active role in identifying and summarizing source materials. Three course "products" will be: a class-generated, threaded discussion, referenced glossary of EPE terms; individual critical reviews of published program evaluations; and  preliminary designs for evaluation of a program component. This semester the target program is New York's Freshwater Wetland regulatory program. 
   

Objectives
    At the conclusion of this course students will be able to:
    1. Articulate the evolving roles of EPE in the policy cycle;
    2. Critically evaluate  completed EPE  analyses;
    3.  Develop a preliminary EPE design for a program component; and
    4.  Relate course concepts to student's individual policy research content  arena.
 

Resources 
   There are three primary reading sources for the course:

    1. Knapp, G. and T. Kim, eds. (1998 or '97), Environmental Program Evaluation: A Primer. U. Ill. Press.
     Students are strongly encouraged to purchase a copy on line from a book source such as:
        http://dogbert.abebooks.com/    ,  http://www.powells.com/   , Barnes and Noble etc.

    2. General Accountability Office, Program Evaluation and Methodology Division:
         Methodology Transfer Papers; and Program Reports:  http://www.gao.gov/     and

         EPA's  Evaluation Support  Div. Gateway:   http://www.epa.gov/evaluate/ 
   
    3. Class reader, complemented by class handouts, and resources identified by the class members.

    There are numerous potentially useful EPE-related web sites. In addition to the GAO site, a few places to start exploring are:
    Basic Guide to Program Eval.:   http://www.mapnp.org/library/evaluatn/fnl_eval.htm
    American Evaluation Assoc.:   http://www.eval.org/
    Congressional Budget Office: http://www.cbo.gov/byclasscat.cfm?cat=18   
    Congressional Research Service reports:          http://www.cnie.org/NLE/CRS/
    National Academies Press:   http://books.nap.edu/v3/makepage.phtml?val1=subject&val2=ev
    State Prog. Eval. Web resources: http://www.csulb.edu/~ddowell/weblinks.htm
    Resources for the Future's library:   http://www.rff.org/rff/Core/Research-Overview.cfm
    National Academy of Public Administration:  http://www.napawash.org/

Grading:
       Class Participation:
             Glossary                        15%
             Discussion                    15
       EPE Critique Paper            35
       Wetlands EPE Proposal   35
SCHEDULE  

 #
 Date
 Topic Readings/Assignments


 I. FRAMEWORK
1
 W 1/19
  Introductions: Class Overview;
   EPE in the Policy Process, Actors and Stakeholders
 Evaluation summaries- due next class
Web

M  24
  What is a "Program'?
 Text: Foreward, Intro.
Web
3
 W  26
  Some Bureaucracy "Background"
 Text: Ch. 2,3
Reader: Sabatier in Mazmanian
Web
4
 M  31
   EPE History  1:
 Text: Ch.1
Reader: Weiss Ch. 1,2
5
 W  2/2
   EPE History  2:
 Text: Ch. 13
Reader: O'Leary- Foreward
Web
6
 M    7
  Course Bias  1:   Environ. Outcomes
 Text: Conclusion
Reader: Russell; Mohr
7
 W   9
  Course Bias  2:  Adaptive Management
  Students select EPE to analyze
Handouts
Web


  II.  A PROGRAM PROCESS  VIEW
8
 M   14
  Useful, Answerable Questions
  Text: Ch. 12
Reader: Rossi, pp.51-65
Web: GAO- "Designing Evaluations" Ch. 1,2,3
9
 W  16
  Inputs 1: Policy Framework
  Reader: Rosenbauam in Mazmanian
10
 M  21
  Inputs 2: Resources
  Reader: Sabatier
11
 W  23
  Throughputs  1:  Causal Models ("theory")
 Reader: Weiss Ch.3; Rossi pp.152-168
12
 M  28
  Throughputs 2: Information/Decision Flows
  Reader: Felleman
13
 W  3/2
  Outcomes 1:  Indicators
 Text: Ch.7
Handouts:
Web
14
 M    7
  Outcomes 2: Profiles/Indices
 Text: Ch. 5,6
Reader: Leach in O'Leary
Handouts: Hodge
Web


  III. NYS FRESHWATER WETLANDS
15
 W   9
  Intro to the NY Freshwater Wetlands Program        Official maps
  Handouts, Class resource "box"
Web


Spring Break
16
 M  21
  Wetland Permit Process
17
 W  23
  DEC Office visit
18
 M  28
  Student ident. of Wetland Evaluation topics
19
 W  30
  no class


 IV. STUDENT EPE REVIEWS
20
 M  4/4

21
 W  6

22
 M  11

23
 W  13
  Monitoring and Adap. Management (again) Web Resources


  V. GUEST LECTURES (TBA)
24 M  18
   Wetland EPE Work in Process-Roundtable
25
 W  20
   "Shifting Paradigms" Conference
  
26
 M  25
 Jack Manno
Qualitative Research:"Water over the Dam"
  EPE Reviews Due
27
 W  27
  Deb Storrings
 Meta-Analysis: "Distance Learning"
  
28
 M  5/2
  Wrapup

Finals
Week
  Student Presentations of Wetland EPE Component Designs

________________________________
WEB Sites:

1. For students who may be unfamiliar with the US government agencies a couple of places to start are:
http://www.firstgov.gov/This is the federal government's primary web portal clearing house. A good point from which to begin to understand the organization and functions of our very complex set of agencies.
 http://www.gpoaccess.gov/gmanual/browse-gm-04.html   The annual U.S. Government Manual includes descriptions of all units of the federal government including key staff titles.

2.     Is a policy a program?:    NEPA - A Study of Effectiveness After 25 Years
               What classes of programs does OMB recognize in PART: http://www.whitehouse.gov/omb/part/index.html
3. Take a quick look at the Executive Summary of the Army's "Ecosystem Management Policy Evaluation" study:
     http://www.aepi.army.mil/internet/dod-ecosystem-mgmt-policy-eval.pdf
5. EPE Hist 2
       Take a look at the Corps of Engineers' PART Document: http://www.gpoaccess.gov/usbudget/fy05/pdf/part/corps.pdf

documents mentioned in class today:
Beirle, T. (2000). The Quality of Stakeholder-Based Decisions: Lessons from the Case Study Record. Wash. D.C. Resources for the Future, Discussion paper 00-56

Kettl, D. (1998). Reinventing Government: A Fifth Year Report Card. Brookings Institution: CPM 98-1.

GAO (1998). Agencies' Annual Performance Plans Under the Results Act. GAO/GGD/AIMD-10.1.18.

GAO (2004). Results Oriented Government: GRPA Has established a Solid Foundation for Achieving Graeter Results. GAO-04-38.

OMB PART. Instructions for Program Assessment Ratings Tool.
    www.whitehouse.gov/omb/budgetintegration/part_assesing2004.html

13
    Go to the National Academy Press, and look at the Exec. Summary, and Ch. 1,2 of " Review of EPA's Environmental Monitoring and Assessment Program": http://books.nap.edu/catalog/4931.html
14
    Go to the National Academy Press, and scan the Exec. Summary of: "Ranking Hazardous-Waste Sites for Remedial Action: http://books.nap.edu/catalog/4781.html
   
Compare this to Wisonsin's pioneering work in developing an Index of Biological Integrety (IBI) to Warm Water Streams, by taking  a quick look at:  http://ncrs.fs.fed.us/pubs/gtr/gtr_nc149.pdf

For a look at how indexing can influence management decisions take another  look at Superfund "success":
       http://www.rff.org/rff/Documents/RFF-RPT-SuperfundSuccess.pdf

15-18
    Go to the NY DEC's Freshwater Wetland site: http://www.dec.state.ny.us/website/dfwmr/habitat/fwwprog.htm
          Peruse the entire site. Print out and review:
                       NY Environmental Conservation Law Article 24- Freshwater Wetlands
                       6NYCRR Part 663: Freshwater Wetlands Permit Requirements Regulations; and
                       6NYCRR Part 664: Freshwater Wetlands Mapping and Classification Regulations
                      

Each year a number of bills are introduced in each house of the state legislature to change the Wetlands law. these provide an excellent snapshot of current issues.
Go to the legislative web site: http://public.leginfo.state.ny.us/menugetf.cgi
    use "word search" to eamine 2005 bills.
23.
Monitoring is a highly controversial topic in both Program Evaluation and  Adaptive Management.
In the US, one of the major ongoing monitoring prgrams is USDA's "National Resources Inventory".
Go to the NRI web page: http://www.nrcs.usda.gov/technical/NRI/
and under "About NRI: Statistical design" take a look at the basics of the Nusser and Goebel paper.

NRI is comprehensive, not program specific. To couple physical monitoring with program Budgeting evaluation, take a look at the "Conservation Effects Assessment project (CEAP): http://www.nrcs.usda.gov/technical/nri/ceap/index.html
      


25
On Wednesday, April 20, Tyrone Hayes will be speaking at the Graduate Student Association's "Shifting Paradigms" symposium.  The lecture will be at 4 p.m., followed by a mixer and dinner beginning at 5:30 p.m.

A short National Geographic "Emerging Explorers" profile about Tyrone Hayes can be found at:
http://www.nationalgeographic.com/emerging/profiles/hayes.html

A story about Tyrone Hayes' research on the pesticide atrazine and his ensuing battle with manufacturer Syngenta can be found at:
http://www.mindfully.org/Pesticide/2003/Syngenta-Tyrone-Hayes31oct03.htm