Evaluation Research

Goal: To apply social research methods to assess the efficacy of social intervention programs and policies.

Special type of explanatory research

Distinctions between basic social and applied evaluation research

Who selects problem to study?

Primary motivation for project?

Primary goal of project?

Rigor of methods?

Typical research arena?

How is info disseminated?

Why is measurement more difficult in evaluation research?

Vague conceptual and operational definitions – broad program goals.

Program may be non-uniform.

Different interest groups with conflicting interests.

Measurement issues to consider

Find/create valid indicators of program outcomes.

Consider timing of outcome measurement.

How to operationalize success or failure.

Example: Tanzanian government program to reduce pop growth and AIDS.

Threats to external validity

Sampling of units

Non-probability samples

Staff qualities

Hawthorne effects

Contextual effects

Stakeholders?

Examples

Types of evaluation research designs

Experimental design

Quasi-experimental design

Time-series

Non-equivalent control groups

Suppose a university decides to initiate a new policy of assigning first-year students to the same residence halls, thereby segregating residence halls into first-year versus upper-class. They have asked you to advise them on how to evaluate the impact of this policy?

Who are the relevant stakeholders in this study?

Explain how you will identify the relevant and potential effects of this new living arrangement.

Carefully describe the general research strategy you would recommend.