Applied research is intended to provide practical information to organizations to suggest actions or increase effectiveness. It uses theory instrumentally to identify concepts and variables that will produce practical results. Evaluation research assesses the impact of programs, policies, or laws, often focusing on whether the intervention succeeded in creating intended changes. Different types of evaluation research include outcome evaluation, cost-benefit analysis, process evaluation, formative analysis, and needs assessments. Careful research design is needed to make causal claims about a program's effects. Participatory action research involves community members in the research process. Politics and organizational interests can influence evaluation findings and their use.

1. Applied Social Research Chapter 14

2. Introduction All social research is designed to increase our understanding of human behavior and can be useful to individuals, groups, or the whole society, but some work is more immediately useful than other research.

3. Introduction Applied research Research intended to be useful in the immediate future and to suggest action or increase effectiveness in some area.

4. Basic and Applied Research Basic research Research designed to add to our fundamental understanding and knowledge of the social world regardless of practical or immediate implications. Practical application can usually be derived from basic research, although these projects are designed to provide greater understanding of our social world and to develop or test theories

5. Basic and Applied Research In applied research, theory is used instrumentally to identify concepts and variables that will produce practical results Applied research is designed to provide organizations such as schools, legislatures, communities, social service agencies, health care institutions, etc. with practical information

6. Social Problems and Social Solutions Awareness of social problems and solutions directed toward them have become increasingly common Problems which have received widespread attention include housing, education, poverty, income disparities, crime, and drug use

7. Social Problems and Social Solutions Applied research attempts to answer these questions Are the programs effective? Should they be continued? Modified or eliminated entirely?

8. Evaluation Research Evaluation research Research specifically designed to assess the impact of a specific program, policy, or legal change Often the focus of an evaluation is whether the program, policy, or law has succeeded in effecting intentional or planned change. Evaluation research is research with a specific purpose

9. Outcome Evaluation Outcome evaluation Research that is designed to “sum up” the effects of a program, policy, or law in accomplishing the goal or intent of the program, policy, or law. Most common type of evaluation research Outcome evaluations typically begin with a question or a hypothesis Example Does the program accomplish its goals?

10. Cost-Benefit Analysis A cost-benefit analysis Research that compares a program’s costs to its benefits A cost-benefit analysis is designed to weigh all the expenses of a program against the monetary estimates of the program’s benefits

11. Cost-Benefit Analysis Cost effectiveness analysis Compares the program costs in delivering desired benefits based on the assumption that the outcome is desirable It estimates the approach that will deliver a desired benefit most effectively (at the lowest cost) without considering the outcome in economic terms

12. Cost Benefit Analysis Typical questions asked How effective is the program? How expensive is it? Is it worth doing? How does this program compare with alternative programs?

13. Needs Assessments and Other Kinds of Evaluations The tasks an evaluation sets out to accomplish are determined by the stage of the program and the needs and interests of the stakeholders Stakeholders People or groups that participate in or are affected by a program or its evaluation, such as funding agencies, policy makers, sponsors, program staff, and program participants

14. Needs Assessments and Other Kinds of Evaluations In some instances, before a program or project is designed, a needs assessment is conducted to determine the needs for various forms of service.

15. Needs Assessments and Other Kinds of Evaluations Needs assessment An analysis of whether a problem exists, its severity, and an estimate of essential services.

16. Focal Research A Needs Assessment of Incarcerated Women in Ecuador by Jill Harrison and Maureen Norton-Hawk Conducted a needs assessment on the conditions in women’s prison

17. Focal Research Ethics While prisons in the United States have their own internal review boards to protect inmates against ethical violations, no such review boards exist in Ecuador. For this reason, the researchers sought and received IRB approval from Suffolk University, Norton-Hawk’s home institution. Of special concern to the IRB was that the gift the researchers intended to give each inmate could be construed as coercive in nature if the gift was too large to refuse and would make the inmate feel obligated to participate. The researchers satisfied the IRB concern by giving a small bag of necessities valued at under $3.00 as a token of appreciation for participation.

18. Needs Assessments and Other Kinds of Evaluations If a new program is funded and there is time in the early stages of design or implementation to make improvements, program staff and developers can benefit from a formative analysis Formative analysis Evaluation research focused on the design or early implementation stages of a program or policy

19. Needs Assessments and Other Kinds of Evaluations Formative analysis Example A high school principal seeks assistance with a new anti-bullying program. A formative analysis would carefully review the program’s goals and its current instructional materials and collect date on the ongoing program. After analysis of the data the evaluator should be able to offer suggestions such as staff development and modifying the materials.

20. Needs Assessments and Other Kinds of Evaluations Process evaluation Research that monitors a program or policy to determine if it is implemented as designed. Also called an implementation study The goal is to try to pinpoint how something works and what aspects of the program contribute to the effect.

21. Evaluation Research Designing Evaluation Research The evaluation researcher needs to select a research strategy A researcher will make specific choices for study design, measurement, data analysis, and the like depending on the specific evaluation, including intended audiences, resources available, ethical concerns, and the project’s time frame.

22. Evaluation Research Designing Evaluation Research When evaluating a program, policy, or law, the following must be decided What is the desired outcome? Is the outcome to be short-or long-term? Are attitudinal changes sufficient, or is it also essential to study behavioral changes? How should change be determined? Should all aspects of the program be studied or only certain parts? Should all the targets of an intervention be studied or only some of them?

23. Evaluation Research Designing Evaluation Research Evaluation researchers almost always have an explanatory purpose and typically test a causal hypothesis about the effect of an independent variable such as a program law, or policy on a desired outcome. Causal hypothesis A testable expectation about an independent variable’s effect on a dependent variable

24. Evaluation Research Designing Evaluation Research Experimental designs with control and experimental groups lend themselves help address the issues of internal validity Internal validity is the agreement between a study’s conclusions about causal connections and what is actually true

25. Evaluation Research Designing Evaluation Research Not all experiments lend themselves to the true experimental model Outside a laboratory the evaluator might not be able to control all aspects of design Practical concerns of time or money can limit design choices Evaluation research can take five or more years from design to the final report

26. Participatory Action Research Explicit goals Participatory action research (PAR) Research done by community members and researchers working as co-participants, most typically within a social justice framework to empower people and improve their lives.

27. Participatory Action Research Explicit goals The social purpose driving PAR is to empower low status people in the organization or community to make decisions and take actions that were previously foreclosed to them.

28. Participatory Action Research Participation and the researcher’s role Doing participatory research means working in partnership with those in the community being studied to obtain and use the knowledge that is generated to empower the community In PAR, the researcher must be involved intimately in the life of a community and its problems and be in dialogue with other researcher collaborators

29. Final Considerations A middle ground Groups might be willing to work in collaboration with researchers to obtain information about effectiveness of programs for their own internal use or to use in seeking recognition or funding for future projects

30. Stop and Think Applied research is usually field research. What do you think are some of the consequence of the real world setting for the research and its outcome?

31. Final Considerations Politics and applied research In evaluation research, the specific choice of research project is affected not only by societal values and the priorities of funding agencies, but also the perspectives of various constituencies and program stakeholders

32. Final Considerations Beyond our control Organizations are not “neutral territory” and in most cases, the researcher is an outsider, working in someone else’s sphere The results of any assessment will have the potential to affect the organization and the individuals under study A special challenge in evaluation research can be obtaining the cooperation of program staff or access to data.

33. Final Considerations Having an Impact Conventional evaluation faces the challenge of implementing change after the research is completed. Ideological and political interests can sometimes have more influence on decisions about the future of social interventions than evaluative feedback. Even is a program is shown to be ineffective it might be kept if it fits with prevailing values, satisfies votes, or pays off political debts

34. Quiz – Question 1 Doing participatory research means getting permission from authorities first. projects that are done with the community not to the community. projects that are heavy top-down from academia and less community based. A and b None of the above

35. Quiz – Question 2 Formative analysis usually occurs when the program is completed. whenever the evaluation team is called in to perform an evaluation. in the early stages of program development and design. after the program has been denied funding. None of the above

36. Quiz – Question 3 Typically, evaluation research is intended to provide information about the history of a program or policy. inform program participants about the goals of a program or policy. create knowledge that is of general interest to scientists studying social policy. assess the impact of a specific program or policy. evaluate the expenses the program incurred.