This document discusses the development of a measure called the KEY-measure to evaluate the effectiveness of adult social work in Finland. It describes a project that tested the use of single-case evaluation designs to measure goal attainment and the impact of social work methods and contextual factors. The project found that single-case evaluation fits well for adult social work but that social workers found it difficult to incorporate systematically. The document presents results of testing the KEY-measure which showed what types of goals were commonly set, levels of goal achievement, and influential contextual factors. It concludes that more work is still needed to develop an effective program theory for evaluating adult social work.

1. Developing adult social work
effectiveness evaluation

2. THL, FinSoc-team 2
Goal-oriented adult social work
as a target for evaluation

3. 3THL, FinSoc-team 3
The characteristics of goal-oriented adult social work
• The goal of social work
• The object of social work
• Social work methods
• Contextual and situational factors
• Mechanisms

4. THL, FinSoc-team 4
Adult social-work effectiveness
evaluation project

5. THL, FinSoc-team 5
The EEA-project (2011-2012)
Coordination:The National Institute for Health and Welfare
Seinäjoki: Developing an
effectiveness evaluation
measure for adult social
work
The Centre Of
Excellence On
Social Welfare In
The Ostrobothnian
Area (SONet
Botnia)
Helsinki: Focusing on
data from client
monitoring forms at West
Helsinki Social Centre
Tuusula:
Evaluating rehabilitative
social work

6. THL, FinSoc-team 6
Design: single-case evaluation

7. THL, FinSoc-team 7
• Single-case design allows social workers to evaluate how
well the social work‟s goals have been reached, what
methods and procedures have been used, and how
important different contextual and situational factors and
mechanisms are for the goal attainment
• The method was originally developed as a practical tool
for social workers rather than as a research method
• Single-case evaluation also seems to be suitable
because it follows the basic idea of empirical research,
but without a control group
• It is possible to gain information about single clients‟
goals but also to obtain quantitative data. It is also
possible to track the client‟s situation using repeated
measurements.

8. Example of how the goals could be evaluated
ordinally
Goal number 5: Subsistence or financial situation management
Question: Managing the subsistence or financial situation.
Choose one of the following options:
Options:
• The goal is to improve the situation
• The goal is that situation remains unchanged
• The item is not a goal
THL, FinSoc-team 8

9. THL, FinSoc-team 9
A framework for the measure:
realist evaluation

10. THL, FinSoc-team 10
• Generative mechanisms
• Generative causality

11. THL, FinSoc-team 11
Developing the KEY-measure for
adult social-work effectiveness
evaluation

12. • Questions about the goals, methods and
procedures of social work and about the
client‟s situation.
• Testing period: November 2011 – August
2012 (10 months)
• Single-case design: Evaluation phase I &
Evaluation phase II
• Evaluation I: 209 client-cases
• Evaluation II: 172 cases
THL, FinSoc-team 12

13. THL, FinSoc-team 13
Basic information
form
Reasons why
evaluation is not
done

14. THL, FinSoc-team 14
Results from adult social-work
effectiveness evaluation

15. THL, FinSoc-team 15
0 10 20 30 40 50 60 70 80 90
Concern about the close personal relationships
Leisure activities and opportunities for participation
The consequences of anti-social behaviour
Physical problems caused by illness
Child's needs
Substance abuse and consequences
Problems related to mental illnessess
Rental housing situation in the area
Substance use
Social skills
Client's relationships
Mental health
Development of service system
Proceeding with further education
Client's possibility to access housing
Inclusion and participation (eg. hobbies)
Self-esteem
Awareness of the problems due to the society
Getting in to further education
Fluency of everyday life
Life management
Access to housing; keeping it
Clearing up the debt situation
Promoting the employment situation (for long-term…
Supporting client's occupational capabilities
Supporting client's problem-solving capabilities
Service management for a client
Employment, searching for a job
Client's control over his/her economic situation
Women (%)
(N=110)
Goals set by clients; stratified by gender (%)
Kivipelto, Blomgren & Suojanen 2013, 43

16. THL, FinSoc-team 16
Goal reached admirably; percentage of levels of achieving goals
(%)
Kivipelto, Blomgren & Suojanen 2013, 44
0
5
18
21
21
23
24
24
25
26
28
29
30
33
33
35
36
36
38
39
39
41
41
42
42
45
49
49
53
0 10 20 30 40 50 60
Client's possibility to access housing
Rental housing situation in the area
Leisure activities and opportunities for participation
Physical problems caused by illness
Promoting the employment situation (for long-term unemployment-people)
The consequences of anti-social behaviour
Concern about the close personal relationships
Mental health
Clearing up the debt situation
Client's relationships
Problems related to mental illness
Awareness of problems due to the society
Employment situation, searching for a job
Inclusion and participation (eg. hobbies)
Development of service system
Client's control over his/her economical situation
Life management
Proceeding with further education
Social skills
Getting into further education
Substance use
Substance abuse and consequences
Supporting client's occupational capabilities
Self-esteem
Fluency of everyday life
Access to housing; keeping it
Supporting client's problem-solving capabilities
Child's needs
Service management for a client
Situation improved

17. THL, FinSoc-team 17
0 20 40 60 80 100
Controlling the client
Supporting client's participation
Supporting client's awareness of social problems
Case management
Dealing with the client's problems
Service plan for the client
Solution focused work
Supportive discussion
Supervision and guidance
Needs assessment
Decision making
Social assistance
Women (%) Men (%)
Most used social work methods. Percentage of all methods; stratified
by gender (%)
Kivipelto, Blomgren & Suojanen
2013, 44

18. THL, FinSoc-team 18
0 10 20 30 40 50 60 70
Values and attitudes in the living area
Access to the environment
Client's cultural background
The demographic structure of living area
Substance abuse of family members
Concern about the close people
Substance abuse / addictions
Client's mental health
Mental health and well-being
Client's employment situation
Satisfaction of relationships/adequacy
Client's lifestyle
Client's economic situation
Client's physical health
Possibility to access health related technology
Awareness of the bacground determinants behind…
Possibility to influence
Leisure activities and opportunities for participation
Client's ability/motive to plan his/her economic situation
Client's occupation
Seeing solutions and alternatives
Employment and occupational situations
Internet connectivity
Client's self-esteem
Living conditions
Client's motivation
Possibilities to be heard
Service provision / availability of services
Client's attitudes towards the services
Men (N=99)
Women (N=110)
Kivipelto, Blomgren & Suojanen
2013, 46
Factors contributing to reaching goals. Stratified by gender (%)

19. THL, FinSoc-team 19
26
10
48
14
27
44
Supportive methods were not used * Used 1-2 supportive methods)* Used 3-4 supportive methods*
Men (%) Women (%)
Effectiveness of supportive methods to the client's problem-solving
capabilities. Comparison of men and women %
Kivipelto, Blomgren & Suojanen 2013, 50

20. THL, FinSoc-team 20
Single-case evaluation fits well
for evaluating social-work with
adults

21. Advantages
• It was seen as important to integrate
systematic inquiry within adult social
work
• The evaluation measure development
was also an attempt to make the data
collection easier and more systematic
THL, FinSoc-team 21

22. Difficulties
• Social workers found it difficult to incorporate the use of the
measure into practical social work, even though the
information required for the measure was largely the same
information they needed in client work
• Single-case evaluation is best suited for those situations that
require long-term social work
• Many clients just “disappeared” and the second phase of
evaluation could not be undertaken
• More work is still needed to develop a programme theory that
works well
THL, FinSoc-team 22

23. THL, FinSoc-team 23
Towards evidence-based
social work

24. THL, FinSoc-team 24
Processing the data into evidence-based knowledge

25. References
• Blomgren S, Kivipelto M (2012) Valtaistus. Aikuissosiaalityön valtakunnallinen
kartoitus [National Survey of Adult Social Work], Report 27, Helsinki: National
Institute for Health and Welfare (THL).
• Bloom M, Fischer J, Orme J G (2009) Evaluating practice: Guidelines for the
accountable professional, 6th edition, Boston: Allyn & Bacon.
• Cohen B J (2011) Design-based practice: A new perspective for social work, Social
Work, 56(4): 337–346.
• Fischer J, Corcoran K (2007) Measures for Clinical Practice and Research. A
sourcebook. Volume 1. Couples, Families and Children, Fourth Edition, New York:
Oxford University Press.
• Gray M, Plath D, Webb S (2009) Evidence-based social work. A critical
stance, London: Routledge.
• Howe D (2009) A brief introduction to social work theory, Basingstoke: Palgrave
Macmillan.
• Juhila K (2008) ‟Aikuisten parissa tehtävän sosiaalityön yhteiskunnallinen paikka‟
[Social Locus of Social Work With Adults], in A Jokinen and K Juhila
(eds), Sosiaalityö aikuisten parissa [Social Work With Adults], Tampere: Vastapaino:
48–81.
THL, FinSoc-team 25

26. • Julnes G, Mark M M (1998) „Evaluation as sensemaking: Knowledge construction in
a realist world‟, in G T Henry, G Julnes and M M Mark (eds) Realist Evaluation: An
Emerging Theory In Support Of Practice: New Directions For Evaluation 1998(78):
33–52.
• Kazi M, Wilson J (1996) „Applying single-case evaluation in social work‟, British
Journal of Social Work 26(5): 699–717.
• Kivipelto M, Blomgren S, Suojanen R (2013)‟ AVAIN-mittarin kehittäminen Seinäjoen
sosiaalivirastossa‟ [Developing the KEY-measure in Seinäjoki social security office].
In M Kivipelto, S Blomgren, P Karjalainen and P Saikkonen. Vaikuttavaa
aikuissosiaalityötä – arviointimalleista mittareihin. [Effective adult social work – from
evaluation models to evaluation measures] Research and development project; final
report. Report 8, Helsinki: National Institute for Health and Welfare (THL).
• Kivipelto M, Blomgren S (2012) „Social work with adults as a tool for tackling
exclusion‟, in S Karvonen, I Keskimäki, M Kuronen and K Wilskman (eds) Annual
review, Helsinki: National Institute for Health and Welfare (THL), 21–23.
• Pawson R, Manzano-Santaella A (2012) „A realist diagnostic workshop‟. Evaluation
8(2), 176–191.
• Pawson R, Tilley N (1997) Realistic Evaluation, London: Sage.
• Payne M (2005) Modern Social Work Theory, 3rd edition, Basingstoke: Palgrave
Macmillan.
• Pedersen L, Rieper O (2008) „Is realist evaluation a realistic approach for complex
reforms?‟ Evaluation 14(3), 271–293.
• Wong S E (2010) ‟Single-case evaluation designs for practitioners‟, Journal of Social
Service Research, 36(3): 248–259.
• THL, FinSoc-team 26