Adult social work, evaluation

1. Effective adult social work – from
evaluation models to evaluation
measures

2. Outline
• Our work at THL
• Adult social work, effectiveness evaluation (project 2011-
2012, reported 2013)
• Results
• Observations
• Challenges
THL, FinSoc 2

3. The Finnish Evaluation of Social Services group
(FinSoc)
• Enhances knowledge of evaluation methodology and
promotes the use of evaluative information in social services
• Several R&D-projects on evaluation since 1997
• Aims
– Provide evaluation methods for the social sector
– Communicate evaluative information on practices and methods in
social services
– Support the development of social services by conducting
evaluation research
• The group co-operates actively with various actors in the
social sector
22.8.2013 3

4. 4
Goal-oriented adult social work as a target for
evaluation
The characteristics of goal-oriented adult social work
• The goal of social work
• The object of social work
• Social work methods
• Contextual and situational factors
• Mechanisms
“What are the mechanism for change triggered by a social
work and how do they counteract the existing social
processes?”
THL, FinSoc 4

5. THL, FinSoc 5
Adult Social Work Effectiveness Evaluation (2011-2012)
Coordination:The National Institute for Health and Welfare
Seinäjoki: Developing an
effectiveness evaluation
measure for adult social
work
The Centre Of
Excellence On
Social Welfare In
The Ostrobothnian
Area (SONet
Botnia)
Helsinki: Focusing on
data from client
monitoring forms at West
Helsinki Social Centre
Tuusula:
Evaluating rehabilitative
social work

6. Single-case design
THL, FinSoc 6
• Single-case design allows social workers to evaluate how
well the social work’s goals have been reached, what
methods and procedures have been used, and how
important different contextual and situational factors and
mechanisms are for the goal attainment
• Single-case evaluation also seems to be suitable
because it follows the basic idea of empirical research
• It is possible to gain information about single clients’
goals but also to obtain quantitative data. It is also
possible to track the client’s situation using repeated
measurements.

7. Developing measure
THL, FinSoc 7
• Questions about the goals, methods and
procedures of social work and about the
client’s situation.
• Testing period: November 2011 – August
2012 (10 months)
• Single-case design: Evaluation phase I &
Evaluation phase II
• Evaluation I: 209 cases (clients)
• Evaluation II: 172 cases

8. THL, FinSoc- 8
Results from adult social work
effectiveness evaluation

9. THL, FinSoc- 9
0 10 20 30 40 50 60 70 80 90
Concern about the close personal relationships
Leisure activities and opportunities for participation
The consequences of anti-social behaviour
Physical problems caused by illness
Child's needs
Substance abuse and consequences
Problems related to mental illnessess
Rental housing situation in the area
Substance use
Social skills
Client's relationships
Mental health
Development of service system
Proceeding with further education
Client's possibility to access housing
Inclusion and participation (eg. hobbies)
Self-esteem
Awareness of the problems due to the society
Getting in to further education
Fluency of everyday life
Life management
Access to housing; keeping it
Clearing up the debt situation
Promoting the employment situation (for long-term…
Supporting client's occupational capabilities
Supporting client's problem-solving capabilities
Service management for a client
Employment, searching for a job
Client's control over his/her economic situation
Women (%)
(N=110)
Men (%) (N=99)
Goals set by clients; stratified by gender (%)
Kivipelto, Blomgren & Suojanen 2013, 43

10. THL, FinSoc 10
Goal reached admirably; percentage of levels of achieving goals
(%)
Kivipelto, Blomgren & Suojanen 2013, 44
0
5
18
21
21
23
24
24
25
26
28
29
30
33
33
35
36
36
38
39
39
41
41
42
42
45
49
49
53
0 10 20 30 40 50 60
Client's possibility to access housing
Rental housing situation in the area
Leisure activities and opportunities for participation
Physical problems caused by illness
Promoting the employment situation (for long-term unemployment-people)
The consequences of anti-social behaviour
Concern about the close personal relationships
Mental health
Clearing up the debt situation
Client's relationships
Problems related to mental illness
Awareness of problems due to the society
Employment situation, searching for a job
Inclusion and participation (eg. hobbies)
Development of service system
Client's control over his/her economical situation
Life management
Proceeding with further education
Social skills
Getting into further education
Substance use
Substance abuse and consequences
Supporting client's occupational capabilities
Self-esteem
Fluency of everyday life
Access to housing; keeping it
Supporting client's problem-solving capabilities
Child's needs
Service management for a client
Situation improved

11. THL, FinSoc- 11
0 20 40 60 80 100
Controlling the client
Supporting client's participation
Supporting client's awareness of social problems
Case management
Dealing with the client's problems
Service plan for the client
Solution focused work
Supportive discussion
Supervision and guidance
Needs assessment
Decision making
Social assistance
Women (%) Men (%)
Most used social work methods. Percentage of all methods; stratified
by gender (%)
Kivipelto, Blomgren & Suojanen
2013, 44

12. THL, FinSoc-team 12
0 10 20 30 40 50 60 70
Values and attitudes in the living area
Access to the environment
Client's cultural background
The demographic structure of living area
Substance abuse of family members
Concern about the close people
Substance abuse / addictions
Client's mental health
Mental health and well-being
Client's employment situation
Satisfaction of relationships/adequacy
Client's lifestyle
Client's economic situation
Client's physical health
Possibility to access health related technology
Awareness of the bacground determinants behind…
Possibility to influence
Leisure activities and opportunities for participation
Client's ability/motive to plan his/her economic situation
Client's occupation
Seeing solutions and alternatives
Employment and occupational situations
Internet connectivity
Client's self-esteem
Living conditions
Client's motivation
Possibilities to be heard
Service provision / availability of services
Client's attitudes towards the services
Men (N=99)
Women (N=110)
Kivipelto, Blomgren & Suojanen
2013, 46
Factors contributing to reaching goals. Stratified by gender (%)

13. THL, FinSoc 13
26
10
48
14
27
44
Supportive methods were not used * Used 1-2 supportive methods)* Used 3-4 supportive methods*
Men (%) Women (%)
Effectiveness of supportive methods to the client's problem-solving
capabilities. Comparison of men and women %
Kivipelto, Blomgren & Suojanen 2013, 50

14. Advantages
• It was seen as important to integrate
systematic inquiry within adult social
work
• The evaluation measure development
was also an attempt to make the data
collection easier and more systematic
THL, FinSoc 14

15. Difficulties
• Social workers found it difficult to incorporate the use of the
measure into practical social work, even though the
information required for the measure was largely the same
information they needed in client work
• Single-case evaluation is best suited for those situations that
require long-term social work
• Many clients just “disappeared” and the second phase of
evaluation could not be undertaken
• More work is still needed to develop a programme theory that
works well
THL, FinSoc 15

16. THL, FinSoc 16
Processing the data into evidence-based knowledge

17. Further challenges
• National instruments and data collection methods
• Knowledge production – decision making; proactivity instead
of reactivity
• Motivation and commitment
• Structures that integrate research, higher education, and
social work practice
• International networks and co-projects around evaluation
research (measure and method development, case-studies,
evidence based social work projects etc.)
• Higher education on social work evaluation (no master level
education in evaluation)
22.8.2013 17

18. Questions?
Our team
• minna.kivipelto@thl.fi (Team leader)
• sanna.blomgren@thl.fi
• pekka.karjalainen@thl.fi
• paula.saikkonen@thl.fi
THL, FinSoc 18
Thank you!

19. References
• Blomgren S, Kivipelto M (2012) Valtaistus. Aikuissosiaalityön valtakunnallinen
kartoitus [National Survey of Adult Social Work], Report 27, Helsinki: National
Institute for Health and Welfare (THL).
• Kivipelto M, Blomgren S, Suojanen R (2013)’ AVAIN-mittarin kehittäminen Seinäjoen
sosiaalivirastossa’ [Developing the KEY-measure in Seinäjoki social security office].
In M Kivipelto, S Blomgren, P Karjalainen and P Saikkonen. Vaikuttavaa
aikuissosiaalityötä – arviointimalleista mittareihin. [Effective adult social work – from
evaluation models to evaluation measures] Research and development project; final
report. Report 8, Helsinki: National Institute for Health and Welfare (THL).
• Kivipelto M, Blomgren S (2012) ‘Social work with adults as a tool for tackling
exclusion’, in S Karvonen, I Keskimäki, M Kuronen and K Wilskman (eds) Annual
review, Helsinki: National Institute for Health and Welfare (THL), 21–23.
• Saikkonen, Paula (2013) Challenges in Evaluating Welfare Services: a Case of Adult
Social Work. Evaluation Connections, June 2013.
THL, FinSoc-team 19