This document summarizes an evaluation of the Family SMILES program, an NSPCC intervention for children with parents who have a mental illness. The evaluation found that after participating in the program, children showed statistically significant reductions in emotional and behavioral difficulties according to parent and practitioner reports. Children's self-esteem also improved significantly. Parents demonstrated enhanced protective behaviors as well. Qualitative interviews revealed that the program improved family communication around mental health issues. The evaluation used various tools to measure outcomes and experienced some facilitators and barriers to the program.

1. Evaluation of Family SMILES: an NSPCC
intervention for children whose parents
have a mental illness
Prakash Fernandes and Rachel Margolis

2. What is the
Family
SMILES
programme?

3. Family SMILES Service
Family SMILES
Family SMILES is targeted at children and young
people aged 8 to 14 whose parents have a major
mental illness that has contributed significantly to
the child having been identified as being
vulnerable or at risk

4. Fa
In Brief
• The original programme was developed in Australia by
Erica Pitman
• 8 Group work sessions
• The aim of the group is to help children understand
about mental health and develop some life skills to help
them cope
• The parent also receives one to one support to
understand their child’s experience of living in a family
with mental illness
• The family then completes a support plan to ensure the
child is safe and their wishes are considered in the
event of the parent becoming ill again

5. • The NSPCC have been running the Family SMILES
service since September 2011.
• Family SMILES is run in the following NSPCC service
centres: Belfast, Bristol, Gillingham, Grimsby, Ipswich,
Jersey, Manchester, Middlesbrough, Prestatyn and
Southampton
• At the end of 2013, 144 parents and 158 children had
gone through the programme.
5
In Brief- cont’d
Where is Family SMILES run and how many people have
used the service?

6. Overview of
the
evaluation

7. 7
To enable children
and young people to
process their
thoughts and
feelings
To enable
children and
young people to
feel better about
themselves
To reduce children and
young people's emotional
and behavioral problems
Objectives
To provide
individual work
with parents
To provide group
support for
children
To provide joint work
with children and
parents
To
improve the
well-being of children
and young people
and reduce isolation
Specific
aims
Overall
aim
Outputs
Group work
Information
Signposting
Peer support
To enhance parents
protective behaviors
Individual work
Information
Signposting
Joint work
Safety plan

8. 8
Evaluation design
Outcome Tool Perspective
To enable children and young people
to feel better about themselves
Adapted Rosenberg Self Esteem Scale Child
To reduce children and young
people’s emotional and behavioural
issues
Goodman’s Strengths and Difficulties
Questionnaire (SDQ)
Health of the Nation Outcome Scales
(HoNOSCA)
Child/ Parent
Practitioner
To enable children to process their
thoughts and feelings
Children’s evaluation wheel Child
To enhance parents’ protective
parenting/ to improve safeguarding
of children
Child Abuse Potential Inventory (CAPI)
Parents evaluation wheel
Parent
Qualitative interviews with children, parents, practitioners and referrers
Quasi experimental design . Full report in December 15

9. Key
Findings
from interim
report

10. To reduce children and young people’s
emotional & behavioural problems
(children/ parent’s perspective)
SDQs completed at both T1 and T2 (n=72)
• The mean at T1 is 15.3and the mean at T2 is 13.6. The change in score is
statistically significant (p=0.01)
The data also indicates a shift from a level of difficulty that is of clinical concern
to that of a normal level.( Exact McNemar’s test p=0.01)
0
2
4
6
8
10
12
14
16
18
T1 T2
Average
score
Time frame
T1
T2
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
T1 T2
Borderline/Clinical
range
Normal range

11. Reduction in children’s
emotional/behavioural difficulties (2)
• There has been a decrease in mean scores across all subscales of the
SDQ. However the change is only statistically significant for conduct
problems (0.04) and peer problems subscale.( p= 0.01)
0
1
2
3
4
5
6
7
8
9
10
Conduct problems Emotional
symptoms
Hyperactivity Peer problems Pro-social
Average score
SDQ subscales
T1
T2

12. To reduce children and young people’s
emotional & behavioural problems
(practitioners' perspective)
56 HoNOSCAs have been completed at T1 and T2.
• The mean score at T1 is 8.04 and the mean at T2 is 6.68. The change in
mean score between T1 and T2 is statistically significant. ( p= 0.02 )
0
1
2
3
4
5
6
7
8
9
T1 T2
Average
score
Time frame
T1
T2

13. 13
Outcome : To enable children and young
people to feel better about themselves
72 Adapted Rosenberg Self-esteem questionnaires have been completed
at T1 and T2
0
5
10
15
20
25
T1 T2
Mean score
Time frame
T1
T2
• There has been an increase from a mean of 19.35 at T1 to 21.29 at T2.
The change is statistically significant p= 0.01

14. Outcome: To enhance Protective Parenting
Subscale Mean
T1
Mean
T2
Statistically significant
(p<=0.05)
Distress (cut off=152) 185.7 168.5 Yes
Rigidity (cut off=30) 20.9 18.1 Yes
Unhappiness (cut off=23) 40 31.6 Yes
Problems with child and self (cut
off=11)
7.5 6.1 No
Problems with family (cut off=18) 17.7 14.8 No
Problems with others (cut off=20) 17.5 17 No
Total score (cut off 215) 289.2 256.2 Yes
Ego strength scale 9.7 12.5 Yes
Loneliness scale 11.5 11.3 No
Overall reduction in behaviours that contributes to risk in children as shown
by the CAPI. CAPIs suitable for evaluation at T1 and T2 (n=22)

15. Outcome: To enable children and young
people to process their thoughts & feelings
• The changes between T1 and T2 are statistically significant for all
criteria except for easily making friends and having a fun time.
Criteria T1 T2
Statistically
significant
I can talk to someone if I'm worried about my
parent's health 3.65 4.07 Yes
I can talk to my parent about how their mental
health affects me 3.25 3.7 Yes
I can easily make friends 3.82 4.03 No
I am able to have a fun time when I want 3.95 4.04 No
I feel supported by others around me 3.85 4.22 Yes
Children’s evaluation wheels (n=94)

16. Parents’ perception of changes to their
parenting
16
22 parents completed the wheel at both Time 1 and Time 2
Criteria T1 T2
Statistically
significant
How much I think that my child is affected
by my behaviour 3.64 3.73 No
How supported I feel in taking care of my
child 3.41 4.05 No
How confident I feel in asking for help
when I need it 3.55 4 No
How much knowledge I have about
children's' needs at different stages of
their development 3.64 4.14 No
How confident I feel that I am doing the
best I can for my child 3.82 4.23 Yes

17. Key Findings: Qualitative interviews with
children and parents
More effective communication within the family about mental
health
“I never really felt comfortable talking to
my mum about anything. My mum is starting
to tell me stuff, and usually before I would
probably go: ‘I don’t really care’. But I do care now
- Young person
17
“But I was able to talk to him then more about it and I learnt that from SMILES. It’s
OK to discuss things with your child and it’s OK to say ‘I’m not feeling well’, which
before I would have said nothing. Just ‘Oh, mummy hurt her back today’ or ‘mummy
tripped and fell’ or something, I’d come up with these reasons for being in bed or if I’d
been crying or whatever. Definitely, definitely made a big difference.”
(Parent, Severe mental health problem)
“I thought I was doing right by
letting her deal with her own
thing, when actually now she’ll
ask me for five minutes and
we’ll sit and talk” (Parent)

18. Key Findings: Facilitators and Barriers
18
• The group work experience: age range ; gender; some awareness of parent’s
mental health problems prior to group starting.
“I could talk about things which
actually other people had
experienced with their own mums.
It was really helpful.” (Child)
I used to bottle it up and I used to find it hard to open myself
up. So I used to kind of sit alone… But then when I had been
to that group for eight weeks somehow I just couldn’t end
up holding it in and maybe that was a better way of me
showing I’m hurting... Now I’ve learnt that telling someone
is the best thing and bottling up is the worst kind of thing”
(Child)
• The individual work with parents: readiness to accept impact of their mental health
on their child; how they are managing their mental health problems; how they perceive
change in the child’s behaviour at home; engagement (or lack of) from the entire
family; inability of the programme to address family dynamics
“She (practitioner) asked me a question and it stuck in my head: “How do the boys feel about
you?” I’ve never asked that because you take it for granted and that was the turning point”
(Parent)

19. Next Steps for the evaluation
19
• Analysis of the comparison group data
• Analysis of the interview data from referrers and
practitioners
• Analysis of Time 3 data

20. 2020
Summaries of all our evaluation plans
Each summary includes:
• Details of the evaluation methods we are using for the service
• Details of any challenges faced and how we overcame them
• A list of the measurement tools we are using
• Contact information of the staff member involved in the evaluation
Tools for measuring outcomes for children and Families
We've outlined our experiences of using each tool, along
with details of what it measures and how easy it is to
use.
Each measure includes:
• An explanation of the measure
• Details of who to use it with
• The time it takes to administer
• Any training or expertise required
• It’s validity and reliability
• Cost
• Our experience of using it
And much, much more….
Our Impact and Evidence Hub can be found at
www.nspcc.org.uk/evidencehub
Join the debate at:
@nspccpro
Search #evidencehub
Our hub features…
The NSPCC Impact and Evidence Hub