This study evaluated a web-based training resource called "Keeping Families and Children in Mind" designed to train clinicians working with families where a parent has a mental illness. Focus groups and questionnaires with participants from urban and rural training sites found that the training increased participants' knowledge, skills, and confidence working with these families. It emphasized the importance of collaborative, family-focused care. Participants responded positively to the interactive design of the resource, including video narratives, but noted that using the extensive materials would require time and commitment. The facilitators highlighted both benefits and challenges to delivering the training using the web-based resource.
‘Keeping families and children in mind’ an evaluation ofa w.docx
1. ‘Keeping families and children in mind’: an evaluation of
a web-based workforce resourcecfs_731 192..200
Andrea Reupert*, Kim Foster†, Darryl Maybery‡, Kylie Eddy§
and Elizabeth Fudge¶
*Senior Lecturer, Department of Rural and Indigenous Health,
Monash University, Moe, Victoria, †Associate Professor,
Mental Health Nursing, University of Sydney, Camperdown,
NSW, ‡Associate Professor of Rural Mental Health,
Department of Rural and Indigenous Health, Monash University
& Gippsland Medical School, Moe, Victoria, and
§Workforce Development Officer, ¶Project Manager, Children
Of Parents with a Mental Illness (COPMI) national
initiative, North Adelaide, South Australia, Australia
A B S T R AC T
This study outlines pilot evaluation data of the web-based
training
resource ‘Keeping Families and Children in Mind’, designed for
clini-
cians who work with families where a parent has a mental
illness. The
resource was developed from scoping existing workforce
packages
and in consultation with consumers, carers, researchers and
mental-
health clinicians. Preliminary evaluation data were collected
2. from an
urban and a rural site in Australia via focus group interviews
and pre-
and post-training questionnaires to ascertain the experiences of
those
who participated in the training. Additionally, training
facilitators
were invited to maintain journals in order to identify planning
and
implementation issues when using the resource. Post-training,
partici-
pants emphasized the need to work collaboratively with others,
as
well as the importance of acknowledging and working with the
family
members of consumers, especially children. Also, participants
reported positive changes in knowledge, skill and confidence
when
working with families affected by parental mental illness.
Facilitators
highlighted technology issues and the need to work interactively
with
participants when using the resource. Recommendations
regarding
policy and future research conclude this paper.
Correspondence:
Andrea Reupert,
Department of Rural and Indigenous
Health,
Monash University,
PO BOX 973,
Moe, Victoria,
Australia
E-mail: [email protected]
Keywords: children, evaluation,
3. families, parental mental illness,
web-based workforce training
Accepted for publication: August 2010
I N T R O D U C T I O N
Mental illness is a family affair, particularly where a
parent, with dependent children, has a mental illness.
Several studies indicate that children where a parent
has a mental illness may be at twice the risk of devel-
oping a mental illness diagnosis compared to other
children (Black et al. 2003; Park et al. 2003; Cunning-
ham et al. 2004; Leschied et al. 2005; Edwards et al.
2006). Other studies highlight the range of behav-
ioural, interpersonal, academic and other difficulties
that children of parents with a mental illness might
face (Rutter & Quinton 1984; Farahati et al. 2003;
Maughan et al. 2007; Reupert & Maybery 2007). An
epidemiological study has estimated that between 21
and 23% of all families have, or have had, at least one
parent with a mental illness (Maybery et al. 2009).
Thus, given the prevalence of families affected by
parental mental illness and the potential difficulties
they face, it is important that the mental-health work-
force is appropriately skilled at identifying and subse-
quently intervening with children and their parents.
This paper describes a web-based training resource,
‘Keeping Families and Children in Mind’ designed for
the mental-health workforce, and the results of a pilot
evaluation of the resource.
While there is ample evidence highlighting the need
for early intervention, children living in families where
5. stance abuse (Glynn et al. 2006). Family-sensitive
practice is beneficial to the consumer as well as other
family members, by reducing a family’s subjective
burden of care and increasing their level of self-care
and emotional functioning (Glynn et al. 2006). Fur-
thermore, acknowledging and working with children
of parent consumers improves family functioning and
children’s understanding of their parent’s disorder as
well as a reduction in children’s internalizing symp-
toms (Beardslee et al. 2008). Given the efficacy of a
family-sensitive approach, it is imperative that training
is designed and developed in ways that addresses
the current skill and knowledge gaps found in the
workforce.
In response to the training needs of the mental-
health workforce, a resource ‘Keeping Families and
Children in Mind’ was developed by the Australian
National COPMI (Children of Parents with a Mental
Illness) initiative through scoping existing workforce
packages and then identifying main themes and issues
across these packages (Reupert et al. 2009). Addition-
ally, the resource was developed using a Delphi
process (see Note 1) with 14 experts consisting
of consumers, carers, researchers and mental-health
clinicians (Whitman et al. 2009). These experts
responded to questions about curriculum content and
teaching processes in three Delphi ‘rounds’ until con-
sensus was reached. In this process, experts were
asked to summarize themes (generated from previous
Delphi rounds) that resulted in the final six core
modules of the resource (see Table 1).
The resulting Keeping Families and Children in Mind:
COPMI Mental HealthWorker Education Resource deliv-
6. ers an interactive, audio and video material using Web
2 technology. Clinicians might focus only on those
modules that are of interest and/or need or undertake
all six modules. The resource includes a variety of
educative web pages and links, and video and audio
inserts of families describing what it is like to live with
parental mental illness as well as clinicians reporting
Table 1 The six modules of ‘Keeping Families and Children in
Mind: COPMI Mental Health Worker Education
Resource’
1 Mental Health and Families – introduces a family where a
parent experiences mental illness. Information is also provided
by parents, children and workers about factors that contribute to
mental health and illness, stigma and mental illness and
family support.
2 The Parent – introduces a second family. Information is
provided about the impact of mental illness on parenting, the
impact of parenting on mental illness and the recovery process.
3 The Child – provides an opportunity to reflect on the
experiences of children in two families where a parent
experiences
mental illness. Information is provided on risk and protective
factors that influence child well-being, including the impact
of parental mental illness on child development. There are
demonstrations of how talking to children can assist in their
understanding of what is happening at home.
4 The Family – highlights the importance and influence of the
family unit in the recovery of a parent who experiences mental
illness, including influences on family functioning, family
resilience and working with families using a strengths-based
approach.
8. the authors). Consequently, as there are likely to be
hundreds and perhaps thousands of clinicians who
will use this resource, it is essential to report initial
data about the utility of the resource. Currently, a
further evaluation of the large-scale roll out is
planned, although it will take some time to collect and
analyse a larger data set. Thus, this paper summarizes
preliminary efficacy data from the piloting of the
training resource.
M E T H O D
Training was conducted at a rural and an urban site,
in two Australian states (Victoria and Tasmania). The
workshops were each presented by two female facili-
tators with extensive backgrounds in mental health
and workforce training. The training for the rural site
took place over two half days with 23 participants,
while training in the urban site went for a full day
and included 14 participants. Training participants
came from a range of services including child and
adolescent and adult mental-health agencies, non-
government agencies and hospitals, and from disci-
plines including psychiatry, social work, education
and consumer and carer groups. The purpose of the
training, at both sites, was to introduce participants to
the broad issues related to families where a parent has
a mental illness, and to identify where to access infor-
mation about mental illnesses for families and clini-
cians. Participation in the training was not dependent
on participation in the evaluation, and ethics approval
was provided by the Monash University Standing
Committee on Ethics in Research Involving Humans.
Data were drawn from three sources. First, partici-
pants were invited to participate in focus group
9. interviews to gain their views on the resource
and, second, to complete anonymous pre- and
post-program questionnaires to quantify short-term
changes in learning. Finally, training facilitators
completed journals in order to identify planning and
implementation issues.
Focus group interviews
Across the two focus groups held at the two sites, there
were a total of 28 participants, eight men and 20
women. Focus group questions aimed to determine
potential changes in knowledge, attitude and practice,
for example:
• What did you learn, if anything, as a result of being
shown the resource?
• Has the resource changed any attitudes or ways of
looking at consumers you previously had, if at all?
• In what ways, if any, do you think the resource will
change the way you practice?
With permission, the focus groups were audiotaped
and subsequently transcribed. Data were then analy-
sed using an open coding system, attaching labels to
lines or paragraphs of data and then describing the
data at a concrete level, before moving to a more
conceptual level (Anfara et al. 2002), first within each
focus group transcript and then across the two tran-
scripts. This descriptive and iterative analytic process
aimed to meaningfully classify codes into themes.
Questionnaires
11. issues, and then after the workshop, to highlight imple-
mentation matters. Pre-training questions included:
• What issues did you encounter in planning the
training? How did you overcome them? Or what
was required to overcome them?
• How did you decide which aspects of the resource
to use/not use?
Post-training questions included:
• What worked well when using the resource? What
didn’t work so well and why?
• What, if anything, do you think needs to change
about the resource?
• What facilitation skills were particularly important
in using the resource in workshop format?
• What issues did you encounter when using the
resource?
Facilitators were instructed to not censure them-
selves, nor be concerned about spelling or grammar.
They were not required to include their name on the
journal. Journals were analysed using the same the-
matic analysis approach employed with the focus
group transcripts.
R E S U LT S
Focus groups
Three themes emerged of participants’ experiences of
12. the training in terms of (i) views on the resource; (ii)
impact of the training on attitude and practice; and
(iii) possibilities for implementation.
Views on the resource
Participants expressed very positive views on the
resource, identifying it as a quality production with
comprehensive content and as a valuable resource for
clinicians from a range of child and adult healthcare
settings, particularly in rural and remote settings. The
emphasis on working together with families, rather
than simply focusing on diagnosis and assessment,
was appreciated. A strength of the resource was its
interactivity and the life-like depiction of issues expe-
rienced by children and families:
I thought that the narratives and the case studies were excel-
lent around mental health and placing that in the context of
the family . . .
Participants particularly enjoyed the video clips,
which they considered very effective for learning, ‘. . .
it holds your attention much more than just reading
. . . it brings it to life’.
Some participants considered the resource to be
quite lengthy and that it took time to learn to navigate
and become familiar with it. They acknowledged that
working through the resource in addition to their
workload would require commitment, ‘. . . if you
don’t block out time, you’ll just be too busy’.
However, the flexibility and ability to choose which
modules and content to read were helpful, ‘. . . the
13. advantage is that you can go in and pick the bits you
need’.
In the workshops, the opportunity to interact and
discuss issues with others, including consumers and
carers, and listen to their insights and practices, added
to the experience.
. . . one of the things that I got the most out of was having a
consumer presence in the training session. It was actually him
and his carer . . . so you were able to get the carer’s perspec-
tive, the consumer’s perspective, and that was good.
The training facilitators’ ability to identify each
group’s needs and tailor the workshop accordingly
was an important aspect of training. Reliable internet
access was integral to its effectiveness, with broadband
access and slow running of computers, a frustration
for some participants.
Impact of the training on attitude and practice
The most prominent impact of the training was par-
ticipants’ heightened awareness of the need to care
for all family members and the importance of taking
time to sit and talk with families. For some, training
reaffirmed that they were on the right pathway in
what they were already practising, while for others,
the resource challenged their attitudes towards
families.
. . . it makes you aware of what you’re bringing to the inter-
15. might impact on individual practice, unless manage-
ment supported child and family-inclusive practice as
core business, the impacts could be limited.
. . . the practice will change the individual, and ideally if
they’re supported by the management structure . . . it spreads
through the organisation . . .
Possibilities for implementation
Participants saw a range of possibilities for imple-
menting the resource within their organization as well
as in other organizations and contexts. This included
using the resource in professional development ses-
sions for staff over a 12-month period, including it as
part of new staff inductions, and using it as a training
resource for students on clinical placement:
. . . I can see the potential for it to change practice within my
organisation.We have a carer support program . . . and I’d like
to deliver [a module] to the carer support team.
Participants made a number of suggestions for
improving the resource, such as adding further infor-
mation on mental illness and drug and alcohol use for
users who might not have a background in mental
health, and adding links to relevant local and/or
regional resources and services for clinicians and
family members. Finally, participants suggested that
the resource should include a section on collaborative
practice and possible protocols for how agencies can
work with each other with families.
16. Questionnaire
Table 2 presents participant mean scores, standard
deviations and paired sample t-statistics for pre- and
post-responses to items about clinicians’ confidence,
knowledge and skill concerning family issues. Almost
all participant responses to knowledge (see Note 2)
items moved in the expected direction (note that some
items are negatively worded) and just under half of the
26 items showed a significant change – again all in the
expected direction. Items shown in bold are signifi-
cant at 0.05 level and actual P-values are shown in the
right-hand column for each item.
Significantly different variables generally reflect
changes that occurred at both the rural and the urban
sites.
Program facilitator journals
Planning issues
The main planning issues for facilitators were related
to information technology and the knowledge to
appropriately download and save different aspects of
the resource. Working with others with technological
expertise was or would have been useful. Facilitators
indicated that having a thorough knowledge of par-
ticipants’ background (profession and organization)
was important in deciding which aspects of the
resource to employ.They also suggested that the work-
shop needed to incorporate interactive as well as
didactic components.
18. with both participants and training facilitators valuing
the interactivity and flexibility of a web-based
resource. Participants, including those from adult
mental health, highlighted the need, post-training,
to work with all family members including children.
This is an important finding given that previous
research has highlighted clinicians’ reluctance to work
with the children of parent consumers (Slack &
Webber 2008). Participants also stressed the necessity
to collaborate with other agencies when working with
families, another substantial result, due to the lack of
inter-agency and inter-sectoral collaboration in this
area (Darlington et al. 2005). While the resource was
not compared in terms of delivery (that is, self-paced
mode vs. a group setting), the focus group data high-
light the significance for clinicians in being trained
alongside others, including consumers and carers, in
order to share insights, experiences and practices.
The quantitative questionnaire data tentatively
shows an improvement in participants’ knowledge,
Table 2 Scores and paired sample t-statistics for pre- and post-
responses to items regarding clinicians’ confidence,
knowledge and skill when working on family issues
Questionnaire item
Pre Post M
t PM (SD) M (SD) diff
I am knowledgeable about how parental mental illness impacts
on children and
families.
19. 5.65 (0.85) 6.04 (0.77) -0.39 -2.30 0.03
I am not confident working with families of consumer-parents.
3.00 (1.87) 2.32 (1.68) 0.68 1.71 0.10
I am not knowledgeable about the key parenting issues for
consumer-parents. 3.04 (1.68) 3.00 (2.04) 0.04 0.12 0.90
I am knowledgeable about the key things that consumer-parents
could do to
maintain the well-being (and resilience) of their children.
4.96 (1.28) 5.38 (1.65) -0.42 -1.15 0.26
I am knowledgeable about the role of family carers and their
influence on
recovery for consumers.
5.38 (1.06) 5.81 (1.17) -0.42 -1.23 0.23
I am not knowledgeable about the role of young carers in
families where parents
experience mental illness
3.27 (1.66) 2.23 (1.56) 1.04 2.78 0.01
I am not confident working with children of consumer-parents.
2.92 (1.77) 2.50 (1.84) 0.42 1.17 0.25
I am knowledgeable about how the role of parenting impacts
mental illness. 5.36 (1.04) 5.84 (0.80) -0.48 -2.39 0.03
I am not confident working with consumer-parents about their
parenting skills. 3.35 (1.79) 2.58 (1.79) 0.77 2.08 0.05
I do not have the skills to work with consumer-parents about
how parental mental
illness impacts on children and families.
3.23 (1.53) 2.31 (1.49) 0.92 3.27 0.00
20. I provide education sessions for adult family members (e.g.
about the illness,
treatment).
4.63 (2.09) 4.42 (2.01) 0.21 0.41 0.69
I am skilled in working with consumer-parents regarding their
parenting. 4.38 (1.84) 4.92 (1.77) -0.54 -2.01 0.06
I provide education sessions for children (e.g. about the illness,
treatment). 3.94 (2.30) 4.06 (2.25) -0.12 -0.32 0.76
I am skilled in providing psychosocial-education to adult family
members about
the mental illness.
4.00 (2.02) 4.86 (1.80) -0.86 -2.42 0.03
I regularly have family meetings (not therapy) with consumer-
parents and their
family.
4.26 (2.26) 4.74 (1.88) -0.47 -1.21 0.24
I consider information from the carer or family when diagnosing
and/or
treating the consumer-parents.
5.37 (1.50) 5.79 (1.47) -0.42 -2.04 0.06
I provide emotional support for family members and children.
5.48 (1.53) 6.09 (0.85) -0.61 -2.37 0.03
I do not refer children of consumer-parents to child-focused
(e.g. peer
support) programs (other than child and adolescent mental
health).
3.19 (1.78) 3.00 (2.05) 0.19 0.43 0.67
21. I do refer consumer-parents and their families for family
therapy or
counselling.
5.09 (1.50) 5.48 (1.34) -0.39 -1.20 0.24
I provide written material (e.g. education, information) about
parenting to
consumer-parents.
4.75 (1.59) 5.38 (1.47) -0.63 -3.72 0.00
I regularly provide information (including written materials)
about mental health
issues to the children of consumer-parents.
4.10 (2.19) 5.29 (1.76) -1.19 -4.23 0.00
I often consider if referral to parent support program (or
similar) is required by
consumer-parents.
5.09 (1.53) 5.65 (1.34) -0.57 -2.02 0.06
Rarely do I consider if referral to peer support program (or
similar) is required
by my consumer-parent’s children.
2.35 (1.47) 2.13 (1.52) 0.22 0.59 0.56
I don’t provide information to the carer and/or family about the
consumer-parent’s medication and/or treatment.
3.82 (2.15) 3.77 (2.09) 0.05 0.09 0.93
23. 2. What are 5 specific facts you took away as a result of this
video?
3. How has your thoughts/opinions changed about the subject?
4. What will you do differently given a situation like this, now
that you have learned more?
5. What are 3 questions you have regarding the content of this
video?
Link: https://www.youtube.com/watch?v=0gAsdEUNUJY
Assignment 2)
The Science of Happiness' Assignment
1. What 3 aspects define happiness? What percentage of your
happiness is genetic?
2. What a few things they have found out about people who are
happy?
3. What is one think people think make them happy? And what
has research shown that proves that to be incorrect?
4. What do they find in research regarding, marriage and
happiness?
5. What do they find in research regarding age and happiness?
6. Dr. Sanderson suggests 10 things to increase happiness. What
are these 10 things?
7. Out of those 10 things what 3 things will you start
implementing in your life to be happier?
Link;
https://dcccd.yuja.com/V/Video?v=276468&node=1320654&a=
1082739204&autoplay=1
Assignment 3)
24. Memory's Assignment
1. What was your overall reaction to how memory works?
2. What are 5 specific things you can do to improve memory?
3. How has your thoughts/opinions changed about how your
memory works?
4. What will you do differently when studying for an exam now
that you have learned some techniques?
5. What are 3 questions you have regarding the content of this
video?
Link: https://www.youtube.com/watch?v=d95dOH-7GHM
Assignment 4)
Born a Boy, Brought up a Girl assignment
1. What is your reaction to this video? What would you have
done in this situation if you were the parents?
2. Why developmental theories/principles are most relevant in
this case?
3. How does this story relate to the notion of nature vs. nurture?
4. What did you used to think about gender development? What
do you think about it now after viewing Born a Boy, Brought up
as a Girl?
6. What did you learn as a result of this video (in terms of how
your learning relates to the course)? Therefore, saying, "I
learned a lot; I learned that you can't do that to kids" etc. are
not appropriate responses.
Link: https://www.youtube.com/watch?v=MUTcwqR4Q4Y
Assignment 5)
Transgender Children Assignment
1. What was your overall reaction to this video regarding
transgender individuals?
2. What are 5 specific facts you took away as a result of this
video?
25. 3. How has your thoughts/opinions changed about
transgender individuals?
4. If you have a child that is transgender, what will you do?
5. What are 3 questions you have regarding the content of this
video?
Link: https://www.youtube.com/watch?v=qV8b8hsQups
https://www.youtube.com/watch?v=DlBOHxRFD0c
Assignment 6)
An Unfinished Lesson's Assignment
1. What was your overall reaction to the podcast? Elaborate.
Saying, "It was fascinating or I learned a lot" are not thought
out responses.
2. What happened in 1918? Approximately how many deaths
were there? Why was it called, The Spanish Flu?
3. What is the understanding behind the U-shape pattern with
the typical flu and death rates versus the W-shaped pattern that
they discovered with Spanish flu of 1918 and death rates? What
age group had a 50% death rate in the 1918 pandemic?
4. What specific things occurred in 1918 in the city of
Philadelphia vs. Milwaukee that created increasingly more
deaths in one of these cities. Explain why the death rates were
so different.
5. Overall, what similarities are we experiencing today in the
COVID-19 pandemic versus. the 1918 Spanish Flu? What are
some differences that we have today versus 1918? Elaborate
your thoughts.
6. What has been your personal experience with the COVID-19
global pandemic occurring right now?
7. What are some things you will now do differently because of
this experience?
8. What are some things you took for granted before
experiencing this?
26. 9. What are some lessons learned from this experience?
10. What are you grateful for today?
Link: https://edge2.pod.npr.org/anon.npr-
podcasts/podcast/npr/hiddenbrain/2020/03/20200323_hiddenbrai
n_nancy_bristow_323_618-3bf25728-044d-4918-a987-
3331c4ef0283.mp3/20200323_hiddenbrain_nancy_bristow_323_
618-3bf25728-044d-4918-a987-
3331c4ef0283.mp3_3c132af9f477c347863b73039a57b218_4714
6087.mp3?orgId=1&topicId=1136&aggIds=423302056,8120549
19&d=2948&p=510308&story=820066211&t=podcast&e=82006
6211&siteplayer=true&size=47060911&awCollectionId=510308
&awEpisodeId=820066211&dl=1&x-ais-
classified=download&hash_redirect=1&x-total-
bytes=47146087&listeningSessionID=0CD_382_125__0dce4d99
3aed49678922078aa332e65d11e23015
Link 2: https://www.npr.org/podcasts/510308/hidden-brain
Complete all assignments