Predictive risk modelling is being developed in New Zealand to identify children at high risk of maltreatment and provide preventative services. However, the accuracy of predictions from models tested so far has been too low for use as a sole mechanism. The main predictors identified have been a caregiver's history of welfare benefits, single parenthood status, and their own history of maltreatment or other children's maltreatment reports. However, these predictors are strongly associated with poverty and may serve to identify many poor families rather than justify child welfare intervention. Improving data quality and developing new datasets specifically for modelling may help increase predictive ability in the future.

1. Big Data Analytics and Predictive Risk
Modelling in Social Work
Lessons from the Reform of Child Protection Services in Aotearoa/New
Zealand
Dr Philip Gillingham, Senior Lecturer in Social Work, University
of Queensland
1

2. SWSD 2014 in Melbourne
What do you think of Predictive Risk Modelling as it is being developed
in New Zealand?
2 years later…
Predictive ability
Insights into child maltreatment
Dr Philip Gillingham, Senior Lecturer in Social Work, University
of Queensland
2

3. Ethical problems
Keddell, E. (2014) The ethics of predictive risk modelling in the Aotearoa/New Zealand child welfare
context: Child abuse prevention or neo-liberal tool? Critical Social Policy, published online 28 July
2014, DOI: 10.1177/0261018314543224
De Haan, I. & Connelly, M. (2014) Another Pandora's box? Some pros and cons of predictive risk
modelling. Children and Youth Services Review, http://dx.doi.org/10.1016/j.childyouth.2014.07.016
Dare, T. (2013) Predictive risk modelling and child maltreatment: an ethical review. Accessed May
2015 at http://www.msd.govt.nz/documents/about-msd-and-our-work/publications-
resources/research/predictive-modelling/00-predicitve-risk-modelling-and-child-maltreatment-an-
ethical-review.pdf
Dr Philip Gillingham, Senior Lecturer in Social Work, University
of Queensland
3

4. What is PRM?
An algorithm that can be applied to children as they enter the social
security system to generate a score that estimates the likelihood of
future child maltreatment.
Aim is to prevent maltreatment by providing support services.
CARE (2012) Vulnerable Children: can administrative data be used to identify children at risk of adverse
outcomes? Centre for Applied Research in Economics, University of Auckland. Available at:
www.msd.govt.nz/documents/about-msd-and-our-work/publications-resources/research/vulnerable-
children/auckland-university-can-administrative-data-be-used-to-identify-children-at-risk-of-adverse-
outcome.pdf
Dr Philip Gillingham, Senior Lecturer in Social Work, University
of Queensland
4

5. What is PRM?
Using historical data (training data) - learns by calculating correlation
between predictors (independent variables) and the outcome
(dependent variable).
Predictors: information about the child, siblings and parents.
Outcome: child maltreatment
But accuracy in training data very important…
Dr Philip Gillingham, Senior Lecturer in Social Work, University
of Queensland
5

6. Predictive ability
Very low.
Data from child protective services about “substantiated cases”.
Children who have been maltreated +
Many more who have not: children considered to be “at risk”,
siblings of those who have been maltreated and those “at risk”,
unaccompanied minors.
Dr Philip Gillingham, Senior Lecturer in Social Work, University
of Queensland
6

7. Insights into child maltreatment
May 2015
MSD (2014) Final report on feasibility of using predictive risk modelling. Wellington, New Zealand:
Ministry of Social Development, Wellington. Accessed May 2015
at http://www.msd.govt.nz/documents/about-msd-and-our-work/publications-
resources/research/predictive-modelling/00-feasibility-study-report.pdf
MSD (2014) The feasibility of using predictive risk modelling to identify new-born children who are
high priority for preventive services – companion technical report. Ministry of Social Development,
Wellington. Accessed May 2015 at https://www.msd.govt.nz/documents/about-msd-and-our-
work/publications-resources/research/predictive-modelling/00-feasibility-study-report-technical-
companion.pdf
TCC Group (2013) Peer Review Report 1. Accessed May 2015 at http://www.msd.govt.nz/
Dr Philip Gillingham, Senior Lecturer in Social Work, University
of Queensland
7

8. 12 different versions of PRM tested
Data from health, corrections, separate model for Maori children
But:
“there are still far too many inaccurate predictions for the current PRM
model to be used as a sole mechanism for the identification of children
at high risk” (TCC Group 2014, p. 4)
Dr Philip Gillingham, Senior Lecturer in Social Work, University
of Queensland
8

9. Insights – main predictors
1. Mother or primary caregiver’s time on benefit in the last five years
2. Relationship status of caregiver – single parent
3. Caregiver’s care and protection history as a child and care and
protection history of other children
Dr Philip Gillingham, Senior Lecturer in Social Work, University
of Queensland
9

10. 1. Mother or primary caregiver’s time on benefit
in the last five years
Poverty and child maltreatment – nothing new
Lindsay, D. (2009) Child Poverty and Inequality: Securing a Better Future for
America's Children. Oxford: Oxford University Press.
Pelton, L. H. (2015) The continuing role of material factors in child maltreatment
and placement. Child Abuse and Neglect, 41, 30-39.
Pelton, L. H. (1994) The role of material factors in child abuse and neglect. In G. B.
Melton, and F. D. Barry (Eds.), Protecting children from abuse and neglect:
Foundations for a new national strategy (pp. 131–181). New York: Guilford Press.
Dr Philip Gillingham, Senior Lecturer in Social Work, University
of Queensland
10

11. Problems with predictors
In addition, if the primary covariate with child welfare is poverty, in
incorporating a multitude of poverty indicators (i.e. corrections data,
benefit data, care and protection history, etc.), this tool serves merely
to identify a large number of poor families in need rather than to justify
a call to action that would involve referral to the child welfare system
(TCC Group 2013, p. 5).
Dr Philip Gillingham, Senior Lecturer in Social Work, University
of Queensland
11

12. 2. Relationship status of caregiver – single parent
Research about caregivers - relationship status not the most important
factor (mental illness and drug dependence).
But single parents more likely to experience poverty.
28% of households are headed by a single parent, 90% of which have
an income below the median household income for all households (as
compared to 55% for two parent families) and 73% are in receipt of
public welfare benefits (MSD, 2010).
Dr Philip Gillingham, Senior Lecturer in Social Work, University
of Queensland
12

13. 3. Caregiver’s care and protection history as a child and
care and protection history of other children
Not new: multiple notifications, investigations and substantiations…
Past maltreatment.
Gillingham, P. (2009) The use of assessment tools in child protection: an ethnomethodological study.
University of Melbourne. http://repository.unimelb.edu.au/10187/4337
Bromfield, L. M., Gillingham, P & Higgins, D. (2007) Cumulative harm and indicators of chronic child
maltreatment. Developing Practice: the Child, Youth and Family Work Journal, 19, Winter/Spring
2007, 34-42.
Bromfield, L. (2005) Chronic child maltreatment in an Australian statutory child protection sample.
Ph.D. thesis, School of Psychology, Deakin University.
Dr Philip Gillingham, Senior Lecturer in Social Work, University
of Queensland
13

14. But…new direction for child protection
reform?
Not new – need to move from forensic investigation to preventative
support services (ex Lonne et al, 2008).
Possibly new – introduce a focus within preventative services on
therapeutic interventions to address harm from childhood
maltreatment.
Dr Philip Gillingham, Senior Lecturer in Social Work, University
of Queensland
14

15. Future developments for PRM
PRM is possible…
Need more accurate datasets to develop reliable algorithms…
Create new datasets within information systems, with PRM in mind…
Separate categorizations for actual maltreatment, at risk, siblings…
Dr Philip Gillingham, Senior Lecturer in Social Work, University
of Queensland
15

16. Reforming child protection services
More emphasis on addressing past maltreatment
Addressing poverty – more than child protection services reform
Dr Philip Gillingham, Senior Lecturer in Social Work, University
of Queensland
16

17. Questions.
Gillingham, P. (2015) Predictive risk modelling to prevent child
maltreatment and other adverse outcomes for service users: inside the
“black box” of machine learning. The British Journal of Social Work.
Published online 9 April 2015. DOI: 10.1093/bjsw/bcv031
Gillingham, P. (under review) Predictive Risk Modelling to Prevent Child
Maltreatment: Insights and Implications from Aotearoa/New Zealand
p.gillingham@uq.edu.au
Dr Philip Gillingham, Senior Lecturer in Social Work, University
of Queensland
17

18. Additional References
Lonne, B., N. Parton, J. Thomson, and M. Harries. 2008. Reforming Child Protection. London: Routledge.
MSD (2010) Sole parenting in New Zealand: An update on key trends and what helps reduce disadvantage.
Centre for Social Research and Evaluation. Te Pokapū Rangahau Arotake Hapori. Ministry of Social
Development, Wellington. Accessed August 2015 at https://www.msd.govt.nz/documents/about-msd-and-our-
work/publications-resources/research/sole-parenting/sole-parenting-in-nz-an-update-on-key-trends-and-
what-helps-reduce-disadvantage.pdf
Dr Philip Gillingham, Senior Lecturer in Social Work, University
of Queensland
18