Evaluation of Youth Development Programme into Young Black Males

1. EVALUATE TO
EDUCATE
E X P L O R I N G T H E E D U C AT I O N A L D E V E L O P M E N T O F
Y O U N G B L A C K M E N T H R O U G H A C A S E S T U D Y
E VA L U AT I O N

2. OVERVIEW OF PRESENTATION
 Context & Background
 The Case Study
 The Benefits of Academic Evaluation for Educational Development Research
 Study Approach
 The Research Perspective
 Preliminary Thoughts
 Discussion
© Bianca Bailey Wilson

3. INTRODUCTION – ABOUT ME
 Final year research postgraduate within Education Office/EDU and School
Professional Development
 Background in linguistics, focus on sociolinguistics and urban youth
language (particularly black Caribbean communities)
 3.5 years Imperial College funded research project in educational
development of young black men
 Evaluation of Amos Bursary programme
 Relevance for HE & educational policy, BME students at Russell Group
Institutions
© Bianca Bailey
Wilson

4. PHD STUDY OVERVIEW
 Community-based evaluation focusing on
the influence of the programme on the
developmental pathways of the young
men;
 Theoretical framework based in work of
Bronfenbrenner’s bioecological model of
human development (bio-psycho-social-
cultural-historical perspective on human
development);
 Developmental outcome of focus -
Bandura’s Self-Efficacy theory (Bandura,
1995);
 Focuses on the role of the environment in
individual development, not just interested
in the what happens but the processes
behind why things happen
A developmental ecological case study exploring the educational and career
experiences of young black men involved in the Amos Bursary.
© Bianca Bailey Wilson
© Bianca Bailey Wilson

5. PHD CASE STUDY - THE AMOS
BURSARY
 AB selects academically able young black men at the age of 16/17 to take part
in their personal and professional development programme.
 Young men supported right through to university graduation (average of 4.5
years).
 Selective process based on social contextual factors such as household
income, FSM, limited higher education attendance in the family.
 Supported by private donors, educational and corporate support, incl. UCL and
Imperial College.
© Bianca Bailey Wilson

6. EVALUATION FRAMEWORK
 Instrumental case study approach;
 Allows the study to operate at two levels, the organisational level and the
individual level (the young black men);
 Interviews - Amos supporters/funders, alumni;
 Observations – Events, workshops, seminars;
 Focus group – 45 students from the Bursary;
 PIPA – Participatory Impact Pathways Analysis Method Workshops
 Biographical approach to interviews – Interactivity and Subjectivity through three
interviews including Tree of Life interview (Bornat, 2008).
 Observations of events, workshops, and recruitment process.
© Bianca Bailey Wilson

7. USING GRAPHIC METHODS TO EXPLORE & ELABORATE ON
PROGRAMME THEORY – WORKING WITH THE PIPA
METHOD
The PIPA (Participatory Impact Pathways Analysis) method was originally developed as a
practical monitoring, evaluation, and planning approach for complex projects in the food
and water sectors (Douthwaite et al, 2008a).
PIPA was developed to allow project evaluators to explore the underlying assumptions
and theories of change behind programmes, as well as incorporate the experiences and
voices of a range of stakeholders.
PIPA is primarily a three part process, which is often done over two to three days.
Day 1: Involves a participatory workshop where participants (stakeholders) outline explicitly what their
assumptions are about how the project will make/has been designed to make impact.
Days 2 & 3: It then moves on to looking at their networks, and finally to developing an outcomes logic
logic model and a future strategy (Douthwaite et al, 2008a).
With its participatory nature, PIPA uses a range of graphic tools to engage stakeholders
and explore their underlying assumptions and beliefs on the programme of focus.
© Bianca Bailey Wilson

8. PROBLEM TREE EXERCISE
The construction of a problem tree is vital to the PIPA process, as its starts with stakeholders
creating their trees to clarify what they think the cause and effect logic of their programme is
(Douthwaite et al, 2008a).
This starts with participants identifying what they think the root cause of the problem they are
addressing in society is. It then moves sequentially from this root cause towards the problems that
they are actually addressing (Douthwaite et al, 2008a; 2008b; 2010).
Problem tree – young black male groupProblem tree exercise – board group
© Bianca Bailey Wilson

9. VISIONING & NETWORK MAPPING EXERCISES
The visioning exercise allows stakeholders to
think about project success and outcomes into the
future
The network mapping gets stakeholders to map
out who they work with now and who they feel
they need to work with in the future to meet their
vision. © Bianca Bailey Wilson

10. POINTS TO CONSIDER
Organisation of groups
It is important to organise groups beforehand, to ensure that
you get as authentic as possible problem trees.
Colour coded badge system worked well for tasks
Managing dominant voices
These activities can cause debate and minor conflict.
It is important to manage dominant voices such as programme
leaders and board members, who can sometimes overpower
other volunteers and project participants (young black men).
© Bianca Bailey
Wilson

11. EXPLORING PARTICIPANT VOICE THROUGH
STORY – NARRATIVE THERAPY
Narrative therapy overview
 Narrative therapy was developed by Michael White and David Epstein
over twenty years ago;
 It is a branch of psychotherapy which focuses on externalising problems
and allowing people to own their life stories.
 ‘Who we are and what we do are influenced by the stories we tell about
ourselves’ (Denborough, 2014: p. 3)
 ‘The person is not the problem, the problem is the problem’ Michael
White, cited in Denborough, 2014: p. 9).
© Bianca Bailey Wilson

12. SUPPORTING UNHEARD YOUNG BLACK MEN TO TELL THEIR
STORIES – STORYTELLING RIGHTS
Storytelling Rights (Denborough, 2014: p. 9)
 Article 1: Everyone has the right to define their experiences and problems in their own words and terms.
 Article 2: Everyone has the right to have their life understood in the context of what they have been
through and in the context of their relationships with others.
 Article 3: Everyone has the right to invite others who are important to them to be involved in the process
of reclaiming their life from the effects of hardship.
 Article 4: Everyone has the right not to have problems caused by trauma and injustice located inside
them, internally, as if there were some deficit in them. The person is not the problem, the problem is the
problem.
 Article 5: Everyone has the right to have their responses to hard times acknowledged. No one is a
passive recipient of hardship. People always protest injustice.
 Article 6: Everyone has the right to have their skills and knowledge of survival respected, honoured, and
acknowledged.
 Article 7: Everyone has the right to know and experience what they have learned through hard times can
make a contribution to the lives of others in similar situations.
© Bianca Bailey Wilson

13. THE TREE OF LIFE
 Tree of life is a graphic elicitation tool that allows participants to
reflect on and speak about their lives.
 It was created in our second interview, and then served as a
reference tool for further discussions, in the final interview.
 The Tree of life is a method used to encourage (vulnerable)
individuals to find their voice and explore their individual lives
and journeys (Denborough, 2014).
© Bianca Bailey Wilson

14. RICH PARTICIPATORY INTERVIEWS WITH THE TREE OF LIFE
Participants draw and tree and illustrate
in a particular way:
 Ground = where you are now, activities
you enjoy, where you live;
 Roots= heritage, family, culture, faith,
favourite childhood activities, significant
teachers/educators;
 Trunk= your skills and abilities
 Leaves = significant people/figures both
past and present (can be fictional or
real);
 Fruit = gifts you have been given by
others (material and non-material);
 Flowers= Gifts you have given others. © Bianca Bailey Wilson

15. POINTS TO CONSIDER
Organisation and location
 This task needs at least one hour and cannot be rushed
 Appropriate location for drawing and for participant to feel comfortable –
especially young people
Emotive process
 Be prepared for detailed exploration of the individual’s life – emotional
connection
 Respect and trust is vital – they are entrusting you with their story, Article 1
and 2
 Article 1: Everyone has the right to define their experiences and problems in
their own words and terms.
 Article 2: Everyone has the right to have their life understood in the context
of what they have been through and in the context of their relationships with
others. © Bianca Bailey Wilson

16. 3E MODEL FOR EVALUATION INSIGHTS
Elicit
Elicit authentic responses using arts-based methods such as the Tree of life, Problem tree,
relational maps.
Explore
Explore individual life stories through the materials produced.
Explore conceptions of impact and change, and underlying assumptions through the materials
produced.
Elaborate
Elaborate on key people, places, and organisations in the individual’s social ecology through
respectful discussion.
Elaborate on implementation and programme processes through respectful discussion.
Evaluation
insights
Authentic
Meaningful
Co-created
Co-owned
© Bianca Bailey Wilson

17. CITATIONS
Bandura, A. (1977) Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review. 84 (2), 191–215.
Bandura, A. (1986) Social foundations of thought and action: A social cognitive theory. U.S: Prentice-Hall, Inc.
Bandura, A. (1995) Exercise of personal and collective efficacy in changing societies. In A. Bandura (Ed.) Self-efficacy in changing societies, (pp. 15, 334). Cambridge, UK:
Cambridge University Press.
Bertschinger, E. (2015) Advancing a Respectful and Caring Community: Learning by doing at MIT. Cambridge, MA: Massachusetts Institute of Technology.
Boliver, V. (2013) How fair is access to more prestigious UK universities. British Journal of Sociology, 64(2), pp.343 -364.
Bronfenbrenner, U. (1977) Toward an experimental ecology of human development. American psychologist, 32(7), 513.
Bronfenbrenner, U. (1986) Ecology of the family as a context for human development: Research perspectives. Developmental psychology, 22(6), 723.
Bronfenbrenner, U. (1979) The ecology of human development: Experiments by nature and design. Cambridge, MA: Harvard University Press.
Bronfenbrenner, U. (2005) Making human beings human: Bioecological perspectives on human development. California, U.S: Sage Publications.
Bronfenbrenner, U., & Morris, P. A. (2006) The bioecological model of human development. Handbook of child psychology. In D. William & R. Lerner (Eds.) Handbook of child
psychology (6th ed.) (pp. 793-828). NJ, U.S: John Wiley & Sons
CoDe. (2013) Addressing Ethnic Inequalities in Social Mobility: Research findings from the CoDe and Cumberland Lodge Policy Workshop. Centre on Dynamics of Ethnicity,
Manchester University: UK.
© Bianca Bailey Wilson

18. CITATIONS
Department for Education. (2013) Permanent and fixed period exclusions from schools and exclusion appeals in England,
2011/12. Department for Education Statistical First Release. London: UK. Available at:
https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/224893/SFR29-2013.pdf [Accessed May 23,
2014].
Harper, S. R. (2012) Black male student success in higher education: A report from the National Black Male College
Achievement Study. Philadelphia: University of Pennsylvania , Center for the Study of Race and Equity in Education. Available
at: https://www.gse.upenn.edu/equity/sites/gse.upenn.edu.equity/files/publications/bmss.pdf
Law, I. Finney, F. & Swann, J. (2012) Searching for autonomy: young black men, schooling and aspirations. Race, Ethnicity, and
Education, pp.1-22.
Office for National Statistics (2012) Annual Population Survey. Available at:
http://www.ons.gov.uk/ons/search/index.html?newquery=annual+population+survey [Accessed June 18, 2014].
Milburn, A. (2012) University Challenge: How Higher Education Can Advance Social Mobility. A progress report by the
reviewer on Social Mobility and Child Poverty. The National Archives. Kew: UK
Zimmerman, B. J. (1995) Self-efficacy and educational development. In A, Bandura (Ed.), Self-efficacy in changing societies,
(pp. 202-231). Cambridge, UK: Cambridge University Press.
© Bianca Bailey Wilson

19. CONTACT
I welcome your thoughts and feedback.
Contact details
Bianca Bailey Wilson
www.biancabaileywilson.net
Bianca.seb@gmail.com
Twitter @LnDnWomanBianca
© Bianca Bailey Wilson