The 8th Annual National Mentoring Summit Washington, DC Workshop on Advanced Methods in Youth Mentoring Research

1. Mixed Methods as a Tool in Mentoring
Research
Nancy L. Deutsch
YOUTH-NEX: The UVA Center to Promote Effective Youth
Development
University of Virginia – Curry School of Education
Workshop on Advanced Methods in Youth Mentoring Research
Wednesday January 24, 2018

2. Session Outline
• Overview of Mixed Methods (15 min.)
• What is it?
• Strengths
• Principles and Designs
• Applications of mixed methods to Mentoring Research (15 min.)
• Study of the Young Women Leaders Program
• Considerations & Challenges
• Resources
• Q & A (10 min.)
January 24, 2018 Workshop on Advanced Methods in Youth Mentoring Research

3. What is Mixed Methods?
January 24, 2018 Workshop on Advanced Methods in Youth Mentoring Research
• Numeric vs. non-numeric ways of representing the world
• Strengths and weaknesses of each
• Complexity of goals of mentoring research
“the combination of words and numbers can bring us closer to the complexity of
developmental change by providing divergent as well as convergent data”
(Yoshikawa, Weisner, Kalil & Way, 2008, p. 345).
“A Baconian search for first-order interactions is no more sensible a strategy than the
empirical searches limited to main effects that I had criticized. Human events reflect
innumerable interactions among planned and unplanned features of situations, so
subtler observation of the interplay between person characteristics and situational
variables is needed. Where once I had called for teaming experimental and
correlational methods to pin down more general and hence more lasting truths, I
now proposed greater reliance on rich description, recognizing that a social-science
"conclusion" is only a record of an event at a moment in history.” [Cronbach, L. J.
(1989). Lee J. Cronbach. In G. Lindzey (Ed.), A history of psychology in autobiography Vol. VIII
(pp. 62–93). Stanford University Press: Stanford University Press, pg. 86]

4. Mixed Methods: Drawing on strengths of
Quant & Qual data and analytic techniques
• Quantitative Studies
• Seek generalizability
• Control for variables that are associated with context
• Positivist & post-positivist paradigms
• Qualitative Studies
• Examines qualities, meanings, and processes within natural
settings
• Exploring the construction of meanings rather than measuring
their quantity
• Transferability, not generalizability
• Interpretive and constructivist paradigms
January 24, 2018 Workshop on Advanced Methods in Youth Mentoring Research

5. The Black Box of Mentoring
Discover. Create. Change.
Mentoring Program
Mentoring Relationship
Youth Background
and
Life Outside
Mentoring Program
Youth
Outcomes

6. Principles behind Mixed Methods designs
• Level of Integration (or mixing) of the methods
• partial or full
• Order in which methods are used
• concurrently or sequentially
• Priority given to one method in relation to each other
• which method is dominant
• Goals of the method with regards to the research questions
• e.g., exploration, confirmation, triangulation, etc.
January 24, 2018 Workshop on Advanced Methods in Youth Mentoring Research

7. Integration issues (Cresswell & colleagues, 2003):
1) the research questions/purpose (e.g., confirmation vs. exploration)
2) the type of data collection (e.g., numeric vs. non-numeric data)
3) the intent of analysis (e.g., descriptive or inferential statistics vs.
descriptive thematic analysis)
4) interpretation of the results (e.g., generalization vs. particularization)
5 major purposes (Greene, Caracelli and Graham, 1989):
1. Triangulation
2. Complementarity
3. Development
4. Initiation
5. Expansion
Issues in Integration and Goals
January 24, 2018 Workshop on Advanced Methods in Youth Mentoring Research

8. January 24, 2018 Workshop on Advanced Methods in Youth Mentoring Research
• QUAL + QUAN
• qual + QUAN
• QUAL + quan
• QUAL QUAN
• qual QUAN
• QUAL quan
• QUAN QUAL
• quan QUAL
• QUAN qual
Mixed Methods Designs
• Sequential explanatory
• Sequential exploratory
• Concurrent triangulation
• Concurrent nested
• Transformative

9. Analysis
Discover. Create. Change.
Separate Integrated

10. Typology of Analysis Techniques
(Teddlie & Tashakkori, 2009)
• Parallel
• Conversion
• Sequential
• Multilevel
• Fully Integrated
• Application of analytical techniques from one tradition to the
other
Discover. Create. Change.

11. Application in Mentoring Research
Study of the Young Women Leaders Program funded by the William T. Grant
Foundation and the U.S. Department of Education
This study was then followed up on 5 with a mixed methods longitudinal study
funded by OJJDP that followed up with the original study participants 5 years later
1) Is participation in YWLP associated with improvements in academic, social, and
mental health outcomes
2) How does the mentoring group function as a context for the development of one-
on-one mentoring relationships
3) What processes occur within the group that contribute to the development of
academic and psychosocial competencies for girls
4) How does the mentoring group function as a context for self and identity
development
5) What group characteristics/resources (composition, facilitator skills) and social
processes (types/qualities of interactions) are associated with positive academic,
social, and mental health outcomes for girls
January 24, 2018 Workshop on Advanced Methods in Youth Mentoring Research

12. • RCT with embedded qualitative component
• Pre and post program surveys
• Mentees
• Control group
• Mentors
• School records (mentees and control group)
• Observations of mentoring groups
• Post-program interviews (mentees and mentors)
• Three levels of mixed methods analysis: Individual, Group, Dyad
January 24, 2018 Workshop on Advanced Methods in Youth Mentoring Research
Application in Mentoring Research

13. Application in Mentoring Research - Individual Level Outcomes - Methods
Research Questions
• Does YWLP have an impact on girls’ academic or socioemotional functioning?
• Mixed Methods design: QUANT QUAL or QUANT qual
Methods
• Quantitative
• 347 Girls from the 2007-2008, 2008-2009, and 2009-2010 school years (T=207; C =143)
• Pre and Post program surveys
• Academic Functioning
• Self-report of grades; School Bonding (Hawkins et al, 1999); School Self-Esteem
(Self-Esteem Questionnaire [SEQ], DuBois et al, 1996)
• Social Functioning
• Peer Self-Esteem Esteem (SEQ, DuBois et al, 1996); Family Self-Esteem Esteem
(SEQ, DuBois et al, 1996); Global Self-Esteem (SEQ, DuBois et al, 1996)
• Qualitative
• 113 Mentees from 2008-2009 and 2009-2010
• Semi-structured post-program interviews
• Data coded in Nvivo for changes girls made and mechanism of
change reported

14. Application in Mentoring Research: Individual Level Outcomes - Findings
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Total (n=113)
attributed to reasons other than
YWLP
Qualitative Findings
Quantitative Findings
CODE Mentors
Mentoring
Group
YWLP
Curriculum
Academics (n=41)
16
(39%)
8
(20%)
8
(20%)
Relational Development
(n=85)
24
(28%)
44
(52%)
28
(33%)
Self-regulation (n=75)
26
(35%)
20
(27%)
29
(39%)
Self-Understanding
(n=75)
29
(39%)
29
(39%)
11
(15%)

15. Application in Mentoring Research: Group Level Processes - Methods
Questions
• Are there differences between mentoring groups in the mentees’ and mentors’ group satisfaction?
• Are there differences between mentoring groups in the mentees’ relational satisfaction?
• If differences exist, what relational processes (patterns of connection and disconnection) occur within the
groups that may influence group and relationship satisfaction?
• Mixed Methods Design: quant QUAL
Methods
• Quantitative
• Surveys of mentors (n=56) and mentees (n=78)
• Mentees’ satisfaction with 1-on-1 relationships (Rhodes, Reddy, Roffman, & Grossman, 2005)
• Mentor and Mentee satisfaction with group (Adapted version of Allen, Kuperminc, Philbur, & Herre’s
(1994) autonomy and relatedness in program scale)
• ANOVAs examined group level differences (PostHoc Tukey comparisons)
• Qualitative
• Observations of mentoring groups
• All field notes coded in NVivo for Connection & Disconnection
• Open coding of data from 4 highest and lowest satisfaction groups for themes

16. January 24, 2018 Workshop on Advanced Methods in Youth Mentoring Research
Application in Mentoring Research: Findings - Group Level Processes - Findings

17. January 24, 2018 Workshop on Advanced Methods in Youth Mentoring Research
Application in Mentoring Research: Findings - Group Level Processes - Findings

18. % of interviews in
which the theme
appeared at least
once
% of references reported by mentors
versus protégés in high and low
satisfaction relationships
Code
Overall
High
Overall
Low
Mentor
High
Protégé
High
Mentor
Low
Protégé
Low
Attunement 69% 50% 54% 46% 75% 25%
Authenticity 69% 50% 46% 54% 75% 25%
Collaboration 50% 50% 38% 62% 63% 37%
Companionship 56% 50% 62% 38% 75% 25%
Empathy 23% 30% 50% 50% 100% 0%
Identification 56% 56% 55% 45% 78% 22%
Misattunement 18% 93% 100% 0% 60% 40%
• Analyzed post-program interviews dyadically (n= 72 mentors; 72 mentees)
• Misattunement differentiated high and low satisfaction relationships
• Mentors in high satisfaction relationships indicated misattunement but
mentees did not
• Those mentors were more self-reflective about the misattunement than
mentors in low satisfaction relationships
Application in Mentoring Research: Dyadic Level

19. • Expertise needed on research team
• Breadth and depth
• Sample Size
• Balancing of resources
• Resource intensive
Considerations & Challenges
January 24, 2018 Workshop on Advanced Methods in Youth Mentoring Research

20. • Dedoose Software
Resources
January 24, 2018 Workshop on Advanced Methods in Youth Mentoring Research
• Creswell, J. W. (2013). Research design: Qualitative,
quantitative, and mixed methods approaches. Thousand
Oaks: Sage Publications.
• Creswell, J. W., & Clark, V. L. P. (2007). Designing and
conducting mixed methods research. Thousand Oaks, CA:
Sage Publications.
• Tashakkori, A. & Teddlie, C. (2003). Handbook of
mixed methods in social & behavioral research. Thousand
Oaks, CA: Sage Publications, Inc.
• Teddlie, C. & Tashakkori, A. (2009). Foundations of Mixed
Methods Research. Thousand Oaks, CA: Sage Publications.
• Tolan, P. & Deutsch, N.L. (2014). Mixed Methods in
Developmental Science. In W. F. Overton & P. C. Molenaar
(Eds.), Theory and Method. Volume 1 of the Handbook of
child psychology and developmental science (7th ed.).
Editor-in-chief: R. M. Lerner. Hoboken, NJ: Wiley.
• Yoshikawa, H., Weisner, T. S., Kalil, A., & Way, N. (2008).
Mixing qualitative and quantitative research in
developmental science: Use and methodological choices.
Developmental Psychology, 44, 344-354. doi: 10.1037/0012-
1649.44.2.344.

21. Q & A
January 24, 2018 Workshop on Advanced Methods in Youth Mentoring Research