From decisions based on intuition
to data-informed decision making
Factors hindering the functioning
of a data team® in higher eduction
Erik Bolhuis, Windesheim University of Applied Sciences, The Netherlands Email: email@example.com.
Joke Voogt, University of Amsterdam & Windesheim University of Applied Sciecnes, The Netherlands. Email: firstname.lastname@example.org
Kim Schildkamp, University of Twente, The Netherlands. Email: email@example.com
Contact details: drs. E.D. Bolhuis, postbus 217, 7500 AE Enschede, The Netherlands. email: firstname.lastname@example.org.
• Context of the research
• Research questions
• Theoretical framework
• Interactive section
• Increase data (-use) in education (OECD,
• Teacher Education Colleges —> data use:
accountability, part of the curriculum
• Knowledge gab: TE —> data use for
school- & instructional improvement
Information that is collected and organized to represent some
aspects of the school (Lai & Schildkamp, 2013, p.10).
▪ Input data: e.g. gender, previous school;
▪ Outcome data: e.g. assessments results, written and oral
exams, portfolio’s, classroom observations, student surveys,
parent interviews, assessment results
▪ Process: e.g. the curriculum,
▪ Context data: eg. data
on school culture
Ways of data use in education / examples of data:
drop- out rates
Drop-out rates, test results,
questionnaires, results form
Test results (formative and
A data team is:
• Teams 6-8 teacher educators and a school leader
• Educational problem: grade repetition, low
• Goals: professional development and school
• Coach guides them through the eight steps (two
• Data analysis courses
• Dropout in the first study year (HE). In the first year drop-out rates from 55% to
• Question: what causes drop-out? Is this related to previous education? To
gender? To the atmosphere in the class (ambitious study climate)?
• Data: test results, questionnaire (students and supervisors), and the curriculum
• Based on the data, they conclude and develop measurements
Depth of inquiry:
More successful teams (i.e. higher student learning gains) —>
more higher level thinking skills (Achinstein 2002; Stokes
2001) —> conversation with a high depth of inquiry.
The depth of inquiry = inquisitive attitude developing new
knowledge and taking action based on data, while reviewing
each step of the procedure critically (Henry 2012).
The conversations —> reasoning, listening, and underpinning
assumptions. Fundamental for making measurements for
improvement, and to the construction of team- and individual
knowledge (Ikemoto & Marsh 2007).
Depth (Henry, 2012)
Depth Participating How Results
No depth Individuals
Sending information No shared knowledge
experiences, and sources
No shared knowledge base and/or
All members are
Actively create a new
No actively test and sharpen this
Depth All members
The discussion focuses
The discussions are not shallow and
lead to a shared explicit knowledge
base. Characteristically the
dialogue is based on concrete
research and/or data.
From literature we know factors influencing data-use
(Schildkamp & Kuipers, 2010)
Which factors enable and constrain depth of inquiry
within the data team?
1. Which factors with regard to data and data information systems
enable and prevent depth of inquiry of data team conversations?
2. Which factors on the level of the user enable and prevent depth
of inquiry of data team conversations?
3. Which factors with regard to the assistance of the data team
enable or prevent the depth of inquiry of data team
1. Which factors are hindering and promoting factors
affecting the depth of the conversations in a data
2. Which factors cause drop-out in first year TE?
•A single-case study: micro-process study
•The data team procedure: 19 meetings in 2 year
•Respondents: The data team, the management
•Instruments: observations of the meetings (taped on audio,
verbatim transcript), documents of the data team and artefacts
•Analyse: coding according to a codebook, analyze in TamsAnalyzer,
analyzed by a pattern matching-and time series strategy (Yin, 2014)
•A within- and a cross-case analyses
•Quality of the study: Kappa Cohen's of 0.79.
Factors influencing data-use
Factors related to data and data-information systems:
Data-information system which provides timely, accurate, relevant,
reliable and valid data, data which coincides with the needs
Data related to the perception of the data team members
Factors related to the user:
Data literacy, buy-in/belief, ownership and locus of control
Being able to handle cognitive conflicts
Clarify prior knowledge
Avoid affective conflicts.
Factors related to the organization:
Support from the data coach e.g. conversations skills
1.Data —> relate to the level of data literacy
2.Stimulating really use data
3.Clarify prior knowledge;
4.Learn from cognitive conflicts —> clarify which knowledge is
conflicting —> manage confusion —> restructure knowledge base;
5.Avoid affective conflicts: but if they do arise, make sure the conflict
can be addressed;
6.Data coach —> get insight level of data literacy —> present the
data that relate to this level —> and intervene in the conversations
to ensure the data team works on a knowledge base together
• The use of data in the teacher education curriculum,
requires teacher educators, who can improve
education based on data;
• Data-use requires active and explicit knowledge-
building. Integrating Theory and theory. Should PD
pay attention to this process?
• The data coach —> supporting the data team as a
team, but also coach to use data to improve their
Which factors cause drop-out?
• Gender? (Not found)
• Atmosphere of the class (Rejected)
• Academic skills (Confirmed)
• —> They accompanied the hardest module with a study course
• Contrasting test schedule —> management making the schedule
• Modules with different test components —> one component
• Climate in the first year (best [pedagogical] teachers in the first year
• Monitoring student progress based on data
Achinstein, B. (2002). Conflict amid community : The micropolitics of teacher collaboration, 104(3), 421–455.
Bernhardt, V.L. (2004). Continuous improvement: It takes more than test scores. Leadership Magazine, 34(2), 16-19.
Bernhardt, V.L. (2005). Data tools for school improvement. Educational Leadership, 62(5), 66-69.
Carlson, D., Borman, G. D., & Robinson, M. (2011). A multistate district-level cluster randomized trial of the impact of data-
driven reform on reading and mathematics achievement. Educational Evaluation and Policy Analysis, 33(3), 378-398.
Earl, L., & Katz, S. (2006). Leading schools in a data-rich world: harnassing data for schoolimprovement. Thousands Oaks, CA:
Henry, S. F. (2012). Instructional Conversations : A Qualitative Exploration of Differences in Elementary Teachers’ Team
Discussions. Dissertation at Harvard University.
Ikemoto, G.S., and J.A. Marsh. 2007. “Cutting Through the ‘Data-Driven’ Mantra: Different Conceptions of Data-Driven
Decision Making.” In Yearbook of the National Society for the Study of Education, edited by P.A. Moss.
Lai, M.K., & Schildkamp, K. (2012). Data-based decision making: An overview. In: Schildkamp, K., Lai, M.K., & Earl, L. (Eds.),
Data-based decision making in education: Challenges and opportunities. London: Springer.
Schildkamp, K., & Kuiper, W. (2010). Data-informed curriculum reform: Which data, what purposes, and promoting and
hindering factors. Teaching and Teacher Education, 26(3), 482–496.
Schildkamp, Poortman, K. C. & Handelzalts, A. (2015). “Data teams for Schoolimprovement.” School effectiveness and School
Improvement. Advanced Online Publication.
Stokes, L. (2001). Lessons from an inquiring school: Forms of inquiry and conditions for teacher learning. Teachers caught in
the action: Professional development that matters, 141-158.
Yin, R.K. (2014). Case study research: Design and methods (5th ed.). London: Sage.