Data in Education: Panacea or problem

Data in Education: Panacea or
Problem
Dai Griffiths
Professor of Education, The University of Bolton
January 15th
, 2018
d.e.griffiths@bolton.ac.uk

Data in Education: Panacea or
Problem
Or “Why should an educational
researcher care about technology and
economics?”
An old qu

Technology: ‘Data’ and ‘Big Data’?

Who has ‘big’ data?
● Which of these has ‘big’ data
– Apple
– Spotify
– Craig Venter
– Ministerio de Education
– El Ayuntamiento de Salamanca
– Comunidad Autonoma de Madrid
– EdX
– El Corte Inglés
– CERN
– Facebook
– La Universidad de Salamanca
● How would you charactarise ‘big’

What is involved in ‘big’ data
● Just moving this data from one place
to another is a huge challenge
● Storing and searching it with
relational databases is not feasible
● New technologies have had to be
developed to deal with this, like
Apache Hadoop

Data created or replicated per year

What does that indicate? No one is
really sure, but...
● About 15 zetabytes
● 1 zetabyte = a trillion gigabytes
(i.e. 1,000,000,000,000)
World population = 7,500,000,000
● Something towards 2000 gigabytes
per person per year?

Every minute on the internet
● Are there more:
– Google searches?
– WhatsApp messages?
– Emails?
● Any idea of the proportions?
● Any idea of the largest figure?

Wow! But why should
educationalists care?

An old quote, but still relevant
Technology is neither good nor bad; nor is it
neutral...technology’s interaction with the
social ecology is such that technical
developments frequently have environmental,
social, and human consequences that go far
beyond the immediate purposes of the
technical devices and practices themselves.
Melvin Kranzberg (1986, p. 545)
Kranzberg, M. (1986) Technology and History:
Kranzberg's Laws, Technology and Culture vol. 27,
no. 3, pp. 544-560
An old qu

So what are the ‘social and human
consequences’ for education?
● We don’t usually have Big Data in
Education
● But this is changing (or being
changed)
● The achievements and methods of
Big Data are being applied to
Education.

Bricks to Clicks Higher Education
Commission UK, 2016 (adapted)
● Worldwide, organisations are realising the
significant value of big data and using data
analytics to improve business.
● Tesco’s Clubcard allows the supermarket chain
to collect an enormous amount of data about
their customers.
● Data used not only to target individual
shoppers – through personalised newsletters
and offers but also to inform broader strategy
● Tesco is also using this data in predictive
analytics to forecast how many products will
be sold when and where.

… and now for education
● The commission believes that:
– The UK HE sector currently possesses a rich
and vast amount of data, but is not making
the most effective use of this valuable
resource.
– The sector should seize the opportunities
that data and analytics presents
immediately.
● But what are the opportunities?

George Siemens
“Education is, today at least, a black box. ... We
don’t know, precisely, which academic practices
need to be curbed and which need to be encouraged.
…
We need a means, a foundation, on which to base
reform activities. In the corporate sector, business
intelligence serves this “decision foundation” role.
In education, I believe learning analytics will serve
this role. Once we better understand the learning
process — the inputs, the outputs, the factors that
contribute to learner success — then we can start to
make informed decisions that are supported by
evidence.”
https://www.oreilly.com/ideas/education-data-analytics-learning

Delivering education
Michael Barber: Prime Minister's Delivery Unit under Tony Blair;
McKinsey’s Global Education Practice; Chief Education Advisor, Pearson
When we take a pill, we expect to feel better. When we hire a car
mechanic, we expect our car to work again. Similarly, when a child goes to
school, she and her parents expect certain outcomes. When a student
goes to university, he expects to learn, grow, and develop. When an adult
receives skills training, she expects to enhance her job prospects.
With our drive towards efficacy, we want to help deliver on those
expectations by continuously improving every single one of our products
On the Road… to Delivering Learner Outcomes
http://efficacy.pearson.com/wp-
content/uploads/2015/03/Pearson_OntheRoad_SPREADS_190315.pdf

We have new ways to research into
learning conversations
http://www.moodlenews.com/2013/the-social-networks-adapting-pedagogical-practice-snapp-tool-and-
visualizing-moodle-forums/

And new data and algorithms for
adaptive learning
https://www.slideshare.net/PaymentsandCommerce/brighterion-overview

Making MOOCs more personalised
● For a Coursera course on Calculus
Fowler and Evans created an adaptive
add-on, called MOOCulus.
● “...the tool is designed to feed
students progressively harder
questions based on previous answers.”
● "Learning calculus involves doing math,
lots of math.... We wanted problems
that would react to students."
https://campustechnology.com/articles/2014/05/14/enhancing-a-mooc-with-adaptive-learning.aspx

Your experience
● What good experiences have you
had of the use of data?
● In education
● In your lives
● Do you have any hopes for the use
of data, in research or in practice?

Has big data created a new
methodology?
● “Out with every theory of human behavior, from
linguistics to sociology. Forget taxonomy, ontology,
and psychology. Who knows why people do what they
do? The point is they do it, and we can track and
measure it with unprecedented fidelity. With enough
data, the numbers speak for themselves.”
●
In other words, correlation is enough.
● This was a polemic, but I’ve seen the results in action
Anderson, C. (2008) The End of Theory, Will the Data Deluge
Makes the Scientific Method Obsolete
http://www.edge.org/3rd_culture/anderson08/
anderson08_index.html

Some problems with Big Data
methods
● Does not use direct measures
– Recidivism (see Kelly, Weapons of Math Destruction)
● In assessing sentence length, the LSI-R instrument in the US
uses indicators such as ‘first contact with the police’ and
‘Family/Marital situation.
● The model itself contributes to a toxic model that helps sustain
it
– Use of the library (surrogate for learning engagement)
– Address (surrogate for financial status)
● Data coverage may be patchy over the
population, and not updated
● Data sets are usually private

Economics
● Like technology, economic and
policy changes have generated huge
forces which are impacting
education
● The use of data in education is tied
in with an educational evidence-
based reform agenda in a complex
relationship with neoliberal
economics and politics

Total student debt in the USA
● How many zeros do I need to add
$1,410,000...

Total federal outstanding student
loans in the USA
● How many zeros do I need to add
$1,410,000,000,000

National student loan
debt
$1,410,000,000,000
Private student loan
debt
$11,600,000,000
Total student debt $1,521,600,000,000
Total credit card
debt
$620,000,000
Total mortgages (Q3) $10,100,000,000,000
Total borrowers
(national & private
46,970,000
https://studentloans.net/student-loan-debt-statistics/

Yes, that is a LOT of money
● If you spent one dollar per second
– $86,400 per day
– $31.5 million per year
– At that rate of spending, it would take you over
32,000 years to spend one trillion dollars

Is this a bubble?
● If so, what would burst it?
– Questioning that learning can be accurately measured, bundled,
packaged and delivered.
● What could keep it going? Further neoliberal
reform
– More commodification of educational products for purchase
through a market
– Quantification of the learning and benefits to be sold, making use
of data
● What opportunities does it offer educational
publishers and service providers?
– Offer to cut costs by providing assessment services & interactive
teaching, and establish private schools and universities

Hayek, as a representative of
neoliberalism
...If you need prices, including the
prices of labor, to direct people to go
where they are needed, you cannot
have another distribution except the
one from the market principle. I
think that intellectually there is just
nothing left of socialism.
Interview with Thomas W. Hazlett in May of 1977, as published in
"The Road to Serfdom, Forseeing the Fall", in Reason magazine
(July 1992). http://reason.com/archives/1992/07/01/the-road-from-
serfdom/3

Managerialism in the public sector
“...driven by an emphasis on target-
setting, performance review, and the
use of incentives and sanctions to
reward appropriate behaviours and
punish inappropriate behaviour or
what is deemed poor performance.”
Stevenson, H. & Wood, P., 2013. Markets, managerialism and
teachers’ work: the invisible hand of high stakes testing in
England. The International Education Journal: Comparative
Perspectives, 12(2), pp.42–61.
https://openjournals.library.sydney.edu.au/index.php/IEJ/article/vi
ewFile/7455/7814
p.50

The rationale for market
information
‘Consumers’ require market information in order to
make rational choices. Published test results, ranked
in league tables, facilitate ‘like-for-like’ comparison,
whilst open enrolment allows parents to exercise
choice.
● Formula-funding, driven overwhelmingly by pupil
numbers, ensures that high performing schools
generate large numbers of parental preferences, and
with them additional resources.
● Schools ranked lower in the league tables are likely to
attract fewer parental preferences and hence, face
falling rolls, and diminishing budgets.
Stevenson, H. & Wood, P., 2013. Markets, managerialism and teachers’ work: the invisible hand of
high stakes testing in England. The International Education Journal: Comparative Perspectives,
12(2), pp.42–61.
https://openjournals.library.sydney.edu.au/index.php/IEJ/article/viewFile/7455/7814 p.49

The need for market information
Within this quasi-market, high stakes
testing is central. Just as the economic
market requires a communicative signal
between producers and consumers, so too
does the educational quasi-market require
an equivalent. Therefore, published test
scores perform a similar, although not
equivalent function to price in the market
for school education.
Stevenson, H. & Wood, P., 2013. Markets, managerialism and teachers’ work:
the invisible hand of high stakes testing in England. The International
Education Journal: Comparative Perspectives, 12(2), pp.42–61.
https://openjournals.library.sydney.edu.au/index.php/IEJ/article/viewFile/7455/
7814
p.49

● Have you heard these arguments?
● Have you had experience of this
kind of process?

Technology and economics work
together
● Technology enables the market
● Data on activities and results can be gathered that
could not previously be gathered
● Data to be communicated to central management
● Data can be analysed in relation to targets, and
target setting

Common Core (see last weeks
lecture)
Peter Cohen, CEO of Pearson's K-12
division, Pearson School:
'It's a really big deal,'
'The Common Core standards are
affecting literally every part of the
business we're involved in.'
http://online.wsj.com/article/SB10001424052702303674
004577434430304060586.html

USA Grades 3-9 annual budget for
assessment: $669,000,000, $27 / pupil
39% of total = roughly
260 million dollars
Chingos, M.M., 2012. Strength in Numbers: State Spending on K-12 Assessment Systems,
Available at: https://www.brookings.edu/wp-
content/uploads/2016/06/11_assessment_chingos_final_new.pdf.

Bridge academies in Africa
● Private, for profit
● Fees aproximately $6 per month
● Bankrolled by Zuckerberg and Gates
● 470 nursery and primary schools with
100,000
● Intended to provide a service
demonstrating that private schools
are more effective than state schools.

How Bridge manages the
educational process
Tablets “not only provide the teachers with
detailed instructions but also monitor their
performance. Teachers must check in via their
tablets when they arrive and run their lessons
almost verbatim from the tablet’s lesson scripts.”
“Back at Bridge’s Nairobi offices and
Massachusetts-based headquarters, all the data is
compiled and analyzed. … Academy Managers
(the “principals” at the schools so to speak) are in
constant contact with headquarters.”
http://www.independent.co.uk/news/world/africa/bridge-international-academies-script
ed-schooling-for-6-a-month-is-an-audacious-answer-to-educating-10420028.html
See also
https://www.theatlantic.com/education/archive/2014/10/is-it-okay-to-make-teachers-rea
d-scripted-lessons/381265/

What do you think?
● What is the balance of opportunities
and perils for the Bridge project

Education is an unusual domain

Education is contested
– What is the product? Learning? Graduates?
Qualifications? Employability? Citizens?
– It has no externally verifiable outcomes (not
engineering tolerance, or even death rates)
– Measured only by its own instruments (exams
and tests)

What brings about success in
education?

Each group has their own theories
which inform representations

Management ascribe success to curricula and
training

Teachers ascribe success to their interventions in
the classroom

Learners ascribe success to their own efforts
Managers, psychologists and pedagogs
typically determine the representations
of the educational process (even if they
are only implicit)

A conflict of representations of
success

Each group has their own theories
which inform representations

Management ascribe success to curricula and
training

Teachers ascribe success to their interventions in
the classroom

Learners ascribe success to their own efforts

Managers, psychologists and pedagogs
typically determine the representations
of the educational process (even if they
are only implicit)

Representations and technology

When these representations are
instantiated in technology, they become
normative representations. They reflect
the interests of their originators, and
intervene in the balance of constraints.

Every Learning Design is a representation of the
education process

Every analytics intervention is a representation
of the education process

Every Learning Management System and every
MOOC is a theory.

Peter Drucker is reported as
saying:
"You can't manage it if you can't measure it"
Quoted to justify data driven decision making.
Also reported as
"What gets measured gets managed"
i.e. The representations used in education by
management (or learning technologists, or
psychologists, or pedagogues) often provide
circular justifications.
See Seddon, J. (2008). Systems Thinking in the Public Sector. Triarchy Press.

Representations of education

In our research into education we should
challenge “What gets measured gets
managed”.

We should be aware that managerial and
economic factors have an increasing
influence on representations

TEL research often reads as if it is conducting
cognitive engineering of students, but
– Teachers are not psychologists, they are professionals
– Students are not patients or subjects, they are
participants in a conversation

What is professional practice for
teachers?

What is professional practice?
‘...to apply skilled service or advice to
others, or to provide technical
managerial or administrative services
to, or within, organisations... ’
Cheetham, G., & Chivers, G. (2005). Professions, Competence And
Informal Learning: The Nature of Professions and the Role of Informal
Learning in Acquiring Professional Competence (p. 360). Cheltenham:
Edward Elgar.

In my view, in education
● Professional practice largely mediates between
– strategies for simplification of management
– the variety of the learners for whom teachers have
professional responsibility.
● Historic accommodation between regulatory
authorities, management and teaching
professionals:
– educational managers indicate the goals for teachers and
learners
– ensure that the results of their activity meet some minimum
standards.
– The rest is left up to the professional skills of teachers and
the ethical integrity of both teachers and learners.

● What happens if increase the ability
of managers to inspect the degree
to which teachers’ results meet
required standards?

I argue that...
The weakness of educational
management in handling detail of
educational interactions may be a
good thing. It allows flexibility which
teachers need to be work with
learners.

Teaching standards
● Among many clauses
● 2012 ‘assess qualified teachers against the standards
of a level that is consistent with what should
reasonably be expected of a teacher in the relevant
role and at the relevant stage of their career’ (DE,
2012, p. 1).
● teachers are ‘accountable’ for ‘pupils’ ‘attainment,
progress and outcomes’
● Head teachers are expected to refer to the standards
when making their judgements and implement those
judgements through the appraisal system.
● Upchurch, M., Moore, P. & Kunter, A., 2014. Marketisation, Commodification and the Implications
for Teachers’ Autonomy in England. In Sraffa and Althusser Reconsidered; Neoliberalism
Advancing in South Africa, England, and Greece. Emerald Insight, pp. 133–153. Available at:
http://www.emeraldinsight.com/doi/10.1108/S0161-723020140000029005.

John Seddon
● Fierce critic of commodification of education
● Argues that targets bring with them their own de
facto purposes.
● There is a danger that targets will force teachers
to focus on the achievement of a constrained set
of criteria for successful outcomes, and the
expense of the pedagogic interventions which
have traditionally formed the core of their
practice.
Seddon, J. (2008). Systems Thinking in the Public
Sector: The Failure of the Reform Regime and a
Manifesto for a Better Way. Axminster: Triarchy
Press.

● How does the use of data in
education support the policy of
teachers’ accountability?
● What is happening in the education
systems that you are studying?

Implications of management by
league table results
● “The commodification of school pupils’
assessment results means that to move
‘upwards’ in the league tables an individual
school must score more highly than
‘competitor’ schools on the proportion of its
pupils attaining grades A C in five GCSE
subjects including English and Maths.”
● This criteria is a decision taken by somebody
Upchurch, M., Moore, P. & Kunter, A., 2014. Marketisation, Commodification and the
Implications for Teachers’ Autonomy in England. In Sraffa and Althusser Reconsidered;
Neoliberalism Advancing in South Africa, England, and Greece. Emerald Insight, pp.
133–153. Available at: http://www.emeraldinsight.com/doi/10.1108/S0161-
723020140000029005.

A secondary teacher said that in
practice this meant:
“As a department we have a meeting round a big
table with all the childrens’ photographs in one year
scattered over the table. We then arrange the
photographs and construct a group out of those
children who are borderline D/C. We record their
names and put them down for extra revision classes
on Saturday mornings and so on. The rest are
subject to a process of ‘principled abandonment’
and not offered the same facilities or officially
expected to receive the same attention from us.”
Upchurch, M., Moore, P. & Kunter, A., 2014. Marketisation, Commodification and the
Implications for Teachers’ Autonomy in England. In Sraffa and Althusser Reconsidered;
Neoliberalism Advancing in South Africa, England, and Greece. Emerald Insight, pp. 133–
153. Available at: http://www.emeraldinsight.com/doi/10.1108/S0161-
723020140000029005.

Teachers pay rises
● “All teachers are awarded a pay rise when
the pay scales and allowances are
updated. Subject to satisfactory
performance, each September, teachers on
the main pay scale move to the next point
on the scale.
Teachers who have recorded excellence
performance may even advance by two
points.”
● How do we know about performance?
Through KPIs and the results of analytics.
https://www.tes.com/jobs/careers-advice/pay-and-conditions/qualified-teachers-pay-scales

● Is performance related pay for
teachers a good idea?

OECD (2012). Does performance-based pay improve teaching? PISA in Focus, (16)
pp.1–4.

Dilbert, as always, has something
to say

Not a panacea, a problem to attend
to
● Education can use the methods of psychology to study
change in the individual, and the factors that bring it
about
– For example, the efficacy of learning materials
– The results can be valuable
● But we should be aware that factors of social
organisation are always involved in the educational
process
● Technology, economics (and politics) are inevitably
entangled with social organisation
● I suggest that all PhD students in education should pay
them attention
● If they are ignored, then the results of research will be
confusing, or misleading

More specifically
●
When you change technology you change a representation
●
When you change data collection, you change the
transparency of the process being managed
●
When you change the transparency of the process to be
managed, you increase the power of management
● When you change the balance of power between manager
and managed, you also empower a particular
representation of the process.
●
Whose representation is empowered, and what are the
consequences for
●
Ministries of education? Directors? Teachers? Learners? Parents?
● What alternative method can we offer for the design of
educational systems?
●
These questions are worthy of PhD research!

Two of my papers that are relevant
to what I have been discussing
● Griffiths, D., 2017. The Use of Models in
Learning Design and Learning Analytics
Interaction Design and Architecture(s)
Journal - IxD&A, N.33, pp. 113-133
http://www.mifav.uniroma2.it/inevent/eve
nts/idea2010/doc/33_6.pdf
● Griffiths, D., 2012. The impact of
analytics in Higher Education on
academic practice. Cetis Analytics Series,
Vol 1, No.10., CETIS. p.6-7. Available at:
http://publications.cetis.ac.uk/2012/532.

Data in Education: Panacea or problem

More Related Content

What's hot

Similar to Data in Education: Panacea or problem

More from Education in the Knowledge Society PhD

Recently uploaded

Data in Education: Panacea or problem