Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013
Upcoming SlideShare
Loading in...5
×
 

Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013

on

  • 562 views

These slides are from a presentaion by Adam Cooper, entitled "Analytics (as if learning mattered)" in the In Focus: Learner analytics and big data symposium, University of London, December 10th ...

These slides are from a presentaion by Adam Cooper, entitled "Analytics (as if learning mattered)" in the In Focus: Learner analytics and big data symposium, University of London, December 10th 2013
The recorded audio from the session is available at: https://soundcloud.com/cdelondon/analytics-as-if-learning

Related blog post at: http://blogs.cetis.ac.uk/adam/2013/10/31/policy-and-strategy-for-systemic-deployment-of-learning-analytics-barriers-and-potential-pitfalls/

Statistics

Views

Total Views
562
Views on SlideShare
379
Embed Views
183

Actions

Likes
1
Downloads
2
Comments
0

3 Embeds 183

http://robwork.wordpress.com 175
https://twitter.com 7
http://feedly.com 1

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • title slideadd title into box shown: add presenters
  • body copy no bullets

Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013 Analytics (as if learning mattered) - RIDE Symposium, University of London 10th December 2013 Presentation Transcript

  • Analytics (as if learning mattered) Adam Cooper, In Focus: Learner analytics and big data symposium, University of London, December 10th 2013
  • Structure of the Talk • My Perspective • Threats • Avoiding the threats Image: cba Jaskirat Singh Bawa
  • PERSPECTIVE
  • Perspective of this Talk Where am I coming from? UK Higher Ed Institution Cetis is based in the Institute for Educational Cybernetics at the University of Bolton. “Our mission is to develop a better understanding of how information and communications technologies affect the organisation of education from individual learning to the global system.”
  • Defining “Analytics” Analytics is the process of developing actionable insights through problem definition and the application of statistical models and analysis against existing and/or simulated future data.
  • Learning vs Business Venn Insights to support educational aims and objectives Insights to support operational and strategic activity.
  • Learning vs Business Venn Retention Student Satisfaction Employability Achievement Applications Easiest Case for Investment
  • Structure to Manage Complexity The SMT – “what makes a viable organisation?” Subject/Discipline Member - “what makes a good chemist?” (knowledge, approach to problems, creativity, attitudes, values...) Course Team – “what curriculum?” (topics, suitable LOs and assessment methods to match our typical student career etc) Class Teacher – “where are my students collectively and individually?” (intellectually, emotionally, meta-cognitively...) Student/Learner – time, place and method of study Self-organising and self-regulating
  • The Intersection Doppelgänger Values Epistemological Retention Student Satisfaction Employability Cause & Effect Models Achievement Moral Imperatives Applications
  • The “Accommodation” “The introduction of these techniques [Learning Analytics] cannot be understood in isolation from the methods of educational management as they have grown up over the past two centuries. These methods are conditioned by the fact that educational managers are limited in their capability to monitor and act upon the range of states which are taken up by teachers and learners in their learning activities. ... Over the years, an accommodation has developed between regulatory authorities, management and teaching professionals: educational managers indicate the goals which teachers and learners should work towards, provide a framework for them to act within, and ensure that the results of their activity meet some minimum standards. The rest is left up to the professional skills of teachers and the ethical integrity of both teachers and learners.” Prof. Dai Griffiths, “Implications of Analytics for Teaching Practice in Higher Education”
  • TWO THREATS (+1 AND A BIT)
  • Threat #1: Interventionism Image: via Wikimedia Commons p
  • Threat #2: BI Tradition Image: via Wikimedia Commons p Hhultgren • Centralised projects • IT + senior leader • Management reports • KPIs Image: Paul Hodge
  • So, you want to optimise student success? • Are the following well-specified? • “optimise student success” • “maximise the probability of success” • What does success look like? • Really? • Says who? • Is a “good education” indicated by student success?
  • Does everyone understand? Attainment: an academic standard, typically shown by test and examination results Progress: how far a learner has moved between assessment events Achievement: takes into account the standards of attainment reached by learners and the progress they have made to reach those standards Enrichment: the extent to which a learner gains professional attitudes, meta-cognition, technical or artistic creativity, delight, intellectual flexibility, ...
  • +1: Technological Solutionism „Recasting all complex social situations either as neatly defined problems with definite, computable solutions or as transparent and self-evident processes that can be easily optimized—if only the right algorithms are in place!—this quest is likely to have unexpected consequences that could eventually cause more damage than the problems they seek to address. I call the ideology that legitimizes and sanctions such aspirations “solutionism.” I borrow this unabashedly pejorative term from the world of architecture and urban planning, where it has come to refer to an unhealthy preoccupation with sexy, monumental, and narrow-minded solutions—the kind of stuff that wows audiences at TED Conferences—to problems that are extremely complex, fluid, and contentious … solutionism presumes rather than investigates the problems that it is trying to solve, reaching “for the answer before the questions have been fully asked.” How problems are composed matters every bit as much as how problems are resolved.‟ Evgeny Morozov, “To Save Everything Click Here: The Folly of Technological Solutionism”, 2013
  • 3M and Six-Sigma “McNerney axed 8,000 workers (about 11% of the workforce), intensified the performancereview process, and tightened the purse strings at a company that had become a profligate spender. He also imported GE's vaunted Six Sigma program—a series of management techniques designed to decrease production defects and increase efficiency. Thousands of staffers became trained as Six Sigma „black belts‟. ... Now his successors face a challenging question: whether the relentless emphasis on efficiency had made 3M a less creative company. ... Those results are not coincidental. Efficiency programs such as Six Sigma are designed to identify problems in work processes—and then use rigorous measurement to reduce variation and eliminate defects. ... Indeed, the very factors that make Six Sigma effective in one context can make it ineffective in another. Traditionally, it uses rigorous statistical analysis to produce unambiguous data that help produce better quality, lower costs, and more efficiency. That all sounds great when you know what outcomes you'd like to control. But what about when there are few facts to go on—or you don't even know the nature of the problem you're trying to define?” Source : Bloomberg Business Week, June 2007.
  • The Bit Data: it may be big but its not clever
  • The Bit: Net Scale Fascination (delusion?) Data: it may be big but its not clever
  • • Hendrik‟s photo Source unknown – thanks to Hendrik Drachsler for passing it on.
  • Threat Rating=UBI*(I+D)UTS Where: UBI = Uncritical BI tradition factor I = Interventionist tendencies D = Big data fascination UTS = Uncritical technological solutionism This is not serious...
  • From: “Analytics in Higher Education: Benefits, Barriers, Progress, and Recommendations” (ECAR Report) Why am I Worried?
  • NEUTRALISING THREATS
  • Data Is Useless Without the Skills to Analyze It „...employees need to become: Ready and willing to experiment: Managers and business analysts must be able to apply the principles of scientific experimentation to their business. They must know how to construct intelligent hypotheses. They also need to understand the principles of experimental testing and design, including population selection and sampling, in order to evaluate the validity of data analyses. ... Adept at mathematical reasoning: How many of your managers today are really “numerate” — competent in the interpretation and use of numeric data?‟ Jeanne Harris, HBR Blog, Sept 2012
  • From: “Analytics in Higher Education: Benefits, Barriers, Progress, and Recommendations” (ECAR Report) Is This a Real Problem?
  • Make Some Progress By: • Directly investing in specialists • E.g. OU, statisticians and data wranglers • ECAR Survey Report 2012: “invest in people over tools... hiring and/or training... analysis” • Operating a “shared service” • Higher Education Statistics Agency (HESA) • Universities and Colleges Admissions Service (UCAS) • Jisc
  • Data Is Useless Without the Skills to Analyze It „...employees need to become: Ready and willing to experiment: Managers and business analysts must be able to apply the principles of scientific experimentation to their business. They must know how to construct intelligent hypotheses. They also need to understand the principles of experimental testing and design, including population selection and sampling, in order to evaluate the validity of data analyses. ... Adept at mathematical reasoning: How many of your managers today are really “numerate” — competent in the interpretation and use of numeric data?‟ Jeanne Harris, HBR Blog, Sept 2012
  • We Get Further with “Soft Capability” • Appreciation of the limitations of analytics technologies • Expertise to comprehend, critique, and converse (as opposed to originate or implement)
  • Visualisations are the Opium of the People (c)2008, Ryan Block/Engadget
  • Sparklines: Tufte vs ...
  • Significance and Sample Size “Flagged for action because 74% of students felt they were well supported by their supervisor compared to 80% in similar institutions (difference >5%)” “We need to talk to Mr. Benn; the drop-out rate doubled in his class this year.” Correlation is not Causation “The predictive model indicates that students with attributes {X, Y, Z} are only 50% likely to complete.”
  • What Does This Give Us? (aside from people less likely to cut themselves on sharp tools)
  • Absorptive Capacity “...the ability of a firm to recognize the value of new, external information, assimilate it, and apply it to commercial ends is critical to its innovative capabilities. We label this capability a firm‟s absorptive capacity and suggest that it is largely a function of the firm‟s level of prior related knowledge. [...] We formulate a model of firm investment in research and development (R&D), in which R&D contributes to a firm’s absorptive capacity, and test predictions relating a firm‟s investment in R&D to the knowledge underlying technical change within an industry.” Cohen and Levinthal, Admin. Science Quarterly 35(1), 1990
  • How to do LA as if learning mattered 1. Build soft capability. 2. Engage in participatory (collaborative) design. 3. Develop & prototype technology (& models) organically. 4. Reflect on the effect LA innovation has on practice. 5. Design and develop with the educator IN the system. 6. Optimise the environment, not only the outcome. 7. Be realistic about scale/quality of data and its meaning. Personal Learning Analytics? • Changed dyamic • Less “big brother”
  • Participatory Design Learning Analytics Image: cbn Jesse Millan
  • Thank-you for your attention These slides are at: http://is.gd/AaiLM
  • Reading and References Jacqueline Bichsel, “Analytics in Higher Education: Benefits, Barriers, Progress, and Recommendations” Available from http://www.educause.edu/ecar Cohen and Levinthal (1990), “Absorptive Capacity: A New Perspective on Learning and Innovation” Admin. Science Quarterly 35(1) http://faculty.fuqua.duke.edu/%7Echarlesw/s591/Bocconi-Duke/Papers/C10/CohenLevinthalASQ.pdf Adam Cooper, “What is Analytics? Definition and Essential Characteristics” http://publications.cetis.ac.uk/2012/521 Jeanne Harris, “Data is Useless Without the Skills to Analyze it” http://blogs.hbr.org/2012/09/data-is-useless-without-the-skills/ Bloomberg Business Week, “At 3M, A Struggle Between Efficiency And Creativity” http://www.businessweek.com/stories/2007-06-10/at-3m-a-struggle-between-efficiency-and-creativity Dai Griffiths, “CETIS Analytics Series: The impact of analytics in Higher Education on academic practice” http://publications.cetis.ac.uk/2012/532 Evgeny Morozov, “To Save Everything Click Here: The Folly of Technological Solutionism” ISBN-10: 9781610391382 Phil Winne, “Improving Education” (lecture) http://www.youtube.com/watch?v=dBxMZoMTIU4
  • Licence “Analytics (as if learning mattered)” by Adam Cooper, Cetis, is licensed under the Creative Commons Attribution 3.0 Unported Licence http://creativecommons.org/licenses/by/3.0/ Case studies and white papers: Cetis Analytics Series, http://publications.cetis.ac.uk/c/analytics Me: http://blogs.cetis.ac.uk/adam a.r.cooper@bolton.ac.uk Cetis: http://www.cetis.ac.uk