Learning Analytics as Interpretive Practice


Published on

Presentation at the 1st International Conference on Learning Analytics and Knowledge in Banff, Canada.

Published in: Education, Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • http://www.flickr.com/photos/ginnerobot/2523448766/
  • http://www.flickr.com/photos/tiswango/120040030/
  • http://www.flickr.com/photos/eeko/3337615282/
  • Positivism doesn ’t provide a way to examine the meaning of phenomena Neither does It account for the the judgments that researchers need to make in scientific practice
  • research is fundamentally about creating meaning through interpretation a tradition of interpreting biblical texts Today, the study of interpretation is applied to many fields Valuable because it couches observations in terms of meaningful social practice, what people are doing
  • Hermeneutics is not… Assigning meaning through metaphor IP metaphor of cognitive science, becomes more real than the observed behavior Good for explaining potential overlaps between what humans and computers do Bad for explaining the mind in terms of creativity, play, exploration Before IP, the steam engine was a popular metaphor for the mind
  • Another way of assigning meaning to phenomena is by removing it all together through reductionism. This means reducing a phenomena down to operationalized, fundamental components and “laws” Is the mind just the sum of its chemicals and the frequencies of its firing synapses? What meaning can be derived from frequency data out of context? Rorty said,…
  • If not abstracting or reducing then what? Hermeneutics is for interpreting behavior in terms of what people are doing. Because behavior transpires within the real world, and not within the mind, it is concrete. A property of the world we live in. Where social practices take place. Why is this important? Behavior is nested with social practice!
  • If behavior is concrete and nested within social practice, how do we observe and interpret behavior in virtual environments?
  • http://www.flickr.com/photos/eeko/3337615282/ Again the point is what people are doing This matters whether we are observing in person or virtually
  • If these are your measures, they need to be organized by what social practices they are a part of.
  • Just as what people are trying to do gives meaning to their actions in the real world, so it does in the virtual world. We propose that observational data be clustered by the tasks and practices the they are a part of.
  • “ testing and estimating causal relations using a combination of statistical data and qualitative causal assumptions”
  • Between, within, and overall slopes
  • Stewardship over these new capabilities to make, as we see, more reliable and valid recommendations.
  • Learning Analytics as Interpretive Practice

    1. 1. Learning Analytics as Interpretive Practice Michael Atkisson and David Wiley Allen Communications and Brigham Young University
    2. 2. Download These Slides http://slideshare.net/opencontent/ http://woknowing.wordpress.com/
    3. 3. Interpretation ≠ Science Just the facts, ma ’am.
    4. 4. Confusion Science with positivism
    5. 5. Quantitative > Qualitative Numbers are objective
    6. 6. Educational Measurement What does s/he know?
    7. 7. Not Directly Observable So we create behavioral proxies and look for evidence
    8. 8. Online Learning Is Even Worse Proxies not available for direct observation
    9. 9. Impoverished Vocabulary “ Observable” behavior = clicks and key presses
    10. 11. Westerman ’s Argument Quantitative inquiry is interpretive
    11. 12. Operational Definitions Fundamental to social science research, including educational measurement
    12. 15. First, Define Constructs Then you can measure them
    13. 17. User ID Page Time 78 /lecture1.html 10:43 78 /lecture2.html 10:49 78 /lecture3.html 10:50 78 /lecture4.html 11:12 78 /quiz.html 11:23
    14. 18. “ Time on Task” Are different definitions possible? Are different results possible? If yes  subjectivity
    15. 19. Philipp Frank <ul><li>“ The uncritical acceptance of empirical data without sophisticated study of assumptions.” </li></ul>Vic Bunderson “ Letting the data tail wag the theory dog”
    16. 20. Learning Analytics Success? <ul><li>If we can predict, but don ’t understand why </li></ul>
    17. 21. If Not Positivism… <ul><li>Then what? </li></ul>
    18. 22. Hermeneutics <ul><li>All about meaning and interpretation </li></ul>
    19. 23. Problems with Metaphor <ul><li>e.g., the information processing model </li></ul>
    20. 24. Problems with Reductionism <ul><li>“ There is really nothing more to be said when neurophysiology has had its say” </li></ul>
    21. 25. Meaning in Concreteness <ul><li>Behavior in context </li></ul>
    22. 26. How Could We Do This? <ul><li>Given abstractness of online behaviors </li></ul>
    23. 28. These observations need to be clustered by practice User ID Page Time 78 /lecture1.html 10:43 78 /lecture2.html 10:49 78 /lecture3.html 10:50 78 /lecture4.html 11:12 78 /quiz.html 11:23
    24. 29. Cluster Behavior Observations by Practice <ul><li>Readings, discussion forum participation, simulation use, homework assignments, taking exams, etc. </li></ul>
    25. 30. How? A combination of… <ul><li>Structural Equation Modeling (Latent Factors) </li></ul><ul><li>Multilevel data structures </li></ul><ul><li>Continuous, longitudinal measures </li></ul>
    26. 32. Tasks nested within within practice B w Within tasks Between Tasks B b B t Total Practice Regression Task 1 Task 2 Task 3
    27. 33. Almost Certainly Better Ways <ul><li>Hopefully you will think of them! </li></ul>
    28. 34. LA  Interventions with People <ul><li>Learning analytics is an ethical activity </li></ul>
    29. 35. Let’s Be Good Stewards <ul><li>And not regress to Behaviorism 2.0 </li></ul>
    30. 36. Thank You michael.atkisson@gmail.com http://woknowing.wordpress.com/ [email_address] http://davidwiley.org/