MedU Plenary Session on Analytics
•Download as PPTX, PDF•
1 like•455 views
This document discusses the history and potential uses of learning analytics in education. It traces the growth of analytics in higher education from 2010-2017. Learning analytics can utilize feedback, prediction, and coaching to support student learning outcomes. Effective analytics requires multiple data sources and will impact the entire educational system. The document suggests analytics could enhance virtual patient learning by adding data metrics to measure feedback effectiveness, predict learning, and enable coaching across a series of virtual patients.
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to MedU Plenary Session on Analytics
Similar to MedU Plenary Session on Analytics (20)
More from Janet Corral
More from Janet Corral (20)
Recently uploaded
Recently uploaded (20)
MedU Plenary Session on Analytics
- 1. Analytics in Decision-Making: Learners, Learning, Life & Analytics What Matters for Learning Analytics? Janet Corral, PhD University of Colorado School of Medicine
- 2. Analytics is a Hot Topic 2011 Journal of Learning Analytics 2012 Georgia State Univ launches student data system 2013 1st international learning analytics conference 2014 Two analytics presentations at AAMC LSL 2015 1st AAMC GIR DATA Symposium 2016 EDUCAUSE Horizon report: Learning analytics 1 yr or less to adoption 2017 1st Ed Tech Working Group presentation on data standards 2010 Purdue University Signals Higher Education Med Schools
- 3. Defining analytics Learning Analytics Education Data Mining Academic Analytics Corral, J. (2017) based on Siemens & Baker (2012)
- 4. Defining analytics Learning Analytics Education Data Mining Academic Analytics Corral, J. (2017) based on Siemens & Baker (2012) Reasoning forward Investigating from the data Business intelligence
- 5. Need multiple data sources To support quality learning outcomes, educational analytics…
- 6. Need multiple data sources Will impact the whole system of stakeholders To support quality learning outcomes, educational analytics…
- 7. Analytics in Decision-Making: Learners, Learning, Life & Analytics What does this mean for Aquifer?
- 8. Continuum of Options Feedback Prediction Coaching
- 9. FEEDBACK
- 11. With gratitude to NYU Educational Data Warehouse: alex.support@med.nyu.edu
- 14. Feedback
- 15. Feedback
- 16. Action is outside the digital system
- 18. Learning analytics impact our systems of Education
- 19. Analytics in Decision-Making: Learners, Learning, Life & Analytics How would you enhance the feedback on one Aquifer VP? What data metrics are important?
- 20. Prediction
- 22. Prediction of learning requires work Pusic, V. et al (2015). Learning curves in health professions education.
- 24. Student Learning Pathways Corral, J. (2012). Learning with Virtual Patients. ! !
- 26. Nudging
- 27. Nudging It’s been 7 days since you logged in. Students who complete VPs 3x/week are 17% more successful in the clerkship.
- 28. Coaching Credit: University of Michigan
- 29. Student Learning Pathways Corral, J. (2012). Learning with Virtual Patients. Students who had difficulty here, found it helpful to: 1. Read _____ 2. Do cases 1 & 5
- 30. Analytics in Decision-Making: Learners, Learning, Life & Analytics What analytics would you add to a series of VPs? What does that type of analytics look like across the VPs? What data metrics are important? Feedback Prediction Coaching
- 33. UC Denver 81 dentistry students 184 medical students 180 pharmacy students Denver Public Schools 81,438 students Facebook 1.71 billion monthly active users University of Phoenix 400,000 students per year Twitter 14 Pedabytes of Data Netflix 40 Pedabytes of Data EDU Data ≠ not BIG Data
- 35. Impact is on the System 10.37% 100 Student Advisors or Coaches increase in B or A awarded “The students who get a red light almost all contact me immediately to ask how to raise their grades.” -Tim Delworth, Mathematics
- 36. Impact is on the System NEW faculty role ‘coach’ NEWcurriculum
- 37. Learning is social Network Awareness Tool of Social Learn Network (Teplov, 2012)
- 38. Learning analytics can detect when emotions impact achievement Delight, Surprise (Rare) Frustration (less associated with poorer learning) Confusion, Engaged Concentration (Watch out for boredom!) Baker, D’Mello, Rodrigo, & Graesser (2004, 2010)
- 39. Learning analytics can detect when emotions impact achievement Delight, Surprise (Rare) Frustration (less associated with poorer learning) Confusion, Engaged Concentration (Watch out for boredom!) Baker, D’Mello, Rodrigo, & Graesser (2004, 2010)
- 40. Need multiple data sources To support quality learning outcomes, educational analytics…
- 41. Need multiple data sources Will impact the whole system of stakeholders To support quality learning outcomes, educational analytics…
- 43. Discussion for the Aquifer Community What type(s) of analytics are most effective for Aquifer to engage?
Editor's Notes
- 2012 – Georgia State – $2M investment nets $10M extra tuition a year, increased URM enrolment (22% hispanic, 15% black male) and increased graduation rate (226% hispanic and 111% in black males). 2013 – LAK in Leuven, Belgium 2014 - Anything from Educause? # of presentations at GIR
- Learning analytics is about what learners are learning, and their contexts they do it in. Education data mining is, in many ways, a quantitative analysis focused on prediction or mining the data first Academic analytics is the business intelligence applied to education data. Here we might move across institutional systems and Student information.
- Learning analytics is about what learners are learning, and their contexts they do it in. Education data mining is, in many ways, a quantitative analysis focused on prediction or mining the data first Academic analytics is the business intelligence applied to education data. Here we might move across institutional systems and Student information.
- Today’s session is about thinking about the data in MedU, and what that could mean to integrate the educational concepts from this morning, with what data MedU collects on learners. We will take a high-level look at examples, and then break for you to work in groups to pick a concept, and disuss its implications for VPs with MedU.
- Any visualized dashbaord is a form of feedback, to whatever user is looking at the dashboard. But what kind of feedback is most optimal for Entrada to take on?
- This is Purdue Course signals, started in 2007 The ‘signals’ are traffic signals, which tell learners if they are at risk or not of doing poorly in a course The metrics (# of assignments done/not done, # of quizzes done/not done) are used to calculate how a student is progressing. Any dashboard is fundamentally feedback. This is feedback, but it’s not making learning visible.
- Let’s make it about learning. Here are final exam scores, reported to learners on a dashboard. What are your thoughts? Turn to the person next to you, behind you, in front of you and discuss for 2 mins.
- This is Vanderbilt VSTAR On this dashboard, we see a series of notices: There is a ‘past due’ task IF learners have started (or not) Any notes the faculty member can make about the learner progress
- Here is another dashboard from Vanderbilt, where learners report their goals against the competencies the program monitors. The learner and their faculty coach see this dashboard, so they can monitor progress.
- The MedU dashboard, at it’s most basic feature, is visualizing learning. What does this mean? It is a type of feedback to learners and faculty. But what do they undersatnd?
- The biggest change was to make it more readable, to provide the necessary information right at faculty members’ fingertips. This is another example of how the dashboard impacts the system.
- This means: Time lag between feedback & action We don’t know what the target user took away A systems level change – more hires, more training
- eedback prompts, outcome measures, reporting to learner, reporting to faculty) Move into groups of 4-6 people Decide which element you want to do What would it look like? What metrics matter to you?
- This is an example of academic analytics. This screen shot is an example of predicting learner graduation rate, based on how they are progressing in a course. What does this look like in VPs?
- The most impactful is making student learning visible. This means, giving the learners an explanation for their answer, key feedback, and the opportunity to practice again.
- Could we predict based on a student’s early progress in the case, based on certain choices, a DDx list, answers to any questions …on how they will do in the rest of the case? Other similar cases? Other different cases? A Learning analytics mindset asks: What are the elements of VPs that you, the MedU community, agree are important features to get right? Do those variables play out as the predictors? Education data mining mindset asks: What data do predict performance within a case, across cases? Do these align with what we know is valuable in clinical reasoning with real patients?
- 40 million nudges a week in 2017?
- The goal is to encourage positive behaviors. We don’t want to shove (push) or slap (cut off) the learner.
- Coaching is the approach that we know your performance – how can we get you to mastery, excellence – or merely on the path you want to be on?
- What about their sequence of elements they got correct & incorrect in the VP predicts performance on future cases?
- What would you add across a series of virtual patients to make learning transparent? e.g. feedback prompts, outcome measures, reporting to learner, reporting to faculty)
- 45 mins
- MedU, however, is primed in the medical education space to provide analysis across institutions, and within insitutions. Across cases, you have the power to analyze learner progress & feedback.
- The signals system sat on top of data feeds from Blackboard. Data signals then sent warnings in 3 colors – green, yellow, red – to learners. Faculty would also compose emails with standard messages, and the emails would send to learners based on their signal color. This approach had a very positive impact on Purdue. Due to Blackboard making changes to their system, Signals will no longer exist after Dec 2017. Purdue recently purchased Kaplan’s eLearning division.
- When doing my PhD, watching learners do the VPs in groups, it was clear that VPs were contagious. MedU doesn’t have control over how the VPs are implemented. Some schools will have learners in small groups. Most will have learners do these independently. But in 2017 – are the learners truly independent? What other tools are they using while doing MedU VPs? Can the coaching tips extend to what resources the learners might use? Which peers to consult? How to critically appriase other resources? OR – said another way – in what ways does bringing in
- Aquifer has key elements: Smart people who ‘get’ med ed Fantastic thinkers who can integrate evidence-based learning principles Large, cross-institutional, longitudinal data set
- End with 10 mins for Q&A