Examining the Hype and Reality of

Learning Analytics"
Patsy	
  Moskal	
  
University	
  of	
  Central	
  Florida	
  
“the use of data, statistical analysis, and
explanatory and predictive models to gain insight
and act on complex issues.”


Analytics!
!
The	
  Analy8cs	
  Landscape	
  in	
  Higher	
  Educa8on,	
  2015	
  (ECAR).	
  Retrieved	
  from	
  hHps://
library.educause.edu/~/media/files/library/2015/5/ers1504cl.pdf	
  
Learning Analytics
“The measurement, collection, analysis and report of
data about learners and their contexts, for purposes of
understanding and optimizing learning and the
environments in which it occurs.”
hHps://solaresearch.org/	
  
“Learning analytics collects and analyzes the
‘digital breadcrumbs’ that students leave as they
interact with various computer systems to look for
correlations between those activities and learning
outcomes.”


Learning Analytics!
!
7	
  Things	
  you	
  should	
  know	
  about	
  first-­‐genera8on	
  learning	
  analy8cs.	
  EDUCAUSE	
  
Learning	
  Ini3a3ve	
  (ELI).	
  2011.	
  Retrieved	
  from	
  hHps://net.educause.edu/ir/library/
pdf/ELI7079.pdf	
  
	
  
Check it out…!
hHps://library.educause.edu/
resources/2015/5/analy8cs-­‐
in-­‐higher-­‐educa8on-­‐2015	
  
Major	
  Priority	
  
Not	
  a	
  Priority	
  
Students’ interest in analytics applications
Student perspectives on using analytics
Helpful
“big mother”
Creepy
“big brother”
Progress	
  toward	
  degree	
  or	
  cer8ficate	
  
Performance	
  in	
  current	
  courses	
  
Performance	
  in	
  past	
  courses	
  
Performance	
  in	
  courses	
  compared	
  to	
  other	
  students	
  
Ac8vity	
  in	
  university	
  applica8on	
  or	
  service	
  
Ac8vity	
  on	
  a	
  college	
  or	
  university	
  website	
  
Campus	
  ac8vi8es	
  logged	
  through	
  student	
  ID/smart	
  card	
  
Campus	
  ac8vi8es	
  logged	
  through	
  smartphone	
  
Proximity	
  to	
  a	
  college	
  building,	
  office,	
  or	
  resource	
  
Loca8on	
  on	
  campus	
  
Social	
  media	
  ac8vi8es	
  
Predictive Analytics in Higher
Education:!
Five Guiding Practices for Ethical Use!
•  Have a vision and plan
•  Build a supportive infrastructure
•  Work to ensure proper use of data
•  Design predictive models and algorithms that avoid bias
•  Meet institutional goals and improve student outcomes by
intervening with care.
Reported	
  in	
  Inside	
  Higher	
  Ed	
  3/6/17	
  –	
  ‘Conversa8on	
  Starter’	
  on	
  Ethical	
  Data	
  Use	
  
	
  
For	
  more	
  informa8on,	
  visit	
  newamerica.org/dataethics.	
  
Learning analytics in
practice!
Available Data!
!SIS	
  
•  Demographics	
  	
  
•  Socio-­‐economic	
  status	
  
•  High	
  school	
  informa8on	
  
•  Placement	
  tests	
  (SAT/ACT)	
  
	
  
LMS	
  
•  Students’	
  behavior/interac8on	
  
with	
  system	
  
•  Access	
  (views,	
  8me)	
  
•  Gradebook	
  data	
  
Other	
  data	
  (varies	
  widely!)	
  
•  Use	
  of	
  advising	
  &	
  tutoring	
  
•  Interac8on	
  with	
  various	
  
campus	
  services	
  (library,	
  
wri8ng	
  center,	
  dining	
  hall,	
  
etc)	
  
•  AHendance	
  at	
  large	
  lecture	
  
classes	
  
•  Financial	
  behavior	
  
•  Orienta8on	
  par8cipa8on	
  
The big issue is…!
Who	
  is	
  at	
  risk???	
  
hHp://www.youthareawesome.com/wp-­‐content/uploads/2010/10/wheres-­‐waldo1.jpg	
  
A more difficult task
is…!
hHp://matrix.wikia.com/wiki/File:Matrix-­‐neo-­‐stops-­‐bullets-­‐wallpaper.jpg	
  
How	
  do	
  we	
  stop	
  them	
  from	
  failing?	
  
How	
  do	
  we	
  stop	
  them	
  from	
  failing?	
  
High Level to !
Home Grown !
Learning Analytics
Approaches!
§  SIS, LMS and other data
§  Model trends across time to identify and ultimately predict
which students are at risk
Trawl net approach!
UCF Student
Performance
Dashboard!
UMBC
Check My
Activity
(John Fritz)!
Fritz & Whitmer (2017) Learning Analytics Research for LMS Course Design: Two
Studies.
http://er.educause.edu/articles/2017/2/learning-analytics-research-for-lms-course-
design-two-studies	
  
Some Prototype !
Adaptive Learners!
Students’ adaptive learning behaviors!
How Students Behave in Personalized
Adaptive Courses!
• Colm Howlin, Principal Researcher
• March 16, 2017
• Realizeit Webinar http://realizeitlearning.com/
community/
Questions still being asked…!
" " " " " "…your mileage may vary!
Issues/strategies	
  with	
  implementaCon	
  
Sources	
  of	
  data	
  to	
  use	
  
Cost	
  issues	
  
InsCtuConal	
  approaches	
  
Improving	
  student	
  outcomes	
  
Governance	
  strategy	
  and	
  milestones	
  
Details,	
  benefits,	
  challenges	
  
Security	
  and	
  ethics	
  concerns	
  
Final Resources to
Check Out…!
Special Learning Analytics issue of Online Learning
http://onlinelearningconsortium.org/read/online-learning-journal/
Editors:
Karen Vignare, Patsy Moskal, Matthew Pistilli and Alyssa Wise
Learning Analytics Bibligraphy "!
• Online Learning list of articles
• Must read articles
• General bibliography
• Early alert system articles
• Articles focused on ethics
Patsy Moskal!
FPatsy.Moskal@ucf.edu!
For	
  more	
  informa8on,	
  contact:	
  
Director,!
Research Initiative for Teaching Effectiveness!
Center for Distributed Learning!
University of Central Florida!

Patsy Moskal: Panel Presentation - Learning Analytics - Examining the Hype and Reality of Learning Analytics

  • 1.
    Examining the Hypeand Reality of
 Learning Analytics" Patsy  Moskal   University  of  Central  Florida  
  • 2.
    “the use ofdata, statistical analysis, and explanatory and predictive models to gain insight and act on complex issues.” Analytics! ! The  Analy8cs  Landscape  in  Higher  Educa8on,  2015  (ECAR).  Retrieved  from  hHps:// library.educause.edu/~/media/files/library/2015/5/ers1504cl.pdf  
  • 6.
    Learning Analytics “The measurement,collection, analysis and report of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.” hHps://solaresearch.org/  
  • 7.
    “Learning analytics collectsand analyzes the ‘digital breadcrumbs’ that students leave as they interact with various computer systems to look for correlations between those activities and learning outcomes.” Learning Analytics! ! 7  Things  you  should  know  about  first-­‐genera8on  learning  analy8cs.  EDUCAUSE   Learning  Ini3a3ve  (ELI).  2011.  Retrieved  from  hHps://net.educause.edu/ir/library/ pdf/ELI7079.pdf    
  • 8.
  • 11.
    Major  Priority   Not  a  Priority  
  • 12.
    Students’ interest inanalytics applications
  • 13.
    Student perspectives onusing analytics Helpful “big mother” Creepy “big brother” Progress  toward  degree  or  cer8ficate   Performance  in  current  courses   Performance  in  past  courses   Performance  in  courses  compared  to  other  students   Ac8vity  in  university  applica8on  or  service   Ac8vity  on  a  college  or  university  website   Campus  ac8vi8es  logged  through  student  ID/smart  card   Campus  ac8vi8es  logged  through  smartphone   Proximity  to  a  college  building,  office,  or  resource   Loca8on  on  campus   Social  media  ac8vi8es  
  • 14.
    Predictive Analytics inHigher Education:! Five Guiding Practices for Ethical Use! •  Have a vision and plan •  Build a supportive infrastructure •  Work to ensure proper use of data •  Design predictive models and algorithms that avoid bias •  Meet institutional goals and improve student outcomes by intervening with care. Reported  in  Inside  Higher  Ed  3/6/17  –  ‘Conversa8on  Starter’  on  Ethical  Data  Use     For  more  informa8on,  visit  newamerica.org/dataethics.  
  • 15.
  • 16.
    Available Data! !SIS   • Demographics     •  Socio-­‐economic  status   •  High  school  informa8on   •  Placement  tests  (SAT/ACT)     LMS   •  Students’  behavior/interac8on   with  system   •  Access  (views,  8me)   •  Gradebook  data   Other  data  (varies  widely!)   •  Use  of  advising  &  tutoring   •  Interac8on  with  various   campus  services  (library,   wri8ng  center,  dining  hall,   etc)   •  AHendance  at  large  lecture   classes   •  Financial  behavior   •  Orienta8on  par8cipa8on  
  • 17.
  • 18.
    Who  is  at  risk???   hHp://www.youthareawesome.com/wp-­‐content/uploads/2010/10/wheres-­‐waldo1.jpg  
  • 19.
    A more difficulttask is…!
  • 20.
    hHp://matrix.wikia.com/wiki/File:Matrix-­‐neo-­‐stops-­‐bullets-­‐wallpaper.jpg   How  do  we  stop  them  from  failing?   How  do  we  stop  them  from  failing?  
  • 21.
    High Level to! Home Grown ! Learning Analytics Approaches!
  • 22.
    §  SIS, LMSand other data §  Model trends across time to identify and ultimately predict which students are at risk Trawl net approach!
  • 23.
  • 24.
    UMBC Check My Activity (John Fritz)! Fritz& Whitmer (2017) Learning Analytics Research for LMS Course Design: Two Studies. http://er.educause.edu/articles/2017/2/learning-analytics-research-for-lms-course- design-two-studies  
  • 25.
  • 27.
  • 28.
    How Students Behavein Personalized Adaptive Courses! • Colm Howlin, Principal Researcher • March 16, 2017 • Realizeit Webinar http://realizeitlearning.com/ community/
  • 29.
    Questions still beingasked…! " " " " " "…your mileage may vary! Issues/strategies  with  implementaCon   Sources  of  data  to  use   Cost  issues   InsCtuConal  approaches   Improving  student  outcomes   Governance  strategy  and  milestones   Details,  benefits,  challenges   Security  and  ethics  concerns  
  • 30.
  • 31.
    Special Learning Analyticsissue of Online Learning http://onlinelearningconsortium.org/read/online-learning-journal/ Editors: Karen Vignare, Patsy Moskal, Matthew Pistilli and Alyssa Wise
  • 32.
    Learning Analytics Bibligraphy"! • Online Learning list of articles • Must read articles • General bibliography • Early alert system articles • Articles focused on ethics
  • 33.
    Patsy Moskal! FPatsy.Moskal@ucf.edu! For  more  informa8on,  contact:   Director,! Research Initiative for Teaching Effectiveness! Center for Distributed Learning! University of Central Florida!