The Connected Intelligence Centre: Human-Centered Analytics for UTS Data Challenges
•
3 likes•2,428 views
The Connected Intelligence Centre at UTS works as a creative incubator to catalyze thinking about the use of data and analytics across various faculties and units at UTS. It coordinates various data challenges and projects involving analyzing real UTS datasets to provide insights. These include predicting student dropout or employment outcomes. It also explores designing analytics for student collaboration and learning dispositions. Ensuring algorithmic accountability and human-centered values are key priorities for its work.
Recommended
More Related Content
What's hot
What's hot (19)
Similar to The Connected Intelligence Centre: Human-Centered Analytics for UTS Data Challenges
Similar to The Connected Intelligence Centre: Human-Centered Analytics for UTS Data Challenges (20)
More from Simon Buckingham Shum
More from Simon Buckingham Shum (20)
Recently uploaded
Recently uploaded (20)
The Connected Intelligence Centre: Human-Centered Analytics for UTS Data Challenges
- 1. The Connected Intelligence Centre: Human-Centered Analytics for UTS Data Challenges Simon Buckingham Shum & CIC Team Director, Connected Intelligence Centre Professor of Learning Informatics @sbuckshum #LearningAnalytics cic.uts.edu.au
- 2. h"ps://twi"er.com/Wiswijzer2/status/414055472451575808 “Note: check the huge difference between knowing and measuring…”
- 5. Blackfriars Campus, Grnd Floor Building 22
- 6. Georgia Markakis Senior Manager Theresa Anderson Senior Lecturer Fac. Arts & Social Sciences Roberto MarLnez-‐Maldonado Research Fellow Xiaolong (Shawn) Wang Web Developer = at today’s Data Science Symposium Ruth Crick Professor of Learning AnalyLcs & EducaLonal Leadership School of EducaLon Simon Knight Research Fellow
- 7. cic.uts.edu.au CIC works as a creative incubator to catalyse thinking about the use of data and analytics among students, educators, researchers and leaders. We do this through research, prototyping, evaluation, embedding and education. So we work with faculties Business Communication Education Engineering Health Information Technology Law Science and support units Careers HELPS Library IML ITD Jumbunna IHL U:PASS Student Services
- 9. UTS Data Challenges 9 Real time analytics for student writing Real time prediction of ‘at risk’ course dropouts Predictive analytics for student employment Analytics for UTS learning.futures Real time analytics for student collaboration Values, Data Infrastructure, Algorithmic Accountability Analytics for student resilience and agency
- 10. What would it mean to convene internal “Data Challenges?” Competitive Teams, Taskforces, or NICs? 10
- 11. 11
- 13. 13 Or a multi-party taskforce, coordinated and collaborating?
- 14. CIC is currently coordinating a pilot Data Challenge on a UTS Graduate Employability dataset
- 15. CIC is currently coordinating a pilot Data Challenge on a UTS Graduate Employability dataset AGS Dataset Can new analytics approaches add insight? Planning & Quality Unit Student Services/Careers The ‘clients’ owning the problem DVC (Education) CIC ‘Knowledge Broker’ School of Math. Sciences FEIT School of Software Advanced Analytics Institute Analytics Experts Clarify dataset, potential contributions, rules of the game, schedule, and operationalise the challenge in quantifiable terms
- 16. CIC is currently coordinating a pilot Data Challenge on a UTS Graduate Employability dataset Textual data (AGS survey comments) Quantitative data (AGS survey item responses) • How accurately can we predict the employed status of a student, given the AGS data? • Do text analytics on student comments add predictive value or other insights? • What are the pros/cons of different statistical and machine learning approaches? • What effort was required, and what obstacles were encountered?
- 17. 17 Or a Networked Improvement Community sharing data and building evidence on a wicked problem? (not acute emergency response • using Improvement Science methodologies for practice-based evidence)
- 18. 18 CIC is working on Networked Improvement Community platforms in education globally, and within UTS http://carnegiefoundationsummit.org http://cdn.carnegiefoundation.org/wp-content/uploads/2014/09/bryk-gomez_building-nics-education.pdf
- 19. Evidence-based Educational Improvement Cycle 19 Picturing the learning system… Human + Automated Analysis Data Data Data Hypotheses, Evidence, Arguments, Insights Prototype Learning Designs Assumptions: what works when and why?
- 20. An Evidence Hub shows who in the community is tackling which parts of the problem People / Organizations / Projects / Claims / Evidence Evidence Hub for Research by Children & Young People: http://rcyp.evidence-hub.net 20
- 21. Impact Map: how much evidence is there to support an improvement hypothesis? http://oermap.org/hypothesis/578/hypothesis-a-performance/ 21
- 22. How do we design and test data science and analytics for 22 ?
- 25. group collaboration analytics 25 Roberto Martinez-Maldonado
- 26. Design drivers: orchestration and awareness MtClassroom learning analytics in a multi-tabletop classroom
- 27. MtClassroom Architecture: enabling collaborative classroom learning analytics
- 28. cic.uts.edu.au In the future, might UTS give collaboration feedback like this? R. Martinez, K. Yacef, J. Kay, and B. Schwendimann. An interactive teacher’s dashboard for monitoring multiple groups in a multi-tabletop learning environment. Proceedings of Intelligent Tutoring Systems, pages 482-492. Springer, 2012. Automatic student tracking from multiple digital tabletops in classroom Analyse the students’ activity traces for significant patterns Visualise on a teacher’s dashboard
- 29. cic.uts.edu.au R. Martinez, K. Yacef, J. Kay, and B. Schwendimann. An interactive teacher’s dashboard for monitoring multiple groups in a multi-tabletop learning environment. Proceedings of Intelligent Tutoring Systems, pages 482-492. Springer, 2012.
- 30. writing analytics 30 Ágnes Sándor Xiaolong Wang Simon Knight
- 31. Writing Analytics product space is growing… 31 http://turnitin.com/en_us/features/turnitin-scoring-engine http://www.pearsonassessments.com/products/100000681/writing-coach.html
- 32. AWA: Academic Writing Analytics tool 32 The Tag Clouds invite you to reflect on the concepts, people and places in the text
- 33. Visualizations of writing cohesion 33 Whitelock, D., D. Field, J. T. E. Richardson, N. V. Labeke and S. Pulman (2014). Designing and Testing Visual Representations of Draft Essays for Higher Education Students. 2nd International Workshop on Discourse-Centric Learning Analytics, Fourth International Conference on Learning Analytics and Knowledge, Indianapolis, Indiana, USA. https://dcla14.files.wordpress.com/2014/03/dcla14_whitelock_etal.pdf
- 34. Rhetorical functions of metadiscourse identified by the Xerox Incremental Parser (XIP) BACKGROUND KNOWLEDGE Recent studies indicate … … the previously proposed … … is universally accepted ... NOVELTY ... new insights provide direct evidence ... ... we suggest a new ... approach ... ... results define a novel role ... OPEN QUESTION … little is known … … role … has been elusive Current data is insufficient … GENERALIZING ... emerging as a promising approach Our understanding ... has grown exponentially ... ... growing recognition of the importance ... CONTRASTING IDEAS … unorthodox view resolves … paradoxes … In contrast with previous hypotheses ... ... inconsistent with past findings ... SIGNIFICANCE studies ... have provided important advances Knowledge ... is crucial for ... understanding valuable information ... from studies SURPRISE We have recently observed ... surprisingly We have identified ... unusual The recent discovery ... suggests intriguing roles SUMMARIZING The goal of this study ... Here, we show ... Altogether, our results ... indicate 34
- 35. AWA: Academic Writing Analytics tool 35 Highlighted sentences are colour- coded according to their broad type Sentences with Function Keys have more precise functions (e.g. Novelty) http://bit.ly/utscicawa
- 36. AWA: Academic Writing Analytics tool 36 Roll over sentences with Fkeys for a popup reminding you of their meaning http://bit.ly/utscicawa
- 37. Could this work for reflective writing? 37
- 38. A “reflection detection NLP architecture” 38Ullmann, T. D. (2011). An Architecture for the Automated Detection of Textual Indicators of Reflection. In W. Reinhardt, T. D. Ullmann, P. Scott, V. Pammer, O. Conlan, & A. Berlanga (Eds.), Proc. 1st European Workshop on Awareness & Reflection in Learning Networks (pp. 138–151). Palermo, IT. http://ceur-ws.org/Vol-790/paper14.pdf Demo: http://reflectr.eduinf.eu/reflectr Based on UIMA http://uima.apache.org
- 40. “It’s more than knowledge and skills. For the innovation economy, dispositions come into play: readiness to collaborate; attention to multiple perspectives; initiative; persistence; curiosity.” Larry Rosenstock High Tech High San Diego hightechhigh.org LearningREimagined project: http://learning-reimagined.com Larry Rosenstock: http://audioboo.fm/boos/1669375-50-seconds-of-larry-rosenstock-ceo-of-hightechhigh-on-how-he-would-re-imagine-learning Knowledge, Skills & Dispositions 40
- 41. Deakin Crick, R., S. Huang, A. Ahmed Shafi and C. Goldspink (2015). Developing Resilient Agency in Learning: The Internal Structure of Learning Power. British Journal of Educational Studies: Published online: 24 Mar 2015. http://dx.doi.org/10.1080/00071005.2015.1006574 41 Evidencing learning dispositions: CLARA survey (Ruth Deakin Crick, UTS)
- 42. Mindful Agency Sense making Creativity CuriosityBelonging Collaboration Hope and optimism Immediate visual analytic generated by CLARA (Individual and cohort profiles + detailed reports + spreadsheets enabling further analysis) Rapid Visual Feedback to Stimulate Self-Directed Change A framework for a coaching conversation x 50,000 profiles => analytics potential
- 43. 43 Structural Equation Model underpinning CLARA Deakin Crick, R., S. Huang, A. Ahmed Shafi and C. Goldspink (2015). Developing Resilient Agency in Learning: The Internal Structure of Learning Power. British Journal of Educational Studies: Published online: 24 Mar 2015. http://dx.doi.org/10.1080/00071005.2015.1006574
- 44. From self-report to activity analytics for dispositions? Proxies for “Conscientiousness”? Shute, V. J. and M. Ventura (2013). Stealth Assessment: Measuring and supporting learning in video games. Cambridge, MA, MIT Press. Figure 5 from report to The John D. and Catherine T. MacArthur Foundation Reports on Digital Media and Learning http://myweb.fsu.edu/vshute/pdf/Stealth_Assessment.pdf
- 45. But personal informatics require personal sensemaking 45 Students need to be placed in control of this kind of analytics, not placed in a big data panopticon Quantified Self Qualified Self I am more than my digital profile. I have ultimate authority over its meaning — and who sees it. Cf. STUDENT PRIVACY SELF-MANAGEMENT: IMPLICATIONS FOR LEARNING ANALYTICS Paul Prinsloo and Sharon Slade, LAK15
- 46. big data, values and algorithmic accountability 46 Theresa Anderson
- 47. cic.uts.edu.au “We must ask difficult questions of Big Data’s models of intelligibility before they crystallize into new orthodoxies.” boyd & Crawford, 2012
- 48. cic.uts.edu.au “Raw data is both an oxymoron and a bad idea; to the contrary, data should be cooked with care.” Geof Bowker, 2005
- 49. cic.uts.edu.au engaging with the bright & dark Where is the ‘human’ in analytics – how can we ensure a humanist element remains present? What human/machine partnerships can/should we enable in computationally intensive work? @uts_mdsi #bright_dark
- 50. Invitation to think out of the box and play!... Multimodal expressions of Big Data @uts_mdsi #bright_dark