Multidisciplinarity vs. Multivocality, the case of “Learning Analytics"


Published on

In this communication presented at LAK2013 (Leuven), we consider an analysis of the TeLearn archive, of the Grand Challenges from the STELLAR Network of Excellence, of two Alpine Rendez-Vous 2011 workshops and research conducted in the Productive Multivocality initiative in order to discuss the notions of multidisciplinarity, multivocality and interidisciplinarity. We use this discussion as a springboard for addressing the term “Learning Analytics” and its relation to “Educational Data Mining”. Our goal is to launch a debate pertaining to what extent the different disciplines involved in the TEL community can be integrated on methodological and theoretical levels.

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Multidisciplinarity vs. Multivocality, the case of “Learning Analytics"

  1. 1. Multidisciplinarity vs. Multivocalitythe case of “Learning Analytics”Nicolas Balacheff1, Kristine Lund2,CNRS1,2, University of Grenoble1, University of Lyon2Learning Analytics & Knowlege ConferenceApril 8-12, 2013Leuven, BelgiumLaboratoire d’Informatique de Grenoble
  2. 2. 2Learning Analytics / Educational Data Mining- Conceptually grounded, coined to respond to research needs- Socially grounded, adopted as common conceptual flagshipsMultidisciplinarity,multivocality andinterdisciplinarityTo what extent can the different disciplines involved in the TELcommunity be integrated on methodological and theoreticallevels?Origin of concepts and methods in the TEL research areaProblématique,theoretical frameworkand methodologyIn which way does each expression solve problemsidentified in the TEL research area and how specific arethey? What relations do they have with other concepts inthe domain?The case ofLA and EDM
  3. 3. 3The TEL dictionary perspective:defining in order to stop reinventing the wheelThe evolution of TEL research is rapid and motivations are diverseThe language is often ill-definedDifferences in terminology: variations among communities or conceptual differencesDifficult to ensure that the wheel is not being reinventedThe case of LA and EDM- LA: introduced in 2009 -- first conference 2011 -- no identified endogenous precursors butstrong heterogeneous (analytics)- EDM: introduced in 2000 -- first conference 2008 – precursor workshops jointly held withITS, AIED, ICALT, etc. – evidence of historical roots in learner modelingDifferent histories, but does that imply semantic differences?
  4. 4. 4(partial view of the links).Colors representthe clusters centered onthe most importantkeywordsData (orange)linkAnalytics (blue)andEducationalData (red)Source: Stellar GrandChallenge problems relating tolearning analytics
  5. 5. 5(partial view of the links).Colors representthe clusters centered onthe most importantkeywordsData (orange)linkAnalytics (blue)andEducationalData (red)Source: Stellar GrandChallenge problems relating tolearning analyticsCloser look at the DataTEL and the Productive Multivocalityworkshops at the Alpine Rendez Vous 2011- although data is at the heart of both, there is almost no sharedvocabulary, apart from cognates of “learning”- in addition, there is a great difference in terms of scope in thetwo workshop’s objectives
  6. 6. 6DataTEL“The research on TEL recommender systems can contribute todecreasing the drop-out rate”“customize existing recommendation algorithms for learning,employ recommender systems in real-life scenarios and developsuitable evaluation criteria for different kinds of recommendersystems”.Productive Multivocality“supportive structure for a dialogical interpretation of the data inorder to make the community and stakeholders aware what resultsconverge among the different data sets and different interpretationsand in order to identify open questions”.The TEL dictionary perspective:defining in order to stop reinventing the wheelData is at the core of both communities, but in different ways- One focuses on improving algorithms to treat data (recommender systems)- The other focuses on interpretation of shared dataSharing data is a potentially productive move for TEL research, but not an easy oneWhat « data » means might be the next questiona challenge illustrated in the second part of this communication
  7. 7. 7Initial conditions for Productive Multivocalityusing the pivotal moment asa boundary objectX 5 Editors: Suthers, D., Lund, K., Rosé, C., Law, N. & Teplovs, C.
  8. 8. 8Multidisciplinarity, interdisciplinarity and multivocality• Neither theoretical perspectives nor actual results from differentparticipating disciplines are integrated during multidisciplinarity subject approached from different angles, using different disciplinary perspectives (vanden Besselaar & Heimerik, 2001) Each research group stays within their own boundaries (Choi & Pak, 2001)• Interdisciplinary research integrates contributing disciplines by creatingits own theoretical, conceptual and methodological identity analyzes, synthesizes and harmonizes links between disciplines into a coordinated andcoherent whole (van den Besselaar & Heimerik, 2001)• Multivocal research performing multiple analyses from different epistemological and methodologicalframeworks on a shared corpus (e.g. group interactions in pedagogical contexts) Productive : analytical concepts were refined, epistemological positions were madeexplicit, and the conditions under which learning occurs were characterized, but withdifferent perspectives, thus allowing discussion about learning
  9. 9. 9Our argument• The LA community is much like the CSCL community Multidisciplinary with a potential for interdisciplinarity• (Our version of ) multivocality is closer to interdisciplinarity than tomultidisciplinarity We will use an example from the Productive Multivocality Initiative to illustrate this Multivocality and interdisciplinarity are approaches that move research fieldsforward– The communities researching “Learning Analytics” are nicely positioned tobenefit from such approaches, much in the same way that CSCL has been
  10. 10. 10How multivocality can tend towards interdisciplinarity• Step 1 3 researchers each designate the moment they call pivotal– Different visions of learning are made explicit– “Moments” are of differing length (cf. unit of analysis/interaction)1 Trausan-Matu2 Shirouzu3 Chiu1 Trausan-Matu1 Trausan-Matu“Fold, then cut out the 3/4 of 2/3 of theorigami paper”
  11. 11. 11Multivocality without convergence• The comparison of two researcher’s pivotal moments lead toprogress in each other’s problématiques, but not to integrating oneither a theoretical or methodological level Taking another researcher’s pivotal moments and interpreting them in one’s ownframework (e.g. Chiu : breakpoints in frequency of new ideas corresponded towhen and how the pedagogical designer’s intentions were actualized by students’behavior - Shirouzu) Neither methodological nor theoretical convergence is achieved, but a discussionhas begun
  12. 12. 12Multivocality with convergence• The comparison of two other researcher’s pivotal moments lead toprogress in one researcher’s problématique, but also to integratingone of the researchers’ approaches on a methodological level Trausan-Matu extended the definition of an analytical concept (e.g. Bakhtinian“voices” include gestures) extended the domain of application (e.g. from just chat to face-to-face interactions) Deeper theoretical integration is more difficult– We do not always aspire to that because tension can be productive
  13. 13. 13“Data” as a boundary object for learning analytics andeducational data miningWhat is shared by a teacher having to manage a lesson on ratio and proportion, andby the dean of the university having to ensure the success of the freshman class?How far is the meaning of “learning” the same in both cases?We join Siemens and Baker in a call for cooperation with the suggestion ofan analysis of…- the nature of data- the problématiques driving underlying commonalities and differencesperhaps for the sake of a new theoretical, conceptual and methodological identity for bothLearning Analytics and Educational Data Mining
  14. 14. 14LA“Learning analytics is the measurement, collection,analysis and reporting of data about learners andtheir contexts, for purposes of understanding andoptimizing learning and the environments in which itoccurs” (Long and Siemens, EDUCAUSE 2011).EDM“Educational Data Mining is a term used forprocesses designed for the analysis of data fromeducational settings to better understand studentsand the settings which they learn in.” (Desmaraisand Baker, TEL Dictionary 2012)The TEL dictionary perspective:defining in order to stop reinventing the wheel- Are the Learning Analytics tools imported from analytics sufficient for relevantlyanalyzing learning data?- Should all data attached to the activities of a student be considered as learning data?- Isn’t Learning Analytics reducing successful learning to the academic success ofstudents in their institutions, limiting de facto the problématique of TEL research?- Compared to the classical problématique of “learner modeling”, what are the specificcontributions of “Learning Analytics”?
  15. 15. 15AfterwordFrom…- 42 papers from LAK12- 24 papers from LAK11 76 docs / articles- 10 papers from JETS12articlessubjectarticlesbody"learninganalytics" 10 564"educationaldata mining" 3 40 LA "learninganalytic“learning (1000.0) learn (909.0) analytic(498.0) learner (372.0) activity (276.0)research (257.0) context (230.0) social (222.0)design (218.0) provide (214.0) community(205.0) knowledge (203.0) practice (196.0)tool (192.0) individual (180.0) datum (175.0)model (169.0) development (160.0)environment (159.0) EDM "educationaldatummining"cluster (140.0) use (125.0) datum (94.0) teacher(91.0) student (85.0) user (81.0) clustering(76.0) project (72.0) mining(66.0) system (64.0)theme (62.0) pattern (57.0) lecture (57.0) study(55.0) video (51.0) online (49.0) feature (43.0)model (43.0) class (41.0) group (40.0) question(39.0) final (39.0) classroom (38.0) tool(37.0) level (35.0) teaching (33.0)instructional (32.0)1. Latent Dirichlet Allocation on the preprocessed corpus (stopwords elimination,discarding parentheses and phrases of less than 2 words & lemmatizing) -- toenforce the search for the specific concepts in the topic model – to consider themcompound words.after this step emerges: "learninganalytic" (without s) and"educationaldatummining" (with datum instead of data)2. Trained incremental LDA topic models starting from only the LAK corpus (1.3MB)with 20 topics (a topic contains all words, with their corresponding weights indescending order with concepts semantically related that emerge from co-occurencerelations3. adding chunks of TASA corpus  results to comeThanks! To Mihail Dascalufor bringing his expertise withinvery short delay! Work in progress!