Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Expert estimation from Multimodal Features


Published on

Grand Challenge on Multimodal Learning Analytics 2013

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

Expert estimation from Multimodal Features

  1. 1. Expertise Estimation based on Simple Multimodal Features Xavier Ochoa, Katherine Chiluiza, Gonzalo Méndez, Gonzalo Luzardo, Bruno Guamán and James Castells Escuela Superior Politécnica del Litoral
  2. 2. Download this presentation
  3. 3. How to (easily) obtain multimodal features? What is already there?
  4. 4. Three Approaches • Literature-based features • Common-sense-based features • “Why not?”-based features
  5. 5. All approaches proved useful Proof that we are in an early stage
  6. 6. Video: Calculator Use (NTCU) • Idea: – Calculator user is the one solving the problem • Procedure: – Obtain a picture of the calculator – Track the position and angle of the image in the video using SURF + FLANN + Rigid Object Transformation (OpenCV) – Determine to which student the calculator is pointing in each frame
  7. 7. was on hat ved inand format ions capabilit ies provided by OpenCV . W hile t here were some frames in which t his mat ching was not possible due t o object occlusions or changes in t he illuminat ion of t he calculat or, in general t he described det ect ion t echnique was robust and provided useful posit ion and direct ion dat a. Video: Calculator Use (NTCU) ing by ent was core ven iffion, ex- at h t et F i gur e 1: D et er m i n at i on of w hi ch st u dent i s u si n g
  8. 8. Video: Total Movement (TM) • Idea: – Most active student is the leader/expert? • Procedure: – Subtract current frame from previous frame – Codebook algorithm to eliminate noise-movement – Add the number of remaining pixels
  9. 9. image out put by t he Codebook algorit hm. T his magnit ude, when comput ed for t he ent ire problem solving session, result s from summing up it s individual values obt ained from each frame t hat compose a problem recording. Video: Total Movement (TM) (a) Original frame (b) Difference frame F i gu r e 2: R esul t s of t he C odeb ook al gor i t hm .
  10. 10. Video: Distance from center table (DHT) • Idea: – If the head is near the table (over paper) the student is working on the problem • Procedure: – Identify image of heads – Use TLD algorithm to track heads – Determine the distance from head to center of table
  11. 11. lem d to cupar- sped as preodeant mall ndihere ned and t hen, t he average of t hese dist ances is obt ained by problem (see Figure 3). A ddit ionally, t he variance of t he average dist ance head t o t able (SD-DHT ), was calculat ed t o det ermine if a part icipant remains most ly st at ic or varies his or her dist ance t o t he t able. Video: Distance from center table (DHT) deary ude, reom F i gur e 3: C al cu l at i on of t h e di st an ce of t h e st u d ent ’ s
  12. 12. Audio: Processing
  13. 13. Audio: Processing
  14. 14. Audio: Features • • • • • Number of Interventions (NOI) Total Speech Duration (TSD) Times Numbers were Mentioned (TNM) Times Math Terms were Mentioned (TMTM) Times Commands were Pronounced (TCP)
  15. 15. Digital Pen: Basic Features
  16. 16. Digital Pen: Basic Features • • • • • Total Number of Strokes (TNS) Average Number of Points (ANP) Average Stroke Path Length (ASPL) Average Stroke Displacement (ASD) Average Stroke Pressure (ASP)
  17. 17. Digital Pen: Shape Recognition Stronium – Sketch Recognition Libraries
  18. 18. Digital Pen: Shape Recognition • • • • • • Number of Lines (NOL) Number of Rectangles (NOR) Number of Circles (NOC) Number of Ellipses (NOE) Number of Arrows (NOA) Number of Figures (NOF)
  19. 19. Features Variation • When the features were evaluated inside a group two variations were usually obtained: – Percentage of the total (e.g. Calculator Use) – Highest / Lowest (e.g. Faster Writer, Lowest Time)
  20. 20. Next step: Find predictive features
  21. 21. Prediction at two levels Problem and Group
  22. 22. Analysis at Problem level Solving Problem Correctly • All available problems were used • Logistic Regression to model Student Solving Problem Correctly • Resulting model was significantly reliable • 60,9% of the problem solving student was identified • 71,8% of incorrectly solved problems were identified
  23. 23. Variable Value for Expert s Discriminat ion Power P CU > 0.41 4.44 C oeffi ci entNof t h e L ogi st i c M od el P r edi ct i ng Od ds for a St u dent Sol v i n g C or r ect l y Ps M > 34.74 3.19 ASP Variable < 38.05 Predict or L B 2.86 W ald df p value exp(B ) Number of Int ervent ions (N OI ) 0.0682.86 24.019 1 0.001 0.934 N OR < 0.13 T imes numbers were ment ioned (T N M ) 0.175 23.816 1 0.001 1.192 T M T M > pronounced (T CP ) 0.3292.65 T imes commands were6.25 4.956 1 0.026 1.390 Analysis at problem level Proport ion of Calculat or Usage (P CU) Fast est St udent in t he Group (F W ) Constant 1.287 0.924 1.654 25.622 18.889 53.462 1 1 1 0.001 0.001 0.001 a 3.622 2.519 0.191 To calculat e t he probability of correct ly solving a problem N um b er of P oi nt s ( A N P ) : Represent s, in s. Classificat provided by of point s t hat composehe following sub-setR st at ist ical ion Trees,[21] for M ac, wer a st udent (P ) t each st roke formula should be used: by rpa number in t he software second part of t he analysis. St r oke T i m e L engt − 11. 7−L0.1N O I + s for N M + 0.3T C P + 1. 3P C U + 0. 9F W h A ST A ccount ehe (st udent) :needed, in 0.2T f milliseconds t hat t avP st= plet e each roke. + e− 11.7− 0. 1N O I + 0.2T N M + 0. 3T C P + 1.3P C U + 0.9F W 1 St r oke P at h L engt h ( A SP L ) : Represent s umber of pixels t hat t he t raject ory of st rokes St r oke D i sp l acem ent ( A SD ) : A ccount s for splacement defined by t he st art ing and ending 4.2 Expert prediction (1) 4.1 Odds of a student solving c problem A Logist ic regression was run wit h St udent
  24. 24. Analysis at Group Level Expertise Estimation • Data from group 2 was removed because there was no expert • Features were feed to a Classification Tree algorithm • Several variables had a high discrimination power between expert and non-experts • Best discrimination (6.53) result in 80% expert prediction and 90% non-expert prediction
  25. 25. ASD AST L ASP MD FW Analysis MP Highest value Lowest value Group Level Highest value at Expertise Estimation T abl e 4: C l assi fi cat i on t r ee sp l i t s w i t h nor m al i zed and non-n or m al i zed feat ur es Variable FW LP P CU MN PN M Value for Expert s > 0.5 > 34.74 > 38.05 > 0.13 > 6.25 Discriminat ion Power 6.53 6.53 4.44 4.03 3.19 classificat ion is maint ained and plat eau at t he final value around t he 12t h problem.
  26. 26. Expert Estimation over Problems Plateau reached after just 4 problems
  27. 27. Main conclusion: Simple features could identify expertise
  28. 28. Conclusions • Three strong features: – Faster Writer (Digital Pen) – Percentage of Calculator Use (Video) – Times Numbers were Mentioned (Audio) • Each mode provide discrimination power to establish expertise • These features maintain its discriminant power at lower levels (solving a problem correctly)
  29. 29. Gracias / Thank you Questions? Xavier Ochoa Twitter: @xaoch