Expertise Estimation based on
Simple Multimodal Features
Xavier Ochoa, Katherine Chiluiza,
Gonzalo Méndez, Gonzalo Luzardo...
Download this presentation

http://www.slideshare.net/xaoch
How to (easily) obtain
multimodal features?
What is already there?
Three Approaches
• Literature-based features
• Common-sense-based features
• “Why not?”-based features
All approaches proved useful
Proof that we are in an early stage
Video: Calculator Use (NTCU)
• Idea:
– Calculator user is the one solving the problem

• Procedure:
– Obtain a picture of ...
was
on
hat
ved
inand

format ions capabilit ies provided by OpenCV . W hile t here
were some frames in which t his mat chi...
Video: Total Movement (TM)
• Idea:
– Most active student is the leader/expert?

• Procedure:
– Subtract current frame from...
image out put by t he Codebook algorit hm. T his magnit ude,
when comput ed for t he ent ire problem solving session, resu...
Video: Distance from center table
(DHT)
• Idea:
– If the head is near the table (over paper) the
student is working on the...
lem
d to
cupar-

sped as
preodeant
mall
ndihere
ned

and t hen, t he average of t hese dist ances is obt ained by problem ...
Audio: Processing
Audio: Processing
Audio: Features
•
•
•
•
•

Number of Interventions (NOI)
Total Speech Duration (TSD)
Times Numbers were Mentioned (TNM)
Ti...
Digital Pen: Basic Features
Digital Pen: Basic Features
•
•
•
•
•

Total Number of Strokes (TNS)
Average Number of Points (ANP)
Average Stroke Path Le...
Digital Pen: Shape Recognition

Stronium – Sketch Recognition Libraries
Digital Pen: Shape Recognition
•
•
•
•
•
•

Number of Lines (NOL)
Number of Rectangles (NOR)
Number of Circles (NOC)
Numbe...
Features Variation
• When the features were evaluated inside a
group two variations were usually obtained:
– Percentage of...
Next step: Find predictive
features
Prediction at two levels
Problem and Group
Analysis at Problem level
Solving Problem Correctly
• All available problems were used
• Logistic Regression to model Stud...
Variable Value for Expert s Discriminat ion Power
P CU
> 0.41
4.44
C oeffi ci entNof t h e L ogi st i c M od el P r edi ct...
Analysis at Group Level
Expertise Estimation
• Data from group 2 was removed because
there was no expert
• Features were f...
ASD
AST L
ASP

MD
FW
Analysis
MP

Highest value
Lowest value
Group Level
Highest value

at
Expertise Estimation

T abl e 4...
Expert Estimation over Problems

Plateau reached after
just 4 problems
Main conclusion:
Simple features could identify
expertise
Conclusions
• Three strong features:
– Faster Writer (Digital Pen)
– Percentage of Calculator Use (Video)
– Times Numbers ...
Gracias / Thank you
Questions?

Xavier Ochoa
xavier@cti.espol.edu.ec
http://ariadne.cti.espol.edu.ec/xavier
Twitter: @xaoc...
Upcoming SlideShare
Loading in …5
×

Expert estimation from Multimodal Features

354 views
255 views

Published on

Grand Challenge on Multimodal Learning Analytics 2013

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
354
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Expert estimation from Multimodal Features

  1. 1. Expertise Estimation based on Simple Multimodal Features Xavier Ochoa, Katherine Chiluiza, Gonzalo Méndez, Gonzalo Luzardo, Bruno Guamán and James Castells Escuela Superior Politécnica del Litoral
  2. 2. Download this presentation http://www.slideshare.net/xaoch
  3. 3. How to (easily) obtain multimodal features? What is already there?
  4. 4. Three Approaches • Literature-based features • Common-sense-based features • “Why not?”-based features
  5. 5. All approaches proved useful Proof that we are in an early stage
  6. 6. Video: Calculator Use (NTCU) • Idea: – Calculator user is the one solving the problem • Procedure: – Obtain a picture of the calculator – Track the position and angle of the image in the video using SURF + FLANN + Rigid Object Transformation (OpenCV) – Determine to which student the calculator is pointing in each frame
  7. 7. was on hat ved inand format ions capabilit ies provided by OpenCV . W hile t here were some frames in which t his mat ching was not possible due t o object occlusions or changes in t he illuminat ion of t he calculat or, in general t he described det ect ion t echnique was robust and provided useful posit ion and direct ion dat a. Video: Calculator Use (NTCU) ing by ent was core ven iffion, ex- at h t et F i gur e 1: D et er m i n at i on of w hi ch st u dent i s u si n g
  8. 8. Video: Total Movement (TM) • Idea: – Most active student is the leader/expert? • Procedure: – Subtract current frame from previous frame – Codebook algorithm to eliminate noise-movement – Add the number of remaining pixels
  9. 9. image out put by t he Codebook algorit hm. T his magnit ude, when comput ed for t he ent ire problem solving session, result s from summing up it s individual values obt ained from each frame t hat compose a problem recording. Video: Total Movement (TM) (a) Original frame (b) Difference frame F i gu r e 2: R esul t s of t he C odeb ook al gor i t hm .
  10. 10. Video: Distance from center table (DHT) • Idea: – If the head is near the table (over paper) the student is working on the problem • Procedure: – Identify image of heads – Use TLD algorithm to track heads – Determine the distance from head to center of table
  11. 11. lem d to cupar- sped as preodeant mall ndihere ned and t hen, t he average of t hese dist ances is obt ained by problem (see Figure 3). A ddit ionally, t he variance of t he average dist ance head t o t able (SD-DHT ), was calculat ed t o det ermine if a part icipant remains most ly st at ic or varies his or her dist ance t o t he t able. Video: Distance from center table (DHT) deary ude, reom F i gur e 3: C al cu l at i on of t h e di st an ce of t h e st u d ent ’ s
  12. 12. Audio: Processing
  13. 13. Audio: Processing
  14. 14. Audio: Features • • • • • Number of Interventions (NOI) Total Speech Duration (TSD) Times Numbers were Mentioned (TNM) Times Math Terms were Mentioned (TMTM) Times Commands were Pronounced (TCP)
  15. 15. Digital Pen: Basic Features
  16. 16. Digital Pen: Basic Features • • • • • Total Number of Strokes (TNS) Average Number of Points (ANP) Average Stroke Path Length (ASPL) Average Stroke Displacement (ASD) Average Stroke Pressure (ASP)
  17. 17. Digital Pen: Shape Recognition Stronium – Sketch Recognition Libraries
  18. 18. Digital Pen: Shape Recognition • • • • • • Number of Lines (NOL) Number of Rectangles (NOR) Number of Circles (NOC) Number of Ellipses (NOE) Number of Arrows (NOA) Number of Figures (NOF)
  19. 19. Features Variation • When the features were evaluated inside a group two variations were usually obtained: – Percentage of the total (e.g. Calculator Use) – Highest / Lowest (e.g. Faster Writer, Lowest Time)
  20. 20. Next step: Find predictive features
  21. 21. Prediction at two levels Problem and Group
  22. 22. Analysis at Problem level Solving Problem Correctly • All available problems were used • Logistic Regression to model Student Solving Problem Correctly • Resulting model was significantly reliable • 60,9% of the problem solving student was identified • 71,8% of incorrectly solved problems were identified
  23. 23. Variable Value for Expert s Discriminat ion Power P CU > 0.41 4.44 C oeffi ci entNof t h e L ogi st i c M od el P r edi ct i ng Od ds for a St u dent Sol v i n g C or r ect l y Ps M > 34.74 3.19 ASP Variable < 38.05 Predict or L B 2.86 W ald df p value exp(B ) Number of Int ervent ions (N OI ) 0.0682.86 24.019 1 0.001 0.934 N OR < 0.13 T imes numbers were ment ioned (T N M ) 0.175 23.816 1 0.001 1.192 T M T M > pronounced (T CP ) 0.3292.65 T imes commands were6.25 4.956 1 0.026 1.390 Analysis at problem level Proport ion of Calculat or Usage (P CU) Fast est St udent in t he Group (F W ) Constant 1.287 0.924 1.654 25.622 18.889 53.462 1 1 1 0.001 0.001 0.001 a 3.622 2.519 0.191 To calculat e t he probability of correct ly solving a problem N um b er of P oi nt s ( A N P ) : Represent s, in s. Classificat provided by of point s t hat composehe following sub-setR st at ist ical ion Trees,[21] for M ac, wer a st udent (P ) t each st roke formula should be used: by rpa number in t he software second part of t he analysis. St r oke T i m e L engt − 11. 7−L0.1N O I + s for N M + 0.3T C P + 1. 3P C U + 0. 9F W h A ST A ccount ehe (st udent) :needed, in 0.2T f milliseconds t hat t avP st= plet e each roke. + e− 11.7− 0. 1N O I + 0.2T N M + 0. 3T C P + 1.3P C U + 0.9F W 1 St r oke P at h L engt h ( A SP L ) : Represent s umber of pixels t hat t he t raject ory of st rokes St r oke D i sp l acem ent ( A SD ) : A ccount s for splacement defined by t he st art ing and ending 4.2 Expert prediction (1) 4.1 Odds of a student solving c problem A Logist ic regression was run wit h St udent
  24. 24. Analysis at Group Level Expertise Estimation • Data from group 2 was removed because there was no expert • Features were feed to a Classification Tree algorithm • Several variables had a high discrimination power between expert and non-experts • Best discrimination (6.53) result in 80% expert prediction and 90% non-expert prediction
  25. 25. ASD AST L ASP MD FW Analysis MP Highest value Lowest value Group Level Highest value at Expertise Estimation T abl e 4: C l assi fi cat i on t r ee sp l i t s w i t h nor m al i zed and non-n or m al i zed feat ur es Variable FW LP P CU MN PN M Value for Expert s > 0.5 > 34.74 > 38.05 > 0.13 > 6.25 Discriminat ion Power 6.53 6.53 4.44 4.03 3.19 classificat ion is maint ained and plat eau at t he final value around t he 12t h problem.
  26. 26. Expert Estimation over Problems Plateau reached after just 4 problems
  27. 27. Main conclusion: Simple features could identify expertise
  28. 28. Conclusions • Three strong features: – Faster Writer (Digital Pen) – Percentage of Calculator Use (Video) – Times Numbers were Mentioned (Audio) • Each mode provide discrimination power to establish expertise • These features maintain its discriminant power at lower levels (solving a problem correctly)
  29. 29. Gracias / Thank you Questions? Xavier Ochoa xavier@cti.espol.edu.ec http://ariadne.cti.espol.edu.ec/xavier Twitter: @xaoch

×