Case study: evaluation of a tool for searching inside a collection of multimodal e-lectures
Upcoming SlideShare
Loading in...5

Like this? Share it with your network

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 3 2 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Case Study: Evaluation of a Tool for Searching inside a Collection of Multimodal e-Lectures Marco Ronchetti Angela Fogarolli Dipartim. Informatica e Telecomunicazioni Universit à di Trento Italy
  • 2. Logical Architecture Synchronous ( webcast ) Asynchronous ( On line , or download - podcast ) Asynchronous( CD - DVD )
  • 3. Acquisition setup
  • 4. Cognitive interface
    • The main focus is on the projected slide ,
    • a clear voice is very important,
    • the video carries additional information like gestures , and can show the environment just enough
    • Navigation is possible
    • It must be also possible to use the blackboard!
  • 5. LODE: the user interface
  • 6. New Flash version
  • 7. Lode - portability Supporto della didattica tradizionale Mac – Win - Linux Player MP4 (no slides yet) Presently attempting a port to mobile phones
  • 8. Student feedback
    • Enthusiastic .
    • We were “forced” to extend the experiment to a second course!
    75% anticipated using the system often or very often 5% 63% total 2 nd exam (23 stud.) 1 st exam (45 stud.) students who… 4% 6% … followed the course completely off-line 87% 51% … used the system to review at least one lecture
  • 9. Advantages from student’s perspective
    • Better time management:
      • ability to recover lectures lost due to forced absence (illness, work or other time-frame incompatibility);
      • ability to better organize their time , deciding not to be present at some lecture (elective absence);
    • Better understanding
      • review some critical point (cases of poor understanding of a section due to concentration drop, excessive speed in an explanation or intrinsic difficulty);
      • ability to check the correctness of notes taken during a lecture;
  • 10. Other advantages
    • Miscellaneous
      • perception by the student of a better service provided by the university ;
      • support foreign (italian) students who might have difficulties with the Italian (English) language ;
      • enrichment of the e-learning portfolio;
      • possibility for the teacher to view himself;
      • possibility to reuse lectures (across time, or in different contexts!)
  • 11. Can we do more?
    • Scenario:
    • Thousands of recorded lectures.
    • Can we dig in them to extract material for (less formal) learning?
    • Cfr. approach by Mike Wald
      • (subtitling for accessibility)
  • 12. Needle - search results
  • 13. Needle - architecture ASR Multimodal search!
  • 14. Questions
    • Is it useful?
    • Is it effective?
  • 15. Scenario
    • 100% accurate transcription (manually corrected)
    • Goal: extract some info from a 2 hour lecture in half of the time…
    • Assess whether the tool could reduce the learning time and help students in their learning task: we do not aim at claiming or proving that a course delivery mode is better than the other.
    • Comparing delivery modes (e.g. distance learning with traditional classroom-based instruction) (Sener 2005) for the purpose of establishing the superiority of one delivery mode over another is specious, irrelevant and counterproductive because there is not an uniformity of practice.
  • 16. Experiment
    • 68 students, after 20 hours of a 50 hour Java course.
    • Group A: LODE
    • Group B: NEEDLE
  • 17. Experiment
    • Pre-Test: 5 questions, 5 minutes
    • LODE group: 70 minutes for the lecture, plus
    • 24 minutes for the post-test (8 questions): total 94 minutes to accomplish the task
    • NEEDLE group: 48 minutes total for responding to the same 8 questions (50% of the time)
  • 18. Pre-test results
  • 19. Test results
  • 20. Analysis
    • In the subset of questions, which require students to have a complete understanding of a part of the lecture both groups performed in the same way.
    • In answering those questions that also required some reasoning about concepts , group LODE – who had listened to the entire lecture – performed better.
    • In the questions where the students where asked to provide the right definition for a concept group NEEDLE performed better.
  • 21. Further investigations
    • Multivariate regression:
    • no significant results
    • Clustering techniques
    • Indication that some feature seem to be present - investigation is still on its way
  • 22. Conclusion
    • NEEDLE tool is more geared towards information finding than towards actual learning.
    • A similar pattern could be reasonably expected in a similar experiment performed by reading some assigned material versus searching quick answer with Google.
    • However, our finding seems to show that Needle works very well as a tool for extracting information from a video source.