Implementing SpokenMedia for the Indian Institute for Human Settlements

Brandon Muramatsu
Brandon MuramatsuAssociate Director, Projects
Implementing SpokenMedia for the  Indian Institute for Human Settlements Brandon Muramatsu Andrew McKinney Peter Wilkins July 2010 Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0  United States License ( creativecommons.org/licenses/by-nc-sa/3.0/us/ ) Citation: Muramatsu, B., McKinney, A., Wilkins, P. (2010). Implementing SpokenMedia for the  Indian Institute for  Human Settlements. Presented at Technology for Education 2010: Mumbai, India, July 1, 2010.
Case Study of Using SpokenMedia for IIHS ,[object Object],[object Object],[object Object],[object Object],[object Object]
MIT Office for Educational Innovation  and Technology ,[object Object],[object Object],[object Object],[object Object],Experiment Incubate Transition Service
SpokenMedia Project Experiment Incubate Transition Service ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
The Indian Institute for Human Settlements (IIHS) will… “create India’s first independent National Innovation University focused on the challenges and opportunities of its urbanisation.” –  Indian Institute for Human Settlements:  Curriculum Framework Version 3.0   January 2010
“ The IIHS Website is our commitment to a different  way of looking at things.” –  Aromar Revi 5 January 2010
“ The Institution will fail or scale based on language.” –  Aromar Revi 5 January 2010
What did we do? Auto Transcribe Edit Translate Play
The Demo
How did we do it? Auto Transcribe Edit Translate Play
How do we do it? Spoken Lecture Research ,[object Object],[object Object],[object Object],[object Object],[object Object],James Glass [email_address] Supported with iCampus MIT/Microsoft Alliance funding
SpokenMedia Process We used a portion of the SpokenMedia process for the demo
How did we do it? Auto Transcribe Edit Translate Play
Edit & Translate: Accuracy Automatic Transcription Hand Transcription Time Adjusted Translated Hindi I I I मेरे खयाल से think think think once one one नयोजन की एक मुख्य चुनौती है and central so challenge central the  of challenger planning challenge of planning is planning nice legitimacy is legitimacy of legitimacy of of government government सरकार की एक ऐसी मुख्य संस्थान के रूप में वैधता  government as as
Automatic Speech Recognition Accuracy ,[object Object],[object Object],[object Object],[object Object],[object Object]
How did we do it? Auto Transcribe Edit Translate Play
The Player ,[object Object],[object Object],[object Object],[object Object],Search within Transcript Highlight text as spoken Switch transcript to different language(s)
SpokenMedia (circa January) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Challenges ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
SpokenMedia (July 2010)
Where are we heading? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Check it out for yourself ,[object Object],[object Object],[object Object]
Thank You! Brandon Muramatsu,  [email_address] Andrew McKinney,  [email_address] Peter Wilkins,  [email_address] ° Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0  United States License ( creativecommons.org/licenses/by-nc-sa/3.0/us/ )
1 of 23

Recommended

Opening Up IIHS Video with SpokenMedia by
Opening Up IIHS Video with SpokenMediaOpening Up IIHS Video with SpokenMedia
Opening Up IIHS Video with SpokenMediaBrandon Muramatsu
1.8K views20 slides
SpokenMedia Project: Media-Linked Transcripts and Rich Media Notebooks for Le... by
SpokenMedia Project: Media-Linked Transcripts and Rich Media Notebooks for Le...SpokenMedia Project: Media-Linked Transcripts and Rich Media Notebooks for Le...
SpokenMedia Project: Media-Linked Transcripts and Rich Media Notebooks for Le...Brandon Muramatsu
1.4K views24 slides
Improving the OER Experience: Enabling Rich Media Notebooks of OER Video and ... by
Improving the OER Experience: Enabling Rich Media Notebooks of OER Video and ...Improving the OER Experience: Enabling Rich Media Notebooks of OER Video and ...
Improving the OER Experience: Enabling Rich Media Notebooks of OER Video and ...Brandon Muramatsu
1.8K views24 slides
Building Community for Rich Media Notebooks: The SpokenMedia Project at NMC 2009 by
Building Community for Rich Media Notebooks: The SpokenMedia Project at NMC 2009Building Community for Rich Media Notebooks: The SpokenMedia Project at NMC 2009
Building Community for Rich Media Notebooks: The SpokenMedia Project at NMC 2009Brandon Muramatsu
1.8K views23 slides
Automated Lecture Transcription at OCW Consortium Global Meeting 2009 by
Automated Lecture Transcription at OCW Consortium Global Meeting 2009Automated Lecture Transcription at OCW Consortium Global Meeting 2009
Automated Lecture Transcription at OCW Consortium Global Meeting 2009Brandon Muramatsu
1.5K views14 slides
Project Greenfield: A New Way of Thinking about MIT OpenCourseWare by
Project Greenfield: A New Way of Thinking about MIT OpenCourseWareProject Greenfield: A New Way of Thinking about MIT OpenCourseWare
Project Greenfield: A New Way of Thinking about MIT OpenCourseWareBrandon Muramatsu
6.6K views22 slides

More Related Content

Viewers also liked

IIHS Open Framework-SpokenMedia by
IIHS Open Framework-SpokenMediaIIHS Open Framework-SpokenMedia
IIHS Open Framework-SpokenMediaBrandon Muramatsu
1.7K views19 slides
Settlement Characteristics by
Settlement CharacteristicsSettlement Characteristics
Settlement Characteristicswhiskeyhj
69.2K views20 slides
Unit 1 planning c oncepts ppt by
Unit 1   planning c oncepts  pptUnit 1   planning c oncepts  ppt
Unit 1 planning c oncepts pptEzhil Tamizh
45.7K views48 slides
Rural housing in india by
Rural housing in indiaRural housing in india
Rural housing in indiakumaresan2704
45.6K views52 slides
Evolution of settlements by
Evolution of settlementsEvolution of settlements
Evolution of settlementsyusra_gul
73.1K views29 slides
Human Settlements by
Human SettlementsHuman Settlements
Human Settlementspeyne
101.9K views54 slides

Viewers also liked(7)

Settlement Characteristics by whiskeyhj
Settlement CharacteristicsSettlement Characteristics
Settlement Characteristics
whiskeyhj69.2K views
Unit 1 planning c oncepts ppt by Ezhil Tamizh
Unit 1   planning c oncepts  pptUnit 1   planning c oncepts  ppt
Unit 1 planning c oncepts ppt
Ezhil Tamizh45.7K views
Rural housing in india by kumaresan2704
Rural housing in indiaRural housing in india
Rural housing in india
kumaresan270445.6K views
Evolution of settlements by yusra_gul
Evolution of settlementsEvolution of settlements
Evolution of settlements
yusra_gul73.1K views
Human Settlements by peyne
Human SettlementsHuman Settlements
Human Settlements
peyne101.9K views

Similar to Implementing SpokenMedia for the Indian Institute for Human Settlements

How to Get Online Course Translations of the Highest Quality by
How to Get Online Course Translations of the Highest QualityHow to Get Online Course Translations of the Highest Quality
How to Get Online Course Translations of the Highest QualityCommLab India – Rapid eLearning Solutions
518 views23 slides
visH (fin).pptx by
visH (fin).pptxvisH (fin).pptx
visH (fin).pptxtefflontrolegdy
32 views21 slides
Rapid Innovative Design Notes by
Rapid Innovative Design NotesRapid Innovative Design Notes
Rapid Innovative Design Notesspotlearning
882 views72 slides
Technology by
TechnologyTechnology
TechnologyGillian Lord
296 views31 slides
Video Captioning for Accessibility: University of Florida and Regis Universit... by
Video Captioning for Accessibility: University of Florida and Regis Universit...Video Captioning for Accessibility: University of Florida and Regis Universit...
Video Captioning for Accessibility: University of Florida and Regis Universit...3Play Media
1.6K views46 slides
IRJET - Analysis on Code-Mixed Data for Movie Reviews by
IRJET - Analysis on Code-Mixed Data for Movie ReviewsIRJET - Analysis on Code-Mixed Data for Movie Reviews
IRJET - Analysis on Code-Mixed Data for Movie ReviewsIRJET Journal
11 views4 slides

Similar to Implementing SpokenMedia for the Indian Institute for Human Settlements(20)

Rapid Innovative Design Notes by spotlearning
Rapid Innovative Design NotesRapid Innovative Design Notes
Rapid Innovative Design Notes
spotlearning882 views
Video Captioning for Accessibility: University of Florida and Regis Universit... by 3Play Media
Video Captioning for Accessibility: University of Florida and Regis Universit...Video Captioning for Accessibility: University of Florida and Regis Universit...
Video Captioning for Accessibility: University of Florida and Regis Universit...
3Play Media1.6K views
IRJET - Analysis on Code-Mixed Data for Movie Reviews by IRJET Journal
IRJET - Analysis on Code-Mixed Data for Movie ReviewsIRJET - Analysis on Code-Mixed Data for Movie Reviews
IRJET - Analysis on Code-Mixed Data for Movie Reviews
IRJET Journal11 views
Tips for Creating an Accessible & Engaging Virtual Classroom by 3Play Media
Tips for Creating an Accessible & Engaging Virtual ClassroomTips for Creating an Accessible & Engaging Virtual Classroom
Tips for Creating an Accessible & Engaging Virtual Classroom
3Play Media741 views
Video Accessibility Toolkit for Success in a Virtual Environment by 3Play Media
Video Accessibility Toolkit for Success in a Virtual EnvironmentVideo Accessibility Toolkit for Success in a Virtual Environment
Video Accessibility Toolkit for Success in a Virtual Environment
3Play Media1.7K views
Vlogging by Aiden Yeh
VloggingVlogging
Vlogging
Aiden Yeh6.1K views
Ebook media-and-entertainment by BnsplBraahmam
Ebook media-and-entertainmentEbook media-and-entertainment
Ebook media-and-entertainment
BnsplBraahmam67 views
Technology supportswrittenlanuagesccec dld by Cheryl Wissick
Technology supportswrittenlanuagesccec dldTechnology supportswrittenlanuagesccec dld
Technology supportswrittenlanuagesccec dld
Cheryl Wissick271 views
Article Summaries by ORhonda
Article SummariesArticle Summaries
Article Summaries
ORhonda706 views
Hindi –tamil text translation by Vaibhav Agarwal
Hindi –tamil text translationHindi –tamil text translation
Hindi –tamil text translation
Vaibhav Agarwal34.1K views
Inclusive E-Learning good practice becoming general practice by Staff Development
Inclusive E-Learning good practice becoming general practiceInclusive E-Learning good practice becoming general practice
Inclusive E-Learning good practice becoming general practice
Staff Development420 views

More from Brandon Muramatsu

Digital Credentials Enabling Mobility and Verification of Educational Achieve... by
Digital Credentials Enabling Mobility and Verification of Educational Achieve...Digital Credentials Enabling Mobility and Verification of Educational Achieve...
Digital Credentials Enabling Mobility and Verification of Educational Achieve...Brandon Muramatsu
239 views21 slides
Sustainability of OER Initiatives: An Interactive Discussion by
Sustainability of OER Initiatives: An Interactive DiscussionSustainability of OER Initiatives: An Interactive Discussion
Sustainability of OER Initiatives: An Interactive DiscussionBrandon Muramatsu
212 views17 slides
Bridging the Gap: Mixing approaches, content and tools to help college students by
Bridging the Gap: Mixing approaches, content and tools to help college studentsBridging the Gap: Mixing approaches, content and tools to help college students
Bridging the Gap: Mixing approaches, content and tools to help college studentsBrandon Muramatsu
201 views11 slides
Federations & Backstage: Thoughts for a Geoscience Education Infrastructure by
Federations & Backstage: Thoughts for a Geoscience Education InfrastructureFederations & Backstage: Thoughts for a Geoscience Education Infrastructure
Federations & Backstage: Thoughts for a Geoscience Education InfrastructureBrandon Muramatsu
176 views9 slides
The Connected Learning Initiative Quality at Scale in India by
The Connected Learning Initiative Quality at Scale in IndiaThe Connected Learning Initiative Quality at Scale in India
The Connected Learning Initiative Quality at Scale in IndiaBrandon Muramatsu
172 views18 slides
The Connected Learning Initiative Quality at Scale in India by
The Connected Learning Initiative Quality at Scale in IndiaThe Connected Learning Initiative Quality at Scale in India
The Connected Learning Initiative Quality at Scale in IndiaBrandon Muramatsu
161 views9 slides

More from Brandon Muramatsu(20)

Digital Credentials Enabling Mobility and Verification of Educational Achieve... by Brandon Muramatsu
Digital Credentials Enabling Mobility and Verification of Educational Achieve...Digital Credentials Enabling Mobility and Verification of Educational Achieve...
Digital Credentials Enabling Mobility and Verification of Educational Achieve...
Brandon Muramatsu239 views
Sustainability of OER Initiatives: An Interactive Discussion by Brandon Muramatsu
Sustainability of OER Initiatives: An Interactive DiscussionSustainability of OER Initiatives: An Interactive Discussion
Sustainability of OER Initiatives: An Interactive Discussion
Brandon Muramatsu212 views
Bridging the Gap: Mixing approaches, content and tools to help college students by Brandon Muramatsu
Bridging the Gap: Mixing approaches, content and tools to help college studentsBridging the Gap: Mixing approaches, content and tools to help college students
Bridging the Gap: Mixing approaches, content and tools to help college students
Brandon Muramatsu201 views
Federations & Backstage: Thoughts for a Geoscience Education Infrastructure by Brandon Muramatsu
Federations & Backstage: Thoughts for a Geoscience Education InfrastructureFederations & Backstage: Thoughts for a Geoscience Education Infrastructure
Federations & Backstage: Thoughts for a Geoscience Education Infrastructure
Brandon Muramatsu176 views
The Connected Learning Initiative Quality at Scale in India by Brandon Muramatsu
The Connected Learning Initiative Quality at Scale in IndiaThe Connected Learning Initiative Quality at Scale in India
The Connected Learning Initiative Quality at Scale in India
Brandon Muramatsu172 views
The Connected Learning Initiative Quality at Scale in India by Brandon Muramatsu
The Connected Learning Initiative Quality at Scale in IndiaThe Connected Learning Initiative Quality at Scale in India
The Connected Learning Initiative Quality at Scale in India
Brandon Muramatsu161 views
Strategic Education Initiatives , MIT Open Learning by Brandon Muramatsu
Strategic Education Initiatives, MIT Open LearningStrategic Education Initiatives, MIT Open Learning
Strategic Education Initiatives , MIT Open Learning
Brandon Muramatsu164 views
Open Embedded Assessments: Play, Author; Anywhere, Anytime by Brandon Muramatsu
Open Embedded Assessments:Play, Author; Anywhere, AnytimeOpen Embedded Assessments:Play, Author; Anywhere, Anytime
Open Embedded Assessments: Play, Author; Anywhere, Anytime
Brandon Muramatsu158 views
Evaluating and Selecting Digital Learning Resources by Brandon Muramatsu
Evaluating and Selecting Digital Learning ResourcesEvaluating and Selecting Digital Learning Resources
Evaluating and Selecting Digital Learning Resources
Brandon Muramatsu161 views
Connected Learning Initiative: Learning at Scale by Brandon Muramatsu
Connected Learning Initiative: Learning at ScaleConnected Learning Initiative: Learning at Scale
Connected Learning Initiative: Learning at Scale
Brandon Muramatsu795 views
The Best of Both Worlds: Transforming OpenCourseWare in an age of Interactivity by Brandon Muramatsu
The Best of Both Worlds: Transforming OpenCourseWare in an age of InteractivityThe Best of Both Worlds: Transforming OpenCourseWare in an age of Interactivity
The Best of Both Worlds: Transforming OpenCourseWare in an age of Interactivity
Brandon Muramatsu1.6K views
Innovative Educational Technology and Educational Infrastructure at MIT by Brandon Muramatsu
Innovative Educational Technologyand Educational Infrastructureat MITInnovative Educational Technologyand Educational Infrastructureat MIT
Innovative Educational Technology and Educational Infrastructure at MIT
Brandon Muramatsu1.6K views
Workshop: Emerging Possibilities and Takeaways for KFUPM by Brandon Muramatsu
Workshop: Emerging Possibilities and Takeaways for KFUPMWorkshop: Emerging Possibilities and Takeaways for KFUPM
Workshop: Emerging Possibilities and Takeaways for KFUPM
Brandon Muramatsu609 views
Workshop: Lessons from Online and edX / MITx Courses by Brandon Muramatsu
Workshop: Lessons from Online and edX / MITx CoursesWorkshop: Lessons from Online and edX / MITx Courses
Workshop: Lessons from Online and edX / MITx Courses
Brandon Muramatsu963 views
Workshop: Design Considerations for Online / Digital Courses by Brandon Muramatsu
Workshop: Design Considerations for Online / Digital CoursesWorkshop: Design Considerations for Online / Digital Courses
Workshop: Design Considerations for Online / Digital Courses
Brandon Muramatsu625 views
Workshop: Educational Technology Opportunities for KFUPM by Brandon Muramatsu
Workshop: Educational Technology Opportunities for KFUPMWorkshop: Educational Technology Opportunities for KFUPM
Workshop: Educational Technology Opportunities for KFUPM
Brandon Muramatsu570 views

Recently uploaded

Lecture: Open Innovation by
Lecture: Open InnovationLecture: Open Innovation
Lecture: Open InnovationMichal Hron
96 views56 slides
Community-led Open Access Publishing webinar.pptx by
Community-led Open Access Publishing webinar.pptxCommunity-led Open Access Publishing webinar.pptx
Community-led Open Access Publishing webinar.pptxJisc
74 views9 slides
Structure and Functions of Cell.pdf by
Structure and Functions of Cell.pdfStructure and Functions of Cell.pdf
Structure and Functions of Cell.pdfNithya Murugan
368 views10 slides
Collective Bargaining and Understanding a Teacher Contract(16793704.1).pptx by
Collective Bargaining and Understanding a Teacher Contract(16793704.1).pptxCollective Bargaining and Understanding a Teacher Contract(16793704.1).pptx
Collective Bargaining and Understanding a Teacher Contract(16793704.1).pptxCenter for Integrated Training & Education
90 views57 slides
GSoC 2024 by
GSoC 2024GSoC 2024
GSoC 2024DeveloperStudentClub10
68 views15 slides
The Open Access Community Framework (OACF) 2023 (1).pptx by
The Open Access Community Framework (OACF) 2023 (1).pptxThe Open Access Community Framework (OACF) 2023 (1).pptx
The Open Access Community Framework (OACF) 2023 (1).pptxJisc
85 views7 slides

Recently uploaded(20)

Lecture: Open Innovation by Michal Hron
Lecture: Open InnovationLecture: Open Innovation
Lecture: Open Innovation
Michal Hron96 views
Community-led Open Access Publishing webinar.pptx by Jisc
Community-led Open Access Publishing webinar.pptxCommunity-led Open Access Publishing webinar.pptx
Community-led Open Access Publishing webinar.pptx
Jisc74 views
Structure and Functions of Cell.pdf by Nithya Murugan
Structure and Functions of Cell.pdfStructure and Functions of Cell.pdf
Structure and Functions of Cell.pdf
Nithya Murugan368 views
The Open Access Community Framework (OACF) 2023 (1).pptx by Jisc
The Open Access Community Framework (OACF) 2023 (1).pptxThe Open Access Community Framework (OACF) 2023 (1).pptx
The Open Access Community Framework (OACF) 2023 (1).pptx
Jisc85 views
The Accursed House by Émile Gaboriau by DivyaSheta
The Accursed House  by Émile GaboriauThe Accursed House  by Émile Gaboriau
The Accursed House by Émile Gaboriau
DivyaSheta158 views
Are we onboard yet University of Sussex.pptx by Jisc
Are we onboard yet University of Sussex.pptxAre we onboard yet University of Sussex.pptx
Are we onboard yet University of Sussex.pptx
Jisc77 views
Class 10 English notes 23-24.pptx by TARIQ KHAN
Class 10 English notes 23-24.pptxClass 10 English notes 23-24.pptx
Class 10 English notes 23-24.pptx
TARIQ KHAN107 views
Universe revised.pdf by DrHafizKosar
Universe revised.pdfUniverse revised.pdf
Universe revised.pdf
DrHafizKosar112 views
Narration lesson plan.docx by TARIQ KHAN
Narration lesson plan.docxNarration lesson plan.docx
Narration lesson plan.docx
TARIQ KHAN104 views
Class 10 English lesson plans by TARIQ KHAN
Class 10 English  lesson plansClass 10 English  lesson plans
Class 10 English lesson plans
TARIQ KHAN257 views
Compare the flora and fauna of Kerala and Chhattisgarh ( Charttabulation) by AnshulDewangan3
 Compare the flora and fauna of Kerala and Chhattisgarh ( Charttabulation) Compare the flora and fauna of Kerala and Chhattisgarh ( Charttabulation)
Compare the flora and fauna of Kerala and Chhattisgarh ( Charttabulation)
AnshulDewangan3316 views

Implementing SpokenMedia for the Indian Institute for Human Settlements

  • 1. Implementing SpokenMedia for the Indian Institute for Human Settlements Brandon Muramatsu Andrew McKinney Peter Wilkins July 2010 Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License ( creativecommons.org/licenses/by-nc-sa/3.0/us/ ) Citation: Muramatsu, B., McKinney, A., Wilkins, P. (2010). Implementing SpokenMedia for the Indian Institute for Human Settlements. Presented at Technology for Education 2010: Mumbai, India, July 1, 2010.
  • 2.
  • 3.
  • 4.
  • 5. The Indian Institute for Human Settlements (IIHS) will… “create India’s first independent National Innovation University focused on the challenges and opportunities of its urbanisation.” – Indian Institute for Human Settlements: Curriculum Framework Version 3.0 January 2010
  • 6. “ The IIHS Website is our commitment to a different way of looking at things.” – Aromar Revi 5 January 2010
  • 7. “ The Institution will fail or scale based on language.” – Aromar Revi 5 January 2010
  • 8. What did we do? Auto Transcribe Edit Translate Play
  • 10. How did we do it? Auto Transcribe Edit Translate Play
  • 11.
  • 12. SpokenMedia Process We used a portion of the SpokenMedia process for the demo
  • 13. How did we do it? Auto Transcribe Edit Translate Play
  • 14. Edit & Translate: Accuracy Automatic Transcription Hand Transcription Time Adjusted Translated Hindi I I I मेरे खयाल से think think think once one one नयोजन की एक मुख्य चुनौती है and central so challenge central the of challenger planning challenge of planning is planning nice legitimacy is legitimacy of legitimacy of of government government सरकार की एक ऐसी मुख्य संस्थान के रूप में वैधता government as as
  • 15.
  • 16. How did we do it? Auto Transcribe Edit Translate Play
  • 17.
  • 18.
  • 19.
  • 21.
  • 22.
  • 23. Thank You! Brandon Muramatsu, [email_address] Andrew McKinney, [email_address] Peter Wilkins, [email_address] ° Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License ( creativecommons.org/licenses/by-nc-sa/3.0/us/ )

Editor's Notes

  1. Citation: Muramatsu, B., McKinney, A., Wilkins, P. (2010). Implementing SpokenMedia for the Indian Institute for Human Settlements. Presented at Technology for Education 2010: Mumbai, India, July 1, 2010. Unless otherwise specified, this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
  2. Spoken Lecture Project Supported by iCampus Includes the browser (which was just demo’d) the processor (back end lecture transcription) and a hand workflow to do the processing Approximately 400 hours of video indexed SpokenMedia •  Technology transfer—get code running outside of original environment • 4 components: automatic lecture transcription, search, player, transcript editor • Suuported by iCampus SpokenMedia as a Service? • Still investigating
  3. Four step process (simple) First we used the SpokenMedia to do automatic transcription. Next we did hand edit and translation steps. Then we created a player for the presentation of the video and transcripts…
  4. Demo In this demo, we see a video interview with Prof. Bish Sanyal from MIT. We’ll see three things in this demo: • As Prof. Sanyal speaks, we will see the text in the transcript highlighted and the highlighting will follow along (“bouncing ball”) • Switching the transcript from English to a hand translation into Hindi that is synchronized with the audio, as the switch occurs the playback is seamless • Searching within the transcript, after entering the search term and selecting the result, the video and transcript seek to that point in the video and playback continues
  5. First we used the SpokenMedia to do automatic transcription.
  6. Lecture Transcription Jim Glass and his group have years of research experience for spoken languages Lectures are a different type of spoken language Much of the speech recognition research has focused on real time transcription of news broadcasts, or interactive voice response systems (telephone) Broadcast news has something like 300 unique words in an hour long broadcast Broadcast news is well structured, prepared copy (in studio via teleprompters), clear transitions between speakers, etc. Lectures are conversational and spontaneous Can use highly specialized vocabularies, engineering, physical sciences, mathematics
  7. We only used part of the process due to time constraints. Audio separation, speech processing, time-coded transcript, and then presentation through a SpokenMedia player.
  8. Next we did hand edit and translation steps.
  9. For this demo, we did computer-based automatic transcription, sent a file to IIHS for “editing” that consisted of performing a hand transcription (due to the format we sent, and the low accuracy of the automatic transcription in this case), a time alignment (though I actually feel that it’s “off” or “slow”) and then a hand translation by IIHS. Automatic transcription is in the ~50-55% accuracy range (by way of comparison YouTube Auto Caption for this same video is ~68% accuracy).
  10. Recognizer Accuracy Base accuracy is approximately 50% (generic domain and speaker models) Increase accuracy with speaker model up to 80-85%, and specific domain model This approach is good for courses with multiple lectures by the same speaker Domain models get more useful as more relevant text documents are indexed (keyword/noun phrase extraction) Initial results indicate that doing one 99% accurate (by hand/manual) transcript can help immensely for additional lectures by the same speaker Better use of limited resources Search accuracy is closer to 90%, searches tend to be for unique words which the processor is better at recognizing
  11. Then we created a player for the presentation of the video and transcripts…
  12. SpokenMedia Player 1.0 • Video-linked transcript • Highlighted text follows along as the speaker speaks • Switch transcript to a different transcript track seamlessly during playback •  Search within a transcript
  13. What we have today It’s not perfect, but a pretty good start Prototype has a number of useful features that demonstrate search interfaces and interaction interfaces
  14. The bright yellow indicates features working in the last two months… • Text processing to create a custom domain model • Creation of custom acoustic models in unsupervised mode •  Updated speech recognition software The gray indicates features we’ve had working since December 2009 and that were used for IIHS The light yellow indicates features on which we’ve just started working. • Accuracy measurement •  SpokenMedia Search (search across multiple videos) • Transcript Editor
  15. Where are we heading? Transition from research project to service Explore new interactions—what we’re calling Rich Media Notebooks
  16. Citation: Muramatsu, B., McKinney, A., Wilkins, P. (2010). Implementing SpokenMedia for the Indian Institute for Human Settlements. Presented at Technology for Education 2010: Mumbai, India, July 1, 2010. Unless otherwise specified, this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License