Implementing SpokenMedia for the Indian Institute for Human Settlements

Brandon Muramatsu
Brandon MuramatsuAssociate Director, Projects
Implementing SpokenMedia for the  Indian Institute for Human Settlements Brandon Muramatsu Andrew McKinney Peter Wilkins July 2010 Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0  United States License ( creativecommons.org/licenses/by-nc-sa/3.0/us/ ) Citation: Muramatsu, B., McKinney, A., Wilkins, P. (2010). Implementing SpokenMedia for the  Indian Institute for  Human Settlements. Presented at Technology for Education 2010: Mumbai, India, July 1, 2010.
Case Study of Using SpokenMedia for IIHS ,[object Object],[object Object],[object Object],[object Object],[object Object]
MIT Office for Educational Innovation  and Technology ,[object Object],[object Object],[object Object],[object Object],Experiment Incubate Transition Service
SpokenMedia Project Experiment Incubate Transition Service ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
The Indian Institute for Human Settlements (IIHS) will… “create India’s first independent National Innovation University focused on the challenges and opportunities of its urbanisation.” –  Indian Institute for Human Settlements:  Curriculum Framework Version 3.0   January 2010
“ The IIHS Website is our commitment to a different  way of looking at things.” –  Aromar Revi 5 January 2010
“ The Institution will fail or scale based on language.” –  Aromar Revi 5 January 2010
What did we do? Auto Transcribe Edit Translate Play
The Demo
How did we do it? Auto Transcribe Edit Translate Play
How do we do it? Spoken Lecture Research ,[object Object],[object Object],[object Object],[object Object],[object Object],James Glass [email_address] Supported with iCampus MIT/Microsoft Alliance funding
SpokenMedia Process We used a portion of the SpokenMedia process for the demo
How did we do it? Auto Transcribe Edit Translate Play
Edit & Translate: Accuracy Automatic Transcription Hand Transcription Time Adjusted Translated Hindi I I I मेरे खयाल से think think think once one one नयोजन की एक मुख्य चुनौती है and central so challenge central the  of challenger planning challenge of planning is planning nice legitimacy is legitimacy of legitimacy of of government government सरकार की एक ऐसी मुख्य संस्थान के रूप में वैधता  government as as
Automatic Speech Recognition Accuracy ,[object Object],[object Object],[object Object],[object Object],[object Object]
How did we do it? Auto Transcribe Edit Translate Play
The Player ,[object Object],[object Object],[object Object],[object Object],Search within Transcript Highlight text as spoken Switch transcript to different language(s)
SpokenMedia (circa January) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Challenges ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
SpokenMedia (July 2010)
Where are we heading? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Check it out for yourself ,[object Object],[object Object],[object Object]
Thank You! Brandon Muramatsu,  [email_address] Andrew McKinney,  [email_address] Peter Wilkins,  [email_address] ° Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0  United States License ( creativecommons.org/licenses/by-nc-sa/3.0/us/ )
1 of 23

Recommended

Opening Up IIHS Video with SpokenMedia by
Opening Up IIHS Video with SpokenMediaOpening Up IIHS Video with SpokenMedia
Opening Up IIHS Video with SpokenMediaBrandon Muramatsu
1.8K views20 slides
SpokenMedia Project: Media-Linked Transcripts and Rich Media Notebooks for Le... by
SpokenMedia Project: Media-Linked Transcripts and Rich Media Notebooks for Le...SpokenMedia Project: Media-Linked Transcripts and Rich Media Notebooks for Le...
SpokenMedia Project: Media-Linked Transcripts and Rich Media Notebooks for Le...Brandon Muramatsu
1.4K views24 slides
Improving the OER Experience: Enabling Rich Media Notebooks of OER Video and ... by
Improving the OER Experience: Enabling Rich Media Notebooks of OER Video and ...Improving the OER Experience: Enabling Rich Media Notebooks of OER Video and ...
Improving the OER Experience: Enabling Rich Media Notebooks of OER Video and ...Brandon Muramatsu
1.8K views24 slides
Building Community for Rich Media Notebooks: The SpokenMedia Project at NMC 2009 by
Building Community for Rich Media Notebooks: The SpokenMedia Project at NMC 2009Building Community for Rich Media Notebooks: The SpokenMedia Project at NMC 2009
Building Community for Rich Media Notebooks: The SpokenMedia Project at NMC 2009Brandon Muramatsu
1.8K views23 slides
Automated Lecture Transcription at OCW Consortium Global Meeting 2009 by
Automated Lecture Transcription at OCW Consortium Global Meeting 2009Automated Lecture Transcription at OCW Consortium Global Meeting 2009
Automated Lecture Transcription at OCW Consortium Global Meeting 2009Brandon Muramatsu
1.5K views14 slides
Project Greenfield: A New Way of Thinking about MIT OpenCourseWare by
Project Greenfield: A New Way of Thinking about MIT OpenCourseWareProject Greenfield: A New Way of Thinking about MIT OpenCourseWare
Project Greenfield: A New Way of Thinking about MIT OpenCourseWareBrandon Muramatsu
6.7K views22 slides

More Related Content

Viewers also liked

IIHS Open Framework-SpokenMedia by
IIHS Open Framework-SpokenMediaIIHS Open Framework-SpokenMedia
IIHS Open Framework-SpokenMediaBrandon Muramatsu
1.7K views19 slides
Settlement Characteristics by
Settlement CharacteristicsSettlement Characteristics
Settlement Characteristicswhiskeyhj
69.2K views20 slides
Unit 1 planning c oncepts ppt by
Unit 1   planning c oncepts  pptUnit 1   planning c oncepts  ppt
Unit 1 planning c oncepts pptEzhil Tamizh
45.7K views48 slides
Rural housing in india by
Rural housing in indiaRural housing in india
Rural housing in indiakumaresan2704
45.7K views52 slides
Evolution of settlements by
Evolution of settlementsEvolution of settlements
Evolution of settlementsyusra_gul
73.2K views29 slides
Human Settlements by
Human SettlementsHuman Settlements
Human Settlementspeyne
102K views54 slides

Viewers also liked(7)

Settlement Characteristics by whiskeyhj
Settlement CharacteristicsSettlement Characteristics
Settlement Characteristics
whiskeyhj69.2K views
Unit 1 planning c oncepts ppt by Ezhil Tamizh
Unit 1   planning c oncepts  pptUnit 1   planning c oncepts  ppt
Unit 1 planning c oncepts ppt
Ezhil Tamizh45.7K views
Rural housing in india by kumaresan2704
Rural housing in indiaRural housing in india
Rural housing in india
kumaresan270445.7K views
Evolution of settlements by yusra_gul
Evolution of settlementsEvolution of settlements
Evolution of settlements
yusra_gul73.2K views
Human Settlements by peyne
Human SettlementsHuman Settlements
Human Settlements
peyne102K views

Similar to Implementing SpokenMedia for the Indian Institute for Human Settlements

How to Get Online Course Translations of the Highest Quality by
How to Get Online Course Translations of the Highest QualityHow to Get Online Course Translations of the Highest Quality
How to Get Online Course Translations of the Highest QualityCommLab India – Rapid eLearning Solutions
519 views23 slides
visH (fin).pptx by
visH (fin).pptxvisH (fin).pptx
visH (fin).pptxtefflontrolegdy
32 views21 slides
Rapid Innovative Design Notes by
Rapid Innovative Design NotesRapid Innovative Design Notes
Rapid Innovative Design Notesspotlearning
882 views72 slides
Technology by
TechnologyTechnology
TechnologyGillian Lord
296 views31 slides
Video Captioning for Accessibility: University of Florida and Regis Universit... by
Video Captioning for Accessibility: University of Florida and Regis Universit...Video Captioning for Accessibility: University of Florida and Regis Universit...
Video Captioning for Accessibility: University of Florida and Regis Universit...3Play Media
1.6K views46 slides
IRJET - Analysis on Code-Mixed Data for Movie Reviews by
IRJET - Analysis on Code-Mixed Data for Movie ReviewsIRJET - Analysis on Code-Mixed Data for Movie Reviews
IRJET - Analysis on Code-Mixed Data for Movie ReviewsIRJET Journal
11 views4 slides

Similar to Implementing SpokenMedia for the Indian Institute for Human Settlements(20)

Rapid Innovative Design Notes by spotlearning
Rapid Innovative Design NotesRapid Innovative Design Notes
Rapid Innovative Design Notes
spotlearning882 views
Video Captioning for Accessibility: University of Florida and Regis Universit... by 3Play Media
Video Captioning for Accessibility: University of Florida and Regis Universit...Video Captioning for Accessibility: University of Florida and Regis Universit...
Video Captioning for Accessibility: University of Florida and Regis Universit...
3Play Media1.6K views
IRJET - Analysis on Code-Mixed Data for Movie Reviews by IRJET Journal
IRJET - Analysis on Code-Mixed Data for Movie ReviewsIRJET - Analysis on Code-Mixed Data for Movie Reviews
IRJET - Analysis on Code-Mixed Data for Movie Reviews
IRJET Journal11 views
Tips for Creating an Accessible & Engaging Virtual Classroom by 3Play Media
Tips for Creating an Accessible & Engaging Virtual ClassroomTips for Creating an Accessible & Engaging Virtual Classroom
Tips for Creating an Accessible & Engaging Virtual Classroom
3Play Media741 views
Intro to watson bluemix services by Vikas Manoria
Intro to watson bluemix servicesIntro to watson bluemix services
Intro to watson bluemix services
Vikas Manoria1.2K views
Video Accessibility Toolkit for Success in a Virtual Environment by 3Play Media
Video Accessibility Toolkit for Success in a Virtual EnvironmentVideo Accessibility Toolkit for Success in a Virtual Environment
Video Accessibility Toolkit for Success in a Virtual Environment
3Play Media1.7K views
Vlogging by Aiden Yeh
VloggingVlogging
Vlogging
Aiden Yeh6.1K views
Ebook media-and-entertainment by BnsplBraahmam
Ebook media-and-entertainmentEbook media-and-entertainment
Ebook media-and-entertainment
BnsplBraahmam67 views
Technology supportswrittenlanuagesccec dld by Cheryl Wissick
Technology supportswrittenlanuagesccec dldTechnology supportswrittenlanuagesccec dld
Technology supportswrittenlanuagesccec dld
Cheryl Wissick271 views
Article Summaries by ORhonda
Article SummariesArticle Summaries
Article Summaries
ORhonda706 views
Hindi –tamil text translation by Vaibhav Agarwal
Hindi –tamil text translationHindi –tamil text translation
Hindi –tamil text translation
Vaibhav Agarwal34.1K views

More from Brandon Muramatsu

Digital Credentials Enabling Mobility and Verification of Educational Achieve... by
Digital Credentials Enabling Mobility and Verification of Educational Achieve...Digital Credentials Enabling Mobility and Verification of Educational Achieve...
Digital Credentials Enabling Mobility and Verification of Educational Achieve...Brandon Muramatsu
239 views21 slides
Sustainability of OER Initiatives: An Interactive Discussion by
Sustainability of OER Initiatives: An Interactive DiscussionSustainability of OER Initiatives: An Interactive Discussion
Sustainability of OER Initiatives: An Interactive DiscussionBrandon Muramatsu
212 views17 slides
Bridging the Gap: Mixing approaches, content and tools to help college students by
Bridging the Gap: Mixing approaches, content and tools to help college studentsBridging the Gap: Mixing approaches, content and tools to help college students
Bridging the Gap: Mixing approaches, content and tools to help college studentsBrandon Muramatsu
201 views11 slides
Federations & Backstage: Thoughts for a Geoscience Education Infrastructure by
Federations & Backstage: Thoughts for a Geoscience Education InfrastructureFederations & Backstage: Thoughts for a Geoscience Education Infrastructure
Federations & Backstage: Thoughts for a Geoscience Education InfrastructureBrandon Muramatsu
176 views9 slides
The Connected Learning Initiative Quality at Scale in India by
The Connected Learning Initiative Quality at Scale in IndiaThe Connected Learning Initiative Quality at Scale in India
The Connected Learning Initiative Quality at Scale in IndiaBrandon Muramatsu
172 views18 slides
The Connected Learning Initiative Quality at Scale in India by
The Connected Learning Initiative Quality at Scale in IndiaThe Connected Learning Initiative Quality at Scale in India
The Connected Learning Initiative Quality at Scale in IndiaBrandon Muramatsu
161 views9 slides

More from Brandon Muramatsu(20)

Digital Credentials Enabling Mobility and Verification of Educational Achieve... by Brandon Muramatsu
Digital Credentials Enabling Mobility and Verification of Educational Achieve...Digital Credentials Enabling Mobility and Verification of Educational Achieve...
Digital Credentials Enabling Mobility and Verification of Educational Achieve...
Brandon Muramatsu239 views
Sustainability of OER Initiatives: An Interactive Discussion by Brandon Muramatsu
Sustainability of OER Initiatives: An Interactive DiscussionSustainability of OER Initiatives: An Interactive Discussion
Sustainability of OER Initiatives: An Interactive Discussion
Brandon Muramatsu212 views
Bridging the Gap: Mixing approaches, content and tools to help college students by Brandon Muramatsu
Bridging the Gap: Mixing approaches, content and tools to help college studentsBridging the Gap: Mixing approaches, content and tools to help college students
Bridging the Gap: Mixing approaches, content and tools to help college students
Brandon Muramatsu201 views
Federations & Backstage: Thoughts for a Geoscience Education Infrastructure by Brandon Muramatsu
Federations & Backstage: Thoughts for a Geoscience Education InfrastructureFederations & Backstage: Thoughts for a Geoscience Education Infrastructure
Federations & Backstage: Thoughts for a Geoscience Education Infrastructure
Brandon Muramatsu176 views
The Connected Learning Initiative Quality at Scale in India by Brandon Muramatsu
The Connected Learning Initiative Quality at Scale in IndiaThe Connected Learning Initiative Quality at Scale in India
The Connected Learning Initiative Quality at Scale in India
Brandon Muramatsu172 views
The Connected Learning Initiative Quality at Scale in India by Brandon Muramatsu
The Connected Learning Initiative Quality at Scale in IndiaThe Connected Learning Initiative Quality at Scale in India
The Connected Learning Initiative Quality at Scale in India
Brandon Muramatsu161 views
Strategic Education Initiatives , MIT Open Learning by Brandon Muramatsu
Strategic Education Initiatives, MIT Open LearningStrategic Education Initiatives, MIT Open Learning
Strategic Education Initiatives , MIT Open Learning
Brandon Muramatsu164 views
Open Embedded Assessments: Play, Author; Anywhere, Anytime by Brandon Muramatsu
Open Embedded Assessments:Play, Author; Anywhere, AnytimeOpen Embedded Assessments:Play, Author; Anywhere, Anytime
Open Embedded Assessments: Play, Author; Anywhere, Anytime
Brandon Muramatsu158 views
Evaluating and Selecting Digital Learning Resources by Brandon Muramatsu
Evaluating and Selecting Digital Learning ResourcesEvaluating and Selecting Digital Learning Resources
Evaluating and Selecting Digital Learning Resources
Brandon Muramatsu161 views
Connected Learning Initiative: Learning at Scale by Brandon Muramatsu
Connected Learning Initiative: Learning at ScaleConnected Learning Initiative: Learning at Scale
Connected Learning Initiative: Learning at Scale
Brandon Muramatsu795 views
The Best of Both Worlds: Transforming OpenCourseWare in an age of Interactivity by Brandon Muramatsu
The Best of Both Worlds: Transforming OpenCourseWare in an age of InteractivityThe Best of Both Worlds: Transforming OpenCourseWare in an age of Interactivity
The Best of Both Worlds: Transforming OpenCourseWare in an age of Interactivity
Brandon Muramatsu1.6K views
Innovative Educational Technology and Educational Infrastructure at MIT by Brandon Muramatsu
Innovative Educational Technologyand Educational Infrastructureat MITInnovative Educational Technologyand Educational Infrastructureat MIT
Innovative Educational Technology and Educational Infrastructure at MIT
Brandon Muramatsu1.6K views
Workshop: Emerging Possibilities and Takeaways for KFUPM by Brandon Muramatsu
Workshop: Emerging Possibilities and Takeaways for KFUPMWorkshop: Emerging Possibilities and Takeaways for KFUPM
Workshop: Emerging Possibilities and Takeaways for KFUPM
Brandon Muramatsu610 views
Workshop: Lessons from Online and edX / MITx Courses by Brandon Muramatsu
Workshop: Lessons from Online and edX / MITx CoursesWorkshop: Lessons from Online and edX / MITx Courses
Workshop: Lessons from Online and edX / MITx Courses
Brandon Muramatsu963 views
Workshop: Design Considerations for Online / Digital Courses by Brandon Muramatsu
Workshop: Design Considerations for Online / Digital CoursesWorkshop: Design Considerations for Online / Digital Courses
Workshop: Design Considerations for Online / Digital Courses
Brandon Muramatsu625 views
Workshop: Educational Technology Opportunities for KFUPM by Brandon Muramatsu
Workshop: Educational Technology Opportunities for KFUPMWorkshop: Educational Technology Opportunities for KFUPM
Workshop: Educational Technology Opportunities for KFUPM
Brandon Muramatsu570 views

Recently uploaded

CUNY IT Picciano.pptx by
CUNY IT Picciano.pptxCUNY IT Picciano.pptx
CUNY IT Picciano.pptxapicciano
56 views17 slides
ICS3211_lecture 09_2023.pdf by
ICS3211_lecture 09_2023.pdfICS3211_lecture 09_2023.pdf
ICS3211_lecture 09_2023.pdfVanessa Camilleri
126 views10 slides
STRATEGIC MANAGEMENT MODULE 1_UNIT1 _UNIT2.pdf by
STRATEGIC MANAGEMENT MODULE 1_UNIT1 _UNIT2.pdfSTRATEGIC MANAGEMENT MODULE 1_UNIT1 _UNIT2.pdf
STRATEGIC MANAGEMENT MODULE 1_UNIT1 _UNIT2.pdfDr Vijay Vishwakarma
87 views68 slides
Computer Introduction-Lecture06 by
Computer Introduction-Lecture06Computer Introduction-Lecture06
Computer Introduction-Lecture06Dr. Mazin Mohamed alkathiri
117 views12 slides
Classification of crude drugs.pptx by
Classification of crude drugs.pptxClassification of crude drugs.pptx
Classification of crude drugs.pptxGayatriPatra14
104 views13 slides
JQUERY.pdf by
JQUERY.pdfJQUERY.pdf
JQUERY.pdfArthyR3
96 views22 slides

Recently uploaded(20)

CUNY IT Picciano.pptx by apicciano
CUNY IT Picciano.pptxCUNY IT Picciano.pptx
CUNY IT Picciano.pptx
apicciano56 views
Classification of crude drugs.pptx by GayatriPatra14
Classification of crude drugs.pptxClassification of crude drugs.pptx
Classification of crude drugs.pptx
GayatriPatra14104 views
JQUERY.pdf by ArthyR3
JQUERY.pdfJQUERY.pdf
JQUERY.pdf
ArthyR396 views
When Sex Gets Complicated: Porn, Affairs, & Cybersex by Marlene Maheu
When Sex Gets Complicated: Porn, Affairs, & CybersexWhen Sex Gets Complicated: Porn, Affairs, & Cybersex
When Sex Gets Complicated: Porn, Affairs, & Cybersex
Marlene Maheu99 views
The basics - information, data, technology and systems.pdf by JonathanCovena1
The basics - information, data, technology and systems.pdfThe basics - information, data, technology and systems.pdf
The basics - information, data, technology and systems.pdf
JonathanCovena1156 views
The Accursed House by Émile Gaboriau by DivyaSheta
The Accursed House  by Émile GaboriauThe Accursed House  by Émile Gaboriau
The Accursed House by Émile Gaboriau
DivyaSheta234 views
GCSE Spanish by WestHatch
GCSE SpanishGCSE Spanish
GCSE Spanish
WestHatch53 views
Create a Structure in VBNet.pptx by Breach_P
Create a Structure in VBNet.pptxCreate a Structure in VBNet.pptx
Create a Structure in VBNet.pptx
Breach_P80 views
11.28.23 Social Capital and Social Exclusion.pptx by mary850239
11.28.23 Social Capital and Social Exclusion.pptx11.28.23 Social Capital and Social Exclusion.pptx
11.28.23 Social Capital and Social Exclusion.pptx
mary850239383 views
REPRESENTATION - GAUNTLET.pptx by iammrhaywood
REPRESENTATION - GAUNTLET.pptxREPRESENTATION - GAUNTLET.pptx
REPRESENTATION - GAUNTLET.pptx
iammrhaywood151 views

Implementing SpokenMedia for the Indian Institute for Human Settlements

  • 1. Implementing SpokenMedia for the Indian Institute for Human Settlements Brandon Muramatsu Andrew McKinney Peter Wilkins July 2010 Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License ( creativecommons.org/licenses/by-nc-sa/3.0/us/ ) Citation: Muramatsu, B., McKinney, A., Wilkins, P. (2010). Implementing SpokenMedia for the Indian Institute for Human Settlements. Presented at Technology for Education 2010: Mumbai, India, July 1, 2010.
  • 2.
  • 3.
  • 4.
  • 5. The Indian Institute for Human Settlements (IIHS) will… “create India’s first independent National Innovation University focused on the challenges and opportunities of its urbanisation.” – Indian Institute for Human Settlements: Curriculum Framework Version 3.0 January 2010
  • 6. “ The IIHS Website is our commitment to a different way of looking at things.” – Aromar Revi 5 January 2010
  • 7. “ The Institution will fail or scale based on language.” – Aromar Revi 5 January 2010
  • 8. What did we do? Auto Transcribe Edit Translate Play
  • 10. How did we do it? Auto Transcribe Edit Translate Play
  • 11.
  • 12. SpokenMedia Process We used a portion of the SpokenMedia process for the demo
  • 13. How did we do it? Auto Transcribe Edit Translate Play
  • 14. Edit & Translate: Accuracy Automatic Transcription Hand Transcription Time Adjusted Translated Hindi I I I मेरे खयाल से think think think once one one नयोजन की एक मुख्य चुनौती है and central so challenge central the of challenger planning challenge of planning is planning nice legitimacy is legitimacy of legitimacy of of government government सरकार की एक ऐसी मुख्य संस्थान के रूप में वैधता government as as
  • 15.
  • 16. How did we do it? Auto Transcribe Edit Translate Play
  • 17.
  • 18.
  • 19.
  • 21.
  • 22.
  • 23. Thank You! Brandon Muramatsu, [email_address] Andrew McKinney, [email_address] Peter Wilkins, [email_address] ° Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License ( creativecommons.org/licenses/by-nc-sa/3.0/us/ )

Editor's Notes

  1. Citation: Muramatsu, B., McKinney, A., Wilkins, P. (2010). Implementing SpokenMedia for the Indian Institute for Human Settlements. Presented at Technology for Education 2010: Mumbai, India, July 1, 2010. Unless otherwise specified, this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
  2. Spoken Lecture Project Supported by iCampus Includes the browser (which was just demo’d) the processor (back end lecture transcription) and a hand workflow to do the processing Approximately 400 hours of video indexed SpokenMedia •  Technology transfer—get code running outside of original environment • 4 components: automatic lecture transcription, search, player, transcript editor • Suuported by iCampus SpokenMedia as a Service? • Still investigating
  3. Four step process (simple) First we used the SpokenMedia to do automatic transcription. Next we did hand edit and translation steps. Then we created a player for the presentation of the video and transcripts…
  4. Demo In this demo, we see a video interview with Prof. Bish Sanyal from MIT. We’ll see three things in this demo: • As Prof. Sanyal speaks, we will see the text in the transcript highlighted and the highlighting will follow along (“bouncing ball”) • Switching the transcript from English to a hand translation into Hindi that is synchronized with the audio, as the switch occurs the playback is seamless • Searching within the transcript, after entering the search term and selecting the result, the video and transcript seek to that point in the video and playback continues
  5. First we used the SpokenMedia to do automatic transcription.
  6. Lecture Transcription Jim Glass and his group have years of research experience for spoken languages Lectures are a different type of spoken language Much of the speech recognition research has focused on real time transcription of news broadcasts, or interactive voice response systems (telephone) Broadcast news has something like 300 unique words in an hour long broadcast Broadcast news is well structured, prepared copy (in studio via teleprompters), clear transitions between speakers, etc. Lectures are conversational and spontaneous Can use highly specialized vocabularies, engineering, physical sciences, mathematics
  7. We only used part of the process due to time constraints. Audio separation, speech processing, time-coded transcript, and then presentation through a SpokenMedia player.
  8. Next we did hand edit and translation steps.
  9. For this demo, we did computer-based automatic transcription, sent a file to IIHS for “editing” that consisted of performing a hand transcription (due to the format we sent, and the low accuracy of the automatic transcription in this case), a time alignment (though I actually feel that it’s “off” or “slow”) and then a hand translation by IIHS. Automatic transcription is in the ~50-55% accuracy range (by way of comparison YouTube Auto Caption for this same video is ~68% accuracy).
  10. Recognizer Accuracy Base accuracy is approximately 50% (generic domain and speaker models) Increase accuracy with speaker model up to 80-85%, and specific domain model This approach is good for courses with multiple lectures by the same speaker Domain models get more useful as more relevant text documents are indexed (keyword/noun phrase extraction) Initial results indicate that doing one 99% accurate (by hand/manual) transcript can help immensely for additional lectures by the same speaker Better use of limited resources Search accuracy is closer to 90%, searches tend to be for unique words which the processor is better at recognizing
  11. Then we created a player for the presentation of the video and transcripts…
  12. SpokenMedia Player 1.0 • Video-linked transcript • Highlighted text follows along as the speaker speaks • Switch transcript to a different transcript track seamlessly during playback •  Search within a transcript
  13. What we have today It’s not perfect, but a pretty good start Prototype has a number of useful features that demonstrate search interfaces and interaction interfaces
  14. The bright yellow indicates features working in the last two months… • Text processing to create a custom domain model • Creation of custom acoustic models in unsupervised mode •  Updated speech recognition software The gray indicates features we’ve had working since December 2009 and that were used for IIHS The light yellow indicates features on which we’ve just started working. • Accuracy measurement •  SpokenMedia Search (search across multiple videos) • Transcript Editor
  15. Where are we heading? Transition from research project to service Explore new interactions—what we’re calling Rich Media Notebooks
  16. Citation: Muramatsu, B., McKinney, A., Wilkins, P. (2010). Implementing SpokenMedia for the Indian Institute for Human Settlements. Presented at Technology for Education 2010: Mumbai, India, July 1, 2010. Unless otherwise specified, this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License