Writing Voice Mashups Thomas Howe - http://www.thomashowe.com
Agenda <ul><li>Introduce the After Hours Doctor’s Office </li></ul><ul><ul><li>Scenario </li></ul></ul><ul><ul><li>The goa...
After Hours Doctor’s Office <ul><li>Mashup written specifically for the Contest </li></ul><ul><ul><li>I quit my job becaus...
Application Scenario <ul><li>Mr. Kraus feels sick, calls Dr. McCarthy </li></ul><ul><ul><li>After hours - no one’s there. ...
The Money <ul><li>Where is the money? </li></ul><ul><ul><li>50 million unnecessary emergency room visits in the US </li></...
Amazon Turks <ul><li>Internet Scale Workforce </li></ul><ul><ul><li>One doctor needs 1/10 of a nurse </li></ul></ul><ul><u...
Bringing a knife to a gun fight… <ul><li>What did I use? </li></ul><ul><ul><li>40 hours, web services and a rapidly aging ...
After Hours Mashup Architecture
VoiceXML <ul><li>Accepts inbound phone calls with rich media experience </li></ul><ul><ul><li>Voice detection, ASR, IVR, r...
After Hours Mashup Architecture
Uploader.php <ul><li>PHP file for accepting inbound request </li></ul><ul><ul><li>Called by VoiceXML file </li></ul></ul><...
After Hours Mashup Architecture
Strike Iron SMS <ul><li>Web Service to send text messages </li></ul><ul><ul><li>One of many online, paid services </li></u...
After Hours Mashup Architecture
Amazon Web Services <ul><li>Amazon Mechanical Turks </li></ul><ul><ul><li>Human Intelligence Tasks </li></ul></ul><ul><ul>...
After Hours Mashup Architecture
Extensions <ul><li>Call Centers </li></ul><ul><ul><li>Expert based call centers, makes response even faster </li></ul></ul...
Call Center : Sylantro <ul><li>Integrating with Sylantro switch </li></ul><ul><ul><li>Accepts inbound calls into expert qu...
Yahoo! Local <ul><li>Nurses page shows location of caller </li></ul><ul><ul><li>Location derived from inbound calling info...
Thank you!!! <ul><li>For more information, source code, and this presentation, see my site at  </li></ul><ul><li>thomashow...
Upcoming SlideShare
Loading in …5
×

Writing Voice Mashups with Amazon Turks

5,375 views

Published on

A few short years ago, communications applications required millions of dollars, teams of highly skilled engineers, access to networks, and many months of time. Telephony Mashups rewrite that equation, and make it possible to blend communications services into applications quickly and easily.

In this workshop, led by 2007 O'Reilly Mashup Winner Thomas Howe, we look at how these mashups are created, and at the implementation details in depth. Using After Hours Doctor's Office as an example, every part of the mashup will be reviewed in code-level detail, including a Voice XML front end, the software interfaces to the Amazon Mechanical Turk nurses, and mapping displays to help direct the patient to a local health care facility. The attendees will leave with a good understanding of the technology and effort required to write their own compelling application.

Published in: Technology

Writing Voice Mashups with Amazon Turks

  1. 1. Writing Voice Mashups Thomas Howe - http://www.thomashowe.com
  2. 2. Agenda <ul><li>Introduce the After Hours Doctor’s Office </li></ul><ul><ul><li>Scenario </li></ul></ul><ul><ul><li>The goals of the Mashup </li></ul></ul><ul><li>Introduce the architecture </li></ul><ul><ul><li>Go over every component </li></ul></ul><ul><ul><li>Functionality, interface, code </li></ul></ul><ul><ul><li>Give you some ingredients, ideas </li></ul></ul><ul><li>What this isn’t </li></ul><ul><ul><li>A complete, and coherent, application </li></ul></ul><ul><ul><li>Utterly scalable, but surprisingly close </li></ul></ul><ul><li>What is this? </li></ul><ul><ul><li>Think of Lewis and Clark </li></ul></ul><ul><ul><li>Today’s telephony is very far from this. Very far. </li></ul></ul><ul><ul><li>With a bit of luck, this is tomorrow’s telephony </li></ul></ul>
  3. 3. After Hours Doctor’s Office <ul><li>Mashup written specifically for the Contest </li></ul><ul><ul><li>I quit my job because I was technically bothered </li></ul></ul><ul><li>Goals </li></ul><ul><ul><li>Show you the money </li></ul></ul><ul><ul><li>Explore the use of Amazon Turks in Telephony Mashups </li></ul></ul><ul><ul><li>Demonstrate the effects of modern Web technologies in telephony applications </li></ul></ul><ul><li>Resources </li></ul><ul><ul><li>Me, 40 hours and a PowerMac G5 Server </li></ul></ul><ul><ul><li>TellMe Studio - Voice XML hosting </li></ul></ul><ul><ul><li>Strike-Iron - SMS Messaging </li></ul></ul><ul><ul><li>Amazon Web Services - Amazon Mechanical Turks </li></ul></ul><ul><ul><li>Web Hosting Provider - PHP and Ruby on Rails </li></ul></ul><ul><li>Source and commentary available at thomashowe.com </li></ul>
  4. 4. Application Scenario <ul><li>Mr. Kraus feels sick, calls Dr. McCarthy </li></ul><ul><ul><li>After hours - no one’s there. </li></ul></ul><ul><ul><li>Decision to make - emergency room? Tomorrow? </li></ul></ul><ul><li>Application takes voice mail message </li></ul><ul><ul><li>Forwards into pool of Amazon Turk Nurses </li></ul></ul><ul><ul><li>SMS messages increase reaction time for small service </li></ul></ul><ul><ul><li>Immediately responds to patient - constant feedback </li></ul></ul><ul><li>Turk Nurse determines urgency </li></ul><ul><ul><li>Listens to the message, summarizes and forwards </li></ul></ul><ul><li>Application gives direction to patient </li></ul><ul><ul><li>Routine? A message from nurse reassuring patient </li></ul></ul><ul><ul><li>Urgent? A call from the Doctor, right away </li></ul></ul><ul><li>Benefits? </li></ul><ul><ul><li>Faster, cheaper with a higher quality of care </li></ul></ul>
  5. 5. The Money <ul><li>Where is the money? </li></ul><ul><ul><li>50 million unnecessary emergency room visits in the US </li></ul></ul><ul><ul><li>Each costs about $900.00 more than a doctor’s office visit </li></ul></ul><ul><li>Where isn’t the money? </li></ul><ul><ul><li>Customer education that the service exists </li></ul></ul><ul><ul><li>Repeated marketing to habituate the service </li></ul></ul><ul><ul><li>Costly market trials </li></ul></ul><ul><li>Why? </li></ul><ul><ul><li>Integration of real time communications into the business process makes it more efficient, saving money for the enterprise and increasing customer satisfaction </li></ul></ul>
  6. 6. Amazon Turks <ul><li>Internet Scale Workforce </li></ul><ul><ul><li>One doctor needs 1/10 of a nurse </li></ul></ul><ul><ul><li>One thousand doctors needs 100. </li></ul></ul><ul><ul><li>More use, more turks, faster service </li></ul></ul><ul><li>Practical? Absolutely </li></ul><ul><ul><li>Nurses get $3.00 for three minutes work == $60/hour </li></ul></ul><ul><ul><li>Down time, family time, sick time, whatever </li></ul></ul><ul><ul><li>Off-shoring? </li></ul></ul><ul><li>Comprehensive Quality </li></ul><ul><ul><li>Amazon Turks automatically </li></ul></ul><ul><ul><li>Record urgency accuracy, compare history </li></ul></ul><ul><ul><li>Transcriptions </li></ul></ul>
  7. 7. Bringing a knife to a gun fight… <ul><li>What did I use? </li></ul><ul><ul><li>40 hours, web services and a rapidly aging geek </li></ul></ul><ul><ul><li>Ruby on Rails, SOA, PHP </li></ul></ul><ul><ul><li>$25.00 to Strike Iron, and I’d probably have to pay x cents a minute to TellMe, Angel, or Voxeo </li></ul></ul><ul><li>What did I not use? </li></ul><ul><ul><li>Big Iron ==> NO LARGE INVESTMENT </li></ul></ul><ul><ul><li>A Contract ==> NO CARRIER INVOLVEMENT </li></ul></ul><ul><ul><li>Marketing ==> NO CUSTOMER EDUCATION </li></ul></ul><ul><ul><li>Risk ==> NO LONG DEVELOPMENT CYCLES </li></ul></ul><ul><li>Traditional carriers will get their clocks cleaned. </li></ul>
  8. 8. After Hours Mashup Architecture
  9. 9. VoiceXML <ul><li>Accepts inbound phone calls with rich media experience </li></ul><ul><ul><li>Voice detection, ASR, IVR, recording audio </li></ul></ul><ul><ul><li>Use CCXML from Voxeo for outdial operations </li></ul></ul><ul><ul><li>May be blended with other voice applications </li></ul></ul><ul><li>Standards Compliant Operation </li></ul><ul><ul><li>I can choose among hosted options, local platform </li></ul></ul><ul><ul><li>Little lock in </li></ul></ul><ul><li>Records audio and posts it to Web Server </li></ul><ul><ul><li>Store it, record it, play it </li></ul></ul><ul><li>Web Server can produce VoiceXML files to play </li></ul><ul><ul><li>Stateful processing </li></ul></ul><ul><li>Many choices for hosted providers </li></ul><ul><ul><li>TellMe, Voxeo </li></ul></ul><ul><ul><li>Some have platform/hosted options </li></ul></ul>
  10. 10. After Hours Mashup Architecture
  11. 11. Uploader.php <ul><li>PHP file for accepting inbound request </li></ul><ul><ul><li>Called by VoiceXML file </li></ul></ul><ul><li>Handles uploading voice files, data </li></ul><ul><li>Calls shell scripts to issue SMS requests </li></ul><ul><li>Returns VoiceXML files back </li></ul><ul><ul><li>Could be dynamic; template procesing </li></ul></ul><ul><li>Complete integration with other Web Tech </li></ul><ul><ul><li>Database integration </li></ul></ul><ul><ul><li>Instant Messaging and Presence Information </li></ul></ul>
  12. 12. After Hours Mashup Architecture
  13. 13. Strike Iron SMS <ul><li>Web Service to send text messages </li></ul><ul><ul><li>One of many online, paid services </li></ul></ul><ul><ul><li>Reverse phone number lookup </li></ul></ul><ul><ul><li>Address verification, currency rate, background check </li></ul></ul><ul><li>Pay per click model </li></ul><ul><ul><li>First 100 clicks are free, small fee thereafter </li></ul></ul><ul><li>168 characters per message </li></ul><ul><ul><li>User interface issues </li></ul></ul><ul><ul><li>URL issues (TinyURL) </li></ul></ul><ul><li>Extraordinarily Powerful </li></ul><ul><ul><li>Ubiquitous </li></ul></ul><ul><ul><li>Immediate </li></ul></ul>
  14. 14. After Hours Mashup Architecture
  15. 15. Amazon Web Services <ul><li>Amazon Mechanical Turks </li></ul><ul><ul><li>Human Intelligence Tasks </li></ul></ul><ul><ul><li>Things that humans can do easily, difficult for computers </li></ul></ul><ul><ul><li>A9 Search Engine </li></ul></ul><ul><li>Artificial Artificial Intelligence </li></ul><ul><ul><li>It’s now OK to cheat </li></ul></ul><ul><ul><li>Humans become subroutines </li></ul></ul><ul><li>S3, EC2 have unique telephony implications </li></ul><ul><ul><li>Mr. Erlang never worked for Amazon </li></ul></ul><ul><ul><li>True defense against DDOS </li></ul></ul><ul><ul><li>S3 == linear scalability == holy grail of telephony </li></ul></ul>
  16. 16. After Hours Mashup Architecture
  17. 17. Extensions <ul><li>Call Centers </li></ul><ul><ul><li>Expert based call centers, makes response even faster </li></ul></ul><ul><ul><li>Better for larger scale installations </li></ul></ul><ul><ul><li>Integrates well with Amazon Turks </li></ul></ul><ul><li>Geolocation </li></ul><ul><ul><li>Give the data right to the nurse, showing hospitals, medical centers, etc. </li></ul></ul><ul><ul><li>Combines well with text messages (text directions) </li></ul></ul><ul><ul><li>Get location from reverse phone number lookup, zip code from VxML, GPS inside phone, or all of the above for fraud detection </li></ul></ul><ul><li>Medical Transcriptions to Testing </li></ul><ul><ul><li>Use medical transcriptions to check for consistency, accuracy, training. </li></ul></ul>
  18. 18. Call Center : Sylantro <ul><li>Integrating with Sylantro switch </li></ul><ul><ul><li>Accepts inbound calls into expert queue </li></ul></ul><ul><ul><li>Nurses login to the system from soft client or PSTN phone </li></ul></ul><ul><ul><li>Turks are used to manage larger schedules </li></ul></ul><ul><li>Even better response for patient </li></ul><ul><ul><li>Nurse is right there to answer questions </li></ul></ul><ul><ul><li>Use VoiceXML to do pre-processing, skills based routing </li></ul></ul><ul><li>SOAP based interface using Java </li></ul><ul><li>Better choice for larger setups </li></ul><ul><ul><li>Dedicated equipment, staff </li></ul></ul><ul><li>Other functionality </li></ul><ul><ul><li>Conferencing in Doctor or other expert </li></ul></ul><ul><ul><li>Past history integration with CRM for accuracy </li></ul></ul><ul><ul><li>Click to dial for post care support through e-mail </li></ul></ul>
  19. 19. Yahoo! Local <ul><li>Nurses page shows location of caller </li></ul><ul><ul><li>Location derived from inbound calling info </li></ul></ul><ul><ul><li>Alternatives are GPS, asked </li></ul></ul><ul><li>Gives nurse immediate information to share with patient </li></ul><ul><ul><li>Nearest clinic, hospital, etc. </li></ul></ul><ul><ul><li>Directions; call ahead; 911 conference </li></ul></ul><ul><li>Flash based API </li></ul><ul><ul><li>Many APIs available… this was wicked simple to use </li></ul></ul>
  20. 20. Thank you!!! <ul><li>For more information, source code, and this presentation, see my site at </li></ul><ul><li>thomashowe.com </li></ul>

×