Your SlideShare is downloading. ×
Conversational Internet - Creating a natural language interface for web pages
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Conversational Internet - Creating a natural language interface for web pages

4,356
views

Published on

Presenting a paper to the 10th International Cross-Disciplinary Conference on Web Accessibility (W4A 2013)

Presenting a paper to the 10th International Cross-Disciplinary Conference on Web Accessibility (W4A 2013)

Published in: Technology, Education

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
4,356
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. © 2009 IBM CorporationConversational Internet:A natural language interface for webpagesDale Lane – IBM United Kingdom14 May 2013dale.lane@uk.ibm.com
  • 2. © 2012 IBM Corporation2Challenge
  • 3. © 2012 IBM Corporation3Challenge
  • 4. © 2012 IBM Corporation4Motivation
  • 5. © 2012 IBM Corporation5Understanding the page : Identifying type
  • 6. © 2012 IBM Corporation6Understanding the page : Identifying navigation options
  • 7. © 2012 IBM Corporation7Understanding the page : Identifying calls-to-action
  • 8. © 2012 IBM Corporation8Understanding the user : Retrieving information
  • 9. © 2012 IBM Corporation9Understanding the user : Mouse actions
  • 10. © 2012 IBM Corporation10Understanding the user : Mouse actions
  • 11. © 2012 IBM Corporation11Understanding the user : Keyboard actions
  • 12. © 2012 IBM Corporation12Understanding the user : Keyboard actions
  • 13. © 2012 IBM Corporation13Understanding the user : Keyboard actions
  • 14. © 2012 IBM Corporation14Conversational Internet Presenting an early-stage prototype being developed to explore the potential forquestion answering as an alternative approach to screen-readers for retrievinginformation from web pages Architecture and approach inspired by active area of research and development inquestion answering on a knowledge derived from a corpus of documents(Ferucci, Lally, Chu-Carroll, et al)http://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=6177717
  • 15. © 2012 IBM Corporation15ImplementationbrowserextensionUIMALanguageWareJavaclient server
  • 16. © 2012 IBM Corporation16Implementationnew request new requestresponsewithconv. idresponsefollow-uprequestrequestwithconv. id“what can I do?”“the options are...”“I want to do...”
  • 17. © 2012 IBM Corporation17Implementation : Client Firefox extension Submits current state ofthe page to server foranalysis Training mode
  • 18. © 2012 IBM Corporation18Implementation : Server Analyzing the page Processing user queries
  • 19. © 2012 IBM Corporation19Implementation : Server : Understanding the page
  • 20. © 2012 IBM Corporation20Implementation : Server : Understanding the pageWhat type of site is this? Machine learning classifiers Whitelists of known domains
  • 21. © 2012 IBM Corporation21Implementation : Server : Understanding the pageWhat can we infer from markup used? Semantic tags ARIA CSS class names
  • 22. © 2012 IBM Corporation22Implementation : Server : Understanding the pageWhat does the structure of the page layout suggest? Machine learning models for common page elements
  • 23. © 2012 IBM Corporation23Implementation : Server : Understanding the pageWhat does the text of the page tell us? Natural Language Processing using LanguageWare to recognize common formsof call-to-action
  • 24. © 2012 IBM Corporation24Implementation : Server : Responding to queries
  • 25. © 2012 IBM Corporation25Implementation : Server : Responding to queriesInterpreting the query NLP rules created with LanguageWare to map to closest known command type WordNet to attempt matches using synonyms of unknown terms
  • 26. © 2012 IBM Corporation26Implementation : Server : Responding to queriesExtracting the requested information Information requested from page extracted from serialized CAS created by firstpipeline, by retrieving sections with relevant annotations
  • 27. © 2012 IBM Corporation27Implementation : Server : Responding to queriesPreparing a response Speech generated using Nuance NDev and streamed to client
  • 28. © 2012 IBM Corporation28Future work / Limitations Use cases Usability testing RIA / AJAX sites
  • 29. © 2012 IBM Corporation29 Paper submitted to W4A– http://goo.gl/3X2iv Overview presentation– http://youtu.be/uS6oquJdgbw Demonstration of the prototype– http://youtu.be/tSGyPCcO-bYDale Lanedale.lane@uk.ibm.com@dalelane

×