Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Similar to Text-mining and Automation
Similar to Text-mining and Automation (20)
More from benosteen
More from benosteen (20)
Recently uploaded
Recently uploaded (20)
Text-mining and Automation
- 1. Automation and Text Mining Ben O'Steen 25 May 2010
- 2. Hi
- 3. Ben O'Steen (Software Engineer, Oxford University Libraries)
- 4. Ben O'Steen (Software Engineer, Oxford University Libraries) Freelance Enthusiast
- 5. “ Text-mining” (and related techniques): Processing natural language to gain direct and contextual information, often with a means to quantify this information's accuracy.
- 6. Automation (in this context): Making decisions , providing additional options and increasing the amount of information understood by a system without the need for human effort.
- 7. “BI Search and Text Analytics”[1] says: “structured data in first place at 47%, trailed by unstructured (31%) and semi-structured data (22%).” 53%! And this was amongst data management professionals! [1] “BI Search and Text Analytics”, 2006 http://tdwi.org/research/2007/07/tdwi-best-practices-reports.aspx Natural Language Processing
- 10. Developing . Every year, new and better solutions are found.
- 12. Developing . Every year, new and better solutions are found.
- 13. Multi-disciplinary . The skills required by the team are broad: Machine-learning, Linguistics, Statistics, Logic and so on.
- 20. Crude, but interesting result.
- 25. (I'll come back to why it is important to note that NLP is heavily multidisciplinary shortly)
- 27. Search-engines (from Google's to Apache Solr)
- 29. And just about any business that trades solely over the web.
- 32. Customer 'churn'
- 33. Credit ratings
- 35. Predictive analytics (game theory)
- 37. TrueKnowledge – NLP + Semantic Knowledge base
- 39. Specific information added to results, formatted based on the meaning of your search query.
- 41. Predictive analytics (game theory)
- 50. Why are people scared of NLP?
- 54. Access to in-depth material is reasonably good – if your library has academic subscriptions.
- 55. There is little middle ground between the two.
- 63. Cannot be blindly applied for good results (yet)
- 65. Requires understanding language syntax!
- 66. Multidisciplinary – makes it very easy for documentation to 'Not State the Obvious'
- 69. Cannot be blindly applied for good results (yet)
- 70. A long learning curve
- 72. Few have someone they know personally who can explain NLP and text-mining analyses.
- 74. If the barrier for new people to join in is too high, then they give up and go elsewhere....
- 76. If the barrier for new people to join in is too high, then they give up and go elsewhere....
- 77. … which means that those writing documentation never get good feedback from those they trust or know 'socially'...
- 79. If the barrier for new people to join in is too high, then they give up and go elsewhere....
- 80. … which means that those writing documentation never get good feedback from those they trust or know 'socially'...
- 86. From my experience, very few developers are starting with Java web apps anymore.
- 87. Simple API? Can we take inspiration from Google? Is it possible to offer some purposeful tools or UIMA workflows in a way that can be accessed by any browser*? (*without a Java applet)
- 88. What 'I' want aka: services that I or people I know have either been asked for, or have seen the desperate need for.
- 90. Enhanced article: http://dx.doi.org/10.1371/journal.pntd.0 000228.x001
- 95. Trace back to find the 'source' of a particular idea.
- 97. Could be applied retroactively.
- 99. Subject and cross-domain classification to aid searching/browsing
- 100. Semantic expression of textual citations
- 102. We need more people to understand how to perform certain tasks.
- 103. We need to make systems more capable of holding heuristically obtained information.
- 104. NLP needs simple entrypoints – more 'Hello World' type examples.
- 105. NLP web apps that can be accessed from the browser.