HLT 2013 - Big Data Navigation and Discovery by Stefan Andreasen & Jeff Godbold
•Download as PPTX, PDF•
0 likes•1,137 views
Last year Basis Technology introduced Odyssey – an analytics solution, which provides an open, scalable platform for search, navigation and discovery. Its purpose is to streamline the development of highly customizable solutions for efficiently discovering relevant information from vast volumes of structured and unstructured content. Basis Technology has recently teamed with Kapow to incorporate their industry leading Big Data integration platform into the Odyssey solution to enhanced both the range of data now available to Odyssey as well as the ease of deployment. During this session, Stefan Andreasen (Kapow) and Jeff Godbold (Basis Technology) will provide an overview of this joint solution, highlighting the many benefits it offers to the world of multilingual, information discovery.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (16)
Similar to HLT 2013 - Big Data Navigation and Discovery by Stefan Andreasen & Jeff Godbold
Similar to HLT 2013 - Big Data Navigation and Discovery by Stefan Andreasen & Jeff Godbold (20)
More from Basis Technology
More from Basis Technology (14)
Recently uploaded
Recently uploaded (20)
HLT 2013 - Big Data Navigation and Discovery by Stefan Andreasen & Jeff Godbold
- 1. Big Data Navigation and Discovery Stefan Andreasen Jeff Godbold Andy Lasko CTO & Founder Director – Fed Solutions Partner Manager Kapow Software Basis Technology Kapow Software
- 2. “ The most meaningful way to differentiate your company from your competition, to put distance between you and the crowd, is to do an outstanding job with information. HOW YOU GATHER, MANAGE AND USE INFORMATION WILL DETERMINE WHETHER YOU WIN OR LOSE. ” Bill Gates
- 4. KNOWN KNOWNS Missing pieces in a familiar world
- 5. KNOWN UNKNOWNS It’s about proactive intelligence
- 6. There will always be unknowns The risky ones are the UNKNOWN UNKNOWNS
- 7. Coming Soon: Toilet Paper Priced Like Airline Tickets Source: Wall Street Journal, September5, 2012, Coming Soon: Toilet Paper Priced Like Airline Tickets
- 9. Data is Spreading Cloud Silos SaaS EAI Portals Centralization Vendor Consolidation SOA Web Legacy Mobile B2B Globalization Social Big Data
- 10. Data is Aging Fast 90% created in the last two years 10X More data in two years
- 12. What if you could access the web as it was a database? Classified Blogs Review sites Emails News Competitors Gov 14 Forums
- 13. Kapow Katalyst If you can see it you have it 15 Odyssey Analytics Platform Kapow Katalyst
- 14. Data to Value Faster with Kapow Get Your Data in Days/Weeks, Not Months/Years Acquisition Months / Day / Weeks Years Distribution Enrichment
- 15. Now What? 17
- 16. Now What? 18
- 19. George Bush? 21
- 20. Odyssey Analytics Platform Entity Resolution 22
- 26. Odyssey Analytics Platform Projects Muslim Brotherhood Blog Blog 28
- 27. Odyssey Analytics Platform User Interface & Workflow 3rd Party Integration Data Processing & Analytics Data Sources 29 DBs
- 28. Demonstration 30
- 29. Questions? 31
Editor's Notes
- [talk about video – how revolutionary this was at the time but how real it is in today’s world, not just for governments but for business….]The Known Known’s are the comfort zone, the missing piece in a world that we know everything about. Inventory levels, ordering history, employees turnover – all information I know I need and I know where and how to get it…. Nothing too unfamiliarBorders & Barnes&Noble – know competitors, successfully rolled-up the smaller bookstores (or put them out of business), they could monitor in the symmetrical wayData feed providers; API connectorsEnterprises focus on pre-selected segments ‘known’ data and generalize & analyze from it to the 'unseen' (unknown).
- The known unknowns are the things you know that you don’t know and proactively trying to get. For example Borders & Barnes&Noble - knew about the internet, knew people were transacting on-line, knew books were being bought but it was an unknown – were they listening? They didn’t know how profound this will be to their business going forwards. As a large media and entertainment company I know my IP is being illegally used online, I don’t know where and how…In eCommerce, I know who my competitors are but I don’t know when are they going to change their prices, of product products, at what frequency, in what regions and SKUsAnd as consumers are becoming so much more social and connected, the decision buying journey involves so many more touch points that are influenced by social commentary. So I know my customers are active on social channels but what are they saying about my brand, what are the positives and negatives etc.This is where you are at as Kapow customers
- There will always be unknowns and the risky ones are the unknown unknows, these are the things under the surface that can have a significant impact… the black swans…]Barnesvs Borders – Amazon is the BS – then eBooks It can be very difficult and can be expensive to collect these unknown (asymmetrical) data sets vs the easy data. But this has the potential to be Black Swan impact full. Very small changes to small pieces of this once unknown - now known data, can have massive impact to a Business. You have to be monitoring. Those of you using Kapow for Competitive pricing understand this.
- Many people are under the impression that Big Data is not real. It’s a great concept that doesn’t apply to me. Let me show you how people are already using it today to gain significant competitive advantage. We all know airline ticket pricing is dynamic but did you know that how much you pay for toilet paper also depends on when you buy it…A recent article from the Wall Street Journal discussed the model of the modern retailer. Leading retailers are dynamically adjusting product prices in the course of a day, monitoring competitors prices and other market conditions and responding in real-time. It’s elasticity of pricing and it’s used with the most commodity products like toilet paper. You can see that if you are not in the game you are in the sideline… What does Amazon know that nobody else know?
- Talk about the growth of the data and how it relates to your original vision when you started the company.
- Big Interaction data is growing rapidly. There are 2.5 quintillion bytes of it created every day. 40k new websites are registered each day with a total of 140 mill.The information is out there for you to leverage
- When it comes to Big Data there is no greatest data source than the Web. This includes blogs, forums, customer review sites, social media, variety of cloud applications and even your competitors, partners and channel portals. There were 170 million websites as of end of last year and 30 million new ones added every month (http://news.netcraft.com/archives/2011/12/09/december-2011-web-server-survey.html). These data sources are a gold mine of very valuable information for any analytical effort allowing companies to monitor customer behavior and buying patterns to improve sales campaigns or predicting market trends allowing financial services companies to trade ahead of their competition and significantly increase revenues. But as I will discuss later in my presentation the data spread across these growing number of sources also creates a challenge beyond volumes.
- Thank you Jeff[Show Muslim Brotherhood site]To get this content, someone needs to search the archives [show], copy all of the documents and metadata [show] into an analytical environment. When you get a few hundred of these sources, the problem gets worse.[Build Robot]So let’s see how we’d build a connector to this data source. Build, debug, deploy, schedule[Show REST/JSON ]Analysts use Kapow to build connectors.IT uses Kapow to build synthetic API’s for application integration, data warehousing and content acquisition.Now that we have our data in Odyssey, I’ll pass it back to Jeff.[Show slide again and build #1 then #2]Speak to bullets 1….The Muslim Brotherhood site is a web application with documents and unstructured metadata. This is a common data source.We see these inside and outside of the Enterprise. Internally these are our CMS, Wiki’s, Intel-link, project repositories and so on.Often there no API’s, the wrong API’s, or means for an analyst to copy the content an analytical application.IT does not have the tools, time or budget to connect to every data source. Manual migration – copy and paste – is too time consuming. Kapow’s workflow development environment we used today provides us rapid access to virtually any content from any application.[Build 2]Let’s look at another example.Let’s say you want to capture a web page or web sites content without writing a connector. Kapow’s ‘Capture’ is a workflow that uses Kapow’s browser connector to request all a web servers browser accessible content.This includes HTML, Images, PDF’s and other file formats, CSS, JS, and even some links and images in Flash.Kapow’s web capture is like Google on Steroids and turns your file system and analytical environment into a mirror of the target web server.The Capture workflow is built in Kapow so it is easy to reconfigure for your exact needs.[Show Google Blog results] http://www.google.com/search?q=muslim+brotherhood&tbm=blgThese are the results of a Google blog search for the term ‘Muslim Brotherhood’. We can call the ‘Web Capture’ from a Browser Plugin like we see here, from a lightweight Kapow Kapplet, from the Kapow Scheduler, or from within Odyssey.[SHOW BUILD SLIDE #3]What if you have something harder? Something that requires logins, multiple coordinated workflows, content in multiple languages and you want to hide your content acquisition activity. [Show Site]I’ll use the Arab Chat room for Egypt as an example.I need to load the site, create a unique name, repeatedly search for conversations, capture the conversations and send them to Odyssey.[Show Robots]Talk through 3 robotsArabChat_Messages.robotArabChat_ChatsToBasis.robotArabChat_People_IP.robotAs Stefan mentioned, Kapow Software can connect to the content of any application – whether it’s a website, an Enterprise Application or something in between like hosted applications, SaaS, and partner portals – in minutes. Kapow can connect to databases, API’s, XML and other file formats.With Kapow Software, Basis users are able to get their content into Odyssey faster, cheaper and they can get content that is difficult to access. Now I’m going to pass it back over to Jeff to review some of this data.