1.
Semantic Search 3.0<br />A Yarn about Twine<br />ISWC 2009 Keynote<br />Nova Spivack<br />Founder, CEO Twine.com<br />nova@twine.com<br />

2.
In 1998 my first Internet companyEarthWebMakers of Developer.com & Dice.comhad a ginormous IPO w00t!!!<br />2<br />1.11.2008 | Product Positioning<br />

3.
At 29 years old, I was set for life**On paper….<br />3<br />1.11.2008 | Product Positioning<br />

4.
But instead of retiring Youth and idealism won out<br />4<br />1.11.2008 | Product Positioning<br />

5.
I wanted to build something REALLY big…A Web of knowledge<br />5<br />1.11.2008 | Product Positioning<br />

6.
So I self-funded my own mini R&D lab, Lucid Ventures<br />6<br />1.11.2008 | Product Positioning<br />

7.
Kris Thorisson (PhD) Jim Wissner (Java whiz)And a bunch of others…<br />7<br />1.11.2008 | Product Positioning<br />

8.
Lucid grew to a few PhD’s And a bunch of engineers We did lots of exploring. In Java.<br />8<br />1.11.2008 | Product Positioning<br />

9.
We eventually built Personal Radar<br />9<br />1.11.2008 | Product Positioning<br /><ul><li>P2P Semantic Desktop for knowledge sharing

10.
Cross-platform, Pure Java

11.
Fully semantic, RDF and OWL based

12.
Homegrown ultra-fast triplestore</li></li></ul><li>We eventually spun the project out as its own company,Radar Networks<br />10<br />1.11.2008 | Product Positioning<br />

13.
11<br />1.11.2008 | Product Positioning<br />

14.
12<br />1.11.2008 | Product Positioning<br />

15.
13<br />1.11.2008 | Product Positioning<br />

16.
14<br />1.11.2008 | Product Positioning<br />

17.
15<br />1.11.2008 | Product Positioning<br />

18.
Screenshots<br />16<br />1.11.2008 | Product Positioning<br />

19.
17<br />1.11.2008 | Product Positioning<br />

20.
18<br />1.11.2008 | Product Positioning<br />

21.
19<br />1.11.2008 | Product Positioning<br />

22.
20<br />1.11.2008 | Product Positioning<br />

23.
21<br />1.11.2008 | Product Positioning<br />

24.
Oh yeah,And I also angel invested in a bunch of startupsAnd bought up more EarthWeb stock. w00t!<br />22<br />1.11.2008 | Product Positioning<br />

25.
Then the Bubble burst…* *WTF???<br />23<br />1.11.2008 | Product Positioning<br />

26.
Nuclear winter for IT and Web venturesBegan…<br />24<br />1.11.2008 | Product Positioning<br />

27.
DON’T TRY THIS AT HOME<br />25<br />1.11.2008 | Product Positioning<br />

28.
Despite the Economy, We Continued to Press ForwardWith our product plan<br />26<br />1.11.2008 | Product Positioning<br />

29.
But there was no venture fundingAvailable anywhere**VC’s are scaredy cats<br />27<br />1.11.2008 | Product Positioning<br />

30.
Luckily,a few days before we ran out of money…<br />28<br />1.11.2008 | Product Positioning<br />

31.
SRI selected us to join the DARPA CALO project<br />29<br />1.11.2008 | Product Positioning<br />

32.
Personal Radar became part of OpenIRIS* See: OpenIRIS.org *Cool project<br />30<br />1.11.2008 | Product Positioning<br />

33.
2004 - 2005We worked on DARPA CALO & IRIS2006Series A venture roundFinished our work on CALO & IRISStarted to work on our own products<br />31<br />1.11.2008 | Product Positioning<br />

34.
Our GoalWeb-Scale Knowledge Networking For Consumers<br />32<br />1.11.2008 | Product Positioning<br />

35.
ObstacleThere was no platform that met our needs at the time. Not even our own.So from 2006 – 2007 we built one**Insane<br />33<br />1.11.2008 | Product Positioning<br />

36.
To accomplish this,using RDF and OWLOn a triplestorewe really had to push the envelope<br />34<br />1.11.2008 | Product Positioning<br />

37.
We built a big federated tuplestoreSupport up to 100 billion tuples Full ACL permissions on the data Capable of serving millions of consumers<br />35<br />1.11.2008 | Product Positioning<br />

38.
In 2008 the platform was ready We started building apps on it<br />36<br />1.11.2008 | Product Positioning<br />

39.
October 2008 Twine went Beta <br />37<br />1.11.2008 | Product Positioning<br />

40.
A Slightly Ambitious Project*Semantic WebNatural language processingEntity detectionGraph theoryClusteringRecommendationsSemantic filteringCollaborative authoringAuto-taggingSummarizationVisualizationPersonalization*What the (bleep) were we thinking?<br />38<br />1.11.2008 | Product Positioning<br />

41.
March 2009 We raised a $14mm Series B round A total of $24mm raised so far…**Ok, that’s what we were thinking…<br />39<br />1.11.2008 | Product Positioning<br />

42.
Twine grew faster than expectedConsumers actually were using it**Hey, this #$%@ works<br />40<br />1.11.2008 | Product Positioning<br />

43.
Within 6 months of launch…Up to 2.2mm unique visitors monthly 4mm pieces of content added 25K interest groups 100’s of articles about it Twine grew faster than Twitter**Our VCs were happy <br />41<br />1.11.2008 | Product Positioning<br />

44.
But Twine was not finished A lot more work was needed And we were gearing up Hiring, designing, coding, spending…<br />42<br />1.11.2008 | Product Positioning<br />

45.
We grew to over 30 people.Mostly engineers. But we really needed more like 70.<br />43<br />1.11.2008 | Product Positioning<br />

46.
But we were optimistic and focused onDesigning the next version of Twine<br />44<br />1.11.2008 | Product Positioning<br />

47.
Then the economy tankedAgain…..<br />45<br />1.11.2008 | Product Positioning<br />

48.
Silicon Valley became a lot less happyAll venture funding dried up<br />46<br />1.11.2008 | Product Positioning<br />

49.
So, like many other venturesWe had to cut back drasticallyon spending, hiring, and our product goals<br />47<br />1.11.2008 | Product Positioning<br />

50.
Lessons LearnedTwine 1.0 tried to solve too many problemsToo many featuresToo hard to figure outOur data revealed that Twine users mainly just want to search and track interests<br />48<br />1.11.2008 | Product Positioning<br />

51.
Lessons LearnedWe should not have relied on a tuplestore so centrally in the Twine 1.0 architectureWe haven’t yet seen a tuplestore that fully meets the needs of a major consumer online web site like Twine<br />49<br />1.11.2008 | Product Positioning<br />

52.
The Web 2.0 party was over.Where to go from here?… A Radical Rethink<br />50<br />1.11.2008 | Product Positioning<br />

53.
Continuing on the path we were onwould be too expensive to scaleMust change the architecture, or change the product,or both**We chose “or both”<br />51<br />1.11.2008 | Product Positioning<br />

54.
The Future of Search<br />Confidential<br />

55.
Where Search is Headed<br />Semantic<br />Personalized<br />Real-time<br />Social<br />Sharing<br />Tracking<br />KM<br />Reasoning<br />53<br />1.11.2008 | Product Positioning<br />

56.
But the Semantic Web Hasn’tHappened Yet…<br />At least not for consumers<br />And most Web developers…<br />54<br />1.11.2008 | Product Positioning<br />

57.
Why hasn’t the Semantic Web taken off yet?<br />55<br />1.11.2008 | Product Positioning<br />

58.
How can we demonstrate the value of the Semantic Web to consumers?And make it easily useful to Web developers?In the near-term?<br />56<br />1.11.2008 | Product Positioning<br />

59.
Let’s Get Real<br />Consumer are not going to add semantic metadata<br />Webmasters are too lazy, or don’t know how<br />Humans are bad at generating good metadata anyway<br />Automated metadata generation is the only practical solution<br />57<br />1.11.2008 | Product Positioning<br />

60.
Index the Semantics of the Web<br />Create the Semantic Web <br />From the non-Semantic Web<br />Automatically<br />Provide it back as a Web service<br />58<br />1.11.2008 | Product Positioning<br />

61.
Provide faceted semantic search and navigation against the indexAt Web-scale<br />59<br />1.11.2008 | Product Positioning<br />

62.
Kind of a Semantic Google<br />IBM.com<br />Web Site<br />Joe<br />Person<br />IBM<br />Company<br />Palo Alto<br />City<br />Lives in<br />Publisher of<br />Fan of<br />Lives in<br />Subscriber to<br />Employee of<br />Sue<br />Person<br />Jane<br />Person<br />Dave.com<br />RSS Feed<br />Coldplay<br />Band<br />Fan of<br />Friend of<br />Member of<br />Design<br />Team<br />Group<br />Depiction of<br />Married to<br />Source of<br />123.JPG<br />Photo<br />Member <br />of<br />Bob<br />Person<br />Dave.com<br />Weblog<br />Depiction of<br />Member of<br />Dave<br />Person<br />Stanford<br />Alumnae<br />Group<br />Member of<br />Author of<br />Member of<br />Person<br />Application<br />

63.
But not on anything close to Google’s budget<br />61<br />1.11.2008 | Product Positioning<br />

64.
What were we smoking?**We were not Freebasing<br />62<br />1.11.2008 | Product Positioning<br />

65.
Well it turns out, it’s not impossible…<br />63<br />1.11.2008 | Product Positioning<br />

66.
T2<br />64<br />1.11.2008 | Product Positioning<br />

67.
The T2 Approach…<br />65<br />1.11.2008 | Product Positioning<br />Focus on search Keep it simple Broad appeal Scale cheaply<br /> Connect with ecosystem<br />

68.
T2 ArchitectureHigh-performanceOpen-ended scalingMore semantic (RDF, OWL)Not built directly on a triplestoreNot built directly on a databaseHeavy use of SOLR (from Lucene)<br />66<br />1.11.2008 | Product Positioning<br />

69.
T2 Stack<br />67<br />1.11.2008 | Product Positioning<br />Twine Properties<br />Partners/3rd Parties<br />WebApp Framework<br />APIs<br />Application Services<br />Resource Service<br />Web Corpus<br />(e.g., BOSS, etc)<br />Semantic Metadata Service<br />Tuple Service<br />SOLR Farm<br />RDBMS<br />

70.
T1 Approach<br />Human<br />Tagging<br />AI<br />Semantic<br />Web<br />Make the Data Smarter<br />Machine<br />Tagging<br />Linguistics<br />Extraction<br />Statistics<br />Make the software smarter<br />

71.
T2 Approach<br />Semantic<br />Web<br />Make the Data Smarter<br />Linguistics<br />Extraction<br />Statistics<br />Make the software smarter<br />

72.
A Tactical Shift<br />Freebase<br />DBpedia<br />Make the Data Smarter<br />Twine <br />T1<br />EVRI<br />Wolfram <br />Alpha<br />Wikipedia<br />Open Calais<br />Endeca<br />Expert System<br />Flickr<br />Bing<br />Siri<br />FAST<br />Powerset<br />Yahoo<br />Delicious<br />Autonomy<br />Inxight<br />Hakia<br />Google<br />Make the software smarter<br />

73.
To a More Mainstream App<br />Freebase<br />Twine <br />T2<br />DBpedia<br />Make the Data Smarter<br />Twine <br />T1<br />EVRI<br />Wolfram <br />Alpha<br />Wikipedia<br />Open Calais<br />Endeca<br />Expert System<br />Flickr<br />Bing<br />Siri<br />FAST<br />Powerset<br />Yahoo<br />Delicious<br />Autonomy<br />Inxight<br />Hakia<br />Google<br />Make the software smarter<br />

74.
Web-Scale Semantic ExtractionAnd search<br />72<br />1.11.2008 | Product Positioning<br />

75.
T2 Semantic Index<br />Videos<br />Music<br />People<br />Reviews<br />Games<br />News<br />Products<br />Recipes<br />Services<br />How-To’s<br />Classifieds<br />Events<br />Hotels<br />Bars<br />Resumes<br />Coupons<br />Reports<br />Photos<br />Help<br />73<br />

76.
Many Uses for this Index<br />Search<br />Interest tracking<br />Recommendations<br />Web application development<br />74<br />1.11.2008 | Product Positioning<br />

77.
1. Do a Keyword Search<br />2. Filter by Type<br />3. Filter by <br /> Attribute<br />Recipe<br />Cuisine<br />Web<br />Page<br />Classified Ad<br />Difficulty<br />Web<br />Page<br />Product <br />Ingredient<br />Prep Time<br />News<br />Web Page<br />Author<br />Web<br />Page<br />Hotel<br />Type of Dish<br />Web<br />Page<br />Song<br />Dietary Option<br />Website<br />Game<br />Web Page<br />Prep Time<br />Help Article<br />Web Page<br />Calories<br />Web Page<br />Etc…<br />Person<br />Etc…<br />75<br />

78.
Use Case: Cooking<br />76<br />

79.
Scenario: Family Dinner<br />Deciding what to cook…<br />

80.
Scenario: Family Dinner<br />

81.
Scenario: Family Dinner<br />

82.
Current Scenario: Family Dinner<br />[INSERT BING RESULTS + EXTERNAL LOGOS]<br />

83.
Current Scenario: Family Dinner<br />[INSERT BING RESULTS + EXTERNAL LOGOS]<br />

84.
Current Scenario: Family Dinner<br />

85.
Current Scenario: Family Dinner<br />

86.

87.
T2 – Tools for Developers<br />API to T2 Index and services<br />Widgets and components for 3rd party sites<br />Web-based ontology community – like Sourceforge<br />Browser plugin for easy site mapping to ontologies<br />85<br />1.11.2008 | Product Positioning<br />

88.
T2 Ontology Site <br />Demo<br />86<br />

89.

90.

91.

92.

93.

94.

95.
T2 Site Mapping Tool<br />Confidential<br />

96.
T2 Site Mapping Tool<br />Confidential<br />

97.
T2 Site Mapping Tool<br />Confidential<br />

98.
T2 Business Model<br />T2 destination<br />Advertising on the site<br />T2 site search & ad network<br />Provide sites with better site search<br />Targeted ads<br />Revshare<br />T2 API<br />Partners pay us, or revshare with us<br />Monetize however they want to<br />96<br />1.11.2008 | Product Positioning<br />

99.
T2 Near-Term Vertical Focus Areas<br />Lifestyle<br />Entertainment<br />Shopping<br />Food (Complete)<br />Health<br />Travel<br />People<br />Gaming (In Dev)<br />Sports<br />Music<br />TV<br />Film <br />Consumer Electronics<br />Health & Beauty<br />Classified Ads<br />Media Products<br />Help & Support<br />

100.
T2 Long-Term Goals<br />Index the structured data on the Web<br />Across all major vertical domains<br />With help from an army of outside developers and partners<br />Grow an ecosystem around the index<br />98<br />1.11.2008 | Product Positioning<br />

101.
Timeline T1 is in Maintenance Mode T2 is in Alpha T2 Goes Beta by Q1<br />99<br />1.11.2008 | Product Positioning<br />

102.
What Next?Find Follow Share<br />100<br />1.11.2008 | Product Positioning<br />

103.
For all types of content Web pages Data records Real-time content<br />101<br />1.11.2008 | Product Positioning<br />

104.
Find stuff (semantic search)Then follow it (interest tracking) Then share it (socialization)<br />102<br />1.11.2008 | Product Positioning<br />

105.
But this time…<br />103<br />1.11.2008 | Product Positioning<br />

106.
We’re starting with just “Find”<br />104<br />1.11.2008 | Product Positioning<br />

107.
Conclusions Web-Scale Semantic Search is the Next-Step for Search and for Twine But after 9 years of this, it’s clear It won’t happen overnight<br />105<br />1.11.2008 | Product Positioning<br />

108.
Stay Tuned…<br />106<br />1.11.2008 | Product Positioning<br />

109.
Contact Information<br />Nova Spivack<br />Twitter: @novaspivack<br />nova@twine.com<br />