Your SlideShare is downloading. ×
A Yarn About Twine -- ISWC 2009 Keynote --   Nova Spivack
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

A Yarn About Twine -- ISWC 2009 Keynote -- Nova Spivack

3,899

Published on

The story of how Twine.com came to be, and where it's going. A candid look behind the scenes. …

The story of how Twine.com came to be, and where it's going. A candid look behind the scenes.

If it doesn't load here on slideshare -- try viewing it at http://novaspivack.com

Published in: Technology, Business
0 Comments
18 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,899
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
7
Comments
0
Likes
18
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Semantic Search 3.0
    A Yarn about Twine
    ISWC 2009 Keynote
    Nova Spivack
    Founder, CEO Twine.com
    nova@twine.com
  • 2. In 1998 my first Internet companyEarthWebMakers of Developer.com & Dice.comhad a ginormous IPO w00t!!!
    2
    1.11.2008 | Product Positioning
  • 3. At 29 years old, I was set for life**On paper….
    3
    1.11.2008 | Product Positioning
  • 4. But instead of retiring Youth and idealism won out
    4
    1.11.2008 | Product Positioning
  • 5. I wanted to build something REALLY big…A Web of knowledge
    5
    1.11.2008 | Product Positioning
  • 6. So I self-funded my own mini R&D lab, Lucid Ventures
    6
    1.11.2008 | Product Positioning
  • 7. Kris Thorisson (PhD) Jim Wissner (Java whiz)And a bunch of others…
    7
    1.11.2008 | Product Positioning
  • 8. Lucid grew to a few PhD’s And a bunch of engineers We did lots of exploring. In Java.
    8
    1.11.2008 | Product Positioning
  • 9. We eventually built Personal Radar
    9
    1.11.2008 | Product Positioning
    • P2P Semantic Desktop for knowledge sharing
    • 10. Cross-platform, Pure Java
    • 11. Fully semantic, RDF and OWL based
    • 12. Homegrown ultra-fast triplestore
  • We eventually spun the project out as its own company,Radar Networks
    10
    1.11.2008 | Product Positioning
  • 13. 11
    1.11.2008 | Product Positioning
  • 14. 12
    1.11.2008 | Product Positioning
  • 15. 13
    1.11.2008 | Product Positioning
  • 16. 14
    1.11.2008 | Product Positioning
  • 17. 15
    1.11.2008 | Product Positioning
  • 18. Screenshots
    16
    1.11.2008 | Product Positioning
  • 19. 17
    1.11.2008 | Product Positioning
  • 20. 18
    1.11.2008 | Product Positioning
  • 21. 19
    1.11.2008 | Product Positioning
  • 22. 20
    1.11.2008 | Product Positioning
  • 23. 21
    1.11.2008 | Product Positioning
  • 24. Oh yeah,And I also angel invested in a bunch of startupsAnd bought up more EarthWeb stock. w00t!
    22
    1.11.2008 | Product Positioning
  • 25. Then the Bubble burst…* *WTF???
    23
    1.11.2008 | Product Positioning
  • 26. Nuclear winter for IT and Web venturesBegan…
    24
    1.11.2008 | Product Positioning
  • 27. DON’T TRY THIS AT HOME
    25
    1.11.2008 | Product Positioning
  • 28. Despite the Economy, We Continued to Press ForwardWith our product plan
    26
    1.11.2008 | Product Positioning
  • 29. But there was no venture fundingAvailable anywhere**VC’s are scaredy cats
    27
    1.11.2008 | Product Positioning
  • 30. Luckily,a few days before we ran out of money…
    28
    1.11.2008 | Product Positioning
  • 31. SRI selected us to join the DARPA CALO project
    29
    1.11.2008 | Product Positioning
  • 32. Personal Radar became part of OpenIRIS* See: OpenIRIS.org *Cool project
    30
    1.11.2008 | Product Positioning
  • 33. 2004 - 2005We worked on DARPA CALO & IRIS2006Series A venture roundFinished our work on CALO & IRISStarted to work on our own products
    31
    1.11.2008 | Product Positioning
  • 34. Our GoalWeb-Scale Knowledge Networking For Consumers
    32
    1.11.2008 | Product Positioning
  • 35. ObstacleThere was no platform that met our needs at the time. Not even our own.So from 2006 – 2007 we built one**Insane
    33
    1.11.2008 | Product Positioning
  • 36. To accomplish this,using RDF and OWLOn a triplestorewe really had to push the envelope
    34
    1.11.2008 | Product Positioning
  • 37. We built a big federated tuplestoreSupport up to 100 billion tuples Full ACL permissions on the data Capable of serving millions of consumers
    35
    1.11.2008 | Product Positioning
  • 38. In 2008 the platform was ready We started building apps on it
    36
    1.11.2008 | Product Positioning
  • 39. October 2008 Twine went Beta
    37
    1.11.2008 | Product Positioning
  • 40. A Slightly Ambitious Project*Semantic WebNatural language processingEntity detectionGraph theoryClusteringRecommendationsSemantic filteringCollaborative authoringAuto-taggingSummarizationVisualizationPersonalization*What the (bleep) were we thinking?
    38
    1.11.2008 | Product Positioning
  • 41. March 2009 We raised a $14mm Series B round A total of $24mm raised so far…**Ok, that’s what we were thinking…
    39
    1.11.2008 | Product Positioning
  • 42. Twine grew faster than expectedConsumers actually were using it**Hey, this #$%@ works
    40
    1.11.2008 | Product Positioning
  • 43. Within 6 months of launch…Up to 2.2mm unique visitors monthly 4mm pieces of content added 25K interest groups 100’s of articles about it Twine grew faster than Twitter**Our VCs were happy
    41
    1.11.2008 | Product Positioning
  • 44. But Twine was not finished A lot more work was needed And we were gearing up Hiring, designing, coding, spending…
    42
    1.11.2008 | Product Positioning
  • 45. We grew to over 30 people.Mostly engineers. But we really needed more like 70.
    43
    1.11.2008 | Product Positioning
  • 46. But we were optimistic and focused onDesigning the next version of Twine
    44
    1.11.2008 | Product Positioning
  • 47. Then the economy tankedAgain…..
    45
    1.11.2008 | Product Positioning
  • 48. Silicon Valley became a lot less happyAll venture funding dried up
    46
    1.11.2008 | Product Positioning
  • 49. So, like many other venturesWe had to cut back drasticallyon spending, hiring, and our product goals
    47
    1.11.2008 | Product Positioning
  • 50. Lessons LearnedTwine 1.0 tried to solve too many problemsToo many featuresToo hard to figure outOur data revealed that Twine users mainly just want to search and track interests
    48
    1.11.2008 | Product Positioning
  • 51. Lessons LearnedWe should not have relied on a tuplestore so centrally in the Twine 1.0 architectureWe haven’t yet seen a tuplestore that fully meets the needs of a major consumer online web site like Twine
    49
    1.11.2008 | Product Positioning
  • 52. The Web 2.0 party was over.Where to go from here?… A Radical Rethink
    50
    1.11.2008 | Product Positioning
  • 53. Continuing on the path we were onwould be too expensive to scaleMust change the architecture, or change the product,or both**We chose “or both”
    51
    1.11.2008 | Product Positioning
  • 54. The Future of Search
    Confidential
  • 55. Where Search is Headed
    Semantic
    Personalized
    Real-time
    Social
    Sharing
    Tracking
    KM
    Reasoning
    53
    1.11.2008 | Product Positioning
  • 56. But the Semantic Web Hasn’tHappened Yet…
    At least not for consumers
    And most Web developers…
    54
    1.11.2008 | Product Positioning
  • 57. Why hasn’t the Semantic Web taken off yet?
    55
    1.11.2008 | Product Positioning
  • 58. How can we demonstrate the value of the Semantic Web to consumers?And make it easily useful to Web developers?In the near-term?
    56
    1.11.2008 | Product Positioning
  • 59. Let’s Get Real
    Consumer are not going to add semantic metadata
    Webmasters are too lazy, or don’t know how
    Humans are bad at generating good metadata anyway
    Automated metadata generation is the only practical solution
    57
    1.11.2008 | Product Positioning
  • 60. Index the Semantics of the Web
    Create the Semantic Web
    From the non-Semantic Web
    Automatically
    Provide it back as a Web service
    58
    1.11.2008 | Product Positioning
  • 61. Provide faceted semantic search and navigation against the indexAt Web-scale
    59
    1.11.2008 | Product Positioning
  • 62. Kind of a Semantic Google
    IBM.com
    Web Site
    Joe
    Person
    IBM
    Company
    Palo Alto
    City
    Lives in
    Publisher of
    Fan of
    Lives in
    Subscriber to
    Employee of
    Sue
    Person
    Jane
    Person
    Dave.com
    RSS Feed
    Coldplay
    Band
    Fan of
    Friend of
    Member of
    Design
    Team
    Group
    Depiction of
    Married to
    Source of
    123.JPG
    Photo
    Member
    of
    Bob
    Person
    Dave.com
    Weblog
    Depiction of
    Member of
    Dave
    Person
    Stanford
    Alumnae
    Group
    Member of
    Author of
    Member of
    Person
    Application
  • 63. But not on anything close to Google’s budget
    61
    1.11.2008 | Product Positioning
  • 64. What were we smoking?**We were not Freebasing
    62
    1.11.2008 | Product Positioning
  • 65. Well it turns out, it’s not impossible…
    63
    1.11.2008 | Product Positioning
  • 66. T2
    64
    1.11.2008 | Product Positioning
  • 67. The T2 Approach…
    65
    1.11.2008 | Product Positioning
    Focus on search Keep it simple Broad appeal Scale cheaply
    Connect with ecosystem
  • 68. T2 ArchitectureHigh-performanceOpen-ended scalingMore semantic (RDF, OWL)Not built directly on a triplestoreNot built directly on a databaseHeavy use of SOLR (from Lucene)
    66
    1.11.2008 | Product Positioning
  • 69. T2 Stack
    67
    1.11.2008 | Product Positioning
    Twine Properties
    Partners/3rd Parties
    WebApp Framework
    APIs
    Application Services
    Resource Service
    Web Corpus
    (e.g., BOSS, etc)
    Semantic Metadata Service
    Tuple Service
    SOLR Farm
    RDBMS
  • 70. T1 Approach
    Human
    Tagging
    AI
    Semantic
    Web
    Make the Data Smarter
    Machine
    Tagging
    Linguistics
    Extraction
    Statistics
    Make the software smarter
  • 71. T2 Approach
    Semantic
    Web
    Make the Data Smarter
    Linguistics
    Extraction
    Statistics
    Make the software smarter
  • 72. A Tactical Shift
    Freebase
    DBpedia
    Make the Data Smarter
    Twine
    T1
    EVRI
    Wolfram
    Alpha
    Wikipedia
    Open Calais
    Endeca
    Expert System
    Flickr
    Bing
    Siri
    FAST
    Powerset
    Yahoo
    Delicious
    Autonomy
    Inxight
    Hakia
    Google
    Make the software smarter
  • 73. To a More Mainstream App
    Freebase
    Twine
    T2
    DBpedia
    Make the Data Smarter
    Twine
    T1
    EVRI
    Wolfram
    Alpha
    Wikipedia
    Open Calais
    Endeca
    Expert System
    Flickr
    Bing
    Siri
    FAST
    Powerset
    Yahoo
    Delicious
    Autonomy
    Inxight
    Hakia
    Google
    Make the software smarter
  • 74. Web-Scale Semantic ExtractionAnd search
    72
    1.11.2008 | Product Positioning
  • 75. T2 Semantic Index
    Videos
    Music
    People
    Reviews
    Games
    News
    Products
    Recipes
    Services
    How-To’s
    Classifieds
    Events
    Hotels
    Bars
    Resumes
    Coupons
    Reports
    Photos
    Help
    73
  • 76. Many Uses for this Index
    Search
    Interest tracking
    Recommendations
    Web application development
    74
    1.11.2008 | Product Positioning
  • 77. 1. Do a Keyword Search
    2. Filter by Type
    3. Filter by
    Attribute
    Recipe
    Cuisine
    Web
    Page
    Classified Ad
    Difficulty
    Web
    Page
    Product
    Ingredient
    Prep Time
    News
    Web Page
    Author
    Web
    Page
    Hotel
    Type of Dish
    Web
    Page
    Song
    Dietary Option
    Website
    Game
    Web Page
    Prep Time
    Help Article
    Web Page
    Calories
    Web Page
    Etc…
    Person
    Etc…
    75
  • 78. Use Case: Cooking
    76
  • 79. Scenario: Family Dinner
    Deciding what to cook…
  • 80. Scenario: Family Dinner
  • 81. Scenario: Family Dinner
  • 82. Current Scenario: Family Dinner
    [INSERT BING RESULTS + EXTERNAL LOGOS]
  • 83. Current Scenario: Family Dinner
    [INSERT BING RESULTS + EXTERNAL LOGOS]
  • 84. Current Scenario: Family Dinner
  • 85. Current Scenario: Family Dinner
  • 86.
  • 87. T2 – Tools for Developers
    API to T2 Index and services
    Widgets and components for 3rd party sites
    Web-based ontology community – like Sourceforge
    Browser plugin for easy site mapping to ontologies
    85
    1.11.2008 | Product Positioning
  • 88. T2 Ontology Site
    Demo
    86
  • 89.
  • 90.
  • 91.
  • 92.
  • 93.
  • 94.
  • 95. T2 Site Mapping Tool
    Confidential
  • 96. T2 Site Mapping Tool
    Confidential
  • 97. T2 Site Mapping Tool
    Confidential
  • 98. T2 Business Model
    T2 destination
    Advertising on the site
    T2 site search & ad network
    Provide sites with better site search
    Targeted ads
    Revshare
    T2 API
    Partners pay us, or revshare with us
    Monetize however they want to
    96
    1.11.2008 | Product Positioning
  • 99. T2 Near-Term Vertical Focus Areas
    Lifestyle
    Entertainment
    Shopping
    Food (Complete)
    Health
    Travel
    People
    Gaming (In Dev)
    Sports
    Music
    TV
    Film
    Consumer Electronics
    Health & Beauty
    Classified Ads
    Media Products
    Help & Support
  • 100. T2 Long-Term Goals
    Index the structured data on the Web
    Across all major vertical domains
    With help from an army of outside developers and partners
    Grow an ecosystem around the index
    98
    1.11.2008 | Product Positioning
  • 101. Timeline T1 is in Maintenance Mode T2 is in Alpha T2 Goes Beta by Q1
    99
    1.11.2008 | Product Positioning
  • 102. What Next?Find Follow Share
    100
    1.11.2008 | Product Positioning
  • 103. For all types of content Web pages Data records Real-time content
    101
    1.11.2008 | Product Positioning
  • 104. Find stuff (semantic search)Then follow it (interest tracking) Then share it (socialization)
    102
    1.11.2008 | Product Positioning
  • 105. But this time…
    103
    1.11.2008 | Product Positioning
  • 106. We’re starting with just “Find”
    104
    1.11.2008 | Product Positioning
  • 107. Conclusions Web-Scale Semantic Search is the Next-Step for Search and for Twine But after 9 years of this, it’s clear It won’t happen overnight
    105
    1.11.2008 | Product Positioning
  • 108. Stay Tuned…
    106
    1.11.2008 | Product Positioning
  • 109. Contact Information
    Nova Spivack
    Twitter: @novaspivack
    nova@twine.com

×