Nova Spivack CEO & Founder Radar Networks Making Sense of the Semantic Web

About This Talk Making sense of the semantic sector Making the Semantic Web more useable Future outlook Twine.com Q & A

Making sense of the semantic sector

Making the Semantic Web more useable

Future outlook

Twine.com

Q & A

The Big Opportunity… And it uses richer semantics to enable: Better search More targeted ads Smarter collaboration Deeper integration Richer content Better personalization The social graph just connects people People Groups The semantic graph connects everything Emails Companies Products Services Web Pages Multimedia Documents Events Projects Activities Interests Places

The third decade of the Web A period in time, not a technology… Enrich the structure of the Web Improve the quality of search, collaboration, publishing, advertising Enables applications to become more integrated and intelligent Transform Web from fileserver to database Semantic technologies will play a key role

A period in time, not a technology…

Enrich the structure of the Web

Improve the quality of search, collaboration, publishing, advertising

Enables applications to become more integrated and intelligent

Transform Web from fileserver to database

Semantic technologies will play a key role

A Higher Resolution Web Coldplay Band Palo Alto City Jane Person IBM Company Dave Person Bob Person Design Team Group Stanford Alumnae Group IBM.com Web Site 123.JPG Photo Dave.com Weblog Sue Person Joe Person Dave.com RSS Feed Lives in Publisher of Friend of Depiction of Depiction of Member of Married to Member of Member of Member of Fan of Lives in Subscriber to Source of Author of Member of Employee of Fan of

The Web IS the Database! Application A Application B Coldplay Band Palo Alto City Jane Person IBM Company Dave Person Bob Person Design Team Group Stanford Alumnae Group IBM.com Web Site 123.JPG Photo Dave.com Weblog Sue Person Joe Person Dave.com RSS Feed Lives in Publisher of Friend of Depiction of Depiction of Member of Married to Member of Member of Member of Fan of Lives in Subscriber to Source of Author of Member of Employee of Fan of

The Intelligence is in the Connections Connections between people Connections between Information Email Social Networking Groupware Javascript Weblogs Databases File Systems HTTP Keyword Search USENET Wikis Websites Directory Portals 2010 - 2020 Web 1.0 2000 - 2010 1990 - 2000 PC Era 1980 - 1990 RSS Widgets PC’s 2020 - 2030 Office 2.0 XML RDF SPARQL AJAX FTP IRC SOAP Mashups File Servers Social Media Sharing Lightweight Collaboration ATOM Web 3.0 Web 4.0 Semantic Search Semantic Databases Distributed Search Intelligent personal agents Java SaaS Web 2.0 Flash OWL HTML SGML SQL Gopher P2P The Web The PC Windows MacOS SWRL OpenID BBS MMO’s VR Semantic Web Intelligent Web The Internet Social Web Web OS

Beyond the Limits of Keyword Search Amount of data Productivity of Search Databases 2010 - 2020 Web 1.0 2000 - 2010 1990 - 2000 PC Era 1980 - 1990 2020 - 2030 Web 3.0 Web 4.0 Web 2.0 The World Wide Web The Desktop Keyword search Natural language search Reasoning Tagging Semantic Search The Semantic Web The Intelligent Web Directories The Social Web Files & Folders

The Future of the Platform… 1980’s -- The Desktop is the platform 1990’s -- The Browser / Server is the platform 2000’s -- Web Services are the platform 2010’s -- The Semantic Web is the platform 2020’s -- The WebOS is the platform 2030’s -- The Human Body is the platform…?

1980’s -- The Desktop is the platform

1990’s -- The Browser / Server is the platform

2000’s -- Web Services are the platform

2010’s -- The Semantic Web is the platform

2020’s -- The WebOS is the platform

2030’s -- The Human Body is the platform…?

Five Approaches to Semantics Tagging Statistics Linguistics Semantic Web Artificial Intelligence

Tagging

Statistics

Linguistics

Semantic Web

Artificial Intelligence

The Tagging Approach Pros Easy for users to add and read tags Tags are just strings No algorithms or ontologies to deal with No technology to learn Cons Easy for users to add and read tags Tags are just strings No algorithms or ontologies to deal with No technology to learn Technorati Del.icio.us Flickr Wikipedia

Pros

Easy for users to add and read tags

Tags are just strings

No algorithms or ontologies to deal with

No technology to learn

Cons

Easy for users to add and read tags

Tags are just strings

No algorithms or ontologies to deal with

No technology to learn

Technorati

Del.icio.us

Flickr

Wikipedia

The Statistical Approach Pros: Pure mathematical algorithms Massively scaleable Language independent Cons: No understanding of the content Hard to craft good queries Best for finding really popular things – not good at finding needles in haystacks Not good for structured data Google Lucene Autonomy

Pros:

Pure mathematical algorithms

Massively scaleable

Language independent

Cons:

No understanding of the content

Hard to craft good queries

Best for finding really popular things – not good at finding needles in haystacks

Not good for structured data

Google

Lucene

Autonomy

The Linguistic Approach Pros: True language understanding Extract knowledge from text Best for search for particular facts or relationships More precise queries Cons: Computationally intensive Difficult to scale Lots of errors Language-dependent Powerset Hakia Inxight, Attensity, and others…

Pros:

True language understanding

Extract knowledge from text

Best for search for particular facts or relationships

More precise queries

Cons:

Computationally intensive

Difficult to scale

Lots of errors

Language-dependent

Powerset

Hakia

Inxight, Attensity, and others…

The Semantic Web Approach Pros: More precise queries Smarter apps with less work Not as computationally intensive Share & link data between apps Works for both unstructured and structured data Cons: Lack of tools Difficult to scale Who makes all the metadata? Radar Networks DBpedia Project Metaweb

Pros:

More precise queries

Smarter apps with less work

Not as computationally intensive

Share & link data between apps

Works for both unstructured and structured data

Cons:

Lack of tools

Difficult to scale

Who makes all the metadata?

Radar Networks

DBpedia Project

Metaweb

The Artificial Intelligence Approach Pros: This is the holy grail!!!! Approximates the expertise and common sense reasoning ability of a human domain expert Reasoning / inferencing, discovery, automated assistance, learning and self-modification, question answering, etc. Cons: This is the holy grail!!!! Computationally intensive Hard to program and design Takes a long time and a lot of work to reach critical mass of knowledge Cycorp

Pros:

This is the holy grail!!!!

Approximates the expertise and common sense reasoning ability of a human domain expert

Reasoning / inferencing, discovery, automated assistance, learning and self-modification, question answering, etc.

Cons:

This is the holy grail!!!!

Computationally intensive

Hard to program and design

Takes a long time and a lot of work to reach critical mass of knowledge

Cycorp

The Approaches Compared Make the software smarter Make the Data Smarter Statistics Linguistics Semantic Web A.I. Tagging

Two Paths to Adding Semantics “ Bottom-Up” (Classic) Add semantic metadata to pages and databases all over the Web Every Website becomes semantic Everyone has to learn RDF/OWL “ Top-Down” (Contemporary) Automatically generate semantic metadata for vertical domains Create services that provide this as an overlay to non-semantic Web Nobody has to learn RDF/OWL -- Alex Iskold

“ Bottom-Up” (Classic)

Add semantic metadata to pages and databases all over the Web

Every Website becomes semantic

Everyone has to learn RDF/OWL

“ Top-Down” (Contemporary)

Automatically generate semantic metadata for vertical domains

Create services that provide this as an overlay to non-semantic Web

Nobody has to learn RDF/OWL

-- Alex Iskold

In Practice: Hybrid Approach Works Best Tagging Semantic Web Top-down Statistics Linguistics Bottom-up Artificial intelligence

Tagging

Semantic Web

Top-down

Statistics

Linguistics

Bottom-up

Artificial intelligence

Smart Data Smart Data is data that carries whatever is needed to make use of it: Definition of intended meaning and schema Policies and permissions Context (links, etc.) History and schedule Authenticity Sentiment Behavior (each piece of data may someday have its own rules and/or agent(s) that seek to move the data to where it is needed, connect it, maintain it, protect it, improve it, etc.) Software can become dumber and more generice, yet ultimately be smarter – the smarts moves into the data itself rather than being hard-coded into the software

Smart Data is data that carries whatever is needed to make use of it:

Definition of intended meaning and schema

Policies and permissions

Context (links, etc.)

History and schedule

Authenticity

Sentiment

Behavior (each piece of data may someday have its own rules and/or agent(s) that seek to move the data to where it is needed, connect it, maintain it, protect it, improve it, etc.)

Software can become dumber and more generice, yet ultimately be smarter – the smarts moves into the data itself rather than being hard-coded into the software

The Semantic Web is a Key Enabler Moves the “intelligence” out of applications, into the data Data becomes self-describing; Meaning of data becomes part of the data: Data = Metadata. Data can be shared and linked more easily Just-in-time schemas – applications can pull the schema for data only when the data is actually needed, rather than having to anticipate it

Moves the “intelligence” out of applications, into the data

Data becomes self-describing; Meaning of data becomes part of the data: Data = Metadata.

Data can be shared and linked more easily

Just-in-time schemas – applications can pull the schema for data only when the data is actually needed, rather than having to anticipate it

The Semantic Web = Open database layer for the Web User Profiles Web Content Data Records Apps & Services Ads & Listings Open Data Mappings Open Data Records Open Rules Open Ontologies Open Query Interfaces

Semantic Web Open Standards RDF – Store data as “triples” OWL – Define systems of concepts called “ontologies” Sparql – Query data in RDF SWRL – Define rules GRDDL – Transform data to RDF

RDF – Store data as “triples”

OWL – Define systems of concepts called “ontologies”

Sparql – Query data in RDF

SWRL – Define rules

GRDDL – Transform data to RDF

RDF “Triples” the subject, which is an RDF URI reference or a blank node the predicate, which is an RDF URI reference the object, which is an RDF URI reference , a literal or a blank node Source: http://www.w3.org/TR/rdf-concepts/#section-triples Subject Object Predicate

the subject, which is an RDF URI reference or a blank node

the predicate, which is an RDF URI reference

the object, which is an RDF URI reference , a literal or a blank node

Semantic Web Data is Self-Describing Linked Data Data Record ID Field 1 Value Field 2 Value Field 3 Value Field 4 Value Definition Definition Definition Definition Definition Definition Definition Ontologies

RDBMS vs Triplestore S P O Person Table f_name jim nova chris lew ID 001 002 003 004 l_name wissner spivack jones tucker Colleagues Table SRC-ID 001 001 001 001 002 002 002 002 003 003 003 003 004 004 004 004 TGT-ID 001 002 003 004 001 002 003 004 001 002 003 004 001 002 003 004 Subject Predicate Object 001 isA Person 001 firstName Jim 001 lastName Wissner 001 hasColleague 002 002 isA Person 002 firstName Nova 002 lastName Spivack 002 hasColleague 003 003 isA Person 003 firstName Chris 003 lastName Jones 003 hasColleague 004 004 isA Person 004 firstName Lew 004 lastName Tucker

Merging Databases in RDF is Easy S P O S P O S P O

Are RDF/OWL the Only Way to Express Semantics? Other contenders: String tags Taxonomies and controlled vocabularies Microformats Ad hoc [name, value] pairs Alternative semantic metadata notations

Other contenders:

String tags

Taxonomies and controlled vocabularies

Microformats

Ad hoc [name, value] pairs

Alternative semantic metadata notations

One Semantic Web or Many? The answer is….Both The Semantic Web is a web of semantic webs Each of us may have our own semantic web…

The answer is….Both

The Semantic Web is a web of semantic webs

Each of us may have our own semantic web…

Why has it Taken So Long? The Dream of the Semantic Web has been slow to arrive The original vision was too focused on A.I. Technologies and tools were insufficient Needs for open data on the Web were not strong enough Keyword search and tagging were good enough…for a while Lack of end-user facing killer apps Lots of misunderstanding to clear up

The Dream of the Semantic Web has been slow to arrive

The original vision was too focused on A.I.

Technologies and tools were insufficient

Needs for open data on the Web were not strong enough

Keyword search and tagging were good enough…for a while

Lack of end-user facing killer apps

Lots of misunderstanding to clear up

Crossing the Chasm… Communicating the vision Focus on open data, not A.I. Technology progress Standards & tools finally maturing Needs were not strong enough Keyword search and tagging not as productive anymore Apps need better way to share data Killer apps and content Several companies are starting to expose data to the Semantic Web. Soon there will be a lot of data. Market Education Show the market what the benefits are

Communicating the vision

Focus on open data, not A.I.

Technology progress

Standards & tools finally maturing

Needs were not strong enough

Keyword search and tagging not as productive anymore

Apps need better way to share data

Killer apps and content

Several companies are starting to expose data to the Semantic Web. Soon there will be a lot of data.

Market Education

Show the market what the benefits are

Future Outlook 2007 – 2009 Early-Adoption A few killer apps emerge Other apps start to integrate 2010 – 2020 Mainstream Adoption Semantics widely used in Web content and apps 2020 + Next big cycle: Reasoning and A.I. The Intelligent Web The Web learns and thinks collectively

2007 – 2009

Early-Adoption

A few killer apps emerge

Other apps start to integrate

2010 – 2020

Mainstream Adoption

Semantics widely used in Web content and apps

2020 +

Next big cycle: Reasoning and A.I.

The Intelligent Web

The Web learns and thinks collectively

A Mainstream Application of the Semantic Web…

Twine.com Overview Helps individuals and groups organize, share and discover information around their interests . Instead of social networking, it’s interest networking Twine is the best service for keeping up with your interests and sharing what interests you with other like-minded people. Twine will generate a large number of vertical interest portals and attract traffic from search engines and partners Twine will monetize through advertising, affiliate commerce, sponsorships and subscriptions

Helps individuals and groups

organize, share and discover

information around their interests .

Instead of social networking, it’s interest networking

Twine is the best service for keeping up with your interests and sharing what interests you with other like-minded people.

Twine will generate a large number of vertical interest portals and attract traffic from search engines and partners

Twine will monetize through advertising, affiliate commerce, sponsorships and subscriptions

Positioning Facebook - For your relationships LinkedIn - For your career Twine - For your interests

Facebook - For your relationships

LinkedIn - For your career

Twine - For your interests

Twine Compared to Other Tools Wikis Document centric, unstructured data Everyone sees the same view of the world Geared towards space (a directory of articles) rather than geared towards time (a sequence of articles) Users have to do all the work of linking and tagging No support for social networking or sharing Only for groups Blogs Document centric, unstructured data Document centric, unstructured data Everyone sees the same view of the world Geared towards time (a sequence of articles) rather than geared towards space (a directory of articles) Users have to do all the work of linking and tagging No support for social networking or sharing Only for individuals Twine Data centric – all data (even unstructured data) becomes structured data Different views for people with different permissions Geared towards spacetime – view documents by directory or sequence Automates tagging, linking and discovery Built in social networking and sharing For individuals and groups – a unified framework

Wikis

Document centric, unstructured data

Everyone sees the same view of the world

Geared towards space (a directory of articles) rather than geared towards time (a sequence of articles)

Users have to do all the work of linking and tagging

No support for social networking or sharing

Only for groups

Blogs

Document centric, unstructured data

Document centric, unstructured data

Everyone sees the same view of the world

Geared towards time (a sequence of articles) rather than geared towards space (a directory of articles)

Users have to do all the work of linking and tagging

No support for social networking or sharing

Only for individuals

Twine

Data centric – all data (even unstructured data) becomes structured data

Different views for people with different permissions

Geared towards spacetime – view documents by directory or sequence

Automates tagging, linking and discovery

Built in social networking and sharing

For individuals and groups – a unified framework

Twine is Smart Emails Bookmarks Documents Products Collaborate & Discuss Search & Discover Track Interests Automatically captures & organizes info Recommends relevant things Helps you navigate and search Tags, crawls & links related information People Photos Videos Places Classifieds Notes Events

Let’s take a look at Twine… (demo of Twine site…)

Twine is Powered by The Semantic Web Twine is built on a new Semantic Web platform 15 patents pending and in process Easily create new Semantic Web apps Written in pure Java Anyone can add & edit ontologies Scale to manage 10’s of billions of RDF triples Developer tools and API’s Open up our platform API’s and open-source in the future Be the center of the Semantic Web ecosystem

Twine is built on a new Semantic Web platform

15 patents pending and in process

Easily create new Semantic Web apps

Written in pure Java

Anyone can add & edit ontologies

Scale to manage 10’s of billions of RDF triples

Developer tools and API’s

Open up our platform API’s and open-source in the future

Be the center of the Semantic Web ecosystem

Radar Networks’ Semantic Web Platform SQL Database Web App KnowledgeBase Bookmarklet & Email User Portal REST API SPARQL Relational database RSS Feeds Object Query & Cache Class inferencing Semantic Object TupleStore service SQL Query Generator Predicate Inferencing Tuple Query Access Control WebDAV File Store Flat File Store AJAX, Jetty, PicoContainer, Java, XML, SPARQL Jena, ATOM RDF, OWL RDF, OWL, SQL Mina Postgres, Solaris webDAV, Isilon cluster Cache Remote Access Cache Cache Twine.com Platform Storage Ontology

Differentiation Focused on sharing knowledge around interests, not just socializing Smarter than other sites – Twine learns, organizes and recommends automatically Powered by the Semantic Web – New capabilities are possible Unified place for all types of information

Focused on sharing knowledge around interests, not just socializing

Smarter than other sites – Twine learns, organizes and recommends automatically

Powered by the Semantic Web – New capabilities are possible

Unified place for all types of information

Target Customer Twine is for active users of the Web, including consumers and professionals, who create, find and share information about their interests Interests : Professional associations Alumni groups Social networks (Facebook, Plaxo, LinkedIn) Volunteer organizations Groups based on interests (hobbies, health, sports, entertainment, culture, family, technology, user groups, etc.) Participating/working in teams at organizations of all sizes Demographics: 18 – 45 years old Have many personal interests and hobbies Social connections are important – family, friends, colleagues Americans with a household income of $100,000 or more Nearly 26 million such consumers used the Internet in August 2003, spending an average of 27.6 hours online -- more than any other income segment. Consume an average of nearly 3,000 pages a month, almost 300 pages more than the average Internet user

Twine is for active users of the Web, including consumers and professionals, who create, find and share information about their interests

Interests :

Professional associations

Alumni groups

Social networks (Facebook, Plaxo, LinkedIn)

Volunteer organizations

Groups based on interests (hobbies, health, sports, entertainment, culture, family, technology, user groups, etc.)

Participating/working in teams at organizations of all sizes

Demographics:

18 – 45 years old

Have many personal interests and hobbies

Social connections are important – family, friends, colleagues

Americans with a household income of $100,000 or more

Nearly 26 million such consumers used the Internet in August 2003, spending an average of 27.6 hours online -- more than any other income segment.

Consume an average of nearly 3,000 pages a month, almost 300 pages more than the average Internet user

Market Opportunities for Twine Individuals Individual consumers Individual professionals Groups, Teams and Communities Interest communities Support groups Content publishers Users groups Hobbyists Social groups Product communities Event communities Communities of practice Customer support Collaborative teams

Individuals

Individual consumers

Individual professionals

Groups, Teams and Communities

Interest communities

Support groups

Content publishers

Users groups

Hobbyists

Social groups

Product communities

Event communities

Communities of practice

Customer support

Collaborative teams

Contact Info Visit www.twine.com to sign up for the invite beta wait-list You can email me at [email_address] My blog is at http://www.mindingtheplanet.net Thanks!

Visit www.twine.com to sign up for the invite beta wait-list

You can email me at [email_address]

My blog is at http://www.mindingtheplanet.net

Thanks!

Rights This presentation is licensed under the Creative Commons Attribution License. Details: This work is licensed under the Creative Commons Attribution 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/3.0/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA. If you reproduce or redistribute in whole or in part, please give attribution to Nova Spivack, with a link to http://www.mindingtheplanet.net

This presentation is licensed under the Creative Commons Attribution License.

Details: This work is licensed under the Creative Commons Attribution 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/3.0/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA.

If you reproduce or redistribute in whole or in part, please give attribution to Nova Spivack, with a link to http://www.mindingtheplanet.net