November 2009 Product Stack

Enterprise Approach

Enterprise Approach Semantic Enterprise based on semantic Web , linked data Leverage existing assets Data, records and instances Taxonomies, structure and schema Layer semantics on to existing systems Develop incrementally Add sophistication, scope over time Keep risks low Integrate with public and Web data (“open world”)

Semantic Enterprise based on semantic Web , linked data

Leverage existing assets

Data, records and instances

Taxonomies, structure and schema

Layer semantics on to existing systems

Develop incrementally

Add sophistication, scope over time

Keep risks low

Integrate with public and Web data (“open world”)

Linked Data “ Linked Data is a set of best practices for publishing and deploying instance and class data using the RDF data model, naming the data objects using uniform resource identifiers (URIs), thereby exposing the data for access via the HTTP protocol, while emphasizing data interconnections, interrelationships and context useful to both humans and machine agents.”

“ Linked Data is a set of best practices for publishing and deploying instance and class data using the RDF data model, naming the data objects using uniform resource identifiers (URIs), thereby exposing the data for access via the HTTP protocol, while emphasizing data interconnections, interrelationships and context useful to both humans and machine agents.”

Layers and Current Products

Current Products the pivotal product; Web services middleware that provides distributed data access and federation Drupal-based structured data linkage to structWSF spreadsheet, JSON and XML authoring and conversion framework reference set of linking subjects and basis for domain vocabularies an ontology- and entity-driven information extraction and tagging system

the pivotal product; Web services middleware that provides distributed data access and federation

Drupal-based structured data linkage to structWSF

spreadsheet, JSON and XML authoring and conversion framework

reference set of linking subjects and basis for domain vocabularies

an ontology- and entity-driven information extraction and tagging system

Fit of Current Products within Layers

Existing Assets Layer

Existing Assets These are the materials that need to be federated, made interoperable, and given a common semantics structured data / databases semi-structure data (XML, Web pages) unstructured data (text)

These are the materials that need to be federated, made interoperable, and given a common semantics

structured data / databases

semi-structure data (XML, Web pages)

unstructured data (text)

Preserving Existing Assets Relational databases (RDBMs) Distributed structured assets spreadsheets lightweight datastores Web pages and Web sites Existing documents and text Web databases and APIs Other databases (RDF, OO, etc.)

Relational databases (RDBMs)

Distributed structured assets

spreadsheets

lightweight datastores

Web pages and Web sites

Existing documents and text

Web databases and APIs

Other databases (RDF, OO, etc.)

Access/Conversion Layer

Conversion Provides in-place access to existing information Translates existing formats and structures to RDF Extracts structured information from unstructured text Aids creation of interoperable datasets Geared almost entirely to records , instances or entities (that is, basic data)

Provides in-place access to existing information

Translates existing formats and structures to RDF

Extracts structured information from unstructured text

Aids creation of interoperable datasets

Geared almost entirely to records , instances or entities (that is, basic data)

Conversion Methods Relational DBs: RDB2RDF RDFizers Information Extraction New Dataset Authoring Direct Use (already in RDF)

Relational DBs: RDB2RDF

RDFizers

Information Extraction

New Dataset Authoring

Direct Use (already in RDF)

Relational DB Conversion Simple mappings of instance records to RDF Methodologies well proven if kept to the instance level RDB schema inform the interoperable layer (“ontologies”) Relational datastores left in place Record data obtained via access layer (structWSF)

Simple mappings of instance records to RDF

Methodologies well proven if kept to the instance level

RDB schema inform the interoperable layer (“ontologies”)

Relational datastores left in place

Record data obtained via access layer (structWSF)

RDFizers General serialization or data format conversions to RDF Mostly applied to: Standard data formats and data structs Web content APIs Some legacy content Sometimes some minor ontology or schema mapping Embodies all conversion steps to linked data We have access to more than 100+ existing formats

General serialization or data format conversions to RDF

Mostly applied to:

Standard data formats and data structs

Web content

APIs

Some legacy content

Sometimes some minor ontology or schema mapping

Embodies all conversion steps to linked data

We have access to more than 100+ existing formats

RDFizers – Listing 1 URN handlers (in addition to IRI and URI): DOI LSID OAI RDF Serialization formats: irON N3 RDF/XML Turtle Languages and ontologies: AB Meta Annotea APML AtomOWL Bibliographic Ontology Creative Commons EXIF FOAF GeoNames GoodRelations Java Javadoc MARC/MODS Meta Standards Music Ontology Natural Language Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) Open Geospatial OWL SIOC SIOCT SKOS UMBEL vCard XML Others (X)HTML pages Embedded Microformats and GRDDL * (see note below): DC eRDF geoURL Google Base hAudio hCalendar hCard hListing hResume hReview HR-XML Ning RDFa relLicense SVG XBRL XFN xFolk XR-XML XSLT Syndication Formats: Atom OPML OCS RSS 1.1 RSS 2.0 XBEL (for bookmarks) REST-style Web service APIs: Alchemy Amazon Apple Best Buy Calais CNet CrunchBase Del.icio.us Digg Discogs Disqus eBay Facebook Flickr Freebase (MQL) FriendFeed Garmin Get Satisfaction Google Google Apps Hoover's HTTP (raw) ISBN DB Last.fm Library Thing Magnolia Meetup MusicBrainz New York Times New York Times Campaign Finance (NYTCF) New York Times tags

URN handlers (in addition to IRI and URI):

DOI

LSID

OAI

RDF

Serialization formats:

irON

N3

RDF/XML

Turtle

Languages and ontologies:

AB Meta

Annotea

APML

AtomOWL

Bibliographic Ontology

Creative Commons

EXIF

FOAF

GeoNames

GoodRelations

Java

Javadoc

MARC/MODS

Meta Standards

Music Ontology

Natural Language

Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH)

Open Geospatial

OWL

SIOC

SIOCT

SKOS

UMBEL

vCard

XML

Others

(X)HTML pages

Embedded Microformats and GRDDL * (see note below):

DC

eRDF

geoURL

Google Base

hAudio

hCalendar

hCard

hListing

hResume

hReview

HR-XML

Ning

RDFa

relLicense

SVG

XBRL

XFN

xFolk

XR-XML

XSLT

Syndication Formats:

Atom

OPML

OCS

RSS 1.1

RSS 2.0

XBEL (for bookmarks)

REST-style Web service APIs:

Alchemy

Amazon

Apple

Best Buy

Calais

CNet

CrunchBase

Del.icio.us

Digg

Discogs

Disqus

eBay

Facebook

Flickr

Freebase (MQL)

FriendFeed

Garmin

Get Satisfaction

Google

Google Apps

Hoover's

HTTP (raw)

ISBN DB

Last.fm

Library Thing

Magnolia

Meetup

MusicBrainz

New York Times

New York Times Campaign Finance (NYTCF)

New York Times tags

RDFizers – Listing 2 Open Library Open Social Open Street OpenLink (facets) O'Reilly Picasa Radio Pop (BBC) Rhapsody Salesforce Slideshare Slidy Technorati Tesco They Work For You Twine Twitter Weather Wikipedia World Bank Yahoo! BOSS Yahoo! Finance Yahoo! Maps Yahoo! Weather Yelp YouTube Zemanta Zillow Files (multitude of file formats and MIME types, including): audio (general) BibJSON BibTEX and others BitTorrent commON CSV Fink Flat files irJSON irXML JPEG JSON images MS Office OpenOffice Open Document Format Palm RDF123 video XLS etc. Metadata extractors: CRW DEB EXIF OCW RPM XMP Email formats: EMail Outlook RFC822 Version control and related systems: Bugzilla Jira POM Subversion Other Web service frameworks: BPEL WSDL XBRL XBEL Data exchange formats: iCalendar LDIF vCalendar vCard Relational databases and related: D2RQ D2RMAP RDF Views Virtuoso VADs OpenLink license files Third party metadata extraction frameworks: Aperture Spotlight Miscellaneous and other related converters: MPEG-7/CS -> OWL Random XSD -> OWL * GRDDL (Gleaning Resource Descriptions from Dialects of Languages) accommodates a wide variety of dialects (see one listing ) and can be combined with arbitrary transformation mechanisms (though currently mostly based on XSLTs).

Open Library

Open Social

Open Street

OpenLink (facets)

O'Reilly

Picasa

Radio Pop (BBC)

Rhapsody

Salesforce

Slideshare

Slidy

Technorati

Tesco

They Work For You

Twine

Twitter

Weather

Wikipedia

World Bank

Yahoo! BOSS

Yahoo! Finance

Yahoo! Maps

Yahoo! Weather

Yelp

YouTube

Zemanta

Zillow

Files (multitude of file formats and MIME types, including):

audio (general)

BibJSON

BibTEX and others

BitTorrent

commON

CSV

Fink

Flat files

irJSON

irXML

JPEG

JSON

images

MS Office

OpenOffice

Open Document Format

Palm

RDF123

video

XLS

etc.

Metadata extractors:

CRW

DEB

EXIF

OCW

RPM

XMP

Email formats:

EMail

Outlook

RFC822

Version control and related systems:

Bugzilla

Jira

POM

Subversion

Other Web service frameworks:

BPEL

WSDL

XBRL

XBEL

Data exchange formats:

iCalendar

LDIF

vCalendar

vCard

Relational databases and related:

D2RQ

D2RMAP

RDF Views

Virtuoso VADs

OpenLink license files

Third party metadata extraction frameworks:

Aperture

Spotlight

Miscellaneous and other related converters:

MPEG-7/CS -> OWL

Random

XSD -> OWL

* GRDDL (Gleaning Resource Descriptions from Dialects of Languages) accommodates a wide variety of dialects (see one listing ) and can be combined with arbitrary transformation mechanisms (though currently mostly based on XSLTs).

scones

Information Extraction scones ( S ubject C oncept O r N amed E ntitie S ) is our IE tagger Information extraction is applied to input Web pages and unstructured text May be applied after structure extraction: (often, at minimum, defluffing ) Settable “window” for snippet (from # of bracketing terms to full document) Extraction is performed for both: Entities (per Wikipedia and enterprise dictionaries) Subject concepts (per UMBEL and domain ontologies) Presently in prototype

scones ( S ubject C oncept O r N amed E ntitie S ) is our IE tagger

Information extraction is applied to input Web pages and unstructured text

May be applied after structure extraction:

(often, at minimum, defluffing )

Settable “window” for snippet (from # of bracketing terms to full document)

Extraction is performed for both:

Entities (per Wikipedia and enterprise dictionaries)

Subject concepts (per UMBEL and domain ontologies)

Presently in prototype

(Named) Entities The places , events , people , objects , and specific things of the real world Literally millions of notable instances Each belongs to one or more subject concept (s) Currently, the predominate basis for linked data Public sources include Wikipedia and Freebase, others Can be readily mixed-and-matched with private entities

The places , events , people , objects , and specific things of the real world

Literally millions of notable instances

Each belongs to one or more subject concept (s)

Currently, the predominate basis for linked data

Public sources include Wikipedia and Freebase, others

Can be readily mixed-and-matched with private entities

Creating New Entity Dictionaries

Triangulating Information Extraction

irON – i nstance r ecord and O bject N otation

irON Dataset Authoring Framework Simple authoring and dataset creation irON includes an abstract notation and vocabulary for instance records Serializations available for: XML (irXML) JSON (irJSON) CSV/spreadsheets (commON) Notations for: Instance records Schema Datasets and metadata Linkages to other schema

Simple authoring and dataset creation

irON includes an abstract notation and vocabulary for instance records

Serializations available for:

XML (irXML)

JSON (irJSON)

CSV/spreadsheets (commON)

Notations for:

Instance records

Schema

Datasets and metadata

Linkages to other schema

Three irON Serializations irXML irJSON commON

irXML

More-or-less Interchangeable Formats

structWSF

structWSF Generally RESTful Web services middleware Uniform, distributed access point Provides the interoperability architecture Based on canonical RDF data model Dataset access orientation Standard tools and services: User permissions and access CRUD (create, read, update, delete) Browse Full-text, faceted search Import / export Many others

Generally RESTful Web services middleware

Uniform, distributed access point

Provides the interoperability architecture

Based on canonical RDF data model

Dataset access orientation

Standard tools and services:

User permissions and access

CRUD (create, read, update, delete)

Browse

Full-text, faceted search

Import / export

Many others

RDF and Data Federation Model

Advantages of a Canonical Model All tools can be driven from a single data format basis Single converters can link in other hubs of data forms ‘ Round-tripping’ thru the canonical form can bring consistency and cleanliness to inputted data RDF is well-suited as the canonical form: Structured data Semi-structured data Unstructured data (after IE) Simple-to-complex data structures Logic and inferencing Suitable to all input data formats Many serializations possible

All tools can be driven from a single data format basis

Single converters can link in other hubs of data forms

‘ Round-tripping’ thru the canonical form can bring consistency and cleanliness to inputted data

RDF is well-suited as the canonical form:

Structured data

Semi-structured data

Unstructured data (after IE)

Simple-to-complex data structures

Logic and inferencing

Suitable to all input data formats

Many serializations possible

A Collaborative, Distributed Network

Flexible User Access Permissions

Access, APIs and Endpoints The resulting linked data may be exposed as: APIs Web services SPARQL endpoints

The resulting linked data may be exposed as:

APIs

Web services

SPARQL endpoints

Ontologies Layer

Ontologies Ontologies provide the basis for: Interoperating Reconciling semantics Multiples may be used at any time Both enterprise (internal) and external ontologies Best built incrementally, with participation Easily modified: OK to test and experiment

Ontologies provide the basis for:

Interoperating

Reconciling semantics

Multiples may be used at any time

Both enterprise (internal) and external ontologies

Best built incrementally, with participation

Easily modified: OK to test and experiment

Ontologies The structural relationships of concepts within a domain Generally class- (or set-) oriented Analogous to relational database schema, only with controlled vocabularies and exact semantics Sets the structure of how to organize the actual data (“instances”) in the domain Semantics and mapping techniques allow disparate ontologies to be inter-related Can inference or reason over the structure

The structural relationships of concepts within a domain

Generally class- (or set-) oriented

Analogous to relational database schema, only with controlled vocabularies and exact semantics

Sets the structure of how to organize the actual data (“instances”) in the domain

Semantics and mapping techniques allow disparate ontologies to be inter-related

Can inference or reason over the structure

Migrating Structure to the Ontology Layer

Ontologies Layer

irON

irON Record Vocabulary irON also provides the standard instance record vocabulary for all federated records Each record source has its own attributes But, irON provides common descriptors: Useful for interoperating Unique, Web-accessible identifiers Standard descriptions and labels Conventions for “driving” user interfaces and tools

irON also provides the standard instance record vocabulary for all federated records

Each record source has its own attributes

But, irON provides common descriptors:

Useful for interoperating

Unique, Web-accessible identifiers

Standard descriptions and labels

Conventions for “driving” user interfaces and tools

UMBEL UMBEL ( U pper M apping and B inding E xchange L ayer) 20,000 defined reference points in information space Means to assert what a given chunk of content is about Enable similar content to be aggregated Place content in context with other content Aggregation points for tying in instances and entities Derived and a subset of the Cyc knowledge base Vocabulary basis for domain-specific subject ontologies

UMBEL ( U pper M apping and B inding E xchange L ayer)

20,000 defined reference points in information space

Means to assert what a given chunk of content is about

Enable similar content to be aggregated

Place content in context with other content

Aggregation points for tying in instances and entities

Derived and a subset of the Cyc knowledge base

Vocabulary basis for domain-specific subject ontologies

Notable Ontologies and Vocabularies

Management Layer

Management/Federation Layer Management/Federation Layer handles: Ontology mapping, management Queries and retrievals All Web services Imports and exports Inferencing and logic Ontology creation and expansion Works off of many RDF datastores Has efficient, full-text indexing with faceting Interface to the system is structWSF Can plug into many options at the Applications Layer (only Drupal with conStruct SCS yet deployed)

Management/Federation Layer handles:

Ontology mapping, management

Queries and retrievals

All Web services

Imports and exports

Inferencing and logic

Ontology creation and expansion

Works off of many RDF datastores

Has efficient, full-text indexing with faceting

Interface to the system is structWSF

Can plug into many options at the Applications Layer

(only Drupal with conStruct SCS yet deployed)

Web-oriented Architecture

Applications Layer

conStruct SCS

conStruct Browse Screen

conStruct Capabilities Based on Drupal Single-click ( cloud ) deployment Theming User and group access and management Data display templates General content management system (CMS) Publishing RDF Open source

Based on Drupal

Single-click ( cloud ) deployment

Theming

User and group access and management

Data display templates

General content management system (CMS)

Publishing RDF

Open source

Re-cap

Summary Incremental, low-risk approach to the semantic enterprise Maximum leverage and re-use of existing information assets Conversion and federation of all available data forms Excellent uses for: Business intelligence Knowledge management Master data management modernization Taxonomy modernization Enterprise content integration All baseline products are open source

Incremental, low-risk approach to the semantic enterprise

Maximum leverage and re-use of existing information assets

Conversion and federation of all available data forms

Excellent uses for:

Business intelligence

Knowledge management

Master data management modernization

Taxonomy modernization

Enterprise content integration

All baseline products are open source

Contacts & Information Michael K. Bergman CEO 319.621.5225 [email_address] blog: www.mkbergman.com Steve Ardire Senior Advisor [email_address] Frédérick Giasson CTO [email_address] blog: fgiasson.com /blog Web Sites structureddynamics.com umbel.org umbel.structureddynamics.com (UMBEL Web services) openstructs.org (open source distros + documentation) constructscs.com (Drupal structured data system)

Michael K. Bergman

CEO

319.621.5225

[email_address]

blog: www.mkbergman.com

Steve Ardire

Senior Advisor

[email_address]

Frédérick Giasson

CTO

[email_address]

blog: fgiasson.com /blog

Web Sites

structureddynamics.com

umbel.org

umbel.structureddynamics.com (UMBEL Web services)

openstructs.org (open source distros + documentation)

constructscs.com (Drupal structured data system)