Your SlideShare is downloading. ×
MarkLogic Semantics
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

MarkLogic Semantics

1,041

Published on

With the release of MarkLogic 7, Semantic Technologies such as RDF and SPARQL are now mainstream. If you’re looking for a Semantics solution, we’ll show how you can use MarkLogic as your Enterprise …

With the release of MarkLogic 7, Semantic Technologies such as RDF and SPARQL are now mainstream. If you’re looking for a Semantics solution, we’ll show how you can use MarkLogic as your Enterprise Triple Store, leveraging MarkLogic’s renowned features for performance, scalability, robustness, and security.

Then we’ll take you One Step Beyond and show you how to query triples, documents, and data in a single combination query that helps you discover, understand, and make decisions over all your information types. And we’ll show you where/how to use Semantics effectively as part of your information access solution, with real-world customer examples.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,041
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
2
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • World Bank Linked Data
    The World Bank data published using the Linked Data design principles. Contains statistical observations and code lists from World Development Indicators, World Bank Finances, World Bank Projects and Operations, and World Bank Climate Change data.
  • Transcript

    • 1. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. MarkLogic Semantics Presented by: Stephen Buxton, Director Product Management, MarkLogic MarkLogic World Tour – 2014
    • 2. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 2 MarkLogic Semantics  MarkLogic Semantics in MarkLogic 7 (and beyond)  Deeper dive? – see MarkLogic Semantics - Under the Hood  What our customers are doing with Semantics  Deeper dive? – see A Field Guide to MarkLogic Semantics  Questions?
    • 3. © COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 3 MARKLOGIC SEMANTICS Powerful, Smarter Applications Faster & Easier
    • 4. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 4 Semantics: A New Way to Organize Data Data is stored in Triples, expressed as: Subject : Predicate : Object John Smith : livesIn : London London : isIn : England Query with SPARQL, gives us simple lookup .. and more! Find people who live in (a place that's in) England "John Smith" "England" livesIn "London" isIn livesIn
    • 5. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 5 Triple Store Enterprise ready. Store RDF triples alongside documents and values Triple Index In addition to value, structure, text, scalar, metadata, security, and geospatial SPARQL Industry-standard language for querying triples SPARQL MARKLOGIC SEMANTICS
    • 6. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 6 DOMAIN WORLD AT LARGE DOCUMENTS
    • 7. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 7 Context from the World at Large “Linking Open Data cloud diagram, by Richard Cyganiak and Anja Jentzsch. http://lod-cloud.net/” Linked Open Data  Facts that are freely available  In a form that’s easily consumed DBpedia (wikipedia as structured information)  Einstein was born in Germany  Ireland’s currency is the Euro GeoNames  Doha is the capital of Qatar  Doha has these lat/long coordinates
    • 8. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 8 Context from Domain Like Open Data, but domain specific  Might be proprietary within a company  Or shared across an industry  Includes data and ontologies Some Examples  A bank's proprietary reference data  A pharmaceutical company's drug ontology  An industry-wide ontology such as FIBO Proprietary Semantic Facts (Facts and Taxonomies in your organization or industry)
    • 9. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 9 Context from Documents Document metadata  Ex: Categories, author, publish date, source Facts in free-flowing text  Entities: this document mentions the person Richard Nixon, the product Advil, the company IBM  Events: this document says that Nixon went to China, John Smith met Jane Doe, Barclays acquired Lehman Brothers  Found automatically or provided at authoring time
    • 10. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 10 The World of Triples Linked Open Data (Free semantic facts available to anyone) Facts from Free-Flowing Text (Derived from semantic enrichment) Proprietary Semantic Facts (Facts and Taxonomies in your organization) Facts in Documents (Part of metadata or added with authoring tools) SemanticWorld DocumentWorld
    • 11. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 11 WHY SEMANTICS?
    • 12. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 12 Why Semantic Technologies?  Triples are atomic – easy to create, manage, combine  Semantic Web shares data as triples  A natural choice for metadata and real-world facts  .. and facts embedded in a document  Adds relationships between facts, between documents  Standards encourage tools and sharing  Graph model – easy to follow links  Ontologies – share information, infer new facts Because …
    • 13. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 13 Why Semantics and Search?  Many use cases need documents, triples, and data  One database means a simple, efficient, powerful architecture  Combination queries – query documents, triples, data in a single query – open up new possibilities Because …
    • 14. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 14 WHAT'S A COMBINATION QUERY?
    • 15. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 15 Two Hemispheres, One Brain
    • 16. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 16 Two Hemispheres, One Brain Triples:  Highly structured  Atomic  Do one thing well XML and JSON:  Flexible structure  Rich documents  Rich applications
    • 17. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 17 Combination query - scenario  You work in an Incident Call Center  A call comes in:  "some maniac in a blue van just tried to run me down"  "I got the first three letters of his license plate: ABC"  You could look up "ABC*" in the license plate database, or …  .. Look for similar incident reports  Reports that mention a "blue van"  … around the same time  … around the same place  … with a license plate that starts with "ABC"
    • 18. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 18 <SAR> <title>Suspicious vehicle…Suspicious vehicle near airport <date> <type> <threat> 2012-11-12Z observation/surveillance <type>suspicious activity <category>suspicious vehicle <location> <lat>37.497075 <long>-122.363319 <subject>IRIID <subject>IRIID <predicate> <predicate> isa value <triple> <triple> <object>license-plate <object>ABC 123 <description>A blue van…A blue van with license plate ABC 123 was observed parked behind the airport sign… </title> </date> </type> </type> </category> </threat> </lat> </long> </location> </subject> </subject> </predicate> </predicate> </object> </object> </description> </SAR> </triple> </triple> An XML or JSON document can represent many information types:
    • 19. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 19 Combination Query: Example <SAR> <title> Suspicious vehicle… <date> 2012-11-12Z <type> <threat> suspicious activity <category> suspicious vehicle <location> <lat> 37.497075 <long> -122.363319 <description> A blue van… <subject> <subject> <predicate> <object> IRIID IRIID isa value license-plate ABC 123<predicate> <object> observation/surveillance <type> <triple> <triple>
    • 20. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 20 WHAT'S IN MARKLOGIC 7?
    • 21. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 21 XQY XSLT SQL SPARQL GRAPH SPARQL Semantics Architecture TRIPLE
    • 22. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 22 What did we build?  Database – Enterprise Triple Store  Store, manage RDF triples  Query - Native SPARQL  SPARQL queries over triples  combination queries across documents, values, triples  Scalability – Indexing  special-purpose Triple Index and Cache  horizontal scaling in a shared-nothing cluster  Application Development  Updated REST APIs, SPARQL end point  SPARQL Query Console  Enterprise Ready  all integrated with MarkLogic's Enterprise NoSQL Database
    • 23. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 23 Technical Drivers -Why MarkLogic? MarkLogic is an Enterprise Triple Store  Robust  Horizontally scalable – billions of triples per box  HA/DR features such as backup/restore, replication, automatic failover  Government-grade security Triples can be embedded in documents  Address problems of provenance and reification  Annotate/add metadata to a triple (or set of triples), then do a combination query  SPARQL queries across facts: search and manage the source documents too  show me all the people John met with  … in the last 6 months, with 70% confidence, where the source is an FBI report that mentions explosives and a place within 100 miles of Paris In the Triple Store world:
    • 24. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 24 Technical Drivers -Why MarkLogic? Add a triple store to your document store  It's easy – MarkLogic combines the features of a document store and a triple store  It's simple – a single architecture for documents, values, and triples  It's powerful – combination queries let you query across documents, triples, and values in the same query Triples add value to your documents  Better search – leverage facts to expand your search  Better User Experience – show facts as well as documents and facets to help users understand, discover, and make decisions  Combination queries – new kinds of queries you cant do with separate document and triple store In the documents/search world:
    • 25. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 25 MarkLogic Semantics: Bringing it all Together Document Store + Data Store + Triple Store
    • 26. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 26 HOW DO I MAKE IT WORK?
    • 27. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 27 Semantic Implementation Details SPARQL -centric:  SPARQL with XQuery built-in functions (including cts:contains)  SPARQL with a search argument  SPARQL with variable bindings  SPARQL with forest-ids XQuery -centric: Inside an XQuery program  sem:sparql( sparql query, search criteria )  cts:triples( subject, predicate, object , search criteria )  cts:triple-range-query( subject, predicate, object , [=,<,>] )
    • 28. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 29 Triples and Documents Documents can contain triples <article> <meta> <title>Man bites dog</title> <sem:triple> <sem:subject>http://example.org/news/42</sem:subject> <sem:predicate>http://example.org/published</sem:predicate> <sem:object>2013-09-10</sem:object> </sem:triple> …
    • 29. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 30 Triples and Documents Triples are persisted in documents <sem:triple> <sem:subject>http://example.org/news/Nixon</sem:subject> <sem:predicate>http://example.org/wentTo</sem:predicate> <sem:object>China</sem:object> </sem:triple> …
    • 30. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 31 Triples and Documents Documents can contain triples <article> <meta> <title>Man bites dog</title> <sem:triple> <sem:subject>http://example.org/news/42</sem:subject> <sem:predicate>http://example.org/published</sem:predicate> <sem:object>2013-09-10</sem:object> </sem:triple> … Triples are persisted in documents <sem:triple> <sem:subject>http://example.org/news/Nixon</sem:subject> <sem:predicate>http://example.org/wentTo</sem:predicate> <sem:object>China</sem:object> </sem:triple> …
    • 31. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 32 Triples and Documents Documents can contain triples <article> <meta> <title>Man bites dog</title> <sem:triple> <sem:subject>http://example.org/news/42</sem:subject> <sem:predicate>http://example.org/published</sem:predicate> <sem:object>2013-09-10</sem:object> </sem:triple> … Triples can be annotated in documents <source>AP Newswire</source> <sem:triple date="1972-02-21" confidence="100"> <sem:subject>http://example.org/news/Nixon</sem:subject> <sem:predicate>http://example.org/wentTo</sem:predicate> <sem:object>China</sem:object> </sem:triple> …
    • 32. import module namespace sem = "http://marklogic.com/semantics" at "/MarkLogic/semantics.xqy"; sem:sparql(' SELECT ?country WHERE { <http://example.org/news/Nixon> <http://example.org/wentTo> ?country } ', (), (), cts:and-query( ( cts:path-range-query( "//sem:triple/@confidence", ">", 80) , cts:path-range-query( "//sem:triple/@date", "<", xs:date("1974-01-01")), cts:or-query( ( cts:element-value-query( xs:QName("source"), "AP Newswire" ), cts:element-value-query( xs:QName("source"), "BBC" ) ) ) ) ) ) Which countries did Nixon visit?  .. before 1974?  .. only show me answers where I have at least 80% confidence  .. and the source is AP Newswire OR BBC
    • 33. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 34 WHAT'S NEXT?
    • 34. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 35 MarkLogic Roadmap – Search and Semantics MarkLogic 7 Semantics  RDF storage and management  RDF bulk load  Specialized triple index  Native SPARQL  SPARQL over REST  Combination queries Search  Custom tokenization  SQL MATCH (driving Tableau)  Dynamic boosting (cf XRANK)  Range index scoring  More-better plans, traces, controls Foundation The essential building blocks of Semantics Search features at least on par with FAST MarkLogic 8 Semantics  Graph traversal and discovery  Automatic Inference  SPARQL 1.1 aggregates  SPARQL 1.1 Update  Basic Visualizations  SPARQL from JavaScript, Node.js Search  Search from JavaScript, Node.js  Entity Enrichment best practices Completeness World-class triple store with inference, SPARQL 1.1 Search and Semantics work together everywhere MarkLogic 9 Search and Semantics  Content Analytics  Advanced read/write visualizations  Graph analytics  Ontology management tools  Ontology-driven concept extraction, classification  More / faster combinations  Extreme Performance and Scale Do More With All Information Content analytics, advanced visualizations, and management tools give you power over all information https://ea.marklogic.com/
    • 35. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 36 WHERE ARE WE GOING WITH THIS?
    • 36. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 37 Search: Understand, Discover, Make decisions Search Fetch documents Extract relevant facts Analyze Old Search Fetch facts, data, and documents in context Analyze and annotate Fetch supporting facts, data, and documents As needed New
    • 37. © COPYRIGHT 2014 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 38
    • 38. © COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 39 QUESTIONS?

    ×