Virtuoso Universal Server Overview

4,851 views
4,687 views

Published on

Virtuoso Universal Server Overview Product Family

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,851
On SlideShare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
51
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Virtuoso Universal Server Overview

  1. 1. Virtuoso Product Family © 2008 OpenLink Software, All rights reserved. Orri Erling - Program Manager, Virtuoso
  2. 2. Virtuoso Product Categories <ul><li>Virtual Database Engine </li></ul><ul><li>Native Data Management (multi-model covering: SQL, RDF, XML, and Free Text) </li></ul><ul><li>Discussion Platform </li></ul><ul><li>Mail Proxy Services </li></ul><ul><li>Client Connectivity Kit </li></ul><ul><li>Virtuoso Universal Server </li></ul>© 2008 OpenLink Software, All rights reserved.
  3. 3. Virtual Database Engine <ul><li>External ODBC or JDBC accessible SQL Data Sources </li></ul><ul><li>External XML based Data Sources </li></ul><ul><li>External SOAP or RESTful Web Services </li></ul><ul><li>External RDF Data (e.g. Oracle) </li></ul><ul><li>Custom Data Sources via Server Extensions API </li></ul>© 2008 OpenLink Software, All rights reserved. RDF, XML, SQL Conceptual Views over:
  4. 4. Virtual Database Engine Contd. <ul><li>SQL Queries over Remote SQL, RDF, XML, and Web Services based Data Sources </li></ul><ul><li>SPARQL Queries over Remote SQL, RDF, XML, and Web Services based Data Sources </li></ul><ul><li>XQuery/XPath Queries over Remote RDF, SQL, and XML based Data Sources </li></ul><ul><li>Web Services based access to Remote RDF, SQL, XML, and other Web Services based Data Sources </li></ul>© 2008 OpenLink Software, All rights reserved.
  5. 5. Virtual Database Engine Contd. <ul><li>Distributed Query Optimization </li></ul><ul><ul><li>Locality Sensitive Query Cost Optimization (Collocated Joins, Pass-Through Queries, and Array Parameters) </li></ul></ul><ul><ul><li>Deductively Abstracts SQL Dialect Differences (via ODBC and JDBC metadata call exploitation) </li></ul></ul><ul><ul><li>Message Latency Factored into Cost Model </li></ul></ul><ul><ul><li>Hash Joins Used When Appropriate, Replacing Multiple Remote Lookups with Single Sequential Read </li></ul></ul><ul><li>2-Phase Commit for Distributed Transactions </li></ul><ul><ul><li>MS DTC for Windows </li></ul></ul><ul><ul><li>Tuxedo on Unix </li></ul></ul>© 2008 OpenLink Software, All rights reserved.
  6. 6. Virtual Database Engine Contd. <ul><li>ATTACH TABLE Statement incorporates Remote Table, Indexes and Statistics into Local Virtuoso Schema </li></ul><ul><li>Allows Incorporation of SQL Functions and Stored Procedures from Remote Relational Database Engines </li></ul><ul><li>Support for Remote XML, Full Text Indexing for Oracle, Microsoft SQL Server </li></ul>© 2008 OpenLink Software, All rights reserved.
  7. 7. Native Data Management – Relational (RDBMS) <ul><li>Native SQL 92/2K Engine </li></ul><ul><li>Rich Procedure Language (PSM-95 based) </li></ul><ul><li>Database Engine Optimized for SMP Performance </li></ul><ul><li>Native Full Text Indexing </li></ul>© 2008 OpenLink Software, All rights reserved.
  8. 8. RDBMS Features - Transactions <ul><li>Full ACID Properties </li></ul><ul><li>Checkpoint + Roll Forward Log, Optional Archiving of Logs </li></ul><ul><li>Uncommitted/Read Committed/Repeatable/Serializable Isolations </li></ul><ul><li>Non-blocking Read Committed Shows Latest Committed Versions of Uncommitted Updated Rows </li></ul><ul><li>Can Work as XA/MS DTC Resource Manager </li></ul>© 2008 OpenLink Software, All rights reserved.
  9. 9. RDBMS Features - SQL <ul><li>Full SQL 92 with many 2K Features </li></ul><ul><li>SQLX, XPATH, XSLT, Xquery </li></ul><ul><li>SQL 2K Objects, Implementation in SQL/Java/.net </li></ul><ul><li>Transparent Mixing of Local and Remote Tables </li></ul>© 2008 OpenLink Software, All rights reserved.
  10. 10. RDBMS Features – Query Optimization <ul><li>Cost Based Optimization </li></ul><ul><li>On The Fly Sampling of Table/Column/Literal Key Cardinalities </li></ul><ul><li>Fixed Statistics for Deterministic Query Plans </li></ul><ul><li>Loop/Hash/Merge Join </li></ul><ul><li>SQL Options for Explicitly Specifying Query Plan </li></ul>© 2008 OpenLink Software, All rights reserved.
  11. 11. RDBMS Features - Storage Engine <ul><li>Rows Stored At Leaves of Primary Key Index Tree </li></ul><ul><li>Non PK Indexes Refer to Row By Value of PK </li></ul><ul><li>Bitmap Index </li></ul><ul><li>Full Text Index </li></ul><ul><li>Striping Across Disks, No Separate Files Per Table/Key </li></ul><ul><li>Incremental Online backup </li></ul>© 2008 OpenLink Software, All rights reserved.
  12. 12. RDBMS Features - Run Time Hosting <ul><li>User Defined Type via Java or .NET Objects Hosted in Process </li></ul><ul><li>User Defined Types Persisted in LOB Columns </li></ul><ul><li>Java/.NET Methods Called Transparently From SQL </li></ul><ul><li>‘ C” based Plugin Mechanism for adding SQL Functions </li></ul>© 2008 OpenLink Software, All rights reserved.
  13. 13. RDBMS Features - Security <ul><li>SQL Role Based Security, Column/Table/View/Procedure Level </li></ul><ul><li>Row Level Security With Policy Functions </li></ul><ul><li>A Policy Function Can Add Extra Conditions to Queries/Updates Depending on User, Time, Other Considerations </li></ul>© 2008 OpenLink Software, All rights reserved.
  14. 14. Data Center Features - Clustering <ul><li>Combine Multiple Servers for Massive Scale and Parallelism </li></ul><ul><li>All Servers Show the Same SQL/RDF Data and Application Logic, A SQL or Web Client Can Connect to Any for the Same Service </li></ul><ul><li>Data Partitioning Specifiable Index by Index </li></ul><ul><li>Optional Replicated Storage of Partitions for More Load Balancing, Fault Tolerance </li></ul><ul><li>Shared Nothing Architecture, Works With Commodity Hardware and Networks </li></ul>© 2008 OpenLink Software, All rights reserved.
  15. 15. Data Center Features - Query Penalization <ul><li>Latency: One Message Round Trip is 20 Single Row Random Lookups </li></ul><ul><li>Virtuoso Divides Queries into Collocated Fragments, Ships All Filtering, Aggregation, Joining to Where the Data Is. </li></ul><ul><li>Sends Arrays of Hundreds of Operations at a Time, Whenever Possible </li></ul>© 2008 OpenLink Software, All rights reserved.
  16. 16. Data Center Features - Transactions <ul><li>Full ACID Properties </li></ul><ul><li>Two Phase Commit with Single Phase Optimization </li></ul><ul><li>Detection of Distributed Deadlocks Without Timing Out </li></ul><ul><li>Each Cluster Node Keeps Own Transaction Log </li></ul><ul><li>No External Monitor, Virtuoso Handles Distributed Recovery Cycle By Itself </li></ul><ul><li>Transactions/Logging Can BE Disabled for Bulk Load etc. </li></ul>© 2008 OpenLink Software, All rights reserved.
  17. 17. Data Center Features - Parallel SQL <ul><li>Transparent Map-Reduce Style Execution of Specified Partitioned SQL Functions/Procedures </li></ul><ul><li>PL Extensions for Async Remote Execution of SQL Code, With and Without Transactional Semantics </li></ul>© 2008 OpenLink Software, All rights reserved.
  18. 18. Data Center Features - Futures <ul><li>Dynamic Deployment, Adding and Removing Cluster Nodes Without Interruption of Service </li></ul><ul><li>Keeping Data in Small, Self-Contained, Easily Relocatable Mini-Partitions </li></ul>© 2008 OpenLink Software, All rights reserved.
  19. 19. SQL Client Connectivity - Data Access Drivers <ul><li>Cross Platform ODBC 3.0 Drivers </li></ul><ul><li>JDBC 2.0 Drivers </li></ul><ul><li>OLE-DB Provider </li></ul><ul><li>ADO.NET Provider </li></ul><ul><li>XMLA Provider </li></ul>© 2008 OpenLink Software, All rights reserved.
  20. 20. Native Data Management - XML <ul><li>Native XML Data Type </li></ul><ul><li>SQLX + Oracle Compatible XML Functions in SQL </li></ul><ul><li>Document Centric Persistence of XML with Special Support in Text Index </li></ul><ul><li>XSLT </li></ul><ul><li>XQuery </li></ul><ul><li>XML Views – XML Mapping Schema based Views of SQL Data Sources </li></ul>© 2008 OpenLink Software, All rights reserved.
  21. 21. Native RDF Data Management <ul><li>Native RDF Quad Storage (Physical Quads) </li></ul><ul><li>SQL Enhanced With RDF IRI and Typed/Language Tagged Data </li></ul><ul><li>Bitmap Indices and Key Compression for Compact Storage </li></ul><ul><li>Selectable Index Scheme, Optionally Allows Queries Against Union of All Graphs </li></ul><ul><li>Optional Full Text Index of Literals </li></ul><ul><li>Reuses SQL Cost Model and Execution Engine With RDF Tailored Statistics </li></ul>© 2008 OpenLink Software, All rights reserved.
  22. 22. RDF Data Services – Client Connectivity <ul><li>SPARQL Protocol </li></ul><ul><li>Jena Storage Provider </li></ul><ul><li>Sesame Storage Provider </li></ul><ul><li>Redland Storage Provider </li></ul><ul><li>Linq2Rdf Storage Provider </li></ul><ul><li>SPASQL </li></ul><ul><ul><li>SPARQL execution within SQL Processor </li></ul></ul><ul><li>Plethora of Built-In Functions, Stored Procedures, Web Services </li></ul>© 2008 OpenLink Software, All rights reserved.
  23. 23. RDF Data Services – SPARQL <ul><li>Full SPARQL, Language and Protocol Support </li></ul><ul><li>Jena Compatible SPARUL for Create Graph, Insert, Update, and Delete </li></ul><ul><li>Extensions for Aggregates & Grouping </li></ul><ul><li>Nested Queries, SQL-Like Existence and Value Subqueries </li></ul><ul><li>Expressions in Result Sets </li></ul><ul><li>Path Expressions for Compact Notation, Also in Expressions </li></ul><ul><li>Full Text & XPath Magic Predicate Extensions </li></ul>© 2008 OpenLink Software, All rights reserved.
  24. 24. RDF Data Services – Inference <ul><li>Backward Chaining Inference Support, No Materialization of Entailed Triples needed for: </li></ul><ul><ul><li>Subclass and Subproperty Hierarchies </li></ul></ul><ul><ul><li>OWL sameAs for Instances, Classes and Properties </li></ul></ul><ul><ul><li>OWL equivalentClass and equivalentProperty </li></ul></ul><ul><ul><li>Inference Enabled at Query or Individual Triple Pattern Level </li></ul></ul>© 2008 OpenLink Software, All rights reserved.
  25. 25. Linked Data Services - RDF-ization Middleware <ul><li>Declarative RDF Views (or Covers) over SQL Data </li></ul><ul><li>In-Built RDF Middleware (Sponger) for RDF-ization of Harvested Web Content (bulk ingest or “on the fly”) </li></ul><ul><li>Extended SPARQL Against Mapped and Stored RDF </li></ul><ul><li>RDF-ization Cartridges for 30+ non RDF data sources </li></ul><ul><ul><li>Used by SPARQL Processor </li></ul></ul><ul><ul><li>Used by in-built Content Crawler </li></ul></ul><ul><li>Cache Invalidation based on HTTP Caching Rules </li></ul><ul><li>Configurable URI dereferencing via pragmas for node selection and path traversal </li></ul>© 2008 OpenLink Software, All rights reserved.
  26. 26. Linked Data Services - Deployment <ul><li>URL Rewrite Rules combined with SPARQL for flexible association of URIs and RDF Data Sets </li></ul><ul><li>Proxy (or wrapper) URIs construction for materializing Linked Data “on the fly” from existing Web information resources </li></ul><ul><li>REST or SOAP based Web Services that expose functionality to Web Clients such as OpenLink Data Explorer, Marbles, Zitgist Data Explorer, DISCO, Tabulator etc. </li></ul>© 2008 OpenLink Software, All rights reserved.
  27. 27. RDF Data Services – RDF Views over SQL Data Sources <ul><li>SPARQL Data Definition Statements for RDB Mapping </li></ul><ul><li>Declare Correspondences Between Graph/Triple Patterns and SQL Objects </li></ul><ul><li>Specify Mapping Between URI's and Keys , Supporting All Data Types, Multipart Keys </li></ul><ul><li>Not Restricted to Table per Class and Column per Property </li></ul><ul><li>Use Arbitrary Joins, SQL Functions and Search Conditions </li></ul><ul><li>Automatically Generate Basic Class per Table, Property per Column Mapping of Given SQL Schema </li></ul>© 2008 OpenLink Software, All rights reserved.
  28. 28. RDF Data Services - RDF Views Contd. <ul><li>Evaluate Arbitrary SPARQL Against an RDF View </li></ul><ul><li>In One Query, Some Graphs May Come from Views, Others From Stored RDF </li></ul><ul><li>RDF Views Generate a Single SQL Statement, The IRI Generation and IRI Parsing is Only in Selection and Constant Expressions </li></ul><ul><li>SQL Has Full Optimization Possibilities and the Generated SQL Does not Depend on Virtuoso Specifics </li></ul><ul><li>Hence, RDF Views Are Efficient for Querying Remote, non-Virtuoso SQL Data </li></ul>© 2008 OpenLink Software, All rights reserved.
  29. 29. RDF Data Services - Clustering <ul><li>Cluster-Optimized RDF Loader and SPARUL </li></ul><ul><li>RDF-Aware Data Partitioning </li></ul><ul><li>Automatic Statistics Sampling Across Cluster for Best Query Plan </li></ul>© 2008 OpenLink Software, All rights reserved.
  30. 30. RDF Benchmarks <ul><li>TPC H With SPARQL Extensions and RDF Views </li></ul><ul><li>LUBM </li></ul><ul><li>Berlin SPARQL Benchmark with Triples and with RDF Views </li></ul>© 2008 OpenLink Software, All rights reserved. Bundled With:
  31. 31. Web Services Platform – HTTP Services <ul><li>HTTP/1.1 and HTTPS Server for Static and Dynamic Content </li></ul><ul><li>Dynamic Web Pages in PHP, Virtuoso SQL Procedures, ASP .net, Others </li></ul><ul><li>SOAP and Rest Web Services in Virtuoso PL, Java, .NET </li></ul><ul><li>DAV </li></ul>© 2008 OpenLink Software, All rights reserved.
  32. 32. Web Services Platform - WebDAV <ul><li>Documents Stored in Virtuoso Database </li></ul><ul><li>ACL Based plus Unix Style Security, SQL User Accounts and Roles Own Documents and Collections </li></ul><ul><li>Automatic RDF Metadata Extraction </li></ul><ul><li>Optional Full Text Indexing and Versioning </li></ul><ul><li>Dynamic Collections for Alternate Views of Directory Hierarchy </li></ul>© 2008 OpenLink Software, All rights reserved.
  33. 33. Web Services Platform – SOAP & REST <ul><li>SOAP 1.1/1.2 End Points Exposing SQL Procedures in All SOAP Styles </li></ul><ul><li>Automatic WSDL Generation </li></ul><ul><li>SQL Extensions for Declaring Full XML Schema Signatures for End Points </li></ul><ul><li>Exposing Java and .net via SOAP </li></ul><ul><li>Dynamic Web Pages and XML Functions for REST Services </li></ul><ul><li>XMLA for SQL Access over SOAP </li></ul>© 2008 OpenLink Software, All rights reserved.
  34. 34. Web Services Platform - Dynamic Server Pages <ul><li>Configure a Virtual Directory as Executable </li></ul><ul><li>Publish Dynamic Web Pages in PHP, Virtuoso PL, Ruby, PERL, ASP .net Without Using External Web Server </li></ul>© 2008 OpenLink Software, All rights reserved.
  35. 35. Administration Services <ul><li>Web Interface for Setup of Web End Points, SQL, XML, RDF Functions </li></ul><ul><li>SQL Functions for Full Programmatic Admin Access </li></ul><ul><li>Simple Tuning, Only Specify File Layout and Amount of Threads and Memory to Use </li></ul>© 2008 OpenLink Software, All rights reserved.
  36. 36. Virtuoso RDF Applications <ul><li>Dbpedia </li></ul><ul><li>BIO2RDF </li></ul><ul><li>Neurocommons </li></ul><ul><li>Zitgist , Pingthesemanticweb , Musicbrainz </li></ul>© 2008 OpenLink Software, All rights reserved.
  37. 37. Product <ul><li>Open Source and Closed Source Versions, Closed Source AddsVirtual Database and Clustering </li></ul><ul><li>All Code, Applications, Samples, Docs in Single Download </li></ul><ul><li>Minimal Installation Consists of Single Executable + Config File </li></ul><ul><li>Web Admin Interface and Bundled ODS Collaborative Apps Suite </li></ul><ul><li>Available for All Linux, Unix, Windows, 32 and 64 bit </li></ul><ul><li>Available Preinstalled on Amazon EC2, With Optional Preloaded Dbpedia, BIO2RDF, Other RDF Data Sets </li></ul>© 2008 OpenLink Software, All rights reserved.

×