The document outlines the Talis platform, a software as a service solution for storing and managing both structured and unstructured data using RDF and SPARQL. It covers key concepts such as platform architecture, metadata management, and querying capabilities while emphasizing flexibility, standards compliance, and access control features. Additional details include content negotiation, data extraction methods, and the use of changesets for managing updates.

1. Getting Started with the Talis Platform Leigh Dodds Platform Programme Manager Talis December 2008 http://creativecommons.org/licenses/by/2.0/uk/

2. Agenda Platform Overview Core Concepts Review of the RDF Model Managing binary data Managing structured metadata Exploring RDF data with SPARQL Extra Features Store Administration Summary

3. Platform Overview

4. Software as a Service Multi-Tenant Data Storage Service

5. Unstructured Data Storage e.g. binary files, including images, documents, etc

6. Structured Data Storage RDF metadata

7. Access Control All data is open (to read) by default Configurable access options

8. Full-Text Searching and Querying

9. Standards Compliance RDF, SPARQL, HTTP

10. Platform Architecture Web API Metabox Contentbox

11. REST, RDF Authentication & Authorization Content Negotiation Core Concepts aka “The Science Bit”

12. REST Re presentational S tate T ransfer Correct Use of HTTP

13. Resource-Centric API Everything has a unique URI

14. Interact with resources using HTTP GET = read PUT = write POST = update/modify DELETE = delete

15. Use HTTP Response Codes 200 = OK 201 = Created (new resource) 202 = Accepted (for processing) 400 = Bad Request 500 = Server Error

16. Mime Types Used to identifiy content & meaning of request and response body

17. Content Negotiation Majority of services support multiple output options, list varies by resource Accept header output parameter

18. Our Service Checklist Consistent URI structure Every service has human interface Plain text error messages for easy debugging Cacheable … etc

19. Authentication HTTP Digest Authentication

20. Authentication Example

21. Authorization By default stores are world-readable, Store owner writable Customisable roles and privileges per-Store

22. Review of the RDF Model

23. Apollo 11 was launched from Cape Canaveral

24. Apollo 11 was launched from Cape Canaveral Subject Predicate Object

25. <http://purl.org/net/schemas/space/spacecraft/apollo-11> <http://purl.org/net/schemas/space/launchsite> <http://purl.org/net/schemas/space/launchsite/capecanaveral>.

26. space: spacecraft/apollo-11 space: launchsite space: launchsite/capecanaveral.

27. space:spacecraft/apollo-11 space:launchsite space:launchsite/capecanaveral. space:spacecraft/apollo-11 rdfs:label “Apollo 11” . space:launchsite/capecanaveral rdfs:label “Cape Canaveral” .

30. Benefits of RDF?

31. Good for Semi-structured Data “Schema-Free” Very Flexible

32. Extensible New properties New resources New types of resource New statements

33. Encourages Convergence Reuse of vocabularies (i.e. properties) Reuse of identifiers (i.e. talk about the same things)

34. Simplifies Data Integration and Aggregation Shared identifiers Common data model Common query language Common data formats

35. Several Different Ways to Serialize RDF Optimized for different purposes

36. Turtle Simple to read and hand-author Used in SPARQL query language

37. @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> @prefix space: <http://purl.org/net/schemas/space/> @ @prefix dc: <http://purl.org/dc/elements/1.1/> <http://purl.org/net/schemas/space/spacecraft/1969-059A> rdf:type <http://purl.org/net/schemas/space/Spacecraft>; dc:description &quot;Apollo 11 was…”; space:agency &quot;United States&quot; .

38. RDF/XML Best for data interchange Harder to read

39. <rdf:RDF xmlns:j.0=&quot;http://xmlns.com/foaf/0.1/“ xmlns:rdf=&quot;http://www.w3.org/1999/02/22-rdf-syntax-ns#&quot; xmlns:space=&quot;http://purl.org/net/schemas/space/&quot; xmlns:dc=&quot;http://purl.org/dc/elements/1.1/&quot; xml:base=&quot;http://purl.org/net/schemas/space&quot;> <rdf:Description rdf:about=&quot;/spacecraft/1969-059A&quot;> <dc:description>Apollo 11 was…</dc:description> <rdf:type rdf:resource=&quot;http://purl.org/net/schemas/space/Spacecraft&quot;/> <space:agency>United States</space:agency> </rdf:Description> </rdf:RDF>

40. The Content Box Managing unstructured, binary data

41. Store any stream of binary data Images, documents, Javascript, etc

42. Full HTTP Caching Support ETags Efficient retrieval Conditional updates

43. Server or Client Assignment of Identifiers Provides full control over how URIs assigned

44. ContentBox URLs /storename/items The Contentbox container /storename/items/<id> An individual item

45. Adding Content

46. Deleting Content

47. Metadata for Contentbox Resources Minimum is URI and ETag Extract height & width of images … more metadata extraction in future

48. The Meta Box Managing structured metadata

49. Full RDF Data Storage Create, read, update, delete RDF resources Query RDF data

50. Configurable Full Text Indexing of RDF Indexes updated whenever new metadata added

51. Versioned and Un-Versioned Updates By submitting data to separate resources Maintain audit trail

52. Can be Divided into Sub-Graphs Separate access control options

53. Metabox URLs /storename/meta The metabox /storename/meta/changesets The collection of changesets associated with this metabox /storename/meta/graphs The collection of sub-graphs /storename/meta/graphs/{id} A sub-graph /storename/meta/graphs/{id}/changesets The collection of changesets associated with a sub-graph /storename/services/sparql SPARQL endpoint for metabox /storename/services/multisparql SPARQL endpoint for querying across all sub-graphs

54. Storing RDF POST application/rdf+xml Changes saved immediately Search indexing asynchronous

55. Triples are Merged into Store Can catch out the unwary Updates happen through separate mechanism

56. Retrieving Metadata /meta?about=…URI… Can select RDF serialization

57. Updating Resources POST application/vnd.talis.changeset+xml

58. ChangeSets Vocabulary that specifies removals/additions to an RDF graph

59. <rdf:RDF xmlns:rdf=&quot;http://www.w3.org/1999/02/22-rdf-syntax-ns#&quot; xmlns:cs=&quot;http://purl.org/vocab/changeset/schema#&quot;> <cs:ChangeSet rdf:about=&quot;http://example.com/changesets#change&quot;> < cs:subjectOfChange rdf:resource=&quot;http://purl.org/net/schema/space/launch/1969-059&quot;/> < cs:createdDate >2008-12-08T00:00:00Z</ cs:createdDate > < cs:creatorName >Leigh Dodds</ cs:creatorName > < cs:changeReason >More accurate launch time</ cs:changeReason > <cs:removal> <rdf:Statement> <rdf:subject rdf:resource=&quot;http://purl.org/net/schema/space/launch/1969-059&quot;/> <rdf:predicate rdf:resource=&quot;http://purl.org/net/schema/space/launched&quot;/> <rdf:object>1969-07-16</rdf:object> </rdf:Statement> </cs:removal> <cs:addition> <rdf:Statement> <rdf:subject rdf:resource=&quot;http://purl.org/net/schema/space/launch/1969-059&quot;/> <rdf:predicate rdf:resource=&quot;http://purl.org/net/schema/space/launched&quot;/> <rdf:object>1969-07-16T13:32:00</rdf:object> </rdf:Statement> </cs:addition> </cs:ChangeSet> </rdf:RDF>

60. <rdf:RDF xmlns:rdf=&quot;http://www.w3.org/1999/02/22-rdf-syntax-ns#&quot; xmlns:cs=&quot;http://purl.org/vocab/changeset/schema#&quot;> <cs:ChangeSet rdf:about=&quot;http://example.com/changesets#change&quot;> <cs:subjectOfChange rdf:resource=&quot;http://purl.org/net/schema/space/launch/1969-059&quot;/> <cs:createdDate>2008-12-08T00:00:00Z</cs:createdDate> <cs:creatorName>Leigh Dodds</cs:creatorName> <cs:changeReason>More accurate launch time</cs:changeReason> < cs:removal > <rdf:Statement> < rdf:subject rdf:resource=&quot;http://purl.org/net/schema/space/launch/1969-059&quot;/> < rdf:predicate rdf:resource=&quot;http://purl.org/net/schema/space/launched&quot;/> < rdf:object >1969-07-16</ rdf:object > </rdf:Statement> </ cs:removal > <cs:addition> <rdf:Statement> <rdf:subject rdf:resource=&quot;http://purl.org/net/schema/space/launch/1969-059&quot;/> <rdf:predicate rdf:resource=&quot;http://purl.org/net/schema/space/launched&quot;/> <rdf:object>1969-07-16T13:32:00</rdf:object> </rdf:Statement> </cs:addition> </cs:ChangeSet> </rdf:RDF>

61. <rdf:RDF xmlns:rdf=&quot;http://www.w3.org/1999/02/22-rdf-syntax-ns#&quot; xmlns:cs=&quot;http://purl.org/vocab/changeset/schema#&quot;> <cs:ChangeSet rdf:about=&quot;http://example.com/changesets#change&quot;> <cs:subjectOfChange rdf:resource=&quot;http://purl.org/net/schema/space/launch/1969-059&quot;/> <cs:createdDate>2008-12-08T00:00:00Z</cs:createdDate> <cs:creatorName>Leigh Dodds</cs:creatorName> <cs:changeReason>More accurate launch time</cs:changeReason> <cs:removal> <rdf:Statement> <rdf:subject rdf:resource=&quot;http://purl.org/net/schema/space/launch/1969-059&quot;/> <rdf:predicate rdf:resource=&quot;http://purl.org/net/schema/space/launched&quot;/> <rdf:object>1969-07-16</rdf:object> </rdf:Statement> </cs:removal> < cs:addition > <rdf:Statement> < rdf:subject rdf:resource=&quot;http://purl.org/net/schema/space/launch/1969-059&quot;/> < rdf:predicate rdf:resource=&quot;http://purl.org/net/schema/space/launched&quot;/> < rdf:object >1969-07-16T13:32:00</ rdf:object > </rdf:Statement> </ cs:addition > </cs:ChangeSet> </rdf:RDF>

62. Versioned Updates POST to /meta/changesets Apply update and stores changeset for later retrieval

63. Batch Updates Combine several changesets into single POST Linked together to define ordering

64. <rdf:RDF xmlns:rdf=&quot;http://www.w3.org/1999/02/22-rdf-syntax-ns#&quot; xmlns:cs=&quot;http://purl.org/vocab/changeset/schema#&quot;> <cs:ChangeSet rdf:about=&quot;http://example.com/changesets/1&quot;> <cs:subjectOfChange rdf:resource=&quot;http://purl.org/net/schema/space/launch/1969-059&quot;/> <cs:changeReason>More accurate launch time</cs:changeReason> < cs:precedingChangeset rdf:resource=&quot; http://example.com/changesets/2 &quot;/> <!– changes --> </cs:ChangeSet> <cs:ChangeSet rdf:about=&quot; http://example.com/changesets/2 &quot;> <cs:subjectOfChange rdf:resource=&quot;http://purl.org/net/schema/space/launch/1969-059&quot;/> <cs: precedingChangeset rdf:resource=&quot; http://example.com/changesets/3 &quot;/> <!– changes --> </cs:ChangeSet> <cs:ChangeSet rdf:about=&quot; http://example.com/changesets/3 &quot;> <cs:subjectOfChange rdf:resource=&quot;http://purl.org/net/schema/space/spacecraft/1969-059D&quot;/> <!– changes --> ... </cs:ChangeSet> </rdf:RDF>

65. Data Extraction & Exploration with SPARQL

66. SPARQL RDF query language; HTTP protocol; Results format 4 different forms of query

67. ASK Test whether the graph contains some data of interest

68. #Was there a launch on 16 th July 1969? PREFIX space: <http://purl.org/net/schemas/space/> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> ASK WHERE { ?launch space:launched &quot;1969-07-16&quot;^^xsd:date. }

69. <?xml version=&quot;1.0&quot;?> <sparql xmlns=&quot;http://www.w3.org/2005/sparql-results#&quot;> <head> </head> <boolean>true</boolean> </sparql>

70. DESCRIBE Generate an RDF description of a resource(s)

71. #Describe launch(es) that occurred on 16 th July 1969 PREFIX space: <http://purl.org/net/schemas/space/> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> DESCRIBE ?launch WHERE { ?launch space:launched &quot;1969-07-16&quot;^^xsd:date. }

72. #Describe spacecraft launched on 16 th July 1969 PREFIX space: <http://purl.org/net/schemas/space/> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> DESCRIBE ?spacecraft WHERE { ?launch space:launched &quot;1969-07-16&quot;^^xsd:date. ?spacecraft space:launch ?launch. }

73. CONSTRUCT Create a custom RDF graph based on query criteria

74. PREFIX space: <http://purl.org/net/schemas/space/> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> PREFIX foaf: <http://xmlns.com/foaf/0.1/> CONSTRUCT { ?spacecraft foaf:name ?name; space:agency ?agency; space:mass ?mass. } WHERE { ?launch space:launched &quot;1969-07-16&quot;^^xsd:date. ?spacecraft space:launch ?launch; foaf:name ?name; space:agency ?agency; space:mass ?mass. }

75. SELECT SQL style result set retrieval

76. PREFIX space: <http://purl.org/net/schemas/space/> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> PREFIX foaf: <http://xmlns.com/foaf/0.1/> SELECT ?name ?agency ?mass WHERE { ?launch space:launched &quot;1969-07-16&quot;^^xsd:date. ?spacecraft space:launch ?launch; foaf:name ?name; space:agency ?agency; space:mass ?mass. }

77. …as XML <?xml version=&quot;1.0&quot;?> <sparql xmlns:rdf=&quot;http://www.w3.org/1999/02/22-rdf-syntax-ns#&quot; xmlns=&quot;http://www.w3.org/2005/sparql-results#&quot; > <head> <variable name=&quot;name&quot;/> <variable name=&quot;agency&quot;/> <variable name=&quot;mass&quot;/> </head> <results> <result> <binding name=&quot;name&quot;> <literal>Apollo 11 Command and Service Module (CSM)</literal> </binding> <binding name=&quot;agency&quot;> <literal>United States</literal> </binding> <binding name=&quot;mass&quot;> <literal>28801.0</literal> </binding> </result> <!– more results --> </results> </sparql>

78. …as JSON { &quot;head&quot;: { &quot;vars&quot;: [ &quot;name&quot; , &quot;agency&quot; , &quot;mass&quot; ] } , &quot;results&quot;: { &quot;bindings&quot;: [ { &quot;name&quot;: { &quot;type&quot;: &quot;literal&quot; , &quot;value&quot;: &quot;Apollo 11 Command and Service Module (CSM)&quot; } , &quot;agency&quot;: { &quot;type&quot;: &quot;literal&quot; , &quot;value&quot;: &quot;United States&quot; } , &quot;mass&quot;: { &quot;type&quot;: &quot;literal&quot; , &quot;value&quot;: &quot;28801.0&quot; } } , { &quot;name&quot;: { &quot;type&quot;: &quot;literal&quot; , &quot;value&quot;: &quot;Apollo 11 SIVB&quot; } , &quot;agency&quot;: { &quot;type&quot;: &quot;literal&quot; , &quot;value&quot;: &quot;United States&quot; } , &quot;mass&quot;: { &quot;type&quot;: &quot;literal&quot; , &quot;value&quot;: &quot;13300.0&quot; } } , { &quot;name&quot;: { &quot;type&quot;: &quot;literal&quot; , &quot;value&quot;: &quot;Apollo 11 Lunar Module / EASEP&quot; } , &quot;agency&quot;: { &quot;type&quot;: &quot;literal&quot; , &quot;value&quot;: &quot;United States&quot; } , &quot;mass&quot;: { &quot;type&quot;: &quot;literal&quot; , &quot;value&quot;: &quot;15065.0&quot; } } ] } }

79. Tour of Extra Features Searching, browsing, augmentation

80. Searching Full text index over RDF literals Configurable indexing options

81. /items?query=[query] &max=[10] &offset=[0] &sort=[comma-separated fieldnames] &xsl=[XSLT stylesheet] &content-type=[mimetype for XSLT results]

82. Query Syntax lunar luna* “ apollo 11” lunar OR apollo name:apollo (lunar OR apollo) AND agency:united states

83. Query Results RSS 1.0 feed OpenSearch extensions (paging, relevance) Full description of each resource

84. <rdf:RDF xmlns=&quot;http://purl.org/rss/1.0/&quot; xmlns:foaf=&quot;http://xmlns.com/foaf/0.1/&quot; xmlns:relevance=&quot;http://a9.com/-/opensearch/extensions/relevance/1.0/&quot; xmlns:rdf=&quot;http://www.w3.org/1999/02/22-rdf-syntax-ns#&quot; xmlns:os=&quot;http://a9.com/-/spec/opensearch/1.1/&quot; xmlns:ns.1=&quot;http://purl.org/net/schemas/space/&quot;> <channel rdf:about=“…&quot;> <title>lunar</title> <link>…</link> <description>Results of a search for lunar on space</description> <items> <rdf:Seq rdf:about=&quot;urn:uuid:eae4ead8-ca6a-4b12-b714-fe631d38e447&quot;> <rdf:li resource=&quot;http://purl.org/net/schemas/space/spacecraft/LUNAR-A&quot; /> </rdf:Seq> </items> < os:startIndex >0</ os:startIndex > < os:itemsPerPage >10</ os:itemsPerPage > < os:totalResults >118</ os:totalResults > </channel> <item rdf:about=&quot;http://purl.org/net/schemas/space/spacecraft/LUNAR-A&quot;> <title>Item</title> <link>http://purl.org/net/schemas/space/spacecraft/LUNAR-A</link> < relevance:score >1.0</ relevance:score > <foaf:name>Lunar-A</foaf:name> <space:mass>520.0</space:mass> <space:internationalDesignator>LUNAR-A</space:internationalDesignator> </item> </rdf:RDF>

85. Facetted Search Similar to Amazon product search, etc Group search results by specific fields

86. /services/facet?query=[query] &fields=[comma-separated fieldnames] &top=[10] &format=[xml|html]

87. <facet-results xmlns=&quot;http://schemas.talis.com/2007/facet-results#&quot;> <head> <query>name:luna*</query> <fields>agency</fields> <top>10</top> <output>xml</output> </head> <fields> <field name=&quot;agency&quot;> <term value=&quot;U.S.S.R&quot; number=&quot;25&quot; facet-uri=“…&quot; search-uri=“…&quot;/> <term value=&quot;United States&quot; number=&quot;9&quot; facet-uri=“…&quot; search-uri=“…&quot;/> <term value=&quot;Japan&quot; number=&quot;1&quot; facet-uri=“…&quot; search-uri=“…&quot;/> <term value=&quot;India&quot; number=&quot;1&quot; facet-uri=“…&quot; search-uri=“…&quot;/> </field> </fields> </facet-results>

88. Augmentation Annotate an RSS 1.0 feed against a store Automatically add a description of each referenced resource

89. Store Administration Job Control, Store Configuration

90. Field Predicate Map Associate a short name to a RDF property Properties in field predicate map are indexed for searching Short name used in query syntax, sort order, etc

91. <rdf:RDF xmlns:rdf=&quot;http://www.w3.org/1999/02/22-rdf-syntax-ns#&quot; xmlns:rdfs=&quot;http://www.w3.org/2000/01/rdf-schema#&quot; xmlns:bf=&quot;http://schemas.talis.com/2006/bigfoot/configuration#&quot; xmlns:frm=&quot;http://schemas.talis.com/2006/frame/schema#“ xml:base=“http://api.talis.com/stores/space”> <bf:FieldPredicateMap rdf:about=&quot;/indexes/default/fpmaps/default&quot;> <frm:mappedDatatypeProperty> <rdf:Description rdf:about=&quot;/indexes/default/fpmaps/default#agency&quot;> <frm:property rdf:resource=&quot;http://purl.org/net/schema/space/agency&quot;/> <frm:name>agency</frm:name> </rdf:Description> </frm:mappedDatatypeProperty> </bf:FieldPredicateMap> </rdf:RDF>

92. Query Profile Assign weightings to fields for searching

93. <rdf:RDF xmlns:rdf=&quot;http://www.w3.org/1999/02/22-rdf-syntax-ns#&quot; xmlns:rdfs=&quot;http://www.w3.org/2000/01/rdf-schema#&quot; xmlns:bf=&quot;http://schemas.talis.com/2006/bigfoot/configuration#&quot; xmlns:frm=&quot;http://schemas.talis.com/2006/frame/schema#“ xml:base=“http://api.talis.com/stores/space”> <bf:QueryProfile rdf:about=&quot;&quot;> <bf:fieldWeight> <rdf:Description rdf:about=&quot;/indexes/default/queryprofiles/default#name&quot;> <bf:weight>10.0</bf:weight> <frm:name>name</frm:name> </rdf:Description> </bf:fieldWeight> <bf:fieldWeight> <rdf:Description rdf:about=&quot;/indexes/default/queryprofiles/default#agency&quot;> <bf:weight>5.0</bf:weight> <frm:name>agency</frm:name> </rdf:Description> </bf:fieldWeight> </bf:QueryProfile> </rdf:RDF>

94. Job Control Reindex, Reset, Snapshot, Restore POST Job Request to /jobs

95. <rdf:RDF xmlns:rdf=&quot;http://www.w3.org/1999/02/22-rdf-syntax-ns#&quot; xmlns:rdfs=&quot;http://www.w3.org/2000/01/rdf-schema#&quot; xmlns:bf=&quot;http://schemas.talis.com/2006/bigfoot/configuration#&quot;> <bf:JobRequest> <rdfs:label>Reset the data in my store</rdfs:label> <bf:jobType rdf:resource=&quot;http://schemas.talis.com/2006/bigfoot/configuration#ResetDataJob&quot;/> <bf:startTime>2008-12-01T15:10:00Z</bf:startTime> </bf:JobRequest> </rdf:RDF>

96. Jobs Each job is a resource, with a URI GET to monitor status, DELETE to remove

97. Summing Up Summary, Additional Resources

98. The Talis Platform… Provides a standards compliant storage infrastructure for structured and unstructured metadata Uses RDF to support widest possible variety of data models and integration options Allow managing of data assets through simple web APIs Offers a range of data extraction options including full-text searching, SPARQL, RSS augmentation Can be tailored to individual applications using the API Can be driven by scheduling jobs to perform data management tasks Is constantly evolving…

99. Additional Resources API Reference http://n2.talis.com/wiki/Platform_API Mailing List http://groups.google.com/group/n2-dev Blog http://blogs.talis.com/n2/

100. Client Libraries (in various states of development) Moriarty http://code.google.com/p/moriarty/ Javascript/JQuery http://n2.talis.com/wiki/Talis_jQuery_plugin Ruby Client http://rubyforge.org/projects/talis-platform/ Java Client http://code.google.com/p/penry/

101. shared innovation