Content-sensitive User Interfaces
for Annotated Web Pages
Applications of Semantic Technologies
4th Int. Workshop, October 2, 2009
Florian Schmedding
University of Freiburg

Outline
1. Motivation
2. Requirements
3. DOM-API extension
4. Use Cases
5. Conclusion
02.10.2009 Florian Schmedding (AST 2009) 2

Motivation Requirements DOM-API extension Use Cases Conclusion
Motivation
• Content of web pages is often related to personal
user data:
Timetables
Addresses
Tasks
• Different applications handle this kind of data
• Why not use the web page itself to interact with
this information?
No switching between applications
Direct interaction with data
User stays with the actual work
02.10.2009 Florian Schmedding (AST 2009) 3

Motivation Requirements DOM-API extension Use Cases Conclusion
9:00 am
9:15 am
(web page)
Conflict
• Feedback: Indicate overlapping events on the web page
• Action: Add events to the personal calendar
02.10.2009 Florian Schmedding (AST 2009) 4

Motivation Requirements DOM-API extension Use Cases Conclusion
Requirements
1) Machine-readable descriptions of the data:
• RDF connector for Outlook
• Annotations for the web page
2) Linkage between document text and data
3) Linkage has to be accessible for user
interactions
02.10.2009 Florian Schmedding (AST 2009) 5

Motivation Requirements DOM-API extension Use Cases Conclusion
Annotation approaches
• Annotea (XPointer)
• MnM, S-Cream (XML)
• GRDDL (XSLT)
• RDFa (embedded)
– Self-contained
– Content, not meta information
– Encodes RDF directly
– Linkage explicitly given
02.10.2009 Florian Schmedding (AST 2009) 6

Motivation Requirements DOM-API extension Use Cases Conclusion
RDFa Example: Event description
<tr about="#l1" typeof="cal:Vevent">
<td property="cal:dtstart" datatype="xsd:dateTime"
content="2008-10-21T09:15:00">
Tuesday, October 21, 2008
<span property="cal:duration" datatype="xsd:duration"
content="PT95M" />
</td>
<td property="cal:summary" datatype="xsd:string">
Introduction
</td>
</tr>
02.10.2009 Florian Schmedding (AST 2009) 7

Motivation Requirements DOM-API extension Use Cases Conclusion
Contained statements:
schedule.html#l1 a cal:Vevent ;
cal:dtstart "2008-10-21T09:15:00"^^xsd:datetime ;
cal:duration "PT95M"^^xsd:duration ;
cal:summary "Introduction"^^xsd:string .
02.10.2009 Florian Schmedding (AST 2009) 8

Motivation Requirements DOM-API extension Use Cases Conclusion
DOM-API extension
• RDFa spec defines extraction rules
• Does not consider the linkage
• Interaction with web pages involves the
document object model
Integration of the document
object model and RDF
• Retrieve DOM elements by SPARQL queries
• Get information about statements from DOM
elements
02.10.2009 Florian Schmedding (AST 2009) 9

Motivation Requirements DOM-API extension Use Cases Conclusion
Augmented DOM-tree
• Attributes with parsed RDF-values
– Subject, Predicates, Object, Literal, Types …
• XML-attributes with RDFa-values unchanged
• Triple stores
– Local (statements)
– Subtree-level (store):
e.store = e.statements c1.store … cn.store
– Global store: root element
02.10.2009 Florian Schmedding (AST 2009) 10

Motivation Requirements DOM-API extension Use Cases Conclusion
Subject e1 Statements
l1 l1 type Vevent
tr
Types Store
Vevent l1 type Vevent
l1 start „2008-10-21“
l1 duration „PT95M“
l1 summary „Introduction“
Statements
Predicates e2 l1 start „2008-10-21“ Predicates e3 Statements
start l1 summary „Introduction“
td Store
start td
Literal Literal
l1 start „2008-10-21“
2008-10-21 l1 duration „PT95M“ Introduction
Tuesday, October Predicates Statements
duration
e4 l1 duration „PT95M“ Introduction
21, 2008 span
Literal
PT95M
02.10.2009 Florian Schmedding (AST 2009) 11

Motivation Requirements DOM-API extension Use Cases Conclusion
Tree construction
• RDFa extraction rules are defined for DOM
• SAX parser suffices, take care of special cases:
– Base-element
– Inner text (for literal properties)
• Additional attributes for XML-elements
– subject, predicates, object, literal, data type, types
– isSkip, isIncomplete, subjectSet
• Statements are stored
• DOM parser uses this to build the augmented
tree
02.10.2009 Florian Schmedding (AST 2009) 12

Motivation Requirements DOM-API extension Use Cases Conclusion
SAX example: Extract triples
RdfaReader doc = new RdfaReader(url);
while (doc.Read())
{
// When there are triples
if (doc.Statements != null)
foreach (Statement s in doc.Statements)
{
// Save statement in a triple store
}
}
doc.Close();
02.10.2009 Florian Schmedding (AST 2009) 13

Motivation Requirements DOM-API extension Use Cases Conclusion
Access Methods
• Provide access to the RDF triples by
– getElementsByResource(Resource r)
– getElementsByStatement(Statement s)
– getElementsByQuery(string q)
• Return a list of XML nodes
• The third method returns a list of variable bindings
– Each binding replaced with its origin
02.10.2009 Florian Schmedding (AST 2009) 14

Motivation Requirements DOM-API extension Use Cases Conclusion
Linkage
• Parser associates RDF resources with their origin in
the document
• Origin is an XML element
• If a resource is used in several RDFa-attributes, it has
also several origins
• The structures of the triple store in the augmented
DOM-tree store these origins
02.10.2009 Florian Schmedding (AST 2009) 15

Motivation Requirements DOM-API extension Use Cases Conclusion
<tr about="#l1" typeof="cal:Vevent">
<td property="cal:dtstart"
content="2008-10-21T09:15:00">
Tuesday, October 21, 2008
<span property="cal:duration"
content="PT95M"/>
</td>
<td property="cal:summary">
Introduction</td>
</tr>
Subject Sori Predicate Pori Object Oori Stori
schedule.html#l1 e1 a e1 cal:Vevent e1 e1
schedule.html#l1 e1 cal:dtstart e2 "2008-10-21T09:15:00" e2 e2
schedule.html#l1 e1 cal:summary e3 "Introduction" e3 e3
schedule.html#l1 e1 cal:duration e4 "PT95M" e4 e4
02.10.2009 Florian Schmedding (AST 2009) 16

Motivation Requirements DOM-API extension Use Cases Conclusion
Use Case: Calendar Integration
Find events on the page
select ?event
where { ?event a cal:Vevent }
Look for conflicts with
Highlight text personal appointments
Get the visible
properties of each event
getElementsByQuery(
select ?literal
where { dbis:l1 ?p ?literal
FILTER(isLiteral(?literal))})
02.10.2009 Florian Schmedding (AST 2009) 17

Motivation Requirements DOM-API extension Use Cases Conclusion
Additionally, show the summaries from the personal calendar:
02.10.2009 Florian Schmedding (AST 2009) 18

Motivation Requirements DOM-API extension Use Cases Conclusion
Further Use Cases
• Handling of related data such as addresses, phone
numbers, tasks, …
• Inline-conversion of units
• Input suggestions based on personal data/profiles for
annotated forms
• Rearrange structured page content based on SPARQL
queries with order by
• Information browsing: relate the content of the current
page to collected data of same kind
• Offer special actions: showing geographical locations or
routes on a map
02.10.2009 Florian Schmedding (AST 2009) 19

Motivation Requirements DOM-API extension Use Cases Conclusion
Conclusion
• New approach to access the linkage between text and
annotations given by RDFa in web pages
• Innovative applications that do not only extract data but use
the linkage to provide content-sensitive user interfaces
– Feedback integration
– Actions for contained data
• Linkage accessed with SPARQL queries
• SAX-parser also available (web crawlers)
• Other annotation approaches can also use the semantic
structures by using the triple stores
02.10.2009 Florian Schmedding (AST 2009) 20

Thanks for your attention!

References
1. Schmedding, F.: Content-sensitive User Interfaces for Annotated Web
Pages. Applications of Semantic Technologies, 4th Int. Workshop. Lübeck,
Germany. (2009)
2. Adida, B., Birbeck, M.: RDFa Primer.
http://www.w3.org/TR/2008/NOTE-xhtml-rdfa-primer-20081014/
(2008)
02.10.2009 Florian Schmedding (AST 2009) 22