This document proposes an open cross-document linking and browsing architecture based on a visual plug-in model. It describes existing limitations in cross-document linking. The proposed architecture uses data and visual plug-ins to support linking across different document formats in a extensible way without changing existing formats. It has been prototyped and evaluated with PDF, XML, text and HTML plug-ins.

1. WEB & INFORMATION
SYSTEMS ENGINEERING
2 December 2005
Open Cross-Document Linking and Browsing
based on a Visual Plug-in Architecture
Ahmed A.O.Tayeh and Beat Signer
Web & Information Systems Engineering Lab (WISE)
Department of Computer Science
Vrije Universiteit Brussel

2. Isolated Digital Documents
×
Ahmed Tayeh - Department October 14, 2014 of Computer Science - atayeh@vub.ac.be
1
HTML document PDF document

3. Isolated Digital Documents …
× ×
Ahmed Tayeh - Department October 14, 2014 of Computer Science - atayeh@vub.ac.be
2
×
Word document PDF document

4. Isolated Digital Documents …
 Limited possibilities to create hyperlinks between snippets
of different document formats
 document formats mainly address links to web resources (HTML)
 simple embedded unidirectional links
 linked documents are not aware of any links that have been
defined from other source documents
Ahmed Tayeh - Department October 14, 2014 of Computer Science - atayeh@vub.ac.be
3

5. Existing Support for Hyperlinks
Ahmed Tayeh - Department October 14, 2014 of Computer Science - atayeh@vub.ac.be
4

6. Cross-Document Linking
Anchor (page index + ec(x,y,w,h)) Anchor (XPointer-like expression)
Anchor (start and end indices) Anchor(XPointer expression)
Ahmed Tayeh - Department October 14, 2014 of Computer Science - atayeh@vub.ac.be
5
PDF (linear-like model) HTML (tree model)
Text (linear model) XML (tree model)

7. Open Hypermedia
 No support for cross-document linking
 Limited extensibility
 on the data as well as
on the visual level
Ahmed Tayeh - Department October 14, 2014 of Computer Science - atayeh@vub.ac.be
6
MADCOW: A Multimedia Digital Annotation System,
Paolo Bottoni et al., Proceedings of AVI 2004

8. Requirements for a Linking Service
 No changes required to existing document formats and
standards
 No assumptions about the document models
 e.g. WYSIWYG, tree models, constrained tree models, …
 Support existing as well as emerging document formats
 Support advanced linking features
Ahmed Tayeh - Department October 14, 2014 of Computer Science - atayeh@vub.ac.be
7

9. Open Cross-Media Architecture
Visual Plug-ins Visual Plug-ins Data Plug-ins
Ahmed Tayeh - Department October 14, 2014 of Computer Science - atayeh@vub.ac.be
8
Annotation/Link
Service
Client Application
Annotation/Link
Browser & Editor
Visual Plug-ins
Resource Plug-in Repository
Data Plug-ins
An Architecture for Open Cross-Media Annotation Services,
Beat Signer and Moria C. Norrie, Proceeding of WISE 2009

10. Data Level Extensibility (RSL Approach)
… …
As We May Link: A General Metamodel for Hypermedia Systems,
Beat Signer and Moria C. Norrie, Proceeding of ER 2007
Ahmed Tayeh - Department October 14, 2014 of Computer Science - atayeh@vub.ac.be
9
… XPointer (s,e) XPointer shape … URL file:// URL file://

11. Visualisation Layer Extensibility
Ahmed Tayeh - Department October 14, 2014 of Computer Science - atayeh@vub.ac.be
10
local visual plug-ins
external visual plug-ins
communicate
Link Browser

12. Visualisation Layer Extensibility …
 Local visual plug-ins vs. external visual plug-ins
Link visual plug-in Link visual plug-in
Link Browser
Ahmed Tayeh - Department October 14, 2014 of Computer Science - atayeh@vub.ac.be
11
Acrobat Reader Google Chrome

13. Visualisation Layer Extensibility …
 DefaultDocument class
 necessary methods to visualise
any document format
- getSelector()
- openDocument()
- …
 event listeners and handlers
 Local visual plug-ins
 extend DefaultDocument class
 might use existing visualisation libraries
 Link browser
 instantiates DefaultDocument
Third-Party Applications
DocFormat3 Application
Browser
Ahmed Tayeh - Department October 14, 2014 of Computer Science - atayeh@vub.ac.be
Visual Plug-ins
12
DocFormat1 DocFormat3 DocFormat2
Visualisation
Gateway DocFormat3
Visualisation
DocFormat1
Visualisation
DocFormat2

14. Visualisation Layer Extensibility …
 External visual plug-ins
 provide methods to handle selections
in third-party applications
 communicate with the link browser
via a specific gateway plug-in
 Gateway plug-ins
 launch third-party application
 offer different communication
protocols
- TCP sockets
- WebSockets
- REST API as a fallback
Third-Party Applications
DocFormat3 Application
DocFormat1 DocFormat3 DocFormat2
Browser
Ahmed Tayeh - Department October 14, 2014 of Computer Science - atayeh@vub.ac.be
Visual Plug-ins
Visualisation
Gateway DocFormat3
Visualisation
DocFormat1
Visualisation
DocFormat2
13

15. Open Cross-Document Link Service Architecture
Data Plug-ins Third-Party Applications
DocFormat3
DocFormat1 DocFormat2
RSL
Database Manager
Database
Visual Plug-ins
DocFormat3 Application
DocFormat1 DocFormat3 DocFormat2
Visualisation
Gateway DocFormat3
Core
Data Layer
Browser
Visualisation
DocFormat1
Visualisation
DocFormat2
Ahmed Tayeh - Department October 14, 2014 of Computer Science - atayeh@vub.ac.be
14

16. Open Service Gateway initiative (OSGi)
 Enhances modularisation of our linking service
 Use OSGi dynamic extensibility for dynamic extensibility
of the linking service
 OSGi versioning and dependency resolution reduces
problems in dealing with multiple libraries (“JAR hell”)
 linking service might provide different visual plug-ins (versions) for
a single document format
Ahmed Tayeh - Department October 14, 2014 of Computer Science - atayeh@vub.ac.be
15

17. Open Service Gateway initiative (OSGi) …
User Interface Manifest
Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: UserInterface
Bundle-SymbolicName: com.rsl.userInterface
Bundle-Version: 1.0.0.qualifier
Bundle-RequiredExecutionEnvironment: JavaSE-1.7
Export-Package: org.rsl.userInterface
Import-Package: org.rsl.core, org.rsl.service,
org.rsl.databasemanager,
org.associations.collections,
org.osgi.framework;version="1.7.0"
PDF Local Visual Plug-in Manifest
Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: Visual
Bundle-SymbolicName: org.rsl.pdf.visual
Bundle-Version: 1.0.0.qualifier
Bundle-RequiredExecutionEnvironment: JavaSE-1.7
Import-Package: org.rsl.core, org.rsl.userInterface,
org.rsl.pdf.data
Extension-Class: org.rsl.pdf.visual.Pdf
Extension-Type: visual
Extension-Format: application/pdf
Ahmed Tayeh - Department October 14, 2014 of Computer Science - atayeh@vub.ac.be
16

18. Link Navigation Scenario
Browser RSL Registry DocFormat1 Gateway DocFormat3 DocFormat2 DocFormat3
getSelection()
selector
getLinkTarget(selector)
getDocument(resource)
getPluginType(formatName)
alt
instantiateClass()
openDocument(resource, selectors, selector)
instantiateClass()
openDocument(resource, selectors, selector)
launchApp()
openDocument(resource, selectors, selector)
[visualPlugin==local]
[else]
Ahmed Tayeh - Department October 14, 2014 of Computer Science - atayeh@vub.ac.be
17

19. Evaluation: PDF, XML, Text and HTML Plug-ins
Ahmed Tayeh - Department October 14, 2014 of Computer Science - atayeh@vub.ac.be
18

20. Conclusion and Future Work
 Cross-document link service prototype
 Extensibility on the data as well as the visualisation layer
 OSGi framework for extensible architecture
 Evaluation with PDF, XML, Text and HTML plug-ins
 Dynamic plug-in extensibility
 Investigate extensibility in a study with developers
 Usability evaluation
Ahmed Tayeh - Department October 14, 2014 of Computer Science - atayeh@vub.ac.be
19