Ontopia tutorial

6,830 views
6,695 views

Published on

A full-day tutorial covering all modules in the Ontopia Topic Maps engine.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,830
On SlideShare
0
From Embeds
0
Number of Embeds
75
Actions
Shares
0
Downloads
96
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Ontopia tutorial

  1. 1. Ontopia Tutorial<br />TMRA 2010-09-29<br />Lars Marius Garshol & Geir Ove Grønmo<br />
  2. 2. Agenda<br />About you<br />who are you?<br />About Ontopia<br />The product<br />The future<br />Participating in the project<br />
  3. 3. Some background<br />About Ontopia<br />
  4. 4. Brief history<br />1999-2000<br />private hobby project for Geir Ove<br />2000-2009<br />commercial software sold by Ontopia AS<br />lots of international customers in diverse fields<br />2009-<br />open source project<br />
  5. 5. The project<br />Open source hosted at Google Code<br />Contributors<br />Lars Marius Garshol, Bouvet<br />Geir Ove Grønmo, Bouvet<br />Thomas Neidhart, SpaceApps<br />Lars Heuer, Semagia<br />Hannes Niederhausen, TMLab<br />Stig Lau, Bouvet<br />Baard H. Rehn-Johansen, Bouvet<br />Peter-Paul Kruijssen, Morpheus<br />Quintin Siebers, Morpheus<br />Matthias Fischer, HTW Berlin<br />
  6. 6. Recent work<br />Ontopia/Liferay integration<br />Matthias Fischer & LMG<br />Various fixes and optimizations<br />everyone<br />Tropics (RESTful web service interface)<br />SpaceApps<br />Porting build system to Maven2<br />Morpheus<br />
  7. 7. Architecture and modules<br />Product overview<br />
  8. 8. The big picture<br />Auto-class.<br />A.N.other<br />A.N.other<br />Other<br />CMSs<br />A.N.other<br />A.N.other<br />DB2TM<br />Portlet support<br />OKP<br />TMSync<br />Engine<br />CMSintegration<br />Data <br />integration<br />Escenic<br />Taxon.import<br />Ontopoly<br />Web<br />service<br />
  9. 9. The engine<br />Core API<br />TMAPI 2.0 support<br />Import/export<br />RDF conversion<br />TMSync<br />Fulltext search<br />Event API<br />tolog query language<br />tolog update language<br />Engine<br />
  10. 10. The backends<br />In-memory<br />no persistent storage<br />thread-safe<br />no setup<br />RDBMS<br />transactions<br />persistent<br />thread-safe<br />uses caching<br />clustering<br />Remote<br />uses web service<br />read-only<br />unofficial<br />Engine<br />Memory<br />RDBMS<br />Remote<br />
  11. 11. DB2TM<br />Upconversion to TMs<br />from RDBMS via JDBC<br />or from CSV<br />Uses XML mapping<br />can call out to Java<br />Supports sync<br />either full rescan<br />or change table<br />TMRAP<br />Nav<br />DB2TM<br />Classify<br />Engine<br />Memory<br />RDBMS<br />Remote<br />
  12. 12. TMRAP<br />Web service interface<br />via SOAP<br />via plain HTTP<br />Requests<br />get-topic<br />get-topic-page<br />get-tolog<br />delete-topic<br />...<br />TMRAP<br />Nav<br />DB2TM<br />Classify<br />Engine<br />Memory<br />RDBMS<br />Remote<br />
  13. 13. Navigator framework<br />Servlet-based API<br />manage topic maps<br />load/scan/delete/create<br />JSP tag library<br />XSLT-like<br />based on tolog<br />JSTL integration<br />TMRAP<br />Nav<br />DB2TM<br />Classify<br />Engine<br />Memory<br />RDBMS<br />Remote<br />
  14. 14. Automated classification<br />Undocumented<br />experimental<br />Extracts text<br />autodetects format<br />Word, PDF, XML, HTML<br />Processes text<br />detects language<br />stemming, stop-words<br />Extracts keywords<br />ranked by importance<br />uses existing topics<br />supports compound terms<br />TMRAP<br />Nav<br />DB2TM<br />Classify<br />Engine<br />Memory<br />RDBMS<br />Remote<br />
  15. 15. Vizigator<br />Viz<br />Ontopoly<br />Graphical visualization<br />VizDesktop<br />Swing app to configure<br />filter/style/...<br />Vizlet<br />Java applet for web<br />uses configuration<br />loads via TMRAP<br />uses “Remote” backend<br />TMRAP<br />Nav<br />DB2TM<br />Classify<br />Engine<br />Memory<br />RDBMS<br />Remote<br />
  16. 16. Ontopoly<br />Viz<br />Ontopoly<br />Generic editor<br />web-based, AJAX<br />meta-ontology in TM<br />Ontology designer<br />create types and fields<br />control user interface<br />build views<br />incremental dev<br />Instance editor<br />guided by ontology<br />TMRAP<br />Nav<br />DB2TM<br />Classify<br />Engine<br />Memory<br />RDBMS<br />Remote<br />
  17. 17. Typical deployment<br />Viewing<br />application<br />Engine<br />Users<br />DB<br />Backend<br />Ontopoly<br />Frameworks<br />Editors<br />DB<br />TMRAP<br />DB2TM<br />HTTP<br />DB<br />External application<br />Application server<br />
  18. 18. APIs<br />The engine<br />
  19. 19. Core APIs<br />net.ontopia.topicmaps.core.*<br />Fairly direct mapping from TMDM<br />TopicIF<br />AssociationIF<br />TopicMapIF<br />...<br />Set/get methods reflect TMDM properties<br />
  20. 20. TopicIF<br />Interface, not a class<br />getTopicNames()<br />addTopicName(TopicNameIF)<br />removeTopicName(TopicNameIF)<br />getOccurrences() + add + remove<br />getSubjectIdentifiers() + add + remove<br />getItemIdentifiers() + add + remove<br />getSubjectLocators() + add + remove<br />getRoles()<br />getRolesByType(TopicIF)<br />
  21. 21. Core interfaces<br />TopicMapStoreIF<br />TopicMapIF<br />TopicIF<br />AssociationIF<br />TopicNameIF<br />OccurrenceIF<br />AssociationRoleIF<br />VariantNameIF<br />
  22. 22. How to get a TopicMapIF<br />Create one directly<br />new net...impl.basic.InMemoryTopicMapStore()<br />Load one from file<br />using an importer (next slide)<br />Connect to an RDBMS<br />covered later<br />Use a topic map repository<br />covered later<br />
  23. 23. TopicMapReaderIF<br />import net.ontopia.topicmaps.core.TopicMapIF;<br />import net.ontopia.topicmaps.core.TopicMapReaderIF;<br />import net.ontopia.topicmaps.utils.ImportExportUtils;<br />public class TopicCounter {<br /> public static void main(String[] argv) throws Exception {<br /> TopicMapReaderIF reader = ImportExportUtils.getReader(argv[0]);<br /> TopicMapIF tm = reader.read();<br /> System.out.println("TM contains " + tm.getTopics().size() + " topics");<br /> }<br />}<br />[larsga@c716c5ac1 tmp]$ java TopicCounter ~/data/bilder/privat/metadata.xtm<br />TM contains 17035 topics<br />[larsga@c716c5ac1 tmp]$ <br />
  24. 24. Supported syntaxes<br />
  25. 25. The utility classes<br />A set of classes outside the core interfaces that perform common tasks<br />a number of these utilities are obsolete now that tolog is here<br />They are all built on top of the core interfaces<br />Some important utilities<br />ImportExportUtils creates readers and writers<br />MergeUtils merges topics and topic maps<br />PSI contains important PSIs<br />DeletionUtils cascading delete of topics<br />DuplicateSuppressionUtils removes duplicates<br />TopicStringifiers find names for topics<br />
  26. 26. Topic Maps repository<br />Uses a set of topic maps sources to build a set of topics maps<br />topic maps can be looked up by ID<br />Many kinds of sources<br />scan directory for files matching pattern<br />download from URL<br />connect to RDBMS<br />...<br />Configurable using an XML file<br />tm-sources.xml<br />Used by Navigator Framework<br />
  27. 27. Event API<br />Allows clients to receive notification of changes<br />Must implement TopicMapListenerIF<br /> static class TestListener extends AbstractTopicMapListener {<br /> public void objectAdded(TMObjectIF snapshot) {<br /> System.out.println("Topic added: " + snapshot.getObjectId());<br /> }<br /> public void objectModified(TMObjectIF snapshot) {<br /> System.out.println("Topic modified: " + snapshot.getObjectId());<br /> }<br /> public void objectRemoved(TMObjectIF snapshot) {<br /> System.out.println("Topic removed: " + snapshot.getObjectId());<br /> }<br /> }<br />
  28. 28. Using the API<br /> // register to listen for events<br /> TestListener listener = new TestListener();<br /> TopicMapEvents.addTopicListener(ref, listener);<br /> // get the store through the reference so the listener is registered<br /> ref.createStore(false);<br /> // let's add a topic<br /> System.out.println("Off we go");<br /> TopicMapBuilderIF builder = tm.getBuilder();<br /> TopicIF newbie = builder.makeTopic(tm);<br /> System.out.println("Let's name this topic");<br /> builder.makeTopicName(newbie, "Newbie topic");<br /> // then let's remove it<br /> System.out.println("And now, the exit");<br /> DeletionUtils.remove(newbie);<br /> System.out.println("Goodbye, short-lived topic");<br />[larsga@dhcp-98 tmp]$ java EventTest bkclean.xtm <br />Off we go<br />Topic added: 3409<br />Let's name this topic<br />Topic modified: 3409<br />Topic modified: 3409<br />And now, the exit<br />Topic removed: 3409<br />Goodbye, short-lived topic<br />
  29. 29. For more information<br />See the Engine Developer's Guide<br />http://www.ontopia.net/doc/current/doc/engine/devguide.html<br />
  30. 30. Persistence & transactions<br />RDBMS backend<br />
  31. 31. RDBMS backend<br />Stores Topic Maps in an RDBMS<br />generic schema<br />access via JDBC<br />Provides full ACID<br />transactions<br />concurrency<br />...<br />Supports several databases<br />Oracle, MySQL, PostgreSQL, MS SQL Server, hsql<br />Clustering support<br />
  32. 32. Core API implementation<br />Implements same API as in-memory impl<br />theoretically, a switch requires only config change<br />Lazy loading of objects<br />objects loaded from DB as needed<br />Considerable internal caching<br />for performance reasons<br />Separate objects for separate transactions<br />in order to provide isolation<br />Shared cache between transactions<br />
  33. 33. Configuration<br />A Java property file<br />Specifies<br />database type<br />JDBC URL<br />username + password<br />cache settings<br />clustering settings<br />...<br />
  34. 34. jdbcspy<br />A built-in SQL profiler<br />Useful for identifying cause of performance issues<br />
  35. 35. tolog<br />The Query Engine<br />
  36. 36. tolog<br />A logic-based query language<br />a mix of Prolog and SQL<br />effectively equivalent to Datalog<br />Two parts<br />queries (data retrieval)<br />updates (data modification)<br />Developed by Ontopia<br />not an ISO standard<br />eventually to be replaced by TMQL<br />
  37. 37. tolog<br />The recommended way to interact with the data<br />API programming is slow and cumbersome<br />tolog queries perform better<br />Available via<br />Java API<br />Web service API<br />Forms interface in Omnigator<br />tolog queries return API objects<br />
  38. 38. Finding all operas by a composer<br /> Collection operas = new ArrayList();<br /> TopicIF composer = getTopicById("puccini");<br /> TopicIF composed_by = getTopicById("composed-by");<br /> TopicIF work = getTopicById("work");<br /> TopicIF creator = getTopicById("composer");<br /> for (AssociationRoleIF role1 : composer.getRolesByType(creator)) {<br /> AssociationIF assoc = role1.getAssociation();<br /> if (assoc.getType() != composed_by)<br /> continue;<br /> for (AssociationRoleIF role2 : assoc.getRoles()) {<br /> if (role2.getType() != work)<br /> continue;<br /> operas.add(role2.getPlayer());<br /> }<br /> }<br />
  39. 39. Finding all operas by a composer<br />composed-by(puccini : composer, $O : work)?<br />composed-by($C : composer, tosca : work)?<br />composed-by($C : composer, $O : work)?<br />composed-by(puccini : composer, tosca : work)?<br />
  40. 40. Features<br />Access all aspects of a topic map<br />Generic queries independent of ontology<br />AND, OR, NOT, OPTIONAL<br />Count<br />Sort<br />LIMIT/OFFSET<br />Reusable inference rules<br />
  41. 41. Chaining predicates (AND)<br />Predicates can be chained<br />born-in($PERSON : person, $PLACE : place),located-in($PLACE : containee, italy : container)?<br />The comma between the predicates means AND<br />This query finds all the people born in Italy<br />It first builds a two-column table of all born-in associations<br />Then, those rows where the place is not located-in Italy are removed<br />(Note that when the PLACE variable is reused above that means that the birthplace and the location must be the same topic in each match)<br />Any number of predicates can be chained<br />Their order is insignificant<br />Actually, the optimizer reorders the predicates<br />It will start with located-in because it has a topic constant<br />
  42. 42. Thinking in predicates<br /><ul><li>Most of you are probably used to functions, which work like this:
  43. 43. function(arg1, arg2, arg3) -> result
  44. 44. Predicates, however, are in a sense bidirectional, because of the way the pattern matching works
  45. 45. predicate(topic : role1, $VAR : role2)
  46. 46. predicate($VAR : role1, topic : role2)
  47. 47. The order of the roles are, on the other hand, insignificant
  48. 48. predicate(topic : role1, $VAR : role2)
  49. 49. predicate($VAR : role2, topic : role1)</li></li></ul><li>Projection<br />Sometimes queries make use of temporary variables that we are not really interested in<br />The way to get rid of unwanted variables is projection<br />Syntax:<br /> select $variable1, $variable2, ... <br />from <query>?<br />The query is first run, then projected down to the requested variables<br />
  50. 50. The instance-of predicate<br /><ul><li>instance-of has the following form:
  51. 51. instance-of (instance,class)
  52. 52. NOTE: the order of the arguments is significant
  53. 53. Like players, instance and class may be specified in two ways:
  54. 54. using a variable ($name)
  55. 55. using a topic reference
  56. 56. e.g. instance-of ( $A, city )
  57. 57. instance-of makes use of the superclass-subclass associations in the topic map
  58. 58. this means that composers will be considered musicians, and musicians will be considered persons</li></li></ul><li>Cities with the most premieres<br />using o for i"http://psi.ontopedia.net/"<br />select $CITY, count($OPERA) from<br /> instance-of($CITY, o:City),<br /> { o:premiere($OPERA : o:Work, $CITY : o:Place)<br /> |<br /> o:premiere($OPERA : o:Work, $THEATRE : o:Place),<br /> o:located_in($THEATRE : o:Containee, $CITY : o:Container)<br /> } order by $OPERA desc?<br />
  59. 59. All non-hidden photos<br />select $PHOTO from<br /> instance-of($PHOTO, op:Photo)<br /> not(ph:hide($PHOTO : ph:hidden)),<br /> not(ph:taken-at($PHOTO : op:Image, $PLACE : op:Place),<br /> ph:hide($PLACE : ph:hidden)),<br /> not(ph:taken-during($PHOTO : op:Image, $EVENT : op:Event),<br /> ph:hide($EVENT : ph:hidden)),<br /> not(ph:depicted-in($PHOTO : ph:depiction, $PERSON : ph:depicted),<br /> ph:hide($PERSON : ph:hidden))?<br />
  60. 60. Demo<br />Show running queries in Omnigator<br />Also show query tracing<br />Shakespeare<br />/* #OPTION: optimizer.reorder = false */<br />
  61. 61. tologspy<br />tolog query profiler<br />shares code with jdbcspy<br />
  62. 62. Using the query engine API<br /><ul><li>The query engine API is really simple to use</li></ul>get a QueryProcessorIF object<br />run a query in the QueryProcessorIF and get a QueryResultIF<br />loop over the results and use them<br />close the result object<br />go back to step 2, or do something else<br /><ul><li>There are two different QueryProcessorIF implementations
  63. 63. the API lets you write code without worrying about that, however
  64. 64. the two implementations behave identically</li></li></ul><li>Running a query with the API<br />TopicMapIF tm = ...;<br />QueryProcessorIF processor = QueryUtils.getQueryProcessor(tm);<br />QueryResultIF result = processor.execute(“instance-of($P, person)?”);<br />try {<br /> while (result.next()) {<br /> TopicIF person = (TopicIF) result.getValue(0);<br /> // do something useful with 'person'<br /> } <br />} finally {<br /> result.close();<br />}<br />
  65. 65. Advanced options<br /><ul><li>It is possible to parse a query once, and then run it many times
  66. 66. the processor returns a ParsedQueryIF object, which can be executed
  67. 67. parameters can be passed to the query on each execution
  68. 68. It is possible to make declarations and use them across executions</li></li></ul><li>Using a parsed query<br />ParsedQueryIF parsedQuery = processor.parse(“instance-of($P, %TYPE%)?”);<br />Map params = Collections.singletonMap(“TYPE”, person);<br />QueryResultIF result = parsedQuery.execute(params);<br />try {<br /> while (result.next()) {<br /> // ...<br /> } <br />} finally {<br /> result.close();<br />}<br />
  69. 69. QueryWrapper<br />Designed to make all of this easier<br />QueryWrapper qw = new QueryWrapper(tm);<br />TopicIF topic = qw.queryForTopic(...);<br />List topics = qw.queryForList(...);<br />List<Person> people = <br />qw.queryForList(..., mapper);<br />
  70. 70. tolog updates<br />Greatly simplifies TM modification<br />Also means you can do modification without API programming<br />useful with RDBMS topic maps<br />useful with TMs in running web servers<br />By performing a sequence of updates, just about any change can be made<br />Potentially allows much more powerful architecture<br />
  71. 71. DELETE<br />Static form<br />delete lmg<br />Dynamic form<br />delete $person from instance-of($person, person)<br />Delete a value<br />delete subject-identifier(topic, “http://ex.org/tst”)<br />
  72. 72. MERGE<br />Static form<br />MERGE topic1, topic2<br />Dynamic form<br />MERGE $p1, $p2 FROM instance-of($p1, person), instance-of($p2, person), <br /> email($p1, $email), <br /> email($p2, $email)<br />
  73. 73. INSERT<br />Static form<br />INSERT lmg isa person; - “Lars Marius Garshol” .<br />Dynamic form<br />INSERT tmcl:belongs-to-schema(tmcl:container : theschema, tmcl:containee: $c) FROM instance-of($c, tmcl:constraint)<br />
  74. 74. INSERT again<br />INSERT <br /> ?y $psi .<br /> event-in-year(event: $e, year: ?y)<br />FROM <br /> start-date($e, $date),<br /> str:substring($y, $date, 4),<br /> str:concat($psi, "http://psi.semagia.com/iso8601", $y)<br />
  75. 75. UPDATE<br />Static form<br />UPDATE value(@3421, “New name”)<br />Dynamic form<br />UPDATE value($TN, “Ontopia”) FROM topic-name(oks, $TN)<br />
  76. 76. More information<br />Look at sample queries in Omnigator<br />tolog tutorial<br />http://www.ontopia.net/doc/current/doc/query/tutorial.html<br />tolog built-in predicate reference<br />http://www.ontopia.net/doc/current/doc/query/predicate-reference.html<br />
  77. 77. Conversion from RDBMS data<br />DB2TM<br />
  78. 78. DB2TM<br />Upconversion of relational data<br />either from CSV files or<br />over JDBC<br />Based on an XML file describing the mapping<br />very highly configurable<br />Support for<br />all of Topic Maps (except variants)<br />value transformations<br />synchronization<br />
  79. 79. Standard use case<br />Pull in data from external source<br />turn it into Topic Maps following some ontology<br />Enrich it<br />usually manually, but not necessarily<br />Resync from source at intervals<br />
  80. 80. DB2TM example<br />Ontopia<br />+<br />=<br />United Nations<br />Bouvet<br /><relation name="organizations.csv" columns="id name url"><br /> <topic type="ex:organization"><br /> <item-identifier>#org${id}</item-identifier><br /> <topic-name>${name}</topic-name><br /> <occurrence type="ex:homepage">${url}</occurrence><br /> </topic><br /></relation><br />
  81. 81. Creating associations<br /> <relation name="people.csv" columns="id given family employer phone"><br /> <topic id="employer"><br /> <item-identifier>#org${employer}</item-identifier><br /> </topic><br /> <topic type="ex:person"><br /> <item-identifier>#person${id}</item-identifier><br /> <topic-name>${given} ${family}</topic-name><br /> <occurrence type="ex:phone">${phone}</occurrence><br /> <player atype="ex:employed-by" rtype="ex:employee"><br /> <other rtype="ex:employer" player="#employer"/><br /> </player><br /> </topic><br /> </relation><br />
  82. 82. Value transformations<br /> <relation name="SCHEMATA" columns="SCHEMA_NAME"><br /> <function-column name='SCHEMA_ID'<br /> method='net.ontopia.topicmaps.db2tm.Functions.makePSI'><br /> <param>${SCHEMA_NAME}</param><br /> </function-column><br /> <topic type="mysql:schema"><br /> <item-identifier>#${SCHEMA_ID}</item-identifier><br /> <topic-name>${SCHEMA_NAME}</topic-name><br /> </topic><br /> </relation><br />
  83. 83. Running DB2TM<br />java net.ontopia.topicmaps.db2tm.Execute<br />command-line tool<br />also works with RDBMS topic maps<br />net.ontopia.topicmaps.db2tm.DB2TM<br />API class to run transformations<br />methods "add" and "sync"<br />
  84. 84. More information<br />DB2TM User's Guide<br />http://www.ontopia.net/doc/current/doc/db2tm/user-guide.html<br />
  85. 85. Synchronizing with other sources<br />TMSync<br />
  86. 86. TMSync<br />Configurable module for synchronizing one TM against another<br />define subset of source TM to sync (using tolog)<br />define subset of target TM to sync (using tolog)<br />the module handles the rest<br />Can also be used with non-TM sources<br />create a non-updating conversion from the source to some TM format<br />then use TMSync to sync against the converted TM instead of directly against the source<br />
  87. 87. How TMSync works<br />Define which part of the target topic map you want,<br />Define which part of the source topic map it is the master for, and<br />The algorithm does the rest<br />
  88. 88. If the source is not a topic map<br />TMSync<br />convert.xslt<br />Simply do a normal one-time conversion<br />let TMSync do the update for you<br />In other words, TMSync reduces the update problem to a conversion problem<br />source.xml<br />
  89. 89. The City of Bergen usecase<br />Norge.no<br />Service<br />Unit<br />Person<br />LOS<br />City of Bergen<br />LOS<br />
  90. 90. Web service interface<br />TMRAP<br />
  91. 91. TMRAP basics<br />Abstract interface<br />that is, independent of any particular technology<br />coarse-grained operations, to reduce network traffic<br />Protocol bindings exist<br />plain HTTP binding<br />SOAP binding<br />Supports many syntaxes<br />XTM 1.0<br />LTM<br />TM/XML<br />custom tolog result-set syntax<br />
  92. 92. get-topic<br />Retrieves a single topic from the remote server<br />topic map may optionally be specified<br />syntax likewise<br />Main use<br />to build client-side fragments into a bigger topic map<br />to present information about a topic on a different server<br />
  93. 93. get-topic<br />Parameters<br />identifier: a set of URIs (subject identifiers of wanted topic)<br />subject: a set of URIs (subject locators of wanted topic)<br />item: a set of URIs (item identifiers of wanted topic)<br />topicmap: identifier for topic map being queried<br />syntax: string identifying desired Topic Maps syntax in response<br />view: string identifying TM-Views view used to define fragment<br />Response<br />topic map fragment representing topic in requested syntax<br />default is XTM fragment with all URI identifiers, names, occurrences, and associations<br />in default view types and scopes on these constructs are only identified by one <*Ref xlink:href=“...”/> XTM element<br />the same goes for associated topics<br />
  94. 94. get-topic-page<br />Returns link information about a topic<br />that is, where does the server present this topic<br />mainly useful for realizing the portal integration scenario<br />result information contains metadata about server setup<br />
  95. 95. get-topic-page<br />Parameters<br />identifier: a set of URIs (subject identifiers of wanted topic)<br />subject: a set of URIs (subject locators of wanted topic)<br />item: a set of URIs (item identifiers of wanted topic)<br />topicmap: identifier for topic map being queried<br />syntax: string identifying desired Topic Maps syntax in response<br />Response is a topic map fragment<br />[oks : tmrap:server = "OKS Samplers local installation"]<br />[opera : tmrap:topicmap = "The Italian Opera Topic Map"]<br /> {opera, tmrap:handle, [[opera.xtm]]}<br />tmrap:contained-in(oks : tmrap:container, opera : tmrap:containee)<br />tmrap:contained-in(opera : tmrap:container, view : tmrap:containee)<br />tmrap:contained-in(opera : tmrap:container, edit : tmrap:containee)<br />[view : tmrap:view-page %"http://localhost:8080/omnigator/models/..."]<br />[edit : tmrap:edit-page %"http://localhost:8080/ontopoly/enter.ted?..."]<br />[russia = "Russia” @"http://www.topicmaps.org/xtm/1.0/country.xtm#RU"]<br />
  96. 96. get-tolog<br />Returns query results<br />main use is to extract larger chunks of the topic map to the client for presentation<br />more flexible than get-topic<br />can achieve more with less network traffic<br />
  97. 97. get-tolog<br />Parameters<br />tolog: tolog query<br />topicmap: identifier for topic map being queried<br />syntax: string identifying desired syntax of response<br />view: string identifying TM-Views view used to define fragment<br />Response<br />if syntax is“tolog”<br />an XML representation of the query result<br />useful if order of results matter<br />otherwise, a topic map fragment containing multiple topics is returned<br />as for get-topic<br />
  98. 98. add-fragment<br />Adds information to topic map on the server<br />does this by merging in a fragment<br />Parameters<br />fragment: topic map fragment<br />topicmap: identifier for topic map being added to<br />syntax: string identifying syntax of request fragment<br />Result<br />fragment imported into named topic map<br />
  99. 99. update-topic<br />Can be used to update a topic<br />add-fragment only adds information<br />update sets the topic to exactly the uploaded information<br />Parameters<br />topicmap: the topic map to update<br />fragment: fragment containing the new topic<br />syntax: syntax of the uploaded fragment<br />identifier: a set of URIs (subject identifiers of wanted topic)<br />subject: a set of URIs (subject locators of wanted topic)<br />item: a set of URIs (item identifiers of wanted topic)<br />Update happens using TMSync<br />
  100. 100. delete-topic<br />Removes a topic from the server<br />Parameters<br />identifier: a set of URIs (subject identifiers of wanted topic)<br />subject: a set of URIs (subject locators of wanted topic)<br />item: a set of URIs (item identifiers of wanted topic)<br />topicmap: identifier for topic map being queried<br />Result<br />deletes the identified topic<br />includes all names, occurrences, and associations<br />
  101. 101. tolog-update<br />Runs a tolog update statement<br />Parameters<br />topicmap: topic map to update<br />statement: tolog statement to run<br />Runs the statement & commits the change<br />
  102. 102. HTTP binding basics<br />The mapping requires a base URL<br />e.g http://localhost:8080/tmrap/<br />This is used to send requests<br />http://localhost:8080/tmrap/method?param1=value1&...<br />GET is used for requests that do not cause state changes<br />POST for requests that do<br />Responses returned in response body<br />
  103. 103. Exercise #1: Retrieve a topic<br />Use the get-topic request to retrieve a topic from the server<br />base URL is http://localhost:8080/tmrap/<br />find the identifying URI in Omnigator<br />just print the retrieved fragment to get a look at it<br />Note: you must escape the “#” character in URIs<br />otherwise it is interpreted as the anchor and not transmitted at all<br />escape sequence: %23<br />Note: you must specify the topic map ID<br />otherwise results will only be returned from loaded topic maps<br />in other words: if the topic map isn’t loaded, you get no results<br />
  104. 104. Solution #1 (in Python)<br />import urllib<br />BASE = "http://localhost:8080/tmrap/tmrap/"<br />psi = "http://www.topicmaps.org/xtm/1.0/country.xtm%23RU"<br />inf = urllib.urlopen(BASE + "get-topic?identifier=" + psi)<br />print inf.read()<br />inf.close()<br />
  105. 105. Solution #1 (response)<br /> <topicMap xmlns="http://www.topicmaps.org/xtm/1.0/" <br /> xmlns:xlink="http://www.w3.org/1999/xlink"><br /> <topic id="id458"><br /> <instanceOf><br /> <subjectIndicatorRef xlink:href="http://psi.ontopia.net/geography/#country"/><br /> </instanceOf><br /> <subjectIdentity><br /> <subjectIndicatorRef xlink:href="http://www.topicmaps.org/xtm/1.0/country.xtm#RU"/><br /> <topicRef xlink:href="file:/.../WEB-INF/topicmaps/geography.xtmm#russia"/><br /> </subjectIdentity><br /> <baseName><br /> <baseNameString>Russia</baseNameString><br /> </baseName><br /> </topic><br />
  106. 106. Processing XTM with XSLT<br />This is possible, but unpleasant<br />the main problem is that the XML is phrased in terms of Topic Maps, not in domain terms<br />this means that all the XPath will talk about “topic”, “association”, ... and not “person”, “works-for” etc<br />The structure is also complicated<br />this makes queries complicated<br />for example, the XPath to traverse an association looks like this:<br />//xtm:association<br /> [xtm:member[xtm:roleSpec / xtm:topicRef / @xlink:href = '#employer']<br /> [xtm:topicRef / @xlink:href = concat('#', $company)]]<br /> [xtm:instanceOf / xtm:topicRef / @xlink:href = '#employed-by']<br />
  107. 107. TM/XML<br />Non-standard XML syntax for Topic Maps<br />defined by Ontopia (presented at TMRA’05)<br />implemented in the OKS<br />XSLT-friendly<br />much easier to process with XSLT than XTM<br />can be understood by developers who do not understand Topic Maps<br />dynamic domain-specific syntaxes instead of generic syntax<br />predictable (can generate XML Schema from TM ontology)<br />
  108. 108. TM/XML example<br /><topicmap ... reifier="tmtopic"><br /> <topicmap id="tmtopic"><br /> <iso:topic-name><tm:value>TM/XML example</tm:value> </iso:topic-name> <br /> <dc:description>An example of the use of TM/XML.</dc:description><br /> </topicmap><br /> <person id="lmg"><br /> <iso:topic-name><tm:value>Lars Marius Garshol</tm:value><br /> <tm:variant scope="core:sort">garshol, lars marius</tm:variant><br /> </iso:topic-name><br /> <homepage datatype="http://www.w3.org/2001/XMLSchema#anyURI"<br /> >http://www.garshol.priv.no</homepage><br /> <created-by role="creator" topicref="tmtopic" otherrole="work"/><br /> <presentation role="presenter"><br /> <presented topicref="tmxml"/> <br /> <event topicref="tmra05"/><br /> </presentation><br /> </person><br /></topicmap><br />
  109. 109. tmphoto<br />Category<br />Person<br />Photo<br />Event<br />Location<br />http://www.garshol.priv.no/tmphoto/<br />A topic map to organize my personal photos<br />contains ~15,000 photos<br />A web gallery runs on Ontopia<br />on www.garshol.priv.no<br />
  110. 110. tmtools<br />http://www.garshol.priv.no/tmtools/<br />Organization<br />An index of Topic Maps tools<br />organized as shown on the right<br />Again, web application for browsing<br />screenshots below<br />Person<br />Software<br />product<br />Platform<br />Category<br />Technology<br />
  111. 111. The person page<br />Boring! No content.<br />
  112. 112. And in tmphoto...<br />
  113. 113. get-illustration<br />A web service in tmphoto<br />receives the PSI of a person<br />then automatically picks a suitable photo of that person<br />Based on<br />vote score for photos,<br />categories (portrait),<br />other people in photo<br />...<br />The service returns<br />a topic map fragment with links to the person page and a few different sizes of the selected photo<br />http://www.garshol.priv.no/blog/183.html<br />
  114. 114. get-illustration<br />Hmmm. Scores, categories, people in photo, ...<br />Do you have a photo of<br />http://psi.ontopedia.net/Benjamin_Bock ?<br />http://www.garshol.priv.no/tmphoto/get-illustration?identifier=http://psi.on....<br />tmphoto<br />tmtools<br />Topic map<br />fragment<br />
  115. 115. Voila...<br />
  116. 116. Points to note<br />No hard-wiring of links<br />just add identifiers when creating people topics<br />photos appear automatically<br />if a better photo is added later, it’s replaced automatically<br />No copying of data<br />no duplication, no extra maintenance<br />Very loose binding<br />nothing application-specific<br />Highly extensible<br />once the identifiers are in place we can easily pull in more content from other sources<br />
  117. 117. My blog<br />Has more content about<br />people (tmphoto & tmtools),<br />events (tmphoto),<br />tools (tmtools),<br />technologies (tmtools)<br />Should be available in those applications<br />
  118. 118. Solution<br />My blog posts are tagged<br />but the tags are topics, which can have PSIs<br />these PSIs are used in tmphoto and tmtools, too<br />The get-topic-page request lets tmphoto & tmtools ask the blog for links to relevant posts<br />given identifiers for a topic, returns links to pages about that topic<br />http://www.garshol.priv.no/blog/145.html<br />
  119. 119. get-topic-page<br />Do you have pages about<br />http://psi.ontopedia.net/TMRA_2008 ?<br />http://www.garshol.priv.no/blog/get-topic-page?identifier=http://psi.on....<br />Blog<br />tmphoto<br />Topic map<br />fragment<br />Topics linking to<br />individual blog posts<br />
  120. 120. In tmphoto<br />
  121. 121. Making web applications<br />Navigator Framework<br />
  122. 122. Ontopia Navigator Framework<br />Java API for interacting with TM repository<br />JSP tag library<br />based on tolog<br />kind of like XSLT in JSP with tolog instead of XPath<br />has JSTL integration<br />Undocumented parts<br />web presentation components<br />some wrapped as JSP tags<br />want to build proper portlets from them<br />
  123. 123. How it works<br />Web server with JSP containere.g. Apache Tomcat<br />JSP page<br />Browser<br />Topic MapEngine<br />TagLibraries<br />JSP page<br />Browser<br />JSP page<br />Browser<br />JSP page<br />Browser<br />Topic Map<br />
  124. 124. The two tag libraries<br /><ul><li>tolog
  125. 125. makes up nearly the entire framework
  126. 126. used to extract information from topic maps
  127. 127. lets you execute tolog queries to extract information from the topic map
  128. 128. looping and control flow structures
  129. 129. template
  130. 130. used to create template pages
  131. 131. separates layout and structure from content
  132. 132. not Topic Maps-aware
  133. 133. optional, but recommended</li></li></ul><li>How the tag libraries work<br /><ul><li>The topic map engine holds a registry of topic maps
  134. 134. collected from the tm-sources.xml configuration file
  135. 135. each topic map has its own id (usually the file name)
  136. 136. Each page also holds a set of variable bindings
  137. 137. each variable holds a collection of objects
  138. 138. objects can be topics, base names, locators, strings, ...
  139. 139. Tags access variables
  140. 140. some tags set the values of variables, while others use them</li></li></ul><li>Building a JSP page<br /><ul><li>The <%@ taglib ... %> tags declare your tag libraries
  141. 141. Tells the page which tag library to include and binds it to a prefix
  142. 142. Prefixes are used to qualify the tags (and avoid name collisions)
  143. 143. Use the <tolog:context> tag around the entire page
  144. 144. The "topicmap" attribute specifies the ID of the current topic map
  145. 145. The first time you access the page in your browser the page gets compiled
  146. 146. If you modify the page then it will be recompiled the next time it is accessed</li></li></ul><li>http://www.ontopia.net/operamap<br />
  147. 147. Navigator tag library example<br /> <%-- assume variable 'composer' is already set --%><br /><p><b>Operas:</b><br/><tolog:foreach query=”composed-by(%composer% : composer, $OPERA : opera), { premiere-date($OPERA, $DATE) }?”> <li> <a href="opera.jsp?id=<tolog:id var="OPERA"/>”<br /> ><tolog:out var="OPERA"/></a><br /> <tolog:if var="DATE"> <tolog:out var="DATE"/> </tolog:if> </li></tolog:foreach></p><br />
  148. 148. Elmer Preview<br />
  149. 149.
  150. 150.
  151. 151.
  152. 152. Possible configuration<br />Application directories<br />webapps<br />myApp/<br />*.jsp<br />omnigator/<br />WEB-INF/<br />config/<br />*.xml<br />i18n/<br />topicmaps/<br />*.xtm, *.ltm<br />web.xml<br />
  153. 153. The navigator configuration files<br /><ul><li>web.xml
  154. 154. where to find the other files, plus plug-ins
  155. 155. tm-sources.xml
  156. 156. tells the navigator where to find topic maps
  157. 157. log4j.properties
  158. 158. configuration of the log4j logging
  159. 159. More details in the "Configuration Guide" document</li></li></ul><li>More information<br />Navigator Framework Configuration Guide<br />http://www.ontopia.net/doc/current/doc/navigator/config.html<br />Navigator Framework Developer's Guide<br />http://www.ontopia.net/doc/current/doc/navigator/navguide.html<br />Navigator Framework Tag Library Reference<br />http://www.ontopia.net/doc/current/doc/navigator/tolog-taglib.html<br />
  160. 160. ...<br />Automated classification<br />
  161. 161. What is automated classification?<br />Create parts of a topic map automatically<br />using the text in existing content as the source<br />not necessarily 100% automatic; user may help out<br />A hard task<br />natural language processing is very complex<br />result is never perfect<br />However, it’s possible to achieve some results<br />
  162. 162. Why automate classification?<br />Creating a topic map requires intellectual effort<br />that is, it requires work by humans<br />Human effort = cost<br />added value must be sufficient to justify the cost<br />in some cases either<br />the cost is too high, or<br />the value added is too limited<br />The purpose of automation is to lower the cost<br />this increases the number of cases where the use of Topic Maps is justified<br />
  163. 163. Automatable tasks<br />Project<br />Person<br />Department<br />Worked on<br />Worked on<br />Jane Doe<br />worked on<br />employed in<br />XYZ Project<br />IT group<br />Ontology<br />hard<br />depends on requirements<br />one time only<br />Instance data<br />hard<br />usually exists in other sources<br />Document keywords<br />easier<br />frequent operation<br />usually no other sources<br />
  164. 164. Two kinds of categorization<br />Broad:<br />Environment, Crisis management<br />Narrow:<br />Water, Norway, drought, Drought Act, Cloud seeding, Morecambe Bay<br />Broad categorization<br />categories are broadly defined<br />include many different subjects<br />Narrow categorization<br />uses very specific keywords<br />each keyword is a single subject<br />
  165. 165. What it does<br />Extract keywords from content<br />goal is to use these for classification<br />Not entity recognition<br />we only care about identifying what the content is about<br />Uses statistical approach<br />no attempt at full formal parsing of the text<br />
  166. 166. Steps of operation<br />Identify format<br />then, extract the text<br />Identify language<br />then, remove stop words<br />stem remaining words<br />Classify<br />can use terms from preexisting Topic Maps<br />exploits knowledge of the language<br />Return proposed keywords<br />
  167. 167. Example of keyword extraction<br />topic maps 1.0<br />metadata 0.57<br />subject-based class. 0.42<br />Core metadata 0.42<br />faceted classification 0.34<br />taxonomy 0.22<br />monolingual thesauri 0.19<br />controlled vocabulary 0.19<br />Dublin Core 0.16<br />thesauri 0.16<br />Dublin 0.15<br />keywords 0.15<br />
  168. 168. Example #2<br />Automated classification 1.0 5<br />Topic Maps 0.51 14<br />XSLT 0.38 11<br />compound keywords 0.29 2<br />keywords 0.26 20<br />Lars 0.23 1<br />Marius 0.23 1<br />Garshol 0.22 1<br />...<br />
  169. 169. So how could this be used?<br />To help users classify new documents in a CMS interface<br />suggest appropriate keywords, screened by user before approval<br />Automate classification of incoming documents<br />this means lower quality, but also lower cost<br />Get an overview of interesting terms in a document corpus<br />classify all documents, extract the most interesting terms<br />this can be used as the starting point for building an ontology<br />(keyword extraction only)<br />
  170. 170. Example user interface<br />The user creates an article<br />this screen then used to add keywords<br />user adjusts the proposals from the classifier<br />
  171. 171. Interfaces<br />java net.ontopia.topicmaps.classify.Chew<br /><topicmapuri><br /><inputfile><br />produces textual output only<br />net.ontopia.topicmaps.classify.SimpleClassifier<br />classify(uri, topicmap) -> TermDatabase<br />classify(uri) -> TermDatabase<br />
  172. 172. Supported formats and languages<br />XML (any schema)<br />HTML (non-XML)<br />PDF<br />Word (.doc, .docx)<br />PowerPoint (.ppt, .pptx)<br />Plain text<br />English<br />Norwegian<br />
  173. 173. Visualization of Topic Maps<br />Vizigator<br />
  174. 174. The Vizigator<br />Graphical visualization of Topic Maps<br />Two parts<br />VizDesktop: Swing desktop app for configuration<br />Vizlet: Java applet for web deployment<br />Configuration stored in XTM file<br />
  175. 175. The uses of visualization<br />Not really suitable for navigation<br />doesn't work for all kinds of data<br />Great for seeing the big picture<br />
  176. 176. Without configuration<br />
  177. 177. With configuration<br />
  178. 178. VizDesktop<br />
  179. 179. The Vizigator<br />The Vizigator uses TMRAP<br />the Vizlet runs in the browser (on the client)<br />a fragment of the topic map is downloaded from the server<br />the fragment is grown as needed<br />Server<br />TMRAP<br />
  180. 180. Embedding the Vizlet<br />Set up TMRAP service<br />Add ontopia-vizlet.jar<br />Add necessary HTML<br /> <applet code="net.ontopia.topicmaps.viz.Vizlet.class"<br /> archive="ontopia-vizlet.jar"><br /> <param name="tmrap" value="/omnigator/plugins/viz/"><br /> <param name="config" value="/omnigator/plugins/viz/config.jsp?tm=<%= tmid %>"><br /> <param name="tmid" value="<%= tmid %>"><br /> <param name="idtype" value="<%= idtype %>"><br /> <param name="idvalue" value="<%= idvalue %>"><br /> <param name="propTarget" value="VizletProp"><br /> <param name="controlsVisible" value="true"><br /> <param name="locality" value="1"><br /> <param name="max-locality" value="5"><br /></applet><br />
  181. 181. Topic Maps debugger<br />Omnigator<br />
  182. 182. Omnigator<br />Generic Topic Maps browser<br />very useful for seeing what's in a topic map<br />the second-oldest part of Ontopia<br />Contains other features beyond simple browsing<br />statistics<br />management console<br />merging<br />tolog querying/updates<br />export<br />
  183. 183. Ontology designer and editor<br />Ontopoly<br />
  184. 184. Ontopoly<br />A generic Topic Maps editor, in two parts<br />ontology editor: used to create the ontology and schema<br />instance editor: used to enter instances based on ontology<br />Features<br />works with both XTM files and topic maps stored in RDBMS backend<br />supports access control to administrative functions, ontology, and instance editors<br />existing topic maps can be imported<br />parts of the ontology can be marked as read-only, or hidden<br />
  185. 185. Ontology designer<br />Create ontology based on<br />topic, association, name, occurrence, and role types<br />Supports iterative ontology development<br />modify and prototype the ontology until it's right<br />Supports ontology annotation<br />add fields to topic types, for example<br />Supports views<br />define restricted views of certain topic types<br />
  186. 186. Instance editor<br />Configured by the ontology editor<br />shows topics as defined by the ontology<br />Has several ways to pick associations<br />drop-down list<br />by search<br />from hierarchy<br />Avoids conflicts<br />pages viewed by one user are locked to others<br />
  187. 187. Ontopoly is embeddable<br />The Ontopoly instance editor can be embedded<br />basically, the main panel can be inserted into another web application<br />uses an iframe<br />Requires only ID of topic being edited<br />can also be restricted to a specific view<br />Makes it possible to build easier-to-use editors<br />so users don't have to learn all of Ontopoly<br />
  188. 188. Adding content features<br />CMS integrations<br />
  189. 189. CMS integration<br />The best way to add content functionality to Ontopia<br />the world doesn’t need another CMS<br />better to reuse those which already exist<br />So far two integrations exist<br />Escenic<br />OfficeNet Knowledge Portal<br />more are being worked on<br />
  190. 190. Implementation<br />A CMS event listener<br />the listener creates topics for new CMS articles, folders, etc<br />the mapping is basically the design of the ontology used by this listener<br />Presentation integration<br />it must be possible to list all topics attached to an article<br />conversely, it must be possible to list all articles attached to a topic<br />how close the integration needs to be here will vary, as will the difficulty of the integration<br />User interface integration<br />it needs to be possible to attach topics to an article from within the normal CMS user interface<br />this can be quite tricky<br />Search integration<br />the Topic Maps search needs to also search content in the CMS<br />can be achieved by writing a tolog plug-in<br />
  191. 191. Articles as topics<br />is about<br />Elections<br />New city council appointed<br />Goal: associate articles with topics<br />mainly to say what they are about<br />typically also want to include other metadata<br />Need to create topics for the articles to do this<br />in fact, a general CMS-to-TM mapping is needed<br />must decide what metadata and structures to include<br />
  192. 192. Mapping issues<br />Article topics<br />what topic type to use?<br />title becomes name? (do you know the title?)<br />include author? include last modified? include workflow state?<br />should all articles be mapped?<br />Folders/directories/sections/...<br />should these be mapped, too?<br />one topic type for all folders/.../.../...?<br />if so, use associations to connect articles to folders<br />use associations to reproduce hierarchical folder structure<br />Multimedia objects<br />should these be included?<br />what topic type? what name? ...<br />
  193. 193. Two styles of mappings<br />Articles as articles<br />Topic represents only the article<br />Topic type is some subclass of “article”<br />“Is about” association connects article into topic map<br />Fields are presentational<br />title, abstract, body<br />Articles as concepts<br />Topic represents some real-world subject (like a person)<br />article is just the default content about that subject<br />Type is the type of the subject (person)<br />Semantic associations to the rest of the topic map<br />works in department, has competence, ...<br />Fields can be semantic<br />name, phone no, email, ...<br />
  194. 194. Article as article<br />Article about building of a new school<br />Is about association to “Primary schools”<br />Topic type is “article”<br />
  195. 195. Article as concept<br />Article about a sports hall<br />Article really represents the hall<br />Topic type is “Location”<br />Associations to<br /><ul><li>city borough
  196. 196. events in the location
  197. 197. category “Sports”</li></li></ul><li>
  198. 198.
  199. 199.
  200. 200.
  201. 201.
  202. 202. Ontopia/Liferay<br />An integration with the Liferay CMS and portal is in progress<br />presented Friday 1130-1150 in Schiller 2<br />
  203. 203. Two projects<br />Real-life usage<br />
  204. 204. The project<br />A new citizen’s portal for the city administration<br />strategic decision to make portal main interface for interaction with citizens<br />as many services as possible are to be moved online<br />Big project<br />started in late 2004, to continue at least into 2008<br />~5 million Euro spent by launch date<br />1.7 million Euro budgeted for 2007<br />Topic Maps development is a fraction of this (less than 25%)<br />Many companies involved<br />Bouvet/Ontopia<br />Avenir<br />KPMG<br />Karabin<br />Escenic<br />
  205. 205. Simplified original ontology<br />Service catalog<br />Escenic (CMS)<br />LOS<br />Form<br />Article<br />nearly<br />everything<br />Category<br />Service<br />Subject<br />Department<br />Borough<br />External<br />resource<br />Employee<br />Payroll++<br />
  206. 206. Data flow<br />Ontopoly<br />Ontopia<br />Escenic<br />LOS<br />Integration<br />TMSync<br />DB2TM<br />Fellesdata<br />Payroll<br />(Agresso)<br />Dexter/Extens<br />Service<br />Catalog<br />
  207. 207. Conceptual architecture<br />Data<br />sources<br />Oracle Portal<br />Application<br />Ontopia<br />Escenic<br />Oracle Database<br />
  208. 208. The portal<br />
  209. 209. Technical architecture<br />
  210. 210. NRK/Skole<br />Norwegian National Broadcasting (NRK)<br />media resources from the archives<br />published for use in schools<br />integrated with the National Curriculum<br />In production<br />delayed by copyright wrangling<br />Technologies<br />OKS<br />Polopoly CMS<br />MySQL database<br />Resin application server<br />
  211. 211. Curriculum-based browsing (1)<br />Curriculum<br />Social studies<br />High school<br />
  212. 212. Curriculum-based browsing (2)<br />Gender roles<br />
  213. 213. Curriculum-based browsing (3)<br />Feminist movement in the 70s and 80s<br />Changes to the family in the 70s<br />The prime minister’s husband<br />Children choosing careers<br />Gay partnerships in 1993<br />
  214. 214. One video (prime minister’s husband)<br />Metadata<br />Subject<br />Person<br />Related<br />resources<br />Description<br />
  215. 215. Conceptual architecture<br />Polopoly<br />HTTP<br />Ontopia<br />MediaDB<br />Grep<br />DB2TM<br />TMSync<br />RDBMS backend<br />MySQL<br />Editors<br />
  216. 216. Implementation<br />Domain model in Java<br />Plain old Java objects built on<br />Ontopia’s Java API<br />tolog<br />JSP for presentation<br />using JSTL on top of the domain model<br />Subversion for the source code<br />Maven2 to build and deploy<br />Unit tests<br />
  217. 217. What we’d like to see<br />The future<br />
  218. 218. The big picture<br />Auto-class.<br />A.N.other<br />A.N.other<br />Other<br />CMSs<br />A.N.other<br />A.N.other<br />DB2TM<br />Portlet support<br />OKP<br />XML2TM<br />Engine<br />CMSintegration<br />Data <br />integration<br />Escenic<br />Taxon.import<br />Ontopoly<br />Web<br />service<br />
  219. 219. CMS integrations<br />The more of these, the better<br />Candidate CMSs<br />Liferay (being worked on at Bouvet)<br />Alfresco <br />Magnolia<br />Inspera<br />JSR-170 Java Content Repository<br />CMIS (OASIS web service standard)<br />
  220. 220. Portlet toolkit<br />Subversion contains a number of “portlets”<br />basically, Java objects doing presentation tasks<br />some have JSP wrappers as well<br />Examples<br />display tree view<br />list of topics filterable by facets<br />show related topics<br />get-topic-page via TMRAP component<br />Not ready for prime-time yet<br />undocumented<br />incomplete<br />
  221. 221. Ontopoly plug-ins<br />Plugins for getting more data from externals<br />TMSync import plugin<br />DB2TM plugin<br />Subj3ct.com plugin<br />adapted RDF2TM plugin<br />classify plugin<br />...<br />Plugins for ontology fragments<br />menu editor, for example<br />
  222. 222. TMCL<br />Now implementable<br />We’d like to see<br />an object model for TMCL (supporting changes)<br />a validator based on the object model<br />Ontopoly import/export from TMCL (initially)<br />refactor Ontopoly API to make it more portable<br />Ontopoly ported to use TMCL natively (eventually)<br />
  223. 223. Things we’d like to remove<br />OSL support<br />Ontopia Schema Language<br />Web editor framework<br />unfortunately, still used by some major customers<br />Fulltext search<br />the old APIs for this are not really of any use<br />
  224. 224. Management interface<br />Import topic maps (to file or RDBMS)<br />
  225. 225. What do you think?<br />Suggestions?<br />Questions?<br />Plans?<br />Ideas?<br />
  226. 226. Setting up the developer environment<br />Getting started<br />
  227. 227. If you are using Ontopia...<br />...simply download the zip, then<br />unzip,<br />set the classpath,<br />start the server, ...<br />...and you’re good to go<br />
  228. 228. If you are developing Ontopia...<br />You must have<br />Java 1.5 (not 1.6 or 1.7 or ...)<br />Ant 1.6 (or later)<br />Ivy 2.0 (or later)<br />Subversion<br />Then<br />check out the source from Subversion<br />svn checkout http://ontopia.googlecode.com/svn/trunk/ ontopia-read-only<br />ant bootstrap<br />ant dist.jar.ontopia<br />ant test<br />ant dist.ontopia<br />
  229. 229. Beware<br />This is fun, because<br />you can play around with anything you want<br />e.g, my build has a faster TopicIF.getRolesByType<br />you can track changes as they happen in svn<br />However, you’re on your own<br />if it fails it’s kind of hard to say why<br />maybe it’s your changes, maybe not<br />For production use, official releases are best<br />
  230. 230. Participating etc<br />The project<br />
  231. 231. Our goal<br />To provide the best toolkit for building Topic Maps-based applications<br />We want it to be<br />actively maintained,<br />bug-free,<br />scalable,<br />easy to use,<br />well documented,<br />stable,<br />reliable<br />
  232. 232. Our philosophy<br />We want Ontopia to provide as much useful more-or-less generic functionality as possible<br />New contributions are generally welcome as long as<br />they meet the quality requirements, and<br />they don’t cause problems for others<br />
  233. 233. The sandbox<br />There’s a lot of Ontopia-related code which does not meet those requirements<br />some of it can be very useful,<br />someone may pick it up and improve it<br />The sandbox is for these pieces<br />some are in Ontopia’s Subversion repository,<br />others are maintained externally<br />To be “promoted” into Ontopia a module needs<br />an active maintainer,<br />to be generally useful, and<br />to meet certain quality requirements<br />
  234. 234. Communications<br />Join the mailing list(s)!<br />http://groups.google.com/group/ontopia<br />http://groups.google.com/group/ontopia-dev<br />Google Code page<br />http://code.google.com/p/ontopia/<br />note the “updates” feed!<br />Blog<br />http://ontopia.wordpress.com<br />Twitter<br />http://twitter.com/ontopia<br />
  235. 235. Committers<br />These are the people who run the project<br />they can actually commit to Subversion<br />they can vote on decisions to be made etc<br />Everyone else can<br />use the software as much as they want,<br />report and comment on issues,<br />discuss on the mailing list, and<br />submit patches for inclusion<br />
  236. 236. How to become a committer<br />Participate in the project!<br />that is, get involved first<br />let people get to know you, show some commitment<br />Once you’ve gotten some way into the project you can ask to become a committer<br />best if you have provided some patches first<br />Unless you’re going to commit changes there’s no need to be a committer<br />
  237. 237. Finding a task to work on<br />Report bugs!<br />they exist. if you find any, please report them.<br />Look at the open issues<br />there is always testing/discussion to be done<br />Look for issues marked “newbie”<br />http://code.google.com/p/ontopia/issues/list?q=label:Newbie<br />Look at what’s in the sandbox<br />most of these modules need work<br />Scratch an itch<br />if there’s something you want fixed/changed/added...<br />
  238. 238. How to fix a bug<br />First figure out why you think it fails<br />Then write a test case<br />based on your assumption<br />make sure the test case fails (test before you fix)<br />Then fix the bug<br />follow the coding guidelines (see wiki)<br />Then run the test suite<br />verify that you’ve fixed the bug<br />verify that you haven’t broken anything<br />Then submit the patch<br />
  239. 239. The test suite<br />Lots of *.test packages in the source tree<br />3795 test cases as of right now<br />test data in ontopia/src/test-data<br />some tests are generators based on files<br />some of the test files come from cxtm-tests.sf.net<br />Run with<br />ant test<br />java net.ontopia.test.TestRunner src/test-data/config/tests.xml test-group<br />
  240. 240. Source tree structure<br />net.ontopia.<br />utils various utilities<br />test various test support code<br />infoset LocatorIF code + cruft<br />persistence OR-mapper for RDBMS backend<br />product cruft<br />xml various XML-related utilities<br />topicmaps next slides<br />
  241. 241. Source tree structure<br />net.ontopia.topicmaps.<br />core core engine API<br />impl engine backends + utils<br />utils utilities (see next slide)<br />cmdlineutils command-line tools<br />entry TM repository<br />nav + nav2 navigator framework<br />query tolog engine<br />viz<br />classify <br />db2tm<br />webed cruft<br />
  242. 242. Source tree structure<br />net.ontopia.topicmaps.utils<br />* various utility classes<br />ltm LTM reader and writer<br />ctm CTM reader<br />rdf RDF converter (both ways)<br />tmrap TMRAP implementation<br />
  243. 243. Let’s write some code!<br />
  244. 244. The engine<br />The core API corresponds closely to the TMDM<br />TopicMapIF, TopicIF, TopicNameIF, ...<br />Compile with<br />ant init compile.ontopia<br />.class files go into ontopia/build/classes<br />ant dist.ontopia.jar # makes a jar<br />
  245. 245. The importers<br />Main class implements TopicMapReaderIF<br />usually, this lets you set up configuration, etc<br />then uses other classes to do the real work<br />XTM importers<br />use an XML parser<br />main work done in XTM(2)ContentHandler<br />some extra code for validation and format detection<br />CTM/LTM importers<br />use Antlr-based parsers<br />real code in ctm.g/ltm.g<br />All importers work via the core API<br />
  246. 246. Find an issue in the issue tracker<br />(Picking one with “Newbie” might be good, <br />but isn’t necessary)<br />Get set up<br />check out the source code<br />build the code<br />run the test suite<br />Then dig in<br />we’ll help you with any questions you have<br />At the end, submit a patch to the issue tracker<br />remember to use the test suite!<br />

×