Better architecture with semantic integration


Published on

This paper presents an approach to using semantic technolo- gies to achieve better and more flexible integration of IT systems. The author believes that the described approach is applicable to a great many organizations, and that it can lead to far more dynamic IT architectures than what is common today.

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Better architecture with semantic integration

  1. 1. Semantic integration TMRA, 2010-10-01 Lars Marius Garshol
  2. 2. Architectural challenges <ul><li>System architecture is not enterprise architecture </li></ul><ul><ul><li>what's good for a single system is not necessarily good for the enterprise as a whole </li></ul></ul><ul><ul><li>local decisions make sense locally, but not necessarily globally </li></ul></ul><ul><li>Organizational reorganizations </li></ul><ul><ul><li>merges, aquisitions, political reorganizations, ... </li></ul></ul><ul><ul><li>these all have implications for IT architecture </li></ul></ul><ul><li>Master data management </li></ul><ul><ul><li>of course you only have a single customer database </li></ul></ul><ul><ul><li>...until you buy a company that has their own </li></ul></ul><ul><li>Architecture dictators </li></ul><ul><ul><li>are a tempting solution to impose some general structure, </li></ul></ul><ul><ul><li>but generally impede progress at the local level </li></ul></ul>
  3. 3. Lego! <ul><li>The goal is not just an architecture that's correct today </li></ul><ul><li>Because we know the situation will soon change </li></ul><ul><li>The goal is an architecture that's easily adapted to tomorrow's environment </li></ul>
  4. 4. A step-by-step process 1. Reference model 2. Master data 3. Reference data 4. Generic services 6. Search 7. Semantic formats 5. Access control
  5. 5. Mapping the IT landscape <ul><li>Which IT systems exist? </li></ul><ul><li>Which entities do they have? </li></ul><ul><li>What are their properties? </li></ul><ul><li>What web services exist? </li></ul><ul><li>What is their input and output? </li></ul>App Entitet Egenskap Tjeneste Format Step #1
  6. 6. The value of the map <ul><li>Living high-level view of systems and services </li></ul><ul><ul><li>navigation/search/visualization, </li></ul></ul><ul><ul><li>connect to relevant documentation, where it exists, </li></ul></ul><ul><ul><li>far superior to PowerPoint/Word </li></ul></ul><ul><li>The main value is in the structure </li></ul><ul><ul><li>that is, the use of a proper semantic model </li></ul></ul><ul><li>No need to build or buy software </li></ul><ul><ul><li>it can all be done with existing open source components </li></ul></ul>Step #1
  7. 7. Problems with the map <ul><li>There is no connection across systems </li></ul><ul><ul><li>shows that 7 systems have the concept &quot;case&quot; </li></ul></ul><ul><ul><li>but so what? </li></ul></ul><ul><li>What is needed is a cross-mapping </li></ul><ul><ul><li>must show to what degree the 7 concepts overlap </li></ul></ul><ul><ul><li>perhaps there are other concepts that mean the same, but have different names? </li></ul></ul>Step #1
  8. 8. Build a reference model <ul><li>Model central concepts </li></ul><ul><ul><li>entities and their properties </li></ul></ul><ul><ul><li>type hierarchy for both </li></ul></ul><ul><ul><li>independent of any specific system; model the organization's understanding </li></ul></ul><ul><li>This is not a canonical data model </li></ul><ul><ul><li>not a format </li></ul></ul><ul><ul><li>not forcing any systems to actually use the model </li></ul></ul><ul><ul><li>systems can use entities/properties which do not (yet) exist in the model </li></ul></ul>Step #1
  9. 9. What is cohabitation? <ul><li>Law on individual pensions </li></ul><ul><ul><li>LOV-2008-06-27-62, § 3-7 </li></ul></ul><ul><ul><ul><li>Med samboer forstås her a) person som kunden har felles bolig og felles barn med, b) person som kunden lever sammen med i ekteskaps- eller partnerskapslignende forhold når det godtgjøres at forholdet har bestått uavbrutt i de siste fem år før kundens død, og det ikke forelå forhold som ville hindre at lovlig ekteskap eller registrert partnerskap ble inngått. </li></ul></ul></ul><ul><li>Regulation on collecting information </li></ul><ul><ul><li>FOR-2005-07-08-826, punkt 1 </li></ul></ul><ul><ul><ul><li>samboere: personer som lever sammen og har felles barn. </li></ul></ul></ul><ul><li>Law on &quot;vergemål&quot; </li></ul><ul><ul><li>LOV-2010-03-26-9, § 2 </li></ul></ul><ul><ul><ul><li>Med samboere menes i denne loven to personer som bor sammen i et ekteskapslignende forhold. </li></ul></ul></ul><ul><li>... </li></ul>Step #1
  10. 10. Modelling Step #1 Cohabitation Cohabitation PENSION Cohabitation INNH Cohabitation VERGE Cohabitation XXX Cohabitation YYY
  11. 11. Properties, too Step #1 Cohabitation Person cohab 1 cohab 2 start date end date ...
  12. 12. Connect the system data App #1 App #2 App #3 COHAB COHABITERS BPG_COHAB Step #1 Cohabitation Cohabitation PENSION Cohabitation INNH Cohabitation VERGE Cohabitation XXX Cohabitation YYY
  13. 13. Degrees of correlation <ul><li>Perfect correlation </li></ul><ul><ul><li>not unusual, neither is it always the case </li></ul></ul><ul><li>Specialization </li></ul><ul><ul><li>that is, B is a narrower concept than A is </li></ul></ul><ul><li>Overlap </li></ul><ul><ul><li>A and B share a common subset </li></ul></ul><ul><li>Resembles </li></ul><ul><ul><li>A and B are related, but the connection is not clear </li></ul></ul>A B Step #1
  14. 14. Uses of the model <ul><li>We describe to understand </li></ul><ul><ul><li>we want to understand so that we can improve </li></ul></ul><ul><li>Analysis of the architecture </li></ul><ul><ul><li>starting point for a restructuring </li></ul></ul><ul><ul><li>identify master data issues </li></ul></ul><ul><ul><li>etc </li></ul></ul><ul><li>A data dictionary </li></ul><ul><ul><li>useful when converting legacy data </li></ul></ul><ul><ul><li>useful for bug fixing </li></ul></ul><ul><ul><li>key personell no longer have to answer questions all the time </li></ul></ul><ul><ul><li>etc </li></ul></ul>Step #1
  15. 15. From documentation to services <ul><li>So far we've only discussed documentation for humans </li></ul><ul><ul><li>this is highly useful in a number of ways </li></ul></ul><ul><ul><li>but it is only the beginning </li></ul></ul><ul><li>The model has a semantic structure </li></ul><ul><ul><li>therefore we can use it to build new kinds of services </li></ul></ul>Trinn #1
  16. 16. Master data control <ul><li>Pick one system to be the master for each kind of data </li></ul><ul><ul><li>where this can really be centralized </li></ul></ul><ul><li>Other systems needing the data must become clients of the master </li></ul><ul><ul><li>this is a gradual transition </li></ul></ul><ul><li>We also need a protocol which the clients can use to retrieve the data </li></ul>Step #2
  17. 17. A service broker <ul><li>A service which routes requests </li></ul><ul><li>A layer above the ESB </li></ul><ul><ul><li>the ESB takes care of transport </li></ul></ul><ul><ul><li>it might also broker between several ESBs </li></ul></ul><ul><li>Uses its knowledge of information and services </li></ul><ul><li>Decouples clients from servers </li></ul><ul><li>Makes the architecture a lot more flexible </li></ul>Broker Reference model ESB Step #2
  18. 18. Master data protocol <ul><li>Used by clients to retrieve data updates </li></ul><ul><li>Makes it possible to gradually migrate </li></ul><ul><li>Master can change without the clients knowing </li></ul>Broker App #1 App #2 App #3 App #4 Sync request: entity + time Referanse-modell Lookup Atom (SDshare) Step #2
  19. 19. Collect reference data <ul><li>Most systems share a number of fairly static lists </li></ul><ul><ul><li>list of countries, list of diagnosis codes, list of provinces, ... </li></ul></ul><ul><li>There is no reason to maintain this in duplicate in different systems </li></ul><ul><ul><li>the lists also need common identifiers for the items </li></ul></ul><ul><li>The lists might as well go into the reference model </li></ul><ul><ul><li>can be retrieved from there by client systems </li></ul></ul>Step #3
  20. 20. Generic lookup service <ul><li>Would not work &quot;out of the box&quot; </li></ul><ul><li>Must be carefully set up so that it works </li></ul><ul><li>Possible because this is a controlled environment </li></ul>Broker App #1 App #2 App #3 App #4 Service #1 Service #2 Service #3 Give me an entity of type X with ID 23414 Who has data about X? Step #4
  21. 21. Generic translation service <ul><li>Sometimes the client wants format X, but the server can only supply Y </li></ul><ul><ul><li>the broker can find a translator service, and </li></ul></ul><ul><ul><li>ensure that the translation happens automatically </li></ul></ul>X->Y Y->Z X->Z Y->X Z->Y Z->X Broker Step #4
  22. 22. Some science fiction <ul><li>We already have </li></ul><ul><ul><li>the structure of XML formats X and Y described, and </li></ul></ul><ul><ul><li>connected to the reference model </li></ul></ul><ul><li>In some cases we can then generate the translator automatically </li></ul><ul><ul><li>made prototype in 2004 </li></ul></ul><ul><ul><li>it worked! </li></ul></ul><ul><ul><li>but it won't always work </li></ul></ul>Step #4
  23. 23. Impact analysis <ul><li>If we register clients and their requests in the model we know more about the uses of the architecture </li></ul><ul><li>It becomes possible to find the answer to questions like </li></ul><ul><ul><li>&quot;can we stop this service?&quot; </li></ul></ul><ul><ul><li>&quot;does anyone use this format?&quot; </li></ul></ul><ul><ul><li>... </li></ul></ul>Step #4
  24. 24. Access control <ul><li>Rules for this are usually </li></ul><ul><ul><li>not documented anywhere, </li></ul></ul><ul><ul><li>encoded in software all over the enterprise </li></ul></ul><ul><li>It can also be represented in the model </li></ul><ul><ul><li>user groups can be connected to the data they are allowed to access/modify </li></ul></ul><ul><ul><li>there is no need to represent individual users </li></ul></ul><ul><li>All this can be retrieved via web services </li></ul>Step #5
  25. 25. Generic querying with SPARQL <ul><li>More advanced lookup </li></ul><ul><ul><li>using SPARQL as the query language </li></ul></ul><ul><ul><li>can do more than just looking up IDs </li></ul></ul><ul><ul><li>doing queries &quot;into the cloud&quot; </li></ul></ul><ul><li>The reference model is used to interpret the query </li></ul><ul><ul><li>splits it up and delegates to different services </li></ul></ul><ul><ul><li>the broker then assembles the result </li></ul></ul><ul><li>SPARQL is not a very powerful language </li></ul><ul><ul><li>that is why this is possible </li></ul></ul>Trinn #6
  26. 26. Semantic formats &quot;on the wire&quot; <ul><li>Ordinary XML formats are static </li></ul><ul><ul><li>semantic formats are dynamic </li></ul></ul><ul><li>New fields and entity types can be added </li></ul><ul><ul><li>without changing the format </li></ul></ul><ul><ul><li>without confusing recipients </li></ul></ul><ul><li>Transparent support for subtyping </li></ul><ul><ul><li>allowing even more flexibility in interpretation </li></ul></ul><ul><li>Support for merging </li></ul><ul><ul><li>again transparent for recipients </li></ul></ul>Step #7
  27. 27. Scenario App #1 Need all X satisfying certain criteria, in format Y Broker SPARQL, formatY App #2 Database SPARQL XTM SPARQL XTM Merge and filter Translator XTM formatY Step #7 Service #1 Service #2
  28. 28. Conclusion <ul><li>Clients don't need to know where the data are </li></ul><ul><ul><li>much looser coupling </li></ul></ul><ul><ul><li>much easier to restructure </li></ul></ul><ul><li>Clients do not need to relate to the many data models that are in use </li></ul><ul><ul><li>instead they need only refer to the reference model </li></ul></ul><ul><li>Clients do not need to worry about the data formats used by servers </li></ul><ul><ul><li>instead, they simply ask for data in the format they want </li></ul></ul>