UKAD forum 2013: What is an API and what might the Discovery API mean for contributing data to Discovery?


Published on

Slides accompanying the 'UKAD forum 2013: What is an API and what might the Discovery API mean for contributing data to Discovery?' podcast. To listen to this podcast, please copy and paste this link into your browser:

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • This is introduction to API – what API is, what is web API. I am going to talk about Discovery system; what are concepts, how Discovery system implements API and provides access to its data, and are our future plans. There are no special requirements. You can start using Discovery API tight away. You’ll need basic understanding of XML data structure, knowledge of the Catalogue, and web browser.
  • With evolution of the Internet term API started to have different meaning. On the web it is about sending standard requests to the systems and getting textual responses back usually as XML.
  • Data is a big part of the API. Recently it was a big explosion in the amount of data. User contributed content is becoming more and more important.
  • Discovery uses different types of the data sources and complex procedures to transform this data into unified data model, which I am going to cover in the next section.
  • User generated content is important component of the data. It helps to improve existing data, enrich existing data, provide new ways of looking at existing data.
  • Information Asset view in search result.
  • Information Asset View in browse.
  • Detailed Information Asset View. Important – each Information Asset has unique persistent ID.
  • Now with all this knowledge we can look at some real examples of using API.
  • API documentation. Only 6 methods, but it is enough to query data in Discovery system.
  • Let’s have a look at couple possible scenarios.
  • Core is the Discovery Platform, which takes care of data ingest, transformation, storage, and availability. Discovery can present data in some certain ways, but it also provides functions for creating custom data views through its output API. Next steps are to enable collection of the data from different sources, including users’ participation which will help to improve existing content with clear separation of different data sources and different authority types.
  • UKAD forum 2013: What is an API and what might the Discovery API mean for contributing data to Discovery?

    1. 1. Aleks DrozdovEnterprise ArchitectWhat is an API and what might the DiscoveryAPI mean for services contributing data toDiscovery
    2. 2. Agenda• API and Data• Discovery architecture• Getting data into Discovery• Discovery data structure• Discovery API (Scenarios)• Next steps
    3. 3. API and DataApplication Programming InterfaceA language and message format used by an application program tocommunicate with the operating system or some other control program such asa database management system (DBMS) or communications protocol.PersonalcomputerApplication Operating SystemUI API
    4. 4. Web API• When used in the context of web development, an API is typically defined as a set ofHypertext Transfer Protocol (HTTP) request messages.• Response messages, which is usually in an Extensible Markup Language (XML) orJavaScript Object Notation (JSON) format.• While "Web API" is virtually a synonym for web service, the recent trend (so-calledWeb 2.0) has been moving away from Simple Object Access Protocol (SOAP) basedservices towards more direct Representational State Transfer (REST) stylecommunications. Source: WikipediaPersonalcomputerApplication Discovery SystemWebbrowserAPI
    5. 5. Data• Explosion in the amount of data• It is common to mashup data from a number of sources• User contributed data• Since ‘80s processor speed increas of 360• RAM price – from $1000/MB to $25/GB• Reduction in RAM size1956 disk drive-5MB-Refrigerator size-250 pounds
    6. 6. Discovery Architecture
    7. 7. Discovery ArchitectureSystem User InterfaceObject Data StoreDatasets and datatransformations
    8. 8. Getting data into Discovery
    9. 9. Data import - databasesRelational (SQL) databasesNonrelationalObject Data StoreDatadenormalisation/TransformationText inputEXCEL TXT CSVXMLXML XML XML
    10. 10. Data import – user
    11. 11. Discovery data model
    12. 12. Core of the Discovery is an Information Asset object. Information Asset models The National Archives data in thehierarchical way and uses Collections and parent-child relationship. Properties of the Information Asset object don’tchange across different data sets and hierarchies. Different data sources are using different schemas, which can leadto missing properties in the Information Assets. Still, the structure of the Information Asset stays unchanged. Themain challenge in the design Information Asset object is to find minimum common set of properties which will workacross all data sources.Information Asset
    13. 13. Discovery data - Search
    14. 14. Discovery data - Browse
    15. 15. Discovery data - Details
    16. 16. Discovery API
    17. 17. Uri Method Descriptioninformationasset/{id} GET Service at[xml][json]informationasset/{ID}children/{id}/{page} GET Service at[xml][json]children/{ID}/{PAGE}totalchildren/{id} GET Service at[xml][json]totalchildren/{ID}parent/{id} GET Service at[xml][json]Parent/{ID}siblings/{id}/{PAGE} GET Service at[xml][json]siblings/{ID}/{PAGE}search/{PAGE}?{query} GET Service at[xml][json]search//{PAGE}/query={QUERY};all={ALLWORDS};exact={EXACTPHRASE};collection={DEPARTMENTS}&startDate={STARTDATE}&endDate={ENDDATE};formerRef={FORMERREFERENCE}Operations at
    18. 18. Information Asset C10127419
    19. 19. API XML response{"assetView" : {"IAID" : "C10127419","SourceLevelId" : 7,"ParentIAID" : "C2389021","Reference" : "1","CatalogueId" : -5596696,"Language" : "English","LegalStatus" : "Public Record(s)","FormerReferenceDep" : "","FormerReferencePro" : "","Title" : "","MapDesignation" : "","CreatorName" : [ ],"CoveringDates" : "1853 October 8","PhysicalDescriptionExtent" : "","PhysicalDescriptionForm" : "","Dimensions" : "","MapScaleNumber" : 0,"PhysicalCondition" : "","HeldBy" : [ {"Corporate_Body_Name_Text" : "The National Archives, Kew"} ],<IAID>C10127419</IAID><SourceLevelId>7</SourceLevelId><ParentIAID>C2389021</ParentIAID><Reference>1</Reference><CatalogueId>-5596696</CatalogueId><Language>English</Language><LegalStatus>Public Record(s)</LegalStatus><FormerReferenceDep/><FormerReferencePro/><Title/><MapDesignation/><CoveringDates>1853 October 8</CoveringDates><PhysicalDescriptionExtent/><PhysicalDescriptionForm/><Dimensions/><MapScaleNumber>0</MapScaleNumber><PhysicalCondition/><HeldBy><HeldBy><Corporate_Body_Name_Text>The National Archives,Kew</Corporate_Body_Name_Text></HeldBy></HeldBy>XML JSON
    20. 20. Scenarios
    21. 21. Scenario – navigating hierarchy<SearchResultList><AssetView><IAID>C3735</IAID><CitableReference>C 203</CitableReference><Title>Chancery: Petty Bag Office: Various Certificates</Title><Description>Chancery: Petty Bag Office: Various Certificates. Miscellaneous returns and certificates in the custody of the Petty Bag Office,including inquisitions and presentments by jurors concerning the killing, sale or eating of meat in Lent; records of the appointment of collectorsof subsidies, possibly connected with their own tax liabilities; documents relating to the sequestration of estates of delinquents; certificatesconfirming the surrender of offices; certificates recording the oath of allegiance, supremacy and abjuration sworn by attornies; certificatesnaming those who failed to swear the oaths required by the Security of the Sovereign Act 1714; certificates concerning concealed deaths underthe Cestui que Vie Act 1707; certificates attesting that members of Parliament met the property qualification required by the Parliament Act1710.</Description><Places /><People /><Subjects><string>C10106 Taxation</string><string>C10039 Food and drink</string></Subjects><StartDate>01-01-1558</StartDate><EndDate>31-12-1839</EndDate><References /><URLParams>066/1/C43/C598/C3735</URLParams><Department>C</Department></AssetView> forInformation Asset
    22. 22. Scenario – navigating hierarchy109 total number ofchildren
    23. 23. Scenario – navigating hierarchy<InformationAssetIdentity><ObjectId>4f24d14055e0f1922313380a</ObjectId><IAID>C3413406</IAID><ParentIAID>C3735</ParentIAID><SourceLevelId>6</SourceLevelId><ReferenceComponent>1</ReferenceComponent><Title>Description available at other catalogue level</Title><PhysicalDescription> </PhysicalDescription><CoveringDates /><HasChildren>true</HasChildren><ChildrenTotal>0</ChildrenTotal><ContextPath /></InformationAssetIdentity> all children
    24. 24. Scenario - searchBuild API request request to APIReceive list ofresultsPage through resultsRetrieve InformationAsset data<TotalResults>53</TotalResults><SearchResultList><AssetView><IAID>C2341588</IAID><CitableReference>DO 119/1373</CitableReference><Title>Ritual murder and &lt;spanclass="highlight"&gt;witchcraft&lt;/span&gt;; Rex vs Chiefs Bereng andGabashane and others</Title><Description>High Commissioner for South Africa, and HighCommissioner for Basutoland, the Bechuanaland Protectorate andSwaziland, and UK High Commissioner for the Union of South Africa:Correspondence. Basutoland. Ritual murder and &lt;spanclass="highlight"&gt;witchcraft&lt;/span&gt;; Rex vs Chiefs Bereng andGabashane and others.</Description><Places /><People /><Subjects><string>C10125 Africa</string><string>C10117 Witchcraft</string><string>C10023 Crime</string></Subjects><StartDate>01-01-1949</StartDate><EndDate>31-12-1949</EndDate><References><string>9657 II</string></References><URLParams>066/1/C86/C495/C6128/C51353/0/C2341588</URLParams><Department>DO</Department></AssetView>
    25. 25. Next steps
    26. 26. Discovery PlatformExternalDataData Import APIData Export APIExternalDataExternalDataData sharing and reuseUser participation API
    27. 27. Thank you!