Downloaded 20 times
NISO/DCMI September 25 Webinar: Implementing Linked Data in Developing Countries and Low-Resource Conditions
The NISO/DCMI webinar on September 25, 2013, discussed the implementation of linked data in developing countries with limited resources. Key topics included the difficulties of data generation, the significance of using standard vocabularies, and the importance of collaboration among institutions to overcome challenges. The discussion highlighted that linked data technologies must become more accessible and resource-efficient to encourage adoption in low-resource contexts.
More Related Content
Similar to NISO/DCMI September 25 Webinar: Implementing Linked Data in Developing Countries and Low-Resource Conditions
More from National Information Standards Organization (NISO)
NISO/DCMI September 25 Webinar: Implementing Linked Data in Developing Countries and Low-Resource Conditions
- 1. NISO/DCMI Webinar: Implementing LinkedData in Developing Countries and Low-Resource Conditions September 25, 2013 Speakers: Johannes Keizer - Information Systems Officer, Food and Agriculture Organization of the United Nations Caterina Caracciolo - Senior Information Specialist at the Food and Agriculture Organization of the United Nations http://www.niso.org/news/events/2013/dcmi/developing
- 2. Implementing Linked Datain Developing Countries and Low Resource Conditions NISO/DCMI Webminar 25 September, 2013 Caterina Caracciolo, Johannes Keizer {caterina.caracciolo},{johannes.keizer}@fao.org
- 3. Goal of thisWebinar • Overview of Linked data stack and components • LOD in low resource conditions – Possible? Why to do it? • What to think of when doing LOD in low resources • Explain some initiatives to enable LOD in low resources • Exemplify a real world LOD Szenario
- 4. The importance ofthe issue Source: United Nations Population Division, World Population Prospects: The 2010 Revision, medium variant (2011).
- 5.
- 6. • ~ 7000languages http://w3techs.com/technologi es/overview/content_language /all And there is something more ~ 7000 languages
- 7. The world bylanguages spoken www.worldmapper.org
- 8. Let’s get intothe nitty gritty
- 9. Implementing Linked Datain Developing Countries and Low Resource Conditions Part 2 NISO/DCMI Webminar 25 September, 2013 Caterina Caracciolo caterina.caracciolo@fao.org
- 10. Today • A bird’seye view on Linked Data lifecycle, from data consumption to data generation • Discussion on major difficulties, especially in the data generation phase • Some considerations on possible solutions, especially from a strategic and organizational point of view • No ambition to have a comprehensive survey of tools!
- 11. What are lowresource conditions really?
- 12. CPU, memory andtechnology constraints...
- 13. Electricity may beunreliable…
- 14.
- 15.
- 16. Internet connection maybe slow...
- 17. … and dependenton the weather…
- 18. Funding... is always aproblem
- 19. IT competencies… Few ITpeople, over-busy, trained on different technologies, with little or no incentives to learn/adopt new ones
- 20. IT and domain-specific competencies •Usually, complete separation between those working on IT and those working on collecting/analysing/maintaining data (domain specialists) • Domain specialists do not want to spend time changing formats, validating conversions, explaining intended meaning of data etc. – Tendency to consider data as “my” data
- 21.
- 23. Scenario An institution hasdata to publish as Linked Data – Data is produced internally, e.g. list of publications produced by the institution, specimens in the local museum, factsheets on local plants, statistics on production, … – Data may be online or inside somebody’s computer – Typically in some RDB, or spreadsheets in file system
- 24. Remark • Although notnecessary, strictly speaking, here we consider RDF as the format for Linked Data
- 25. A typical LinkedData flow SPARQL endpoint HTML/RDF Content negotiation RDF store RDF dump LOD based applications Data consumptionData exposureData storageData lifecycle Data conversion Data linking Data maintenance
- 26.
- 27. Building LOD basedapplications is easy… (relatively)
- 28. Relatively easy… • Itis about making mash up applications… • But interfacing with the data may be an issue – Developers need to know SPARQL – And how to use it within his/her framework of choice
- 29. A pointer • Researchto Impact Hackathon, Kenya, Jan 2013 – @iHub Research, Kenya • local agricultural and nutritional sector – Comments on that in Tim Davies’ blog • http://www.timdavies.org.uk/ • Other blogs around … (search for them!)
- 30. Data exposure canbe done in various ways
- 31. Exposing de-referenceable URIs •Need to set up content negotiation mechanism – Serving content for URIs • In our experience, not a big problem – Simple back-ends are available, e.g. Pubby • Still, need server 24/7… properly configured
- 32. Provide an RDFdump • Always a good choice – Data is downloaded for inclusion in applications – Efficiency of access to data is under control – Perhaps not always clear how to produce the dump, what to include in it… • Only the data? Also the links?
- 33. Expose SPARQL endpoint •Endpoint typically provided by triple store • Heavy on server side • Query processing is left to the SPARQL engine – Implementation of reasoning – Implementation of order in clause processing – filters, unions, select • Require 24/7 server availability
- 34. Expose Web Services •Known technology • May be built on top RDF stores • Good performances • Control on what data may be accessed • API formats to simplify use of linked data by web developers https://code.google.com/p/linked-data-api/
- 35. Data storage istricky
- 36. Triple stores arewell known resource-guzzlers • Intense use of CPU, memory • Server configuration needs to be appropriate • Internet connection may be a bottleneck • Again, some tech know-how needed to choose the best solution – Also considering other technologies, e.g. NoSQL
- 37. The Semantic Webis resource guzzler! Downscale the Semantic Web! http://worldwidesemanticweb.org/events/downscale2012/ http://worldwidesemanticweb.org/events/downscale2013/
- 38.
- 39. Producing RDF maybe a daunting task
- 40. Getting to RDF…from what? • In many cases, RDF means an abrupt jump from formats that we consider long abandoned • From a recent survey, we learn that some AGROVOC users (libraries, institutions) use the paper version – Last published in 1992
- 41. RDF generation • Itis a simple format, simply triples • But requires some familiarity with the technology, and especially acquaintance with the mentality around, especially on standards and reuse
- 42. A much simplifiedexample from AGROVOC TermCode 1 TermCode 2 TermSpell1 TermSpell2 LangCode 1 LangCode 2 LinkType 1 2 Irrigated farm Farm EN EN BT 1 3 Irrigated farm irrigation EN EN RT
- 43. Can be turnedinto some RDF… Subject Predicate Object Entity1 TermSpell Irrigated farm Entity1 BT Entity2 Entity2 TermSpell Farm Entity3 TermSpell Irrigation Entity2 BT Entity3
- 44. The problem isthe middle column • These are locally defined predicates • One has to guess what they stand for! Predicate TermSpell BT TermSpell TermSpell BT
- 45. Better something likethat.. Subject Predicate Object URI_1 rdfs:label “Irrigated farm” URI_1 skos:broader URI_2 URI_2 rdfs:label “Farm” URI_3 rdfs:label “Irrigation” URI_1 skos:related URI_3
- 46. Using standard vocabulariesis the key • Standard, or de facto standard • Only a few of them: – Dublin Core, BIBO, FOAF, SKOS, .. • Ensure possibility of reuse of data
- 47. Standard vocabularies asStep 0 of Linked Data • Reusing existing vocabularies is the first step to have some indications of what data may be linked and what not – E.g. dct:subject in a bibliographic record indicates the “topic” of the record
- 48. How to knowwhat vocabulary to use? • And how to know if the right vocabulary exists? – We very often receive questions about this from local institutions (who expect to use AGROVOC for that…) • This is probably the very first conceptual blocker!
- 49. Need to supportdata managers • Initiatives such as Linked Open Vocabularies (LOV) are useful: – http://lov.okfn.org/dataset/lov/index.html • But also need usable and stable tools to support data managers
- 50. Drupal’s way tosupport small users • Allows one to import data from other sources, create RDF, and expose RDF dumps • At conversion time, one can chose the vocabulary to use • Then, it becomes the tool for data maintenance • No programming skill required, still some competency on Drupal! And you need to understand RDF and your data!
- 51. Other attempts alongthe same line • AgriDrupal – Drupal especially customized for small institutions – And bibliographic data, data on people, organizations • ScratchPad – Customized for biodiversity data
- 52.
- 53. Is assigning URIsalso a problem? • Often not a technical issue… • Choice may have to do with the languages of the data – AGROVOC uses numbers because it was not possible to chose one language over the others, but software developers often complain • Or with the internal organizations’ asset • It may require longer time than one would expect…
- 54.
- 55. Linking data isa bottleneck
- 56. Example of linkingfrom AGROVOC http://aims.fao.org/aos/agrovoc/c_2808 skos:exactMatch http://www.caas.net.cn/caas/cat/c_33429 “farmland” from AGROVOC exact match …chinese term…
- 57. Linking entities • Stillactive research area • Maintenance still an issue – see example of AGROVOC linked to Chinese thesaurus… • Data validation usually outside the rest of the data lifecycle
- 58. Data maintenance • Choice:keep everything in your db and continue periodic generation of rdf • Move maintenance in different tools
- 59. In what languageis your data?
- 60. Certainly, there aremany languages beyond English…
- 61. Written in variousways… 汉语/漢語
- 62.
- 63. Some considerations froma managerial perspective…
- 64. Assuming an institutionwith constrained resources has already planned to go Linked Data, what to do?
- 65. Options • Go aheadon your own • Organize a collaboration – A network creation effort
- 66. AGRIS is anexample of network Data coordination Partner Partner Partner Partner Partner Partner Can be much smaller or bigger! Partner Partner
- 67.
- 68. 1) Semantic Webis energy intensive • Because of infrastructure requirements • The biggest bottleneck is often on the side of IT competencies, and at the interface between IT and domain knowledge, especially for data modeling • Linked Data-related technologies must become lighter in order to be adoptable in low resource conditions
- 69. 2) In lowresource conditions… • Do a careful assessment of your data and in- house skills • It is a good idea to organize your effort in collaboration • Start mobilizing IT specialists, data curators
- 70. 3) Start withStep 0: identify and use standards to describe your data • Mobilize IT specialists, data curators
- 71.
- 72. ……a bibliographical recordoriginal
- 73. …the same recordtransformed
- 74.
- 75.
- 76. How is linkeddata produced
- 77. ……using title andauthor
- 78. ……using title andauthor
- 79. ……using the keywords
- 80. ……using the keywords
- 81.
- 83.
- 84.
- 85.
- 86.
- 87. NISO/DCMI Webinar Implementing LinkedData in Developing Countries and Low-Resource Conditions NISO/DCMI Webinar • September 25, 2013 Questions? All questions will be posted with presenter answers on the NISO website following the webinar: http://www.niso.org/news/events/2013/dcmi/developing
- 88. Thank you forjoining us today. Please take a moment to fill out the brief online survey. We look forward to hearing from you! THANK YOU
Editor's Notes
- #5 Definition varyUsually based on socio-economic parametersGDP, …In any case, they are the majority of the world193 members of UNBut more countries and territories in the world…Not only the majority of countries, also the majority of people….In 2009: "Out of every 100 persons added to the population in the coming decade, 97 will live in developing countries.“Hania Zlotnik, UN Population Division
- #72 AGRIS is a central repository aggregating and centralizing data from more than 200 bibliographic collections worldwide, some of them of a huge relevance in the agricultural domain.AGRIS ingests data from collections varying from National Research Centres, open access repositories of full-text scholarly literature, publishers of scientific electronic journals in agriculture, and so on.Open Access repositories in 2012.. 29, 355 records from the Wageningen UR, Library (Netherlands)28,582 from the Open Knowledge Repository of the World Bank, which recently opened up to OA to ensure that their research projects and publications are widely available13,000 from R4D: Research for Development - Department for International Development in UK11,600 from AgEcon open access repository15,000 resources from EMBRAPA’s Open Repository
- #75 - AGRIS consumes metadata provided by the community and publishes it as open data The metadata is captured either via a client harvester collecting the data from OAI-PMH client services and open repositories or by delivery (via email or ftp) of database dumps from other information systems and cross reference tools.The data is thus ingested, validated, processed and indexed/stored in two different repositories (the XML and the RDF store). In the next few months, data will be stored only in the RDF repositoryThe data is disseminated via the OpenAGRIS application