Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Use of Open Data to Improve the Repeatability of Adaptivity and Personalisation Experiment

12 views

Published on

Workshop Position Paper
Harshvardhan J. Pandit, Ramisa Hamed, Seamus Lawless, David Lewis.
Published in the proceedings of the 1st EvalUMAP workshop - Towards comparative evaluation in user modeling, adaptation and personalization, held in conjunction with the 24th Conference on User Modeling, Adaptation and Personalization, UMAP 2016.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

The Use of Open Data to Improve the Repeatability of Adaptivity and Personalisation Experiment

  1. 1. The Use of Open Data to Improve the Repeatability of Adaptivity and Personalisation Experiment Harshvardhan Pandit, Roghaiyeh Gachpaz Hamed, Shay Lawless, David Lewis ADAPT, Trinity College Dublin Dublin, Ireland The ADAPT Centre is funded under the SFI Research Centres Programme (Grant 13/RC/2106) and is co-funded under the European Regional Development Fund.
  2. 2. www.adaptcentre.ieReproducibility Reproducibility Reproducibility of results is a key element for the verification of scientific experiments and an important indicator of the quality of a published experiment.
  3. 3. www.adaptcentre.ieUsage license and rights Usage license and rights • Personalisation experiment data cannot be openly published • Some of it may be anonymised, but what was the process? • Who can use it? Under what conditions? • Can someone reuse their own dataset to run the experiment? • Is the usage license of the data different and separate from the usage license of the experiment?
  4. 4. www.adaptcentre.ieAdopt Best Practices Linked Open Data • Emerging linked data standards can be applied to the description and data of published adaptivity and personalisation experiments • Can be linked from publications and easily located, accessed and reused to repeat an experiment. • The approach also provides possibilities for published experiments to be extended or modified to provide a firmer grounding for publishing new results and conclusions.
  5. 5. www.adaptcentre.ieNLP / ML Best Practices NLP / ML experiments • Highly structured, Highly repeatable • Most research based on adaptation and modification of existing experiments, which leads to rapid pace of experimentation • Data dependant with Shared tasks and competitions
  6. 6. www.adaptcentre.ieNLP / ML Best Practices NLP / ML experiments • Uses Linked Open Data ontologies - MEX, NIF • LingHub aggregates all experiment metadata • Shared experiments make it easy to extend existing research
  7. 7. www.adaptcentre.ieLinked Open Data Ontologies • RDF / OWL: • knowledge representation • base of linked open data • DCAT: • Authorship, Publication • Declare authors and contributors of experiment • ODRL: • Digital Rights and Licensing • Attach terms of usage, publication etc. to data or experiment • PROV / P-Plan: • Provenance - declare generation of steps • OPMW: • Experiment Workflow - uses PROV / P-Plan to declare experiments
  8. 8. www.adaptcentre.ieLinked Open Data Technology: • CSVW (CSV for the Web) • structured linked data using CSV • share data sets • CSV format universally known • JSON-LD (based on JSON) • metadata representation using JSON • share metadata for experiments • JSON is lightweight and popular • JSON-LD is valid JSON
  9. 9. www.adaptcentre.ieWhy Linked Open Data? Why Linked Open Data?? The machine readable nature of metadata makes it easy for an automated system to verify the correct- ness of the data, or perform other operations that may prove helpful in validations such as checking of data formats and alterations through replacement of key steps, which provides opportunities for new outcomes and results.
  10. 10. www.adaptcentre.ieWhy Linked Open Data? Separation of Concerns • An experiment is multiple steps connected by the data exchanged between steps. • Possible to repeat or verify certain steps without repeating the entire experiment. • Replacement of a step with other comparable approaches and to evaluate comparative results between them. • Ease of variation in experiment repeatability, possible to compare results across a range of similar experiments. • The abstraction of experiment workflows from individual steps also allows each step to be implemented using different technologies.
  11. 11. www.adaptcentre.ieAn Example Personalised Multilingual Information Retrieval (PMIR) Application of DCAT, PROV, P-PLAN and CSVW metadata to tabular data sets arising from an experiment
  12. 12. www.adaptcentre.ieAn Example Personalised Multilingual Information Retrieval (PMIR)
  13. 13. www.adaptcentre.ieFor UMAP community • The creation of an ontology specific to the description of the steps and data associated with a published adaptivity and personalisation experiment offers a cohesive model that can be linked together to form a corpus of knowledge of re- lated experiments. • The efforts required to create such an ontology are heavily based on adopting existing ontologies and standards and have the advantage of best practices for models that easily locate, access, reuse and repeat an experiment. • Sharing experiment workflows and experimental data allows broader sharing of knowledge linked across do- mains. • This brings new ideas into the UMAP community while exposing the ongoing work in new research avenues to other fields of related disciplines. • Ultimately, scientific research can only progress through sharing of knowledge in a usable and repeatable manner and hence must endure efforts towards the same.
  14. 14. www.adaptcentre.ieAcknowledgements This work has been supported partially by the European Commission as part of the FALCON project (contact number 610879) and the ADAPT Centre for Digital Content Technology which is funded under the SFI Research Centres Programme (Grant 13/RC/2106) and is co-funded under the European Regional Development Fund.
  15. 15. Scientific Research can only progress through sharing of knowledge in a usable and repeatable manner and hence must endure efforts towards the same

×