Hobbit in a Nutshell - EDF2016
•
1 like•263 views
Presentation of HOBBIT Joint Event Post-EDF 2016. Eindhoven, Netherlands (This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 688227.)
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (15)
Similar to Hobbit in a Nutshell - EDF2016
Similar to Hobbit in a Nutshell - EDF2016 (20)
More from Holistic Benchmarking of Big Linked Data
More from Holistic Benchmarking of Big Linked Data (20)
Recently uploaded
Recently uploaded (20)
Hobbit in a Nutshell - EDF2016
- 1. HOBBIT in a Nutshell Axel Ngonga Horizon 2020 GA No 688227 01/12/2016–30/11/2018 Joint Event Post-EDF 2016 Eindhoven, Netherlands July 1st, 2016 Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 1 / 17
- 2. A Lot of Data 1 1http://www.ibmbigdatahub.com/infographic/four-vs-big-data Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 2 / 17
- 3. A Lot of Tools 2 2https://cdn.datafloq.com/cms/os_big_data_open_source_tools-v2.png Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 3 / 17
- 4. Core Questions Developers: How good is my tool? Vendors: Who is my tool good for? Users: Which tool(s) should I use for my application? Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 4 / 17
- 5. Many Questions Where are the current bottlenecks? Which steps of the data lifecycle are critical? Which solutions are available? Which key performance indicators are relevant? How well do or should tools perform? How do existing solutions perform w.r.t. relevant indicators? Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 5 / 17
- 6. GERBIL Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 6 / 17
- 7. GERBIL Evaluation platform for NER/NEL 9 reference annotation systems 11 reference datasets Benchmarking 10× faster Archiving of results Citeable URIs Additional analysis Open-source project Local deployment Normalized implementation of KPIs Online instance at http://gerbil.aksw.org/ Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 6 / 17
- 8. GERBIL Annotator Tasks NIF-based Annotators 2519 Babelfy 958 DBpedia Spotlight 922 TagMe 2 811 WAT 787 Kea 763 Wikipedia Miner 714 NERD-ML 639 Dexter 587 AGDISTIS 443 Entityclassifier.eu NER 410 FOX 352 Cetus 1 Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 7 / 17
- 9. HOBBIT Rationale A community-driven benchmarking framework for the community Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 8 / 17
- 10. HOBBIT Rationale A community-driven benchmarking framework for the community Focus on Big Linked Data Cover all steps of the Linked Data lifecycle Used by a growing number of companies Mature and maturing technologies Open benchmarks based on industrial data and use cases Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 8 / 17
- 11. HOBBIT 36-month project Project begin: Dec. 1st, 2015 Project volume: ca. 4 million Euros 10 partners Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 9 / 17
- 12. Aims 1 Gather real requirements Performance indicators Performance thresholds 2 Provide universal benchmarking platform Standardized hardware Comparable results 3 Develop benchmarks based on real data 4 Periodic benchmarking challenges 5 Periodic reporting 6 Found independent Hobbit association Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 10 / 17
- 13. Overview Data Collection Industry data Measure Collection Benchmark Creation Benchmark 1 KPIs Tasks KPIs Tasks KPIs Tasks KPIs Tasks KPIs Tasks KPIs Tasks Benchmark 2 Benchmark n HOBBIT Platform Solution 1 Solution k Solution 2 Challenges Reports Participants/Community Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 11 / 17
- 14. We offer a benchmarking platform Controller Data Generator Task Generator Data Generator Data Generator Task Generator Task Generator FrontendSystem Adapter System data flow creates component Store SPARQL Endpoint Analysis Benchmark Evaluator Module Eval. Store Message Bus Node Observer Logging Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 12 / 17
- 15. We offer a benchmarking platform Addresses all steps of the Linked Data Lifecycle Benchmarks derived from industry use cases Real data under the benchmarks Scalable size of benchmarks Open-source implementation Local instance on server cluster Uses established deployment technologies Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 13 / 17
- 16. We offer benchmarks Streaming and static deterministic benchmarks Realistic benchmarks Controlled volume and velocity Generation and Acquisition Conversion of XML into RDF Entity recognition and linking Relation extraction Analysis and Processing Link Discovery Machine Learning Supervised and unsupervised Storage and Curation Triple stores Versioning Incl. updates Visualization and Services Question Answering Faceted Browsing Usage-based benchmarks Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 14 / 17
- 17. We offer datasets Twitter7 dataset ca. 476 million tweets ca. 17 million users ClueWeb12 ca. 733 million websites 1+ billion annotations Printing Machinery ca. 6.5 trillion events 1500 printing machines LIVED ca. 2.5 billion measurements 6 households, two years Injection molding industry ca. 120 million measurements Traffic data archive ca. 15 trillion speed measurements 100+ million road segments Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 15 / 17
- 18. We need ... Your use cases Participate in the survey Join the HOBBIT community Provide KPIs Provide datasets Join the platform development Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 16 / 17
- 19. Thank You http://project-hobbit.eu/get-involved/ http://goo.gl/forms/1iRIoG4Xpb https://twitter.com/hobbit_project Ngonga (InfAI) HOBBIT in a nutshell July 1st, 2016 17 / 17