Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Big Data Europe Concept and Platform


Published on

BDE concept and platform as presented during the 3rd BDE SC5 online hangout.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Big Data Europe Concept and Platform

  2. 2. Big Data Europe (CSA: 2015-17)  Show societal value of Big Data o Across all societal challenges addressed by Horizon 2020  Lower barrier for using big data technologies o Effort and resources to convert tools and workflows o Skills and expertise H l bli h d l h i Help establish data value chains o Across languages, organizations and domains
  3. 3. Consortium NCSR DEMOKRITOS
  4. 4. Stakeholder Engagement Stakeholder engagement workshops:g g p  Present the action and its showcase deployments  Raise awareness about BDE results and what they mean for stakeholders  Collect requirements to drive further development
  5. 5. SocietalSoc e a Challenges
  6. 6. Architecture  Big Data Integrator (BDI): o The prototype developed by BDE  Main points of the architecture Main points of the architecture o Dockerization Support layer including integrated UIo Support layer, including integrated UI o Semantification layer
  7. 7. Docker containers  Docker offers lightweight virtualization o Docker containers can be shared to be provisioned on different Linux variations and versions  Identical base sys not required  All BDI components: Docker containers Docker containers
  8. 8. BDI components  Processing and storage components o Re-used existing docker containers where available o Dockerized by BDE where not o Ensured all can be provisioned through Docker Swarm  Components by BDE: o Support Layer o Semantic Layer
  9. 9. BDE Docker Containers D t i HDFS C d 4 t Data serving: HDFS, Cassandra, 4store, PostGIS, Strabon, Elastic Search, Hive, SemagrowSemagrow  Processing: Spark, Flink, Sansa  Stream ingestion middleware: Flume, Kafka
  10. 10. Semantic layer tools  BDE tooling for Semantic Data Lake:g o Swagger: Semantics of RESTful APIs o Semantic Analytics Stack (SANSA):y ( ) Distributed data processing for large- scale RDF data o Semagrow: SPARQL perspective over Big Data stores
  11. 11. Questions?  BigDataEurope Web site:  Thank you for your attention!