Getting the best insights from your data using Apache Metamodel by Alberto Rodríguez at Big Data Spain 2015

•

0 likes•1,365 views

We live in an age of an over-abundance of data stores choices and we also live in an age where the data has more value than ever.We think that data has no value until it’s shown and everybody is nowadays looking for simple ways to get insights from its data through self-explanatory visualizations. Would it be nice if we were able to create awesome graphs regardless the underlying technology that stores our data? Session presented at Big Data Spain 2015 Conference 15th Oct 2015 Kinépolis Madrid http://www.bigdataspain.org Event promoted by: http://www.paradigmatecnologico.com Abstract: http://www.bigdataspain.org/program/thu/slot-12.html

Technology

BIG DATA
SPAIN
Getting the best insights
from your data using
Apache MetaModel

AGENDA
Where am I
working for
Stratio Viewer
Inception
Apache
Metamod
el
Viewer is
much more
Live demo
1 2 3 4 5

II
GO TO SPACE
STRATIO
OPEN-SOURCE SOLUTIONS
Our enterprises solutions
are based on open source
technologies
PURE SPARK
The only pure Spark
platform, the only global
solution
ENTERPRISE SPARK
On – premise & cloud, our
platform is geared towards
helping companies
SPARK-BASED BD
PLATFORM
The first Spark-Based big
data platform released

STRATIO VIEWER INCEPTION
“Data has no value
until its shown”

STRATIO VIEWER INCEPTION
“Data has no value
until its shown”
STRATIO VIEWER INCEPTION
ClearnessSimplicity
Standards
Freedom
Prototyping
Dashboards
Widgets
DataSources
DataViews
Attractive
Pretty

STRATIO VIEWER INCEPTION
“Data has no value
until its shown”
STRATIO VIEWER INCEPTION

STRATIO VIEWER INCEPTION
“Data has no value
until its shown”
STRATIO VIEWER INCEPTION
Oh wait! @Stratio...We're polyglots

APACHE METAMODEL
“... a uniform connector and query API to
many very different datastore types...”

One language to rule them all:
SQL4everything!
APACHE METAMODEL

APACHE METAMODEL
Simple jar
Sometimes the work will be pushed down to
the connectors and sometimes we'll be using MetaModel's
own query engine

APACHE METAMODEL
DataContext is your friend
DataContext dc = new MongoDbDataContext(db)
DataSet dataSet = dc.query().from(table)
.select(FunctionType.SUM,“foo”)
.select(“baz”).groupBy(“baz”).execute()

APACHE METAMODEL
DataContext
Schema
Table
Column
Metadata discovery

APACHE METAMODEL
What about performance?

THANK YOU
UNITED STATES
Tel: (+1) 408
5998830
EUROPE
Tel: (+34) 91 828 64
73
contact@stratio.com
www.stratio.com

Getting the best insights from your data using Apache Metamodel by Alberto Rodríguez at Big Data Spain 2015

Title: Your organization needs an OSPO, or be ready to fail! Open source is becoming the main ingredient for companies to success. To achieve it, companies need to manage efficiently their relationship with open source projects. And that’s the main goal for companies’ Open Source Program Office (OSPO). During this talk, you will learn about the benefits of having an OSPO in your organization, why should companies adapt to open source and adopt an OSPO, how it makes a difference to have a team responsible for viewing, managing, making critical decisions, contributions back to open source projects, and providing oversight for open source initiatives on their company and where they should start. Also, we will give real examples of how companies are doing this today and their impact for the community, like Samsung Open Source Group, Uber, and others. Additionally, you will learn about communities and initiatives to help you having a successful OSPO, like TODO Group and CHAOSS, the importance of CHAOSS to give actual data and insights about open source projects and a bigger perspective with analytics dashboards, and how data and metrics from the OSPO can help companies tackle their corporate strategy.

Overcoming the challenges of multiple data frameworks in a multiple cloud env...

Thomas Pauly

MuleSoft London Community October 2019 - Preparing for London Connect 2019

Pace Integration

Executing Digital Platform Strategies- Al Grimes and Richard Glew

Thoughtworks

In 2017 most companies with significant technology footprints have made some attempt to adopt agile methods to improve organisational performance, yet the rate of failure remains high. IT departments continue to search for the holy grail of modular and scalable systems, yet remain encumbered with legacy systems that are anything but these things. Richard and Al explore some of the challenges around why these problems persist despite enormous organisational effort and expense. They will share how an effective digital platform strategy may have some of the answers organisations need to evolve.

The New Reality: the Role of PaaS in Technology Innovation - Franklin Herbas

jaxconf

Today, we are in the midst of a mobile, social, and big data revolution – a new era with the promise of never before imagined possibilities. We require a platform that can deliver these required capabilities in an easy to use manner. In-memory computing has becoming the leading solution for how these applications can bring together the huge amounts of varying data now available. You need a platform that offers services to develop real-time applications with mobile, social or big data capabilities.

Step-by-step into the world of Hybrid integration with Azure

delaware BeLux

http://www.bigdataspain.org/2014/conference/state-of-play-data-science-on-hadoop-in-2015-keynote Machine Learning is not new. Big Machine Learning is qualitatively different: More data beats algorithm improvement, scale trumps noise and sample size effects, can brute-force manual tasks. Session presented at Big Data Spain 2014 Conference 18th Nov 2014 Kinépolis Madrid http://www.bigdataspain.org Event promoted by: http://www.paradigmatecnologico.com Slides: https://speakerdeck.com/bigdataspain/state-of-play-data-science-on-hadoop-in-2015-by-sean-owen-at-big-data-spain-2014

Intro to the Big Data Spain 2014 conference

Big Data Spain

ToroDB: Scaling PostgreSQL like MongoDB by Álvaro Hernández at Big Data Spain...

Big Data Spain

NoSQL databases have emerged as a response to some perceived problems in the RDBMSs: agile/dynamic schemas; and transparent, horizontal scaling of the database. The former has been promptly targeted with the introduction of unstructured data types, but scaling a relational databases is still a very hard problem. As a consequence, all NoSQL databases have been built from scratch: their storage engines, replication techniques, journaling, ACID support (if any). They haven't leveraged the previously existing state-of-the-art of RDBMSs, effectively re-inventing the wheel. Isn't this sub-optimal? Wouldn't it be possible to construct a NoSQL database by layering it on top of a relational database? Session presented at Big Data Spain 2015 Conference 16th Oct 2015 Kinépolis Madrid http://www.bigdataspain.org Event promoted by: http://www.paradigmatecnologico.com Abstract: http://www.bigdataspain.org/program/fri/slot-37.html

Dataflows: The abstraction that powers Big Data by Raul Castro Fernandez at ...

Big Data Spain

http://www.bigdataspain.org/2014/conference/dataflows-the-abstraction-that-powers-the-big-data-technology Dataflows are an omnipresent abstraction across many big data technologies due to its suitability for representing programs in a way that is easy to parallelize. All dataflow models---such as those of Spark or MapReduce---are stateless, which facilitates achieving fault tolerance, a crucial property when running at large-scale. However, this stateless dataflow models have a negative impact on the programming models they expose, which need to adapt to match the stateless nature of the underlying platforms. With the “democratization of data”, different types of users with different skills want answers from their big datasets, but sometimes they lack the skills required to write programs adapted to these specific frameworks: A familiar programming model becomes crucial to open big data value to a broader set of users. Video https://www.youtube.com/watch?v=blY7EleXW6U

Big Data the potential for data to improve service and business management by...

Big Data Spain

Big Data Web applications for Interactive Hadoop by ENRICO BERTI at Big Data...

Big Data Spain

Data warehouse modernization programme by TOBY WOOLFE at Big Data Spain 2014

Big Data Spain

The top five questions to ask about NoSQL. JONATHAN ELLIS at Big Data Spain 2012

Big Data Spain

Location analytics by Marc Planaguma at Big Data Spain 2014

Big Data Spain

http://www.bigdataspain.org/2014/conference/location-analytics While the implementation of analytic operations on distributed computing frameworks has been widely describing, enabling the computational core of a Big Data system with capabilities for supporting geospatial querying on data is yet a challenging issue. This session aims to target that specific aspect by reviewing how researchers at BDigital Technology Centre have designed and implemented a stack for advanced Machine Learning on Urban Data by providing a way to geoquery massive amounts of HDFS data from Spark processes without hindering the overall system performance.

Convergent Replicated Data Types in Riak 2.0

Big Data Spain

Talk by Gordon Guthrie, Senior Software Engineer at Basho Summary A review of the CAP Theorem and the difficulties of resolving conflicts in highly distributed systems. Covering the issues and various theories on how to resolve including the use CRDTs in Riak Details CRDTs are used to replicate data across multiple computers in a network, executing updates without the need for remote synchronisation. This leads to merge conflicts in systems using conventional eventual consistency technology, but CRDTs are designed such that conflicts are mathematically impossible. Under the constraints of the CAP theorem they provide the strongest consistency guarantees for available/partition-tolerant (AP) settings. The CRDT concept was first formally defined in 2007 by Marc Shapiro and Nuno Preguiça in terms of operation commutativity, and development was initially motivated by collaborative text editing. The concept of semilattice evolution of replicated states was first defined by Baquero and Moura in 1997, and development was initially motivated by mobile computing. The two concepts were later unified in 2011. Basho has worked with the EU and Marc Shapiro's team to push CRDTs into distributed systems. Riak v2.x is the first commercial product to include this functionality

CloudMC: A cloud computing map-reduce implementation for radiotherapy. RUBEN ...

Big Data Spain

IAd-learning: A new e-learning platform by José Antonio Omedes at Big Data Sp...

Big Data Spain

Purpose of the talk: Describing the use of Machine Learning and Big Data Techniques to improve the performance of elearning students. Presenting an existing case of an elearning platform (iAdLearn¡ng) and the technology used behind the scenes, to make adaptive/high performance elearning a reality. Session presented at Big Data Spain 2015 Conference 15th Oct 2015 Kinépolis Madrid http://www.bigdataspain.org Event promoted by: http://www.paradigmatecnologico.com Abstract: http://www.bigdataspain.org/program/thu/slot-17.html

Essential ingredients for real time stream processing @Scale by Kartik pParam...

Big Data Spain

At LinkedIn, we ingest more than 1 Trillion events per day pertaining to user behavior, application and system health etc. into our pub-sub system (Kafka). Another source of events are the updates that are happening on our SQL and No-SQL databases. For e.g. every time a user changes their linkedIn profile, a ton of downstream applications need to know what happened and need to react to it. We have a system (DataBus) which listens to changes in the database transaction logs and makes them available for down stream processing. We process ~2.1 Trillion of such database change events per week. We use Apache Samza for processing these event-streams in real time. In this presentation we will discuss some of challenges we faced and the various techniques we used to overcome them. Session presented at Big Data Spain 2015 Conference 15th Oct 2015 Kinépolis Madrid http://www.bigdataspain.org Event promoted by: http://www.bigdataspain.org/program/thu/slot-3.html

Data Science in 2016: Moving up by Paco Nathan at Big Data Spain 2015

Big Data Spain

The term 'Data Science' was first described in scientific literature about 15 years ago. It started to become a major trend in industry about 7 years ago. O'Reilly Media surveys the industry extensively each year. In addition we get a good birds-eye view of industry trends through our conference programs and publications, working closely with some of the best practitioners in Data Science. By now, the field has evolved far beyond its origins eclipsing an earlier generation of Business Intelligence and Data Warehousing approaches. Data Science is moving up, into the business verticals and government spheres of influence where it has true global impact. This talk considers Data Science trends from the past three years in particular. What is emerging? Which parts are evolving? Which seem cluttered and poised for consolidation or other change? Session presented at Big Data Spain 2015 Conference 15th Oct 2015 Kinépolis Madrid http://www.bigdataspain.org Event promoted by: http://www.paradigmatecnologico.com Abstract: http://www.bigdataspain.org/program/thu/slot-2.html

How to integrate Big Data onto an analytical portal, Big Data benchmarking fo...

Big Data Spain

DKS EAP (Enterprise Analytical Portal) was first thought of as an integrated analytical portal; a conjunction of BI and BA products (ranging from advanced customer intelligence to marketing analytics) assembled under a unified interface, specifically designed to better assist business in their decision-making. We believe DKS EAP is of special interest for the Big Data community nowadays since it is currently leveraging technologies such as Hadoop, Cassandra, Spark and Storm to improve its analytic capabilities. Accordingly, the talk will briefly present DKS EAP as an integrated Big Data analytic environment, focusing on how the fast-growing needs for real-time, social media and geomarketing data analysis intensified its necessity to embrace Big Data technologies. Session presented at Big Data Spain 2015 Conference 15th Oct 2015 Kinépolis Madrid http://www.bigdataspain.org Event promoted by: http://www.paradigmatecnologico.com Abstract: http://www.bigdataspain.org/program/thu/slot-8.html#spch9.1

A new streaming computation engine for real-time analytics by Michael Barton ...

Big Data Spain

Apache Spark has successfully built on Hadoop infrastructure to encompass real-time processing, moving from rigid Map-Reduce operations to general purpose functional operations distributed across a cluster of machines. However data storage has become a black box. The source data for a query has to be retrieved in full and sent through the analysis pipeline rather than processing the data where it is stored, as in traditional database systems. This introduces significant cost, both in network utilisation and in the time taken to produce a result. Session presented at Big Data Spain 2015 Conference 15th Oct 2015 Kinépolis Madrid http://www.bigdataspain.org Event promoted by: http://www.paradigmatecnologico.com Abstract: http://www.bigdataspain.org/program/thu/slot-14.html

Analyzing organization e-mails in near real time using hadoop ecosystem tools...

Big Data Spain

Big Data, analytics and 4th generation data warehousing by Martyn Jones at Bi...

Big Data Spain

The ultimate business success of Big Data in business will depend on our ability to successfully bring about the realignment and placement of Big Data into a more generalized architectural framework, one that coalesces strategic, technical and management elements of data warehousing (DW 3.0), business intelligence, textual analysis and statistical analysis into a coherent, synergistic and usable whole. Session presented at Big Data Spain 2015 Conference 16th Oct 2015 Kinépolis Madrid http://www.bigdataspain.org Event promoted by: http://www.paradigmatecnologico.com Abstract: http://www.bigdataspain.org/program/fri/slot-26.html

Geospatial and bitemporal search in C* with pluggable Lucene index by Andrés ...

Big Data Spain

Stratio presented its open source Lucene-based implementation of Cassandra’s secondary indexes at Cassandra Summit London 2014, which provided several search engine features. It used to be distributed as a fork of Apache Cassandra, which was a huge problem both for users and maintainers. Nowadays, due to some changes introduced at C* 2.1.6, we are proud to announce that it has become a plugin that can be attached to the official Apache Cassandra. With the plugin we have been able to provide C* with geospatial capabilities, making it possible to index geographical positions and perform bounding box and radial distance queries. This is achieved through Lucene’s geospatial module. Session presented at Big Data Spain 2015 Conference 15th Oct 2015 Kinépolis Madrid http://www.bigdataspain.org Event promoted by: http://www.paradigmatecnologico.com Abstract: http://www.bigdataspain.org/program/thu/slot-9.html

Processing large-scale graphs with Google(TM) Pregel by MICHAEL HACKSTEIN at...

Big Data Spain

WSO2 Platform Introduction

WSO2

CWIN17 Rome / The sas platform for the new generation of ai and ml

Capgemini

Viewers also liked

State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...

Big Data Spain

Intro to the Big Data Spain 2014 conference

Big Data Spain

ToroDB: Scaling PostgreSQL like MongoDB by Álvaro Hernández at Big Data Spain...

Big Data Spain

Dataflows: The abstraction that powers Big Data by Raul Castro Fernandez at ...

Big Data Spain

Big Data the potential for data to improve service and business management by...

Big Data Spain

Big Data Web applications for Interactive Hadoop by ENRICO BERTI at Big Data...

Big Data Spain

Data warehouse modernization programme by TOBY WOOLFE at Big Data Spain 2014

Big Data Spain

The top five questions to ask about NoSQL. JONATHAN ELLIS at Big Data Spain 2012

Big Data Spain

Location analytics by Marc Planaguma at Big Data Spain 2014

Big Data Spain

Convergent Replicated Data Types in Riak 2.0

Big Data Spain

CloudMC: A cloud computing map-reduce implementation for radiotherapy. RUBEN ...

Big Data Spain

IAd-learning: A new e-learning platform by José Antonio Omedes at Big Data Sp...

Big Data Spain

Essential ingredients for real time stream processing @Scale by Kartik pParam...

Big Data Spain

Data Science in 2016: Moving up by Paco Nathan at Big Data Spain 2015

Big Data Spain

How to integrate Big Data onto an analytical portal, Big Data benchmarking fo...

Big Data Spain

A new streaming computation engine for real-time analytics by Michael Barton ...

Big Data Spain

Analyzing organization e-mails in near real time using hadoop ecosystem tools...

Big Data Spain

Big Data, analytics and 4th generation data warehousing by Martyn Jones at Bi...

Big Data Spain

Geospatial and bitemporal search in C* with pluggable Lucene index by Andrés ...

Big Data Spain

Processing large-scale graphs with Google(TM) Pregel by MICHAEL HACKSTEIN at...

Big Data Spain

Viewers also liked (20)