Hadoop, SQL and NoSQL, No longer an either/or question

  • 394 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
394
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
15
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. © Copyright Ovum. All rights reserved. Ovum is a subsidiary of Informa plc.1 Hadoop, SQL & NoSQL – No longer an either or question Tony Baer Hadoop Summit 2014 June 4, 2014
  • 2. © Copyright Ovum. All rights reserved. Ovum is an Informa business.2  Where we’ve come – Twins separated at birth & joyous reunion  Why/how the convergence?  Loose ends Agenda
  • 3. © Copyright Ovum. All rights reserved. Ovum is an Informa business.3 SQL RDBMS File systems Hierarchical Data stores OODBMS SQL, NoSQL, Hadoop 1970s 1980s 1990s 2000s 2010s Network Data stores
  • 4. © Copyright Ovum. All rights reserved. Ovum is an Informa business.4 Early Development Commercialization Ecosystem Formation 1960s 1980s 1990s 2000s “Prehistoric” EF Codd publishes seminal RDBMS model IBM System R, Ingres DB2, Oracle, Teradata, PC-based DBMSs SQL becomes de facto enterprise standard data platform Tooling emerges SQL market consolidates: Oracle, DB2, SQL Server, Teradata NewSQL analytic platforms emerge Mainframe era Midranges & PCs emerge Big Data 2014 DBMSs add multiple engines Database timeline 1970s Client/server & n-Tier Ecosystem Broadens CODASYL, IMS MySQL/ LAMP stack emerges J2EE, .NET
  • 5. © Copyright Ovum. All rights reserved. Ovum is an Informa business.5 Early Development Commercialization Ecosystem Formation 2003 - 2005 2009 2011 2012 2013 First Advanced SQL platforms emerge Hadoop emerges Other NoSQL platforms emerge Cloudera intros comm’l Hadoop support Major vendors enter Big Data market Tooling emerges 2nd wave NewSQL platforms emerge Big Data Tools emerge Internet firm early adopters Enterprise early adopters (FS & Media) Mainstream adoption begins 2014 Big Data Apps emerge Big Data platform timeline Hortonworks enters market MongoDB, Cassandra emerge
  • 6. © Copyright Ovum. All rights reserved. Ovum is an Informa business.6 Platform proliferation = Data processing silos SQL RDBMS NewSQL RDBMS NoSQL Key-Value NoSQL JSON Hadoop OLTP (ACID) OLTP (Non- ACID) BI Query & Report Analytics OLTP (Non- ACID) Advanced Analytics Operational Decision Support Operational Decision Support MapReduce- based Advanced Analytics
  • 7. © Copyright Ovum. All rights reserved. Ovum is an Informa business.7  Where we’ve come – Twins separated at birth & joyous reunion  Why/how the convergence?  Loose ends Agenda
  • 8. © Copyright Ovum. All rights reserved. Ovum is an Informa business.8 Analytic SLA requirements vary Batch Periodic Interactive Real-time Exploratory Analytics Standard reporting Days/Hours Seconds Split seconds Interactive query Search Streaming Decision Support Modeling Operational Decision Support Hours/Minutes
  • 9. © Copyright Ovum. All rights reserved. Ovum is an Informa business.9 Analytics problems cross silos – Operational examples  Customer engagement  Interaction – Customer 360 query in DW  Behavior – Enrich with sentiment analysis on Hadoop  Engagement – Manage real-time engagement on NoSQL database  Risk mitigation  Baseline – Model party & transactional risk on DW or Hadoop  Enrich – Analyze, rank impact of externalities on Hadoop  Ingest – Real-time market feeds via streaming in-memory  Define – Decision processes offline via BPM  Act – Allow/deny credit on system of record
  • 10. © Copyright Ovum. All rights reserved. Ovum is an Informa business.10 Architecture – Common threads  Aggressive tiering  Multiple storage engines  Multiple workload types  On the horizon:  Federated query  Workload/query orchestration  Loose ends:  Common security?
  • 11. © Copyright Ovum. All rights reserved. Ovum is an Informa business.11 SQL Databases adding multiple personas  IBM DB2  BLU architecture adds columnar, data skipping, advanced tiering  New MongoDB-compliant JSON data store  Oracle Database 12c  “In-Memory” option adds DRAM-based columnar, extreme compression  Microsoft SQL Server  PDW adds columnar indexing  PolyBase feature adds Hadoop integration  Teradata  Teradata 14.10 adds “Intelligent Memory” data tiering, columnar, Hadoop integration  Aster 6 adds graph, file store, “SNAP” framework for choreographing SQL, MapReduce, graph & Hadoop processing  SAP  “Smart Data Access” federated query over HANA, Sybase IQ, Teradata & Hadoop
  • 12. © Copyright Ovum. All rights reserved. Ovum is an Informa business.12 Hadoop growing beyond MapReduce  Apache Hadoop 2.0’s new YARN resource allocation framework allows multiple workloads  Interactive SQL – lots of flavors  Spark – The new MapReduce & more…  Search  Streaming  Loose ends:  Graph ready for prime time?
  • 13. © Copyright Ovum. All rights reserved. Ovum is an Informa business.13 Emerging NewSQL + NoSQL databases  JSON data stores exploding  Intuitive for representing Internet data  MongoDB, Couchbase  IBM, Teradata… potentially Oracle adding JSON  New transaction stores … not full ACID  Cassandra for NoSQL (integrated to Hadoop)  NuoDB, Clustrix, MemSQL & others reinvent OLTP for distributed Internet apps  HBase  DynamoDB, Berkeley DB (Oracle NoSQL database) & other key-value stores
  • 14. © Copyright Ovum. All rights reserved. Ovum is an Informa business.14 A variety of overlapping choices NewSQL JSON Graph Hadoop SQL Deep analytics Stream Graph NoSQL Account/user profiles Interactive content Graph Machine data JSON SQL RDBMS OLTP DW JSON Distributed OLTP Fast, deep analytics Active Archiving SQLRDBMS NewSQLRDBMS NoSQLKey-Value NoSQLJSON Hadoop From To
  • 15. © Copyright Ovum. All rights reserved. Ovum is an Informa business.15 A variety of overlapping choices – But… Who owns the logical hub? SQL RDBMS NewSQL Hadoop NoSQL OLTP DW Active Archiving JSON Distributed OLTP Fast, deep analytics JSON Graph SQL Deep analytics Stream Graph Account/user profiles Interactive content Graph Machine data JSON
  • 16. © Copyright Ovum. All rights reserved. Ovum is an Informa business.16  Where we’ve come – Twins separated at birth & joyous reunion  Why/how the convergence?  Loose ends Agenda
  • 17. © Copyright Ovum. All rights reserved. Ovum is an Informa business.17 Loose ends  Ideally, policy-based federated query will be the solution  Who owns federated query?  Data platform?  BI tool?  Application?  Who owns workload management?  Who owns security? Tug of war between data platforms likely
  • 18. © Copyright Ovum. All rights reserved. Ovum is an Informa business.18 Takeaways  Analytics no longer limited by platform constraints  Data platforms are taking multiple personas –  Platform choice is not either/or  But  Analytics are no longer silo’ed  Execution remains silo’ed  The brass ring will be a logical hub for  Policy/SLA-based workload targeting & management  Security & operations/performance management
  • 19. © Copyright Ovum. All rights reserved. Ovum is a subsidiary of Informa plc.19 Thank you Tony Baer Ovum (646) 546-5330 @TonyBaer tony.baer@ovum.com