Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Metanautix Quest and Couchbase: Scalable Analytics Across NoSQL, RDBMS, and Hadoop: Couchbase Connect 2015

1,204 views

Published on

Enterprises are increasingly faced with a variety of data silos, each optimized for specific workloads and data types. For example, product catalogs may reside in Couchbase as JSON-structured documents, sales transactions may be stored in an RDBMS as flat relational tables, and web logs may be stored in Hadoop’s HDFS. Business analysts must combine data from these disparate silos quickly, at scale, while leveraging years of investment in standard SQL. This session will present a new approach that makes it easy to quickly read and write big data from Couchbase, RDBMS, and Hadoop HDFS; use standard SQL for extract-transform and load (ETL); run standard machine learning algorithms like clustering, prediction, and classification; support nested repeated data in Couchbase; perform visual, interactive analysis using leading BI tools like Tableau and Excel; and deploy in virtualized environments like VMware.

Published in: Technology
  • Be the first to comment

Metanautix Quest and Couchbase: Scalable Analytics Across NoSQL, RDBMS, and Hadoop: Couchbase Connect 2015

  1. 1. Metanautix Quest and Couchbase: Scalable Analytics Across NoSQL, RDBMS, and Hadoop Jim Adler, VP Products & Marketing, CPO
  2. 2. 2 © 2015 METANAUTIX.
  3. 3. “Data, data everywhere, How all the drops to drink?” with apologies to Samuel Taylor Coleridge Rime of the Ancient Mariner OR
  4. 4. 4 © 2015 METANAUTIX. About Metanautix Pioneers from Google and Facebook Theo Vassilakis Founder, CEO Led development of Dremel at Google, used by 40,000 Googlers on 100,000 cores Toli Lerios Founder, CTO Developed infrastructure at Facebook behind the world’s largest photo repository with 300 billion images
  5. 5. 5 © 2015 METANAUTIX. The Complexity Conundrum is Everywhere DIFFERENT Formats DISCONNECTED Silos LARGE Volumes .. .
  6. 6. 6 ©2014 METANAUTIX. “How to drink all the drops” DATA TYPES DATA SHAPES DATA STORAGE Logs Web Documents Mobile Social Media Flat Nested Array RDBMS No SQL/Hadoop OLAP SaaS File System Data Scientists Analysts Business Users Audit/ Compliance Executives Statistical IDE BI Tools Spreadsheet SQL Client VisibilitySpeed Generality On-prem Cloud Hybrid Imperative Declarative Custom Standard
  7. 7. 7 © 2015 METANAUTIX. INTRODUCING METANAUTIX QUEST The Industry’s First Enterprise-Ready Data Compute Engine Connect – N1QL Speed at Scale Standard SQL
  8. 8. 8 ©2014 METANAUTIX. Your Organization’s Data Supply Chain Complex, Slow Data Pipelines Grow Organically On Prem NFS/NetApp RDBMS/Oracle Informatica Teradata Netezza SAP Hana Excel Tableau Microstrategy Cloud AWS/S3 HDFS/QFS Hadoop MapReduce Hive AWS/Redshift Google/BigQuery AWS/Dynamo Tableau Online GoodData Storage Gather ETL Analyze Ad-Hoc Share In Memory Visual Analysis
  9. 9. 9 ©2014 METANAUTIX. On Prem NFS/NetApp RDBMS/Oracle Excel Tableau Microstrategy Cloud AWS/S3 HDFS/QFS Tableau Online GoodData Quest Data Compute Engine Standards-based Solution for End-to-End Analysis Storage ETL Ad-Hoc In Memory Visual Analysis Gather Analyze Share Integrated, Managed, Scalable NOT: transactional, OLTP, CEP
  10. 10. 10 © 2015 METANAUTIX. SQL Declarative Processing • SQL Literacy – Reads like English – Venn diagrams – Sufficiently precise to execute • Optimizers can distribute computation • JSON support in a standard way SELECT I.category AS category, sum(SS.sales_price) AS sales FROM StoreSales AS SS JOIN Items AS I ON SS.item_sk = I.item_sk WHERE SS.sold_date = extract( QUARTER FROM current_date()) GROUP BY I.category ORDER BY sales DESC HAVING bookings > 20000000 -- $2M category sales Jewelry $78,862,109 Shoes $42,026,958 Books $24,349,021
  11. 11. 11 © 2015 METANAUTIX. Declarative Media Processing Generating Mosaics using SQL pipelines SELECT m.x*g.size + g.x AS x, m.y*g.size + g.y AS y, g.r, g.g, g.b, m.scale_r, m.scale_g, m.scale_b FROM SourceMatchedPixels m JOIN GalleryPixels g ON m.gallery = g.image https://metanautix.com/tr/01_big_data_techniques_for_media_graphics.pdf
  12. 12. 16 © 2015 METANAUTIX. Uncoupling Compute From Storage • Next generation analytical applications combine data from many sources • Storage system options are multiplying • Where the data is stored matters less than what you’re doing with it
  13. 13. 17 ©2014 METANAUTIX.CONFIDENTIAL. ©2014 METANAUTIX. Before Metanautix Quest Tableau Server Data Query Single Server Join Teradata Couchbase Server No Couchbase Support ODBC Tableau Server Data Query Distributed Join Teradata Couchbase Server VMware/vCloud Size of cluster tunable from VMware/vCloud After Metanautix Quest TPT N1QL
  14. 14. 18 ©2014 METANAUTIX. Demo Setup Store Sales ItemsCustomers Tableau Server Data Query Distributed Join Teradata Couchbase Server TPT N1QL Standard SQL Smart Flatten™
  15. 15. 19 ©2014 METANAUTIX. Quest Couchbase Demo 1 TPC-DS Store Sales Check it out at on quest.metanautix.com: https://goo.gl/8XrwOC
  16. 16. 20 © 2015 METANAUTIX. Quest Couchbase Demo 2 Detecting Failures in Utility Water Pipes Metadata Points
  17. 17. 21 © 2015 METANAUTIX. Quest Couchbase Demo 3 Twitter #TalkPay Data Analysis Check it out at on quest.metanautix.com: https://goo.gl/TQ05xH
  18. 18. 22 © 2015 METANAUTIX. Selected Customer Use-Cases “… Analyses that had taken days now take only minutes with Quest.” Ben Weitzel, VP Customer Insights & Analytics, Shutterfly Collect And Understand Financial Data Across Divisions, Geographies, and Functions Make Better Decisions Build Great Products Anomalous Behavior Detection for Security Huge Enterprise Software and Hardware Company Preventative Maintenance of Field Assets Regional Utility Accelerate Hive Analytics on Billions of Events and 100+M Users Web-scale Online Company Attribute Marketing Spend in multi-channel E-commerce
  19. 19. Thank You! Follow https://metanautix.com/blog @Metanautix Check out Quest

×