• Share
  • Email
  • Embed
  • Like
  • Private Content
The Perfect Storm: The Impact of Analytics, Big Data and Analytics
 

The Perfect Storm: The Impact of Analytics, Big Data and Analytics

on

  • 583 views

The Briefing Room with Barry Devlin and NuoDB ...

The Briefing Room with Barry Devlin and NuoDB

Live Webcast on Oct. 23, 2012

Three major factors in enterprise computing are combining to rewrite how data is stored, accessed and managed: 1) the demand of analytics that now spreads across hundreds, even thousands of users; 2) the pervasiveness of Big Data in all its forms and sizes; and 3) the rise of the commodity data center, aka Cloud computing. The convergence of these forces calls for a new data foundation, one that can handle the scalability and workload issues that face today's information managers.

Check out this episode of The Briefing Room to learn from veteran Analyst Barry Devlin, one of the very first architects of data warehousing, who will explain how today's information architectures require a radically different approach. He'll be briefed by Barry Morris, Founder and CEO of NuoDB, who will tout his company's product, described as a peer-to-peer messaging system that acts as a database. It behaves just like a traditional relational database, but was designed with a completely distributed and scalable architecture.

http://www.insideanalysis.com

Statistics

Views

Total Views
583
Views on SlideShare
583
Embed Views
0

Actions

Likes
1
Downloads
5
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    The Perfect Storm: The Impact of Analytics, Big Data and Analytics The Perfect Storm: The Impact of Analytics, Big Data and Analytics Presentation Transcript

    • Eric.kavanagh@bloorgroup.comTwitter Tag: #briefr The Briefing Room
    • !   Reveal the essential characteristics of enterprise software, good and bad !   Provide a forum for detailed analysis of today s innovative technologies !   Give vendors a chance to explain their product to savvy analysts !   Allow audience members to pose serious questions... and get answers!Twitter Tag: #briefr The Briefing Room
    • November: Cloud December: Innovators January: Big Data February: Performance March: IntegrationTwitter Tag: #briefr The Briefing Room
    • !  Historically, databases have been built around SQL, a declarative query language targeted at organizing data in two- dimensional tables !  The ever increasing variety, volume and velocity of data has taxed traditional relational databases and created performance bottlenecks, particularly around CPU, memory, disk I/O and network saturation !  Alternatives like NoSQL and NewSQL have emerged to better support extreme and diverse workloads without suffering hits in performanceTwitter Tag: #briefr The Briefing Room
    • Dr. Barry Devlin is a founder of the data warehousing industry and among the foremost authorities worldwide on business intelligence (BI). He is a widely respected consultant, lecturer and author of “Data Warehouse—from Architecture to Implementation.” Barry has 30 years of experience in the IT industry, previously with IBM, as an architect, consultant, manager and software evangelist. As founder and principal and 9sight Consulting (www.9sight.com), Barry provides strategic consulting and thought leadership to buyers and vendors of BI solutions. He is currently developing a new architectural model for fully consistent business support—from informational to operational and collaborative—Business Integrated Insight (BI2). He is based in Cape Town, South Africa.Twitter Tag: #briefr The Briefing Room
    • ! NuoDB is an ACID-compliant NewSQL relational database management system !  It is architected to scale elastically on the cloud !  It leverages a peer-to-peer, distributed architecture ! NuoDB currently has 1000+ users in betaTwitter Tag: #briefr The Briefing Room
    • Barry is an accomplished software CEO with over 25 years of industry experience in running private and public companies around industry- changing paradigm shifts in technology. He had leadership roles at IONA Technologies, which helped lay the groundwork for modern SOA- based systems, and StreamBase Systems, a pioneer of complex event processing. Barry’s early career included technical, management and business development roles. Barry does a great deal of consulting and has served on a variety of boards for startup companies in Boston, Ireland and South Africa. He earned his Degree in Engineering from New College Oxford University and holds an Honorary Doctorate in Business Administration from the IMCA.Twitter Tag: #briefr The Briefing Room
    • The Elastically Scalable Database™ Copyright © NuoDB 2012 1
    • NuoDB The Database for the 21st Century NuoDB is a revolutionary database system based on a patented Emergent Architecture. NuoDB is designed for modern datacenters, workloads and business models.NuoDB delivers all of the capabilities and services of the 20th Century RDBMS.NuoDB has a SQL personality but it could just as easily be a Document Database, an Object Database, a Graph Database or something else.NuoDB Inc is building next generation capabilities that will redefine the role of databases in next generation applications. Copyright © NuoDB 2012 2
    • 20th Century Database 9% Powerful Query 3% 4% Language 19% 44% Industry Standards Data GuaranteesORACLE 21%IBMMicrosoft Employee SkillsSybaseTeradataOthers Existing Data Tools Copyright © NuoDB 2012 3
    • 21st Century Problem Commodity Datacenters ✗ Big Data ✗ Powerful Query Modern Workloads ✗ Language 24x7 Operation ✗ Industry Standards Geo-distribution ✗ Data Guarantees Developer Employee SkillsEmpowerment ✗ 3% 4% 9% Existing Data 19% 44% 21% Tools ORACLE IBM Microsoft Sybase Teradata Others Copyright © NuoDB 2012 4
    • Database CrisisWikipedia Flickr Facebook Amazon Google Source: Marc Bojoly Copyright © NuoDB 2012 5
    • Jim Starkey“Elastically Scalable Transactions represent the biggest breakthrough in database technology in 25 years” ‣ DEC RDB/ELN ‣ InterBase ‣ Firebird ‣ Falcon ‣ BLOBS ‣ MVCC ‣ etc Copyright © NuoDB 2012 6
    • Emergent Database Architecture “An emergent behavior can appear when a number of simpleentities operate in an environment, forming morecomplex behaviors as a collective.” - Wikipedia Copyright © NuoDB 2012 7
    • Poleposition - Single Node NotesMySQL 5.1NuoDB Beta 3 - Single Nodehttp://www.polepos.orgIn early tests NuoDB on asingle node was 2x to 20x faster than MySQL 5.1 running the industry standard Poleposition Benchmarks. Your mileage may vary. ‣ Time taken for given benchmark, normalized to NuoDB = 1 ‣ Less is Better Copyright © NuoDB 2012 8
    • Adding a Second Machine• Second machine typically doubles TPS• Second machine is added to live database while it is running at 1,000’s of TPS• Performance increase is immediate• BTW - you can take either machine away and the database keeps running without data loss Second Machine Instant Performance Increase Copyright © NuoDB 2012 9
    • Adding a Third Machine• Third machine typically triples single machine TPS• Third machine is added to live database while it is running at 1,000’s of TPS• Performance increase is immediate Second & Third Machine• BTW - you can take any Instant Performance Increase machine away and the database keeps running without data loss Copyright © NuoDB 2012 10
    • More Machines? Bring ‘em On Nodes TPS &!!!!" MySQL 1 3,000 %#!!!" NuoDB 1 4,500 %!!!!" NuoDB 9 27,000 $#!!!"Technical Details: TPS $!!!!"‣ 2-9 Tx engines #!!!"‣ 1 storage manager‣ Best sustained TPS and !" # clients combination $" %" &" " #" (" )" *" +"‣ 50% updates Number of Nodes NuoDB running on 9 nodes was approx. 9x faster than MySQL running on 1 node. Copyright © NuoDB 2012 11
    • Or Scale-out on IAAS #!!!"‣ Nuodb scales linearly on !!!!" EC2‣ Per-node performance on &!!!" m1.large nodes approx 50% of our commodity servers %!!!" TPS‣ Just started on optimizing $!!!"‣ RDS runs on 1 node, and gets overloaded with 10+ #!!!" connections !" " #" (" $" )" %" *" &" +" Number of EC2 Nodes Copyright © NuoDB 2012 12
    • Standard SQL - Favorite Tools MS Excel (and other MS tools) Squirrel SQLYou already know how to use NuoDB DBVisualizer Copyright © NuoDB 2012 13
    • NuoDB The Elastically Scalable Database™Applications Brokers Transaction Engines Storage Managers Copyright © NuoDB 2012 14
    • NuoDB Architecture Copyright © NuoDB 2012 15
    • The 21st Century Database OldSQL NoSQL NuoDB Powerful Query Language (SQL) ✓ ✓ Industry Standards (SQL, JDBC, ODBC etc) ✓ ✓ 20th C. Data Guarantees (ACID Database Transactions) ✓ ✓ Employee Skills ✓ ✓ Existing Data ✓ ✓ On-demand Capacity ✓ ✓ Commodity Datacenters / Virtualization / Cloud ✓ ✓ Modern Workloads (Concurrency, TPS, Latency) ½ ✓ Big Data ½ ✓ ✓ 21st C. 100% Uptime ✓ ✓ Database Online Maintenance, Admin and Schema Evolution ✓ ✓ Geo-distribution ✓ ✓ Developer Empowerment ✓ ✓ Zero Touch Backup ✓ ✓ “Zero” Admin ✓ ✓ Copyright © NuoDB 2012 16
    • The Elastically Scalable Database™ Copyright © NuoDB 2012
    • Twitter Tag: #briefr The Briefing Room
    • The Perfect Storm: The Impact ofAnalytics, Big Data and CloudThe Briefing Room, 23 October 2012 Dr Barry Devlin Founder & Principal 9sight Consulting Copyright © 2012 9sight Consulting, All Rights Reserved
    • Three key trends in business are driving rapid change. 1.  Closed-loop business – strategy to execution –  Merge operational, informational & collaborative –  Extreme flexibility in adapting to change 2.  Massive information volumes for use –  Volumes, sources, types 3.  Collaborate to innovate –  Millennials move into power –  Mobile users and applications Faster Bigger Distributed More flexible More personal11 Copyright © 2010-12 9sight Consulting
    • Recent technology advances offer new ways to address emerging business needs. 1.  Closed-loop business – strategy to execution 2.  Massive information volumes for use 3.  Collaborate to innovate 4.  SOA, Mobile Apps and Analytics –  Adaptive IT and design flexibility 5.  Advances in “Data Processing” –  RDBMS advances, Big Data and Cloud 6.  Web / Enterprise 2.0 and beyond –  Collaborative tools, semantic web and more12 Copyright © 2010-12 9sight Consulting
    • Big data is really all data Three domains Business Analytics Flexibility §  Process-mediated data –  “Traditional” operational & informational data –  Via data entry & cleansing Human-sourced information §  Machine-generated data –  Output of machines & sensors –  High-speed, high-volume (Traditional) –  The Internet of Things Business Processes §  Human-sourced information Machine- –  Subjectively interpreted record generated of personal experiences Process- data –  Model unknown before usage mediated –  From Tweets to Videos data §  See: bit.ly/Big_Data_Zoo Timeliness [In the context of these domains, “data” signifies well-structured and/or modeled and “information” is more loosely structured and human-centric.]13 Copyright © 2012 9sight Consulting
    • Technology drives and dictates progress §  Vast improvements in price-performance for memory –  Critical data for most businesses can fit in main memory –  Traditional database design is disk-centric – Commit means on disk – Disk I/O bottleneck is a key design point §  Single processors cannot go any faster; the move to multi- core / multi-processing has been ongoing for over 5 years –  Traditional programming is single-CPU-centric –  MPP – from specialized / high-cost to wide-spread / low-cost §  Physical data representation back at the forefront –  Row store vs column store vs key-value store –  Compression ratios –  Are column stores slow for update?14 Copyright © 2012 9sight Consulting
    • Database – Innovation and evolution §  “Post-relational” Features / –  Flexibility Performance Cumulative progress –  Scalability §  Relational –  A logical model of data’s relationships to “reality” –  Predefined model Next wave? §  Hierarchical & Network –  Speed of record update and access –  Physical storage optimization Relational Niche? Disruptive change Hierarchical & Network Sustaining change “Post-relational” 1960 1970 1980 1990 2000 2010 2020 Clayton M. Christensen, “The Innovator’s Dilemma”, 199715 Copyright © 2012 9sight Consulting
    • The emerging biz-tech ecosystem §  Fully symbiotic existence of business and IT 1.  Interdependence –  New technology enables new business possibilities; new business opportunities drive technology advances 2.  Reintegration –  Silos in business and IT deter Web-savvy customers; coherence becomes mandatory 3.  Cross-over –  Business people need IT skills to see how to recreate the business with new technology; IT people need business acumen to see how to satisfy business needs in new ways with emerging technology16
    • Questions (1) 1.  You emphasize the object-oriented / distributed / message-oriented nature of NuoDB as well as in-memory operation. With improving memory price-performance and the possibility that many businesses will be able to fit all business-critical data in memory, why do you need both? 2.  It seems that disk storage is replaced first by distributed computer storage, and then “failback” to disk. Are you replacing disk I/O latencies with network latencies? How is this an advantage? 3.  As an in-memory database, how do you position NuoDB vs. SAP HANA? 4.  With advances in memory, MPP, columnar stores, etc., I see the possible end of the old operational vs informational split. What is your view? Where does NuoDB fit in that scenario? 5.  Big data – what do you mean by the term? On which aspects of big data does NuoDB focus?17 Copyright © 2012 9sight Consulting
    • Questions (2) 6.  “NoSQL” databases emphasize flexibility to changing data structures mainly by exposing a key-value store to applications. Is that why you use a KV store? How do you benefit from the KV store as it is “locked behind” the relational model? 7.  The query optimizer is perhaps the key to database performance. For most new DBs, it has proven to be a long road to build an optimized optimizer – how will NuoDB address this? 8.  In your white paper you say “database designers don’t need to compromise on schema design by de-normalizing tables, removing joins” for performance… sounds like magic. Why not? 9.  You support indexing. Why do you need it / use it in an in-memory database? 10.  You put Multiversion Concurrency Control (MVCC) forward as the solution to ACID requirements. Do you always insert rather than update?18 Copyright © 2012 9sight Consulting
    • Dr Barry Devlin Founder & Principal 9sight ConsultingCopyright © 2012 9sight Consulting, All Rights Reserved
    • Barry Devlin Founder and Principal 9sight Consulting, www.9sight.com Dr. Barry Devlin is a founder of the data warehousing industry and among the foremost authorities worldwide on business intelligence (BI) and beyond. He is a widely respected consultant, lecturer and author of “Data Warehouse—from Architecture to Implementation”. Barry has 30 years of experience in the IT industry, previously with IBM, as an architect, consultant, manager and software evangelist. As founder and principal of 9sight Consulting (www. 9sight.com), Barry provides strategic consulting and thought- leadership to buyers and vendors of BI solutions. He is currently developing a new architectural model for fully consistent business support—from informational to operational and collaborative—Business Integrated Insight (BI2). Based in Cape Town, South Africa, Barry’s knowledge and expertise are in demand both locally and internationally. Email: barry@9sight.com Twitter: @BarryDevlin20 Copyright © 2012 9sight Consulting
    • Twitter Tag: #briefr The Briefing Room
    • This Month: Database November: Cloud December: Innovators January: Big Data 2013 Editorial Calendar (www.insideanalysis.com)Twitter Tag: #briefr The Briefing Room
    • Twitter Tag: #briefr The Briefing Room