Introducing dashDB MPP: The Power of Data Warehousing in the Cloud
•
8 likes•12,043 views
dashDB Enterprise MPP is a new fully managed cloud data warehouse service with massive scale and performance. Powered by IBM's network cluster architecture, dashDB MPP is an easy to use, self service solution for building: standalone data warehouses; data science data marts; hybrid warehousing; development and QA environments; and analytics for NoSQL. It is available through IBM Bluemix along with IBM's other Cloud Data Services, including Cloudant and SQL DB.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Introducing dashDB MPP: The Power of Data Warehousing in the Cloud
Similar to Introducing dashDB MPP: The Power of Data Warehousing in the Cloud (20)
More from IBM Cloud Data Services
More from IBM Cloud Data Services (17)
Recently uploaded
Recently uploaded (20)
Introducing dashDB MPP: The Power of Data Warehousing in the Cloud
- 1. 1 © 2015 IBM Corporation Introduction to dashDB MPP: The Power of Data Warehousing in the Cloud
- 2. 2 © 2015 IBM Corporation IBM Cloud Data Services
- 3. 3 © 2015 IBM Corporation IBM Cloud Data Services Provides a comprehensive set of rich, integrated cloud data services covering content, data and analytics. Get started: IBM.com/Bluemix
- 4. 4 © 2015 IBM Corporation IBM Cloud Data Services IBM Cloudant NoSQL DBaaS IBM BigInsights on Cloud Enterprise Hadoop as a Service IBM SQL DB relational database service
- 5. 5 © 2015 IBM Corporation IBM dashDB a fully managed cloud data warehouse service designed for performance and scale, compatible with a wide range of BI toolsets and advanced analytics tooling.
- 6. 6 © 2015 IBM Corporation ! Extend your on-premises data warehouse to the cloud
- 7. 7 © 2015 IBM Corporation ! Extend your on-premises data warehouse to the cloud ! Build new, self service cloud warehousing infrastructure
- 8. 8 © 2015 IBM Corporation ! Extend your on-premises data warehouse to the cloud ! Build new, self service cloud warehousing infrastructure ! Ingest structured and unstructured data and run advanced analytics
- 9. 9 © 2015 IBM Corporation dashDB is available as a fully managed service through IBM Bluemix. IBM manages the setup, configuration, tuning and disaster recovery operations, so you can get straight to building…
- 10. 10 © 2015 IBM Corporation dashDB features: " BLU in-memory acceleration " Columnar technology " In-database predictive analytics " Cloudant integration for JSON analytics " Built-in Netezza algorithms " SoftLayer bare metal infrastructure
- 11. 11 © 2015 IBM Corporation IBM dashDB is designed with the greater data ecosystem in mind.
- 12. 12 © 2015 IBM Corporation Integrate: Your Data Sources
- 13. 13 © 2015 IBM Corporation Integrate: Your Analytics
- 14. 14 © 2015 IBM Corporation Integrate: Your BI Tools
- 15. 15 © 2015 IBM Corporation Case in point: Simple data pipe for Salesforce IBM developer advocates have built an open source app for moving your Salesforce CRM data to dashDB
- 16. 16 © 2015 IBM Corporation End result: Learn more: https://developer.ibm.com/clouddataservices/dashdb/
- 17. 17 © 2015 IBM Corporation “With IBM's dashDB and Cloudant, we can leverage a modern and complete cloud-based data analytics portfolio, which allows us to accelerate our delivery of products and services for analytically savvy media companies.”
- 18. 18 © 2015 IBM Corporation “dashDB reduced data queries that used to take all day down to four seconds, totally transforming our business. IBM customized a solution for us, and provided greater support than competing offerings.”
- 19. 19 © 2015 IBM Corporation What’s New " R & Netezza analytics at in-database speed " Oracle compatibility " Data prep with Dataworks Forge " Open data and Twitter data integration " Elastic scalability and ultra high performance via new MPP architecture
- 20. 20 © 2015 IBM Corporation
- 21. 21 © 2015 IBM Corporation MPP = Massively Parallel Processing
- 22. 22 © 2015 IBM Corporation MPP breaks up very large, complex tasks into smaller pieces that can be worked on by multiple resources in parallel
- 23. 23 © 2015 IBM Corporation MPP breaks up very large, complex tasks into smaller pieces that can be worked on by multiple resources in parallel Whose cows are going to have a brand new home first?
- 24. 24 © 2015 IBM Corporation Introducing IBM dashDB Enterprise MPP " Multi-node data warehousing " Analytics performance that scales " As simple to use as non-MPP dashDB configurations " Fully managed system shielding you from the complexities of cluster maintenance Can be ordered now!
- 25. 25 © 2015 IBM Corporation dashDB MPP architecture
- 26. 26 © 2015 IBM Corporation " Multiple nodes, each divided into 8 slices " Dedicated CPU cores, memory, and storage " Queries can be broken up across all data slices for ultra fast query processing CPUsCPUsCPUsCPUs BLU Acceleration Dynamic In-Memory Processing CPUsCPUsCPUsCPUs BLU Acceleration Dynamic In-Memory Processing dashDB MPP architecture
- 27. 27 © 2015 IBM Corporation Case in point: a customer is bringing 60 on prem data marts to the cloud with dashDB MPP
- 28. 28 © 2015 IBM Corporation Five Ways to Use dashDB MPP
- 29. 29 © 2015 IBM Corporation 1. Standalone cloud data warehouse
- 30. 30 © 2015 IBM Corporation 2. Development and QA Systems
- 31. 31 © 2015 IBM Corporation 3. Augmenting the existing data warehouse (Hybrid)
- 32. 32 © 2015 IBM Corporation 4. Analysis of NoSQL Data
- 33. 33 © 2015 IBM Corporation 5. Data science data store
- 34. 34 © 2015 IBM Corporation *Composed of 30 deep analytic queries that a sales report analyst would run to generate reports or dashboard analysis. 60 concurrent user streams, each running a random order of these 30 queries. The proof of the pudding is in the eating… ! IBM data warehouse benchmark* showed speedup of 10 times when compared to a dashDB single-server instance!
- 35. 35 © 2015 IBM Corporation Enterprise MPP • Dedicated, single tenant environment • Bare metal • 24 cores per node • 256 GB memory per node • SSD storage (for about 4 TB of preload data per node) Entry • Shared, multi-tenant environment • 20 GB SAN storage capacity • Freemium: < 1 GB of raw data is free Enterprise - 1TB • Dedicated, single tenant environment • Virtual environment • 16 cores • 64 GB memory • SAN storage (for about 1 TB of preload data) Enterprise - 4 TB • Dedicated, single tenant environment • Bare metal • 32 cores • 256 GB memory • SAN storage (for about 4 TB of preload data) Enterprise - 12 TB • Dedicated, single tenant environment • Bare metal • 32 cores • 256 GB memory • SAN storage (for about 12 TB of preload data) • Fully managed • High performance data warehousing and analytics in the cloud • Scale out to meet your data growth needs A wide range of dashDB plans to suit your needs
- 36. 36 © 2015 IBM Corporation Learn more: www.dashdb.com https://developer.ibm.com/clouddataservices/dashdb Watch demos, tutorials and more: Follow us: http://bit.ly/dashDBYT @GetdashDB
Editor's Notes
- Hi and welcome everyone to another dashDB webcast from IBM Cloud Data Services. My name is Doug Flora, and today I’ll be joined by Sam Lightstone, distinguished engineer and chief architect for IBM Data Warehousing. Sam will be providing an in depth overview of dashDB Enterprise MPP, a new offering from IBM that brings even more scale and performance to the dashDB cloud data warehousing service through a network cluster architecture.
- But before diving into the nitty gritty of dashDB MPP, I’m going to briefly introduce our new viewers to IBM Cloud Data Services and the dashDB cloud data warehouse service.
- Launched in 2014, IBM Cloud Data Services provides a comprehensive set of rich, integrated cloud data services covering content, data and analytics that let you innovate faster, reduce your risk of failure and reduce costs. With CDS, all of IBM’s renowned, trusted technical leadership is now available to you via the cloud, instantly provisioned and managed for you, so you can stay focused on creating new systems that wow your customers. Our fully managed services run on the IBM Softlayer bare metal cloud platform, with industry leading performance that gives you faster throughput, lower latency and more consistency. And we offer the flexibility to deploy your cloud data on public, private or hybrid cloud environments in order to maximize cost efficiency, control and performance. All IBM Cloud Data Services are available on IBM Bluemix, and you can get started with them by just heading over to IBM.com/Bluemix and signing up for a free account.
- IBM Cloud Data Services include IBM Cloudant, the NoSQL database as a service optimized for handling workloads for web and mobile apps; IBM SQL DB, a relational database service that handles web and transactional workloads; IBM BigInsights on Cloud, an enterprise Hadoop as a service that simplifies the adoption and scaling of Hadoop so you can build big data stores to analyze relational and no relational data.
- And, last but not least, IBM dashDB is a fully managed cloud data warehouse service designed for performance and scale, and compatible with a wide range of business intelligence tools and analytics. With dashDB, you have a 24/7 fully managed service that enables a wide variety of data warehousing operations for today’s builders – including developers, database administrators, solutions architects, data scientists and many more.
- dashDB lets you extend your on premises data warehouse to the cloud, so you can bring the cloud’s scale and agility to the simplicity and performance of a data warehouse appliance.
- dashDB also lets you build entirely new, self service and on demand data warehousing infrastructure in the cloud, optimal for ingesting born-on-the-cloud data like web and mobile data.
- dashDB is also a staging ground for analytics, and can ingest both structured and unstructured data from a wide variety of sources, on prem and in the cloud.
- And like all cloud data services, dashDB is available through IBM Bluemix, starting for free. IBM manages the setup, configuration, tuning and disaster recovery operations, so you can get straight to building your newest solutions, applications and architectures and bringing them to market, without having to invest in costly new on premises data warehousing infrastructure.
- The keys to dashDB’s performance lie in its secret sauce – a combination of powerful technologies including IBM BLU Acceleration dynamic in-memory column store, which minimizes input output operations and achieves an order of magnitude in speedup compared to conventional row store databases. dashDB is also directly integrated with Cloudant, making it easy to ingest JSON documents to incorporate unstructured data into your analytics and business intelligence operations; and embedded IBM Netezza in-database analytics, allowing you to run analytics natively in the database, where the data resides, and gain huge efficiencies. Underneath all of this is IBM Softlayer bare metal infrastructure, providing a high degree of performance, control and flexibility.
- What’s more, because no cloud services should exist in isolation, dashDB is designed with the greater business intelligence ecosystem in mind, and is highly compatible with the technologies and toolsets you already know.
- That means you can easily integrate dashDB with your existing data management solutions, ranging from Oracle on premises systems to IBM Pure Data for Analytics appliances to Cloudant NoSQL as a service.
- And you can seamlessly connect to analytics toolkits including aginity, Watston Analytics, esri and R, to realize more value from your data
- And finally, you can integrate with a wide variety of business intelligence tools, including IBM solutions like Cognos and third party offerings like Looker and Tableau.
- Let’s take a quick look at why these integrations are so essential to getting the most out of dashDB. IBM’s own developer advocates recently built an open source application, called the simple data pipe, which enables you to easily move Salesforce CRM data to dashDB, and then build reports on that data using Looker. This solution brings together today’s leading cloud based services to provide a simple way to get a daily readout of Salesforce data – and it’s all made possible because IBM experts are constantly working to build new integrations that help accelerate your business.
- The simple data pipe is available for free on github, and we’ll be announcing more about it soon – in the meantime, if you’d like to learn more, head on over to the URL displayed here and connect with our IBM Cloud Data Services developer advocates.
- Of course, you don’t have to take our word – there are many innovative companies that are already redefining their businesses using dashDB and other solutions in the Cloud Data Services portfolio.
- These companies are finding that dashDB’s ease of use, performance, great support and advanced analytics capabilities are reducing the time and resources they spend building and managing data warehousing infrastructure, leaving them with more bandwidth to build the systems that will give them a competitive edge in today’s data-driven world.
- Now that you have a feel for the basics of dashDB, I’m going to hand the mic to Sam Lightstone. Sam is going to take a deeper dive into the newly available dashDB Enterprise MPP offering. Welcome, Sam.
- One dashDB MPP customer maintains customer data for sports and entertainment venues in a large number of small on-premise SQL Server data marts, and they leverage this data for analytics to help improve operations. The company is now incrementally moving their data marts to the cloud, but they need to scale beyond the confines of a single server in short order. The dashDB MPP service is enabling them to seamlessly migrate their data, and to accelerate their analytics and reporting, without having to manage the system themselves. Long term, the customer plans to consolidate their data on dashDB and move away from their legacy on-premises environment entirely.
- dashDB’s scalability and performance mean you can use it as a standalone, fully managed cloud data warehousing service – regardless of your size, and if you want to build a data mart, a development environment, or an enterprise data warehouse
- If you have a powerful data warehouse on-premise, which you’re using for critical workloads, you may not want your developers testing new code there. With dashDB, your developers can experiment, build new application code and test it on the cloud without disrupting on-premise operations. Because dashDB is compatible with Oracle, DB2, Netezza and PostgreSQL, you can have confidence that code developed and tested on dashDB will run well on premises, too. For data sets up to about 50 GB you can even use dashDB for development and QA free of charge.
- With dashDB you can build your hybrid information management strategy and extend on-premise data warehouse environments to the cloud. Since you pay for capacity as you need it, the platform is elastic and can grow with your business needs.
- You can easily synchronize JSON documents within IBM Cloudant to structured data within dashDB, providing a way to bring BI and analytics to your unstructured data.
- The dashDB service maintains a robust set of predictive analytic algorithms for data scientists and analysts, and includes R runtime and RStudio built in. This makes dashDB an optimal data warehouse to support data analysis and statistical software development. Approximate pre-load uncompressed data.
- An IBM internal data warehouse benchmark* shows that a dashDB MPP instance (3-server configuration) yields a performance speedup of 10 times when compared to a dashDB Enterprise single-server instance (4TB server configuration) in a throughput run of 60 users This stunning speedup is achieved by dashDB MPP scale-out technology Newest generation of Intel E5 v3 chipset 2 times more cores and 3 times more memory Better IO subsystem with higher IOPS and more … * Internal benchmark Composed of 30 deep analytic queries that a sales report analyst would run to generate reports or dashboard analysis 60 concurrent user streams, each running a random order of these 30 queries
- Thanks for joining us today, and please visit us at dashdb.com, where you’ll be just a few clicks away from trying dashDB free of charge. And make sure to follow us on Twitter at getdashDB to get the latest updates and resources on dashDB – see you next time.