Better Together: The New Data Management Orchestra

•Download as PPTX, PDF•

1 like•1,259 views

This document discusses the evolution of data management technologies from early systems like Hadoop and NoSQL databases to modern approaches that combine these technologies. It notes that Hadoop focused on processing large amounts of multi-structured data using Apache components on commodity hardware, while NoSQL databases addressed use cases that were difficult for relational databases. The document outlines how companies like Teradata, Cloudera, and MongoDB now offer unified platforms that bring together relational, NoSQL, and analytics capabilities. A panel discussion is advertised on better integrating these technologies to enable data-driven enterprises.

Technology

1
Better Together: The New Data
Management Orchestra

2
Moderator
Colin White
• President and Founder of BI
Research and DataBase
Associates
• Covers data management,
information integration and
BI

3
Hadoop Beginnings: Apache
Source: Microsoft
“A framework for running
applications on a large
hardware cluster built of
commodity hardware.”
wiki.apache.org/hadoop/
Focus was on programmatic and batch-oriented applications that processed large
amounts of multi-structured data (the original “big data”)
Systems were deployed by assembling Apache components or using Hadoop
distributions from companies such as Cloudera, Hortonworks and MapR

4
NoSQL Beginnings (Examples)
Focus was on use cases and
types of data that were difficult
to implement using a relational
database approach – “non-relational”
is a more appropriate
term to use than “NoSQL.”
Non-relational systems are not
new, but earlier products were
usually proprietary.

The DW Today: Teradata Example
Past Today
Single Platform to Solve All Problems Unified Architecture Data
Query Single Database Query Multi-Databases and Sources
SQL Multi Language (SQL/R/Java/Perl/Ruby/Python)
Structured Data Structured and XML/JSON/Weblogs
Business Users Business Users, Data Scientists, & Developers
Disk Data Storage Hybrid Storage with Solid State Drives
Standard Caching Intelligent Memory
Row Data Storage Hybrid Row/Column Data Storage
On-prem Dedicated Systems
Public, Private and Hosted CloudAgile Phased;
Incremental Delivery; BI SS

6
A Lot Has Changed: Cloudera Example
Relational and NoSQL Database
Enterprise Data Hub
Data Applications
Data Sources
Custom
Applications
Cloudera positioning: “The Enterprise Data Hub Complements the Ecosystem”

7
Our Panelists
Kelly Stirman,
Director of Products, MongoDB
Chris Twogood,
VP of Product and Services
Marketing, Teradata
Charles Zedlewski,
VP of Products, Cloudera

13
Join us
• Hadoop World
– Oct. 15-17, NYC
• Teradata PARTNERS
– Oct. 19-23, Nashville
• MongoDB Days
– Coming to a city near
you

To ingest, store, process and leverage big data for maximum business impact requires integrating systems, processing frameworks, and analytic deployment options. Learn how Cloudera’s enterprise data hub framework, MongoDB, and Teradata Data Warehouse working in concert can enable companies to explore data in new ways and solve problems that not long ago might have seemed impossible. Gone are the days of NoSQL and SQL competing for center stage. Visionary companies are driving data subsystems to operate in harmony. So what’s changed? In this webinar, you will hear from executives at Cloudera, Teradata and MongoDB about the following: How to deploy the right mix of tools and technology to become a data-driven organization Examples of three major data management systems working together Real world examples of how business and IT are benefiting from the sum of the parts Join industry leaders Charles Zedlewski, Chris Twogood and Kelly Stirman for this unique panel discussion, moderated by BI Research analyst, Colin White.

Data lake

GHAZOUANI WAEL

A Data Lake is a storage repository that can store large amount of structured, semi-structured, and unstructured data. It is a place to store every type of data in its native format with no fixed limits on account size or file. It offers high data quantity to increase analytic performance and native integration. Data Lake is like a large container which is very similar to real lake and rivers. Just like in a lake you have multiple tributaries coming in, a data lake has structured data, unstructured data, machine to machine, logs flowing through in real-time.

Scaling Multi-Cloud Deployments with Denodo: Automated Infrastructure Management

Denodo

Evolution from Apache Hadoop to the Enterprise Data Hub by Cloudera - ArabNet...

ArabNet ME

A new foundation for the Modern Information Architecture. Speaker: Amr Awadallah, CTO & Cofounder, Cloudera Our legacy information architecture is not able to cope with the realities of today's business. This is because it is not able to scale to meet our SLAs due to separation of storage and compute, economically store the volumes and types of data we currently confront, provide the agility necessary for innovation, and most importantly, provide a full 360 degree view of our customers, products, and business. In this talk Dr. Amr Awadallah will present the Enterprise Data Hub (EDH) as the new foundation for the modern information architecture. Built with Apache Hadoop at the core, the EDH is an extremely scalable, flexible, and fault-tolerant, data processing system designed to put data at the center of your business.

Leading the Change

Amazon Web Services

Todd Weatherby discusses leading an organization's transition to cloud computing. He outlines the opportunity presented by cloud services in reducing costs and increasing agility. The journey involves planning a phased approach to migrate production applications. The new IT model shifts away from large internal staff and outsourcing contracts towards managed cloud platforms and services. Key steps include assessments, proof of concepts, and training to fully transition the organization.

DataStax on Azure: Deploying an industry-leading data platform for cloud apps...

DataStax

Learn how DataStax Enterprise (DSE) on Microsoft Azure delivers experiences to cloud applications beyond customer expectations. Powered by the industry’s best version of Apache Cassandra™ and leveraging the global scale, hybrid deployment capabilities, and ease of integration of Azure, DSE is the always-on data platform that allows you to focus on what matters most to you by ensuring your applications scale reliably and effortlessly while delivering actionable insight in real-time. View recording: https://youtu.be/kLEkqTH_2Bc Explore all DataStax webinars: http://www.datastax.com/resources/webinars

Unlocking big data with Hadoop + MySQL

Ricky Setyawan

Optimize the performance, cost, and value of databases.pptx

IDERA Software

The document discusses elastic data warehousing using Snowflake's cloud-based data warehouse as a service. Traditional data warehousing and NoSQL solutions are costly and complex to manage. Snowflake provides a fully managed elastic cloud data warehouse that can scale instantly. It allows consolidating all data in one place and enables fast analytics on diverse data sources at massive scale, without the infrastructure complexity or management overhead of other solutions. Customers have realized significantly faster analytics, lower costs, and the ability to easily add new workloads compared to their previous data platforms.

Enterprise Data Hub: The Next Big Thing in Big Data

Cloudera, Inc.

If you missed Strata + Hadoop World, you missed quite a bit. This year's event was packed with Big Data practitioners across industries who shared their experiences and how they are driving new innovations like never before. Just because you weren't there, doesn't mean you missed out. In this session, we'll touch on a few of the key highlights from the show, including: Key trends in Big Data adoption The enterprise data hub How the enterprise data hub is used in practice

Introducing the Snowflake Computing Cloud Data Warehouse

Snowflake Computing

Introducing Snowflake, an elastic data warehouse delivered as a service in the cloud. It aims to simplify data warehousing by removing the need for customers to manage infrastructure, scaling, and tuning. Snowflake uses a multi-cluster architecture to provide elastic scaling of storage, compute, and concurrency. It can bring together structured and semi-structured data for analysis without requiring data transformation. Customers have seen significant improvements in performance, cost savings, and the ability to add new workloads compared to traditional on-premises data warehousing solutions.

Analyzing Semi-Structured Data At Volume In The Cloud

Robert Dempsey

Presentation from Snowflake Computing at the November 2015 Data Wranglers DC meetup. The Cloud, Mobile and Web Applications are producing semi-structured data at an unprecedented rate. IT professionals continue to struggle capturing, transforming, and analyzing these complex data structures mixed with traditional relational style datasets using conventional MPP and/or Hadoop infrastructures. Public cloud infrastructures such as Amazon and Azure provide almost unlimited resources and scalability to handle both structured and semi-structured data (XML, JSON, AVRO) at Petabyte scale. These new capabilities coupled with traditional data management access methods such as SQL allow organizations and businesses new opportunities to leverage analytics at an unprecedented scale while greatly simplifying data pipeline architectures and providing an alternative to the "data lake".

Case Study: Big Data Analytics

Abhinav Das

The client is a US-based product company that offers an enterprise search and discovery platform based on big data technologies. They added a new SQL interface and wanted to benchmark its performance against other alternatives like open source Hadoop+Hive. TenXLabs researched the TPC-H benchmark, generated test data, built clusters for the client's product and Hadoop+Hive, and developed a test harness and web tool to run queries and collect performance statistics to compare the systems. The client was able to identify areas for improvement and was surprised their product performed well compared to MySQL in some cases.

The Future of Data Management: The Enterprise Data Hub

Cloudera, Inc.

The document discusses the future of data management through the use of an enterprise data hub (EDH). It notes that an EDH provides a centralized platform for ingesting, storing, exploring, processing, analyzing and serving diverse data from across an organization on a large scale in a cost effective manner. This approach overcomes limitations of traditional data silos and enables new analytic capabilities.

Cloud Storage Spring Cleaning: A Treasure Hunt

Steven Moy

1) The document discusses how Yelp analyzed their S3 access logs stored in AWS to optimize their cloud storage costs. 2) They used Spark to convert the log files to Parquet format for easier analysis. AWS Athena was then used to run SQL queries on the Parquet files to understand access patterns and age of data. 3) This analysis found that 20% of their data was rarely accessed after 90-400 days and could be moved to the cheaper Infrequent Access storage tier, while 50% of their data was over 400 days old and could be archived to Glacier to reduce costs by around 25% ongoing.

Demystifying Data Warehouse as a Service

Snowflake Computing

Building a Data Lake - An App Dev's Perspective

GeekNightHyderabad

Speaker: Geetha Balasundaram, Developer at ThoughtWorks From tools and technology to people and requirements, what's different in the data engineering space? App development is traditional now. All enterprises want to become data-guided. Data lake is good start yet the know-hows and do-hows are so many. Experiences from building a data lake in the retail domain, the talk will be covering. - What is this vast new space of data engineering, - Why it is critical to think in terms of data rather than features - How important it is to understand these technologies and create a data lake that is usable and insightful to business

Getting Started with Data Virtualization – What problems DV solves

Denodo

Experts and analysts agree that data virtualization's strategic role in enterprise architecture for increasing agility and flexibility in the delivery of information. In this presentation, you will find how data virtualization enables organizations to access, manage, and integrate data from a wide variety of data sources. This presentation is part of the Fast Data Strategy Conference, and you can watch the video here goo.gl/IS9RGK.

Snowflake: Your Data. No Limits (Session sponsored by Snowflake) - AWS Summit...

Amazon Web Services

Snowflake is a cloud-based data warehouse that is built for the cloud. It was founded in 2012 and has raised $1 billion in funding. Snowflake's architecture separates storage, compute, and metadata services, allowing it to offer unlimited scalability, multiple clusters that can access shared data with no downtime, and full transactional consistency across the system. Snowflake has over 2000 customers including large enterprises that use it for analytics, data science, and sharing large volumes of data securely.

Big Data Hadoop Training- Multisoft Systems

Multisoft Systems

Why Data Lake should be the foundation of Enterprise Data Architecture

Agilisium Consulting

Changing the game with cloud dw

elephantscale

This document provides an agenda and overview of a presentation on cloud data warehousing. The presentation discusses data challenges companies face today with large and diverse data sources, and how a cloud data warehouse can help address these challenges by providing unlimited scalability, flexibility, and lower costs. It introduces Snowflake as a first cloud data warehouse built for the cloud, with features like separation of storage and compute, automatic query optimization, and built-in security and encryption. Other cloud data warehouse offerings like Amazon Redshift are also briefly discussed.

Data Ninja Webinar Series: Realizing the Promise of Data Lakes

Denodo

Watch the full webinar: Data Ninja Webinar Series by Denodo: https://goo.gl/QDVCjV The expanding volume and variety of data originating from sources that are both internal and external to the enterprise are challenging businesses in harnessing their big data for actionable insights. In their attempts to overcome big data challenges, organizations are exploring data lakes as consolidated repositories of massive volumes of raw, detailed data of various types and formats. But creating a physical data lake presents its own hurdles. Attend this session to learn how to effectively manage data lakes for improved agility in data access and enhanced governance. This is session 5 of the Data Ninja Webinar Series organized by Denodo. If you want to learn more about some of the solutions enabled by data virtualization, click here to watch the entire series: https://goo.gl/8XFd1O

Exploiting Data Lakes: Architecture, Capabilities & Future

Agilisium Consulting

Webinar - The Agility Challenge - Powering Cloud Apps with Multi-Model & Mixe...

DataStax

Building and managing cloud applications is not easy. Teams come face to face with these challenges: agility, manageability, performance, scalability, continuous availability and of course, security. Join us for “The Agility Challenge: Powering Cloud Applications with Multi-Model & Mixed Workloads” webinar where we will deep dive into challenges customers face with multiple data models such as graph, mixed workloads and how DataStax Enterprise can help. Video: https://youtu.be/1tKDxkexzFE

Modern data warehouse

Stephen Alex

The document discusses the modern data warehouse and key trends driving changes from traditional data warehouses. It describes how modern data warehouses incorporate Hadoop, traditional data warehouses, and other data stores from multiple locations including cloud, mobile, sensors and IoT. Modern data warehouses use multiple parallel processing (MPP) architecture and the Apache Hadoop ecosystem including Hadoop Distributed File System, YARN, Hive, Spark and other tools. It also discusses the top Hadoop vendors and Oracle's technical innovations on Hadoop for data discovery, transformation, discovery and sharing. Finally, it covers the components of big data value assessment including descriptive, predictive and prescriptive analytics.

I Am MongoDB – And So Can You!

Better Together: The New Data Management Orchestra

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (8)

Similar to Better Together: The New Data Management Orchestra

Similar to Better Together: The New Data Management Orchestra (20)

More from MongoDB

More from MongoDB (20)

Recently uploaded

Recently uploaded (20)

Better Together: The New Data Management Orchestra

Editor's Notes