Filling the Data Lake - Strata + HadoopWorld San Jose 2016 Preview Presentation

•Download as PPTX, PDF•

4 likes•8,329 views

Preview of the Strata + Hadoop World Strata San Jose 2016 session about truly scalable and automated data onboarding for Hadoop Attend the presentation at the conference to learn how to tackle repeatable, self-service Hadoop ingestion without coding Filling the Data Lake Thursday, March 31 11:50a-12:30p Room 230B http://conferences.oreilly.com/strata/hadoop-big-data-ca/public/schedule/detail/50677

Filling the
Data Lake
Chuck Yarbrough, Director, Pentaho Solutions
Mark Burnette, Enterprise Sales Engineer
Preview presentation for Strata +
HadoopWorld San Jose 2016 session
Thursday, March 31 at 11:50 am, room 230B

Hadoop is Hard…
Empower team
members to
integrate and
process Hadoop
Data
Establish a
modern data on
boarding process
that is flexible and
scalable
Deliver governed
analytic insights
for large
production use
bases
Things that can help ease the pain

Proper Care and Feeding of the Data Lake

How do we effectively scale data pipelines to accommodate
exploding data sources, volumes, and complexity?
More Data, More Problems
Have you ever had the pleasure of…
Migrating hundreds of tables between databases?
Enabling business users to onboard a variety of data themselves?
Ingesting hundreds of changing data sources into Hadoop?

More Data, More Problems
Modern data onboarding is more than
just “connecting” or “loading” – it includes:
Managing a changing array of data sources
Establishing repeatable processes at scale
Maintaining control and governance

CSVCSV
RDBM
S
Big Data On Boarding
Ingest Procedures
Hadoop
AVRO
RDBM
S
Disparate Data Sources Dynamic Integration Processes Dynamic Transformations
RDBM
S

Continuous Big Data On Boarding
Blueprint
Streamline data
ingest from wide
variety of source data
Reduce dependence
on hard coded data
movement procedures
Simplify regular data
movement at scale
into Hadoop

Visit Pentaho in the exhibit hall
booth #1025
Attend the session Filling the Data Lake
Thursday, March 31 11:50a-12:30p
Room 230B
http://conferences.oreilly.com/strata/hadoop-big-data-
ca/public/schedule/detail/50677

Watch full webinar here: https://bit.ly/32TT2Uu Data virtualization is not just for self-service, it’s also a first-class citizen when it comes to modern data platform architectures. Technology has forced many businesses to rethink their delivery models. Startups emerged, leveraging the internet and mobile technology to better meet customer needs (like Amazon and Lyft), disrupting entire categories of business, and grew to dominate their categories. Schedule a complimentary Data Virtualization Discovery Session with g2o. Traditional companies are still struggling to meet rising customer expectations. During this webinar with the experts from g2o and Denodo we covered the following: - How modern data platforms enable businesses to address these new customer expectation - How you can drive value from your investment in a data platform now - How you can use data virtualization to enable multi-cloud strategies Leveraging the strategy insights of g2o and the power of the Denodo platform, companies do not need to undergo the costly removal and replacement of legacy systems to modernize their systems. g2o and Denodo can provide a strategy to create a modern data architecture within a company’s existing infrastructure.

Pentaho Suite Analysis

Kymberly Grayson-Perry

Modern data warehouse

Stephen Alex

Data Integration Alternatives: When to use Data Virtualization, ETL, and ESB

Denodo

Data integration is paramount, in this presentation you will find three different paradigms: using client-side tools, creating traditional data warehouses and the data virtualization solution - the logical data warehouse, comparing each other and positioning data virtualization as an integral part of any future-proof IT infrastructure. This presentation is part of the Fast Data Strategy Conference, and you can watch the video here goo.gl/1q94Ka.

Designing an Agile Fast Data Architecture for Big Data Ecosystem using Logica...

Denodo

Autodesk designed a modern data architecture that heavily uses data virtualization to integrate both legacy data sources and contemporary big data analytics like Spark into a single unified logical data warehouse. In this presentation, you will learn how to build a logical data warehouse using data virtualization and create a single, unified enterprise-wide access and governance point for any data used within the company. This presentation is part of the Fast Data Strategy Conference, and you can watch the video here goo.gl/Ab4PDB.

Enabling Cloud Data Integration (EMEA)

Denodo

Watch full webinar here: https://bit.ly/3FcgiyK Denodo recently released the Denodo Cloud Survey 2021. Learn about some of the insights we have from the survey as well as some of the use cases Denodo comes across in the cloud. We will also conduct a brief product demonstration highlighting how easy it is to migrate to the cloud and support access to data in hybrid cloud architectures. In this session not only will we look at what you, the customers are saying in the Denodo Cloud Survey but also: - We will explore how, in reality, many organizations are already operating in a hybrid or multi-cloud environment and how their needs are being met through the use of a logical data fabric and data virtualization - We will discuss how easy it is to reduce the risk and minimize disruption when migrating to the cloud - We will educate you on why a uniform security layer removes regulatory risk in data governance. - Finally we will demonstrate some of the key capabilities of the Denodo Platform to support the above.

Big Data and Data VirtualizationKenneth Peeples

How Denodo Complement’s Logical Data Lake in Cloud ● Denodo does not substitute data warehouses, data lakes, ETLs... ● Denodo enables the use of all together plus other data sources ○ In a logical data warehouse ○ In a logical data lake ○ They are very similar, the only difference is in the main objective ● There are also use cases where Denodo can be used as data source in a ETL flow

Enterprise Data Lake - Scalable Digital

sambiswal

This white paper will present the opportunities laid down by data lake and advanced analytics, as well as, the challenges in integrating, mining and analyzing the data collected from these sources. It goes over the important characteristics of the data lake architecture and Data and Analytics as a Service (DAaaS) model. It also delves into the features of a successful data lake and its optimal designing. It goes over data, applications, and analytics that are strung together to speed-up the insight brewing process for industry’s improvements with the help of a powerful architecture for mining and analyzing unstructured data – data lake.

Hadoop and Your Data Warehouse

Caserta

Presentation at Data Summit 2015 in NYC. Elliott Cordo shared real-world insights across a range of topics, including the evolving best practices for building a data warehouse on Hadoop that also coexists with multiple processing frameworks and additional non-Hadoop storage platforms, the place for massively parallel-processing and relational databases in analytic architectures, and the ways in which the cloud offers the ability to quickly and cost-effectively establish a scalable platform for your Big Data warehouse. For more information, visit www.casertaconcepts.com

Analyst View of Data Virtualization: Conversations with Boulder Business Inte...

Denodo

In this presentation, executives from Denodo preview the new Denodo Platform 6.0 release that delivers Dynamic Query Optimizer, cloud offering on Amazon Web Services, and self-service data discovery and search. Over 30 analysts, led by Claudia Imhoff, provide input on strategic direction and benefits of Denodo 6.0 to the data virtualization and the broader data integration market. This presentation is part of the Fast Data Strategy Conference, and you can watch the video here goo.gl/DR6r3m.

Designing Fast Data Architecture for Big Data using Logical Data Warehouse a...

Denodo

Companies such as Autodesk are fast replacing the once-true- and-tried physical data warehouses with logical data warehouses/ data lakes. Why? Because they are able to accomplish the same results in 1/6 th of the time and with 1/4 th of the resources. In this webinar, Autodesk’s Platform Lead, Kurt Jackson,, will describe how they designed a modern fast data architecture as a single unified logical data warehouse/ data lake using data virtualization and contemporary big data analytics like Spark. Logical data warehouse / data lake is a virtual abstraction layer over the physical data warehouse, big data repositories, cloud, and other enterprise applications. It unifies both structured and unstructured data in real-time to power analytical and operational use cases.

Hadoop: Extending your Data Warehouse

Cloudera, Inc.

Data Lake,beyond the Data Warehouse

Data Science Thailand

The Future of Data Warehousing: ETL Will Never be the Same

Cloudera, Inc.

Traditional data warehouse ETL has become too slow, too complicated, and too expensive to address the torrent of new data sources and new analytic approaches needed for decision making. The new ETL environment is already looking drastically different. In this webinar, Ralph Kimball, founder of the Kimball Group, and Manish Vipani, Vice President and Chief Architect of Enterprise Architecture at Kaiser Permanente will describe how this new ETL environment is actually implemented at Kaiser Permanente. They will describe the successes, the unsolved challenges, and their visions of the future for data warehouse ETL.

Taming Big Data With Modern Software Architecture

Big Data User Group Karlsruhe/Stuttgart

Big Data kennen sehr viele IT-Experten, wenigstens haben Sie eine Vorstellung davon. In der Praxis arbeiten damit in Deutschland derzeit nur wenige. Dabei bringt Big Data ein ganz neues Momentum in moderne Softwarelösungen und ist im Kontext der Mobil-, Cloud- und Social-Veränderungen nicht wegzudenken. Big Data macht Software intelligent und damit auf eine ganz neue Art für die Benutzer erlebbar. Mit Big Data entstehen neue Softwarearchitekturen, weil Informationen völlig anders verarbeitet werden - nämlich schneller, differenzierter und oft mit dem Ziel, Schlüsse zu ziehen und Vorhersagen zu treffen. In diesem Vortrag wird erläutert, wie moderne Softwarearchitekturen gestaltet werden, sodass Sie Big Data Paradigmen erfolgreich umsetzen und welche Vorteile sich für die zunehmend mobilen Softwarelösungen ergeben. Wir werfen zudem einen Blick auf die Potentiale und Optionen in Branchen wie Banken, Versicherung oder Handel.

Denodo DataFest 2016: What’s New in Denodo Platform – Demo and Roadmap

Denodo

Watch the full session: Denodo DataFest 2016 sessions: https://goo.gl/ptGwp7 Curious about product roadmap? In this session, we will review some of the new key features introduced this year in the Denodo Platform in areas such as performance, self-service, security and monitoring. We will also take a sneak peek at the most exciting features in the roadmap for Denodo 7.0. In this session, you will learn: • New performance-related features in big data scenarios • New governance and self-service features • New connectivity, data transformation, and enterprise-wide deployment features This session is part of the Denodo DataFest 2016 event. You can also watch more Denodo DataFest sessions on demand here: https://goo.gl/VXb6M6

Gartner Cool Vendor Report 2014

jenjermain

Big Data Day LA 2015 - Data Lake - Re Birth of Enterprise Data Thinking by Ra...

Data Con LA

Data Virtualization: From Zero to Hero

Denodo

Watch full webinar here: https://bit.ly/2xc6IO0 To solve these challenges, according to Gartner "through 2022, 60% of all organizations will implement data virtualization as one key delivery style in their data integration architecture". It is clear that data virtualization has become a driving force for companies to implement agile, real-time and flexible enterprise data architecture. In this session we will look at the data integration challenges solved by data virtualization, the main use cases and examine why this technology is growing so fastly. You will learn: - What data virtualization really is - How it differs from other enterprise data integration technologies - Why data virtualization is finding enterprise-wide deployment inside some of the largest organizations

Minimizing the Complexities of Machine Learning with Data Virtualization

Denodo

Watch full webinar here: https://buff.ly/309CZ1Y Advanced data science techniques, like machine learning, have proven an extremely useful tool to derive valuable insights from existing data. Platforms like Spark, and complex libraries for R, Python and Scala put advanced techniques at the fingertips of the data scientists. However, these data scientists spent most of their time looking for the right data and massaging it into a usable format. Data virtualization offers a new alternative to address these issues in a more efficient and agile way. Attend this webinar and learn: *How data virtualization can accelerate data acquisition and massaging, providing the data scientist with a powerful tool to complement their practice *How popular tools from the data science ecosystem: Spark, Python, Zeppelin, Jupyter, etc. integrate with Denodo *How you can use the Denodo Platform with large data volumes in an efficient way *About the success McCormick has had as a result of seasoning the Machine Learning and Blockchain Landscape with data virtualization

Denodo DataFest 2016: Comparing and Contrasting Data Virtualization With Data...

Denodo

Watch the full session: Denodo DataFest 2016 sessions: https://goo.gl/Bvmvc9 Data prep and data blending are terms that have come to prominence over the last year or two. On the surface, they appear to offer functionality similar to data virtualization…but there are important differences! In this session, you will learn: • How data virtualization complements or contrasts technologies such as data prep and data blending • Pros and cons of functionality provided by data prep, data catalog and data blending tools • When and how to use these different technologies to be most effective This session is part of the Denodo DataFest 2016 event. You can also watch more Denodo DataFest sessions on demand here: https://goo.gl/VXb6M6

Where does Fast Data Strategy Fit within IT Projects

Denodo

Fast Data Strategy is a must for organizations to become and be competitive. There are four use cases where Fast Data Strategy fits within IT Projects - Agile BI, Big Data/ Cloud, Data Services, and Single View. In this presentation, you will discover how four customers used data virtualization and Fast Data Strategy for these use cases. This presentation is part of the Fast Data Strategy Conference, and you can watch the video here goo.gl/UxHMuJ.

Building a Logical Data Fabric using Data Virtualization (ASEAN)

Denodo

Watch full webinar here: https://bit.ly/3FF1ubd In the recent Building the Unified Data Warehouse and Data Lake report by leading industry analysts TDWI, we have discovered 64% of organizations stated the objective for a unified Data Warehouse and Data Lakes is to get more business value and 84% of organizations polled felt that a unified approach to Data Warehouses and Data Lakes was either extremely or moderately important. In this session, you will learn how your organization can apply a logical data fabric and the associated technologies of machine learning, artificial intelligence, and data virtualization can reduce time to value. Hence, increasing the overall business value of your data assets. KEY TAKEAWAYS: - How a Logical Data Fabric is the right approach to assist organizations to unify their data. - The advanced features of a Logical Data Fabric that assist with the democratization of data, providing an agile and governed approach to business analytics and data science. - How a Logical Data Fabric with Data Virtualization enhances your legacy data integration landscape to simplify data access and encourage self-service.

Designing modern dw and data lake

punedevscom

From Traditional Data Warehouse To Real Time Data Warehouse

Osama Hussein

Data Lakes: 8 Enterprise Data Management Requirements

SnapLogic

2016 is the year of the data lake. As you consider adopting an enterprise data lake strategy to manage more dynamic, poly-structured data, your data integration strategy must also evolve to handle new requirements. Thinking you can simply hire more developers to write code or rely on your legacy rows-and-columns centric tools is a recipe to sink in a data swamp instead of swimming in a data lake. In this presentation, you'll learn about eight enterprise data management requirements that must be addressed in order to get maximum value from your big data technology investments. To learn more, visit: https://www.snaplogic.com/big-data

Data Mashups for Analytics

Pentaho

Explore how data integration (or “mashups”) can maximize analytic value and help business teams create streamlined data pipelines that enables ad-hoc analytic inquiries. You’ll learn why businesses increasingly focused on blending data on demand and at the source, the concrete analytic advantages that this approach delivers, and the type of architectures required for delivering trusted, blended data. We provide a checklist to assess your data integration needs and capabilities, and review some real-world examples of how blending various data types has created significant analytic value and concrete business impact.

Big Data for Product Managers

Pentaho

What's hot

Unlock Your Data for ML & AI using Data Virtualization

Denodo

Enterprise Data Lake - Scalable Digital

sambiswal

Hadoop and Your Data Warehouse

Caserta

Analyst View of Data Virtualization: Conversations with Boulder Business Inte...

Denodo

Designing Fast Data Architecture for Big Data using Logical Data Warehouse a...

Denodo

Hadoop: Extending your Data Warehouse

Cloudera, Inc.

Data Lake,beyond the Data Warehouse

Data Science Thailand

The Future of Data Warehousing: ETL Will Never be the Same

Cloudera, Inc.

Taming Big Data With Modern Software Architecture

Big Data User Group Karlsruhe/Stuttgart

Denodo DataFest 2016: What’s New in Denodo Platform – Demo and Roadmap

Denodo

Gartner Cool Vendor Report 2014

jenjermain

Big Data Day LA 2015 - Data Lake - Re Birth of Enterprise Data Thinking by Ra...

Data Con LA

Data Virtualization: From Zero to Hero

Denodo

Minimizing the Complexities of Machine Learning with Data Virtualization

Denodo

Denodo DataFest 2016: Comparing and Contrasting Data Virtualization With Data...

Denodo

Where does Fast Data Strategy Fit within IT Projects

Denodo

Building a Logical Data Fabric using Data Virtualization (ASEAN)

Denodo

Designing modern dw and data lake

punedevscom

From Traditional Data Warehouse To Real Time Data Warehouse

Osama Hussein

Data Lakes: 8 Enterprise Data Management Requirements

SnapLogic

What's hot (20)

Unlock Your Data for ML & AI using Data Virtualization

Enterprise Data Lake - Scalable Digital

Hadoop and Your Data Warehouse

Analyst View of Data Virtualization: Conversations with Boulder Business Inte...

Designing Fast Data Architecture for Big Data using Logical Data Warehouse a...

Hadoop: Extending your Data Warehouse

Data Lake,beyond the Data Warehouse

The Future of Data Warehousing: ETL Will Never be the Same

Taming Big Data With Modern Software Architecture

Denodo DataFest 2016: What’s New in Denodo Platform – Demo and Roadmap

Gartner Cool Vendor Report 2014

Big Data Day LA 2015 - Data Lake - Re Birth of Enterprise Data Thinking by Ra...

Data Virtualization: From Zero to Hero

Minimizing the Complexities of Machine Learning with Data Virtualization

Denodo DataFest 2016: Comparing and Contrasting Data Virtualization With Data...

Where does Fast Data Strategy Fit within IT Projects

Building a Logical Data Fabric using Data Virtualization (ASEAN)

Designing modern dw and data lake

From Traditional Data Warehouse To Real Time Data Warehouse

Data Lakes: 8 Enterprise Data Management Requirements

Viewers also liked

Data Mashups for Analytics

Pentaho

Big Data for Product Managers

Pentaho

Up Your Analytics Game with Pentaho and Vertica

Pentaho

Big Data is a game-changer. In the face of exploding volumes and varieties of data, traditional data management and ETL systems just aren’t cutting it anymore. A new way of sifting through vast volumes of data to find the most relevant info, combining this data with other data sources to extract faster insights is desperately needed. Enter HP|Vertica and Pentaho with a proven solution for lightning fast queries and blended data and analytics capabilities for your business users.

Big Data Predictions for 2015

Pentaho

Improving the Business of Healthcare through Better Analytics

Pentaho

The Next Big Thing in Big Data

Pentaho

James Dixon has a unique perspective on the big data space - he coined the term "data lake." In this on-demand webinar the Big Data Maverick talks big data - watch to learn more about the technology landscape and evolving use cases. He covers topics such as: - What are today's technologies of choice - where did they come from and why? - Why is the emergence and definition of these use cases so important? - What technologies are likely to come next? - Why did the data explosion start and will it continue? - Why are data scientists in such huge demand? - What is the role of open source in big data, and the role of big data in open source?

Data Integration and Advanced Analytics for MongoDB: Blend, Enrich and Analyz...MongoDB

Competitive edgewithmongod bandpentaho_2014sep_v3[1]

Pentaho

With the combination of Pentaho and MongoDB, it’s drastically simpler and faster to build single analytical views of clients by aggregating and blending data from a variety of internal sources (customer, transaction, position data) and external sources (social networking, central bank, news, pricing) with fast response times. Webinar covers: An insider’s view of new ways financial services companies are using MongoDB to rapidly store and consume unlimited shapes and sizes of data How Pentaho makes it easy to enrich data in MongoDB with predictive scoring, visual data integration tools, reports, interactive dashboards, and data visualizations A live demo of blending Twitter, equity pricing, and news data into a single analytical view that unlocks market intelligence to create investment opportunities

Why Your Product Needs an Analytic Strategy

Pentaho

Data Is Your Next Product Opportunity

Pentaho

Open data day 2017

Daniele Mondello

Eighty20 open data standards for financial inclusion

Jessica Robey

Pentaho Analytics for MongoDB - presentation from MongoDB World 2014

Pentaho

01 2017 lacquadifiori

L'acqua di Fiori

OneMap Myanmar Open Data Training

Neil Sorensen

OneMap project works with government, civil society, ethnic groups and communities, to produce, enhance, and share high quality and accuracy data on land and other natural resources. The open-access, online OneMap spatial data platform democratizes access to multi-sectoral data. It aims to function as an effective basis for transparent and accountable land governance and development planning by government and citizens. By supporting government and civil society alike, OneMap provides space for multi-stakeholders based production and verification of key datasets, and thereby allows different perspectives on land to be equally represented.

Open data De Lijn

Open Knowledge Belgium

Introduction To Pentaho

DataminingTools Inc

Machine learning workshop @DYP Pune

Ganesh Raskar

Data Governance

SambaSoup

Big Data and Data Science @ BNL - D. Morgagni & L. Dell'Anna

Data Driven Innovation

Sono molteplici le sfide che una banca deve essere in grado di fronteggiare per riuscire a valorizzare al meglio i propri dati generando nuove opportunità di Business. In quest’ottica i Big Data e l’Analitycs svolgono un ruolo chiave nella definizione di un’azienda “data driver”. Il proseguimento di questi obiettivi passa attraverso l’evoluzione architettura, l’introduzione di nuove metodologie e l’inserimento di nuove competenze. L’obiettivo di questo intervento è quello di presentare come queste tematiche si calano all’interno del contesto BNL.

Viewers also liked (20)

Data Mashups for Analytics

Big Data for Product Managers

Up Your Analytics Game with Pentaho and Vertica

Big Data Predictions for 2015

Improving the Business of Healthcare through Better Analytics

The Next Big Thing in Big Data

Data Integration and Advanced Analytics for MongoDB: Blend, Enrich and Analyz...

Competitive edgewithmongod bandpentaho_2014sep_v3[1]

Why Your Product Needs an Analytic Strategy

Data Is Your Next Product Opportunity

Open data day 2017

Eighty20 open data standards for financial inclusion

Pentaho Analytics for MongoDB - presentation from MongoDB World 2014

01 2017 lacquadifiori

OneMap Myanmar Open Data Training

Open data De Lijn

Introduction To Pentaho

Machine learning workshop @DYP Pune

Data Governance

Big Data and Data Science @ BNL - D. Morgagni & L. Dell'Anna

Similar to Filling the Data Lake - Strata + HadoopWorld San Jose 2016 Preview Presentation

Pervasive DataRushtempledf

WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRobertsJane Roberts

Haddop in Business IntelligenceHGanesh

Planning your Next-Gen Change Data Capture (CDC) Architecture in 2019 - Strea...

Impetus Technologies

Traditional databases and batch ETL operations have not been able to serve the growing data volumes and the need for fast and continuous data processing. How can modern enterprises provide their business users real-time access to the most up-to-date and complete data? In our upcoming webinar, our experts will talk about how real-time CDC improves data availability and fast data processing through incremental updates in the big data lake, without modifying or slowing down source systems. Join this session to learn: What is CDC and how it impacts business The various methods for CDC in the enterprise data warehouse The key factors to consider while building a next-gen CDC architecture: Batch vs. real-time approaches Moving from just capturing and storing, to capturing enriching, transforming, and storing Avoiding stopgap silos to state-through processing Implementation of CDC through a live demo and use-case You can view the webinar here - https://www.streamanalytix.com/webinar/planning-your-next-gen-change-data-capture-cdc-architecture-in-2019/ For more information visit - https://www.streamanalytix.com

Hadoop data-lake-white-paper

Supratim Ray

Accelerating Big Data Analytics

Attunity

Is the traditional data warehouse dead?

James Serra

With new technologies such as Hive LLAP or Spark SQL, do I still need a data warehouse or can I just put everything in a data lake and report off of that? No! In the presentation I’ll discuss why you still need a relational data warehouse and how to use a data lake and a RDBMS data warehouse to get the best of both worlds. I will go into detail on the characteristics of a data lake and its benefits and why you still need data governance tasks in a data lake. I’ll also discuss using Hadoop as the data lake, data virtualization, and the need for OLAP in a big data solution. And I’ll put it all together by showing common big data architectures.

Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...

Hortonworks

Many enterprises are turning to Apache Hadoop to enable Big Data Analytics and reduce the costs of traditional data warehousing. Yet, it is hard to succeed when 80% of the time is spent on moving data and only 20% on using it. It’s time to swap the 80/20! The Big Data experts at Attunity and Hortonworks have a solution for accelerating data movement into and out of Hadoop that enables faster time-to-value for Big Data projects and a more complete and trusted view of your business. Join us to learn how this solution can work for you.

Modern data warehouse

Stephen Alex

"Beyond the Data Lake", Matthias Korn, Technical Consultant at datavirtuality

Dataconomy Media

"Beyond the Data Lake", Matthias Korn, Technical Consultant at datavirtuality YouTube Link: https://www.youtube.com/watch?v=P5StySlZTzU Watch more from Data Natives 2015 here: http://bit.ly/1OVkK2J Visit the conference website to learn more: www.datanatives.io Follow Data Natives: https://www.facebook.com/DataNatives https://twitter.com/DataNativesConf Stay Connected to Data Natives by Email: Subscribe to our newsletter to get the news first about Data Natives 2016: http://bit.ly/1WMJAqS Presentation Slides: https://www.slideshare.net/secret/sis... About the Author: Matthias Korn, Technical Consultant at datavirtuality. His talk, "Beyond the Data Lake" will talk about the shift in digital era, which harnesses large amounts of data to make astute business decisions and improve operations, which is now an imperative. While our ability to generate data still far outstrips our ability to analyze it, we are making strides. Exciting new approaches are merging big data solutions with traditional enterprise data strategies. Logical data warehouses, in which there is no single data repository, hold enormous promise. By offering an ecosystem of multiple best-fit repositories, technologies, and tools, business can effectively analyze data for powerful insight.

Choosing technologies for a big data solution in the cloud

James Serra

Has your company been building data warehouses for years using SQL Server? And are you now tasked with creating or moving your data warehouse to the cloud and modernizing it to support “Big Data”? What technologies and tools should use? That is what this presentation will help you answer. First we will cover what questions to ask concerning data (type, size, frequency), reporting, performance needs, on-prem vs cloud, staff technology skills, OSS requirements, cost, and MDM needs. Then we will show you common big data architecture solutions and help you to answer questions such as: Where do I store the data? Should I use a data lake? Do I still need a cube? What about Hadoop/NoSQL? Do I need the power of MPP? Should I build a "logical data warehouse"? What is this lambda architecture? Can I use Hadoop for my DW? Finally, we’ll show some architectures of real-world customer big data solutions. Come to this session to get started down the path to making the proper technology choices in moving to the cloud.

Filling the Data Lake

DataWorks Summit/Hadoop Summit

Big data and apache hadoop adoption

faizrashid1995

Asterix Solution’s Hadoop Training is designed to help applications scale up from single servers to thousands of machines. With the rate at which memory cost decreased the processing speed of data never increased and hence loading the large set of data is still a big headache and here comes Hadoop as the solution for it. http://www.asterixsolution.com/big-data-hadoop-training-in-mumbai.html Duration - 25 hrs Session - 2 per week Live Case Studies - 6 Students - 16 per batch Venue - Thane

Stratebi Big Data

Stratebi

Building a Big Data Solution

James Serra

As a follow-on to the presentation "Building an Effective Data Warehouse Architecture", this presentation will explain exactly what Big Data is and its benefits, including use cases. We will discuss how Hadoop, the cloud and massively parallel processing (MPP) is changing the way data warehouses are being built. We will talk about hybrid architectures that combine on-premise data with data in the cloud as well as relational data and non-relational (unstructured) data. We will look at the benefits of MPP over SMP and how to integrate data from Internet of Things (IoT) devices. You will learn what a modern data warehouse should look like and how the role of a Data Lake and Hadoop fit in. In the end you will have guidance on the best solution for your data warehouse going forward.

ETL big data with apache hadoop

Maulik Thaker

From Relational Database Management to Big Data: Solutions for Data Migration...

Cognizant

Big data migration testing for transferring relational database management files is a very time-consuming, high-compute task; we offer a hands-on, detailed framework for data validation in an open source (Hadoop) environment incorporating Amazon Web Services (AWS) for cloud capacity, S3 (Simple Storage Service) and EMR (Elastic MapReduce), Hive tables, Sqoop tools, PIG scripting and Jenkins Slave Machines.

Bridging the Big Data Gap in the Software-Driven World

CA Technologies

Implementing and managing a Big Data environment effectively requires essential efficiencies such as automation, performance monitoring and flexible infrastructure management. Discover new innovations that enable you to manage entire Big Data environments with unparalleled ease of use and clear enterprise visibility across a variety of data repositories. To learn more about Mainframe solutions from CA Technologies, visit: http://bit.ly/1wbiPkl

Data_Migration_to_SAP_S4_HANA.pptx

subbulokam

How Hadoop Revolutionized Data Warehousing at Yahoo and Facebook

Amr Awadallah

Similar to Filling the Data Lake - Strata + HadoopWorld San Jose 2016 Preview Presentation (20)

Pervasive DataRush

WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRoberts

Haddop in Business Intelligence

Planning your Next-Gen Change Data Capture (CDC) Architecture in 2019 - Strea...

Hadoop data-lake-white-paper

Accelerating Big Data Analytics

Is the traditional data warehouse dead?

Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...

Modern data warehouse

"Beyond the Data Lake", Matthias Korn, Technical Consultant at datavirtuality

Choosing technologies for a big data solution in the cloud

Filling the Data Lake

Big data and apache hadoop adoption

Stratebi Big Data

Building a Big Data Solution

ETL big data with apache hadoop

From Relational Database Management to Big Data: Solutions for Data Migration...

Bridging the Big Data Gap in the Software-Driven World

Data_Migration_to_SAP_S4_HANA.pptx

How Hadoop Revolutionized Data Warehousing at Yahoo and Facebook

More from Pentaho

30 for 30: Quick Start Your Pentaho Evaluation

Pentaho

Embedded Analytics in CRM and Marketing

Pentaho

Embedded Analytics in Customer Success

Pentaho

Embedded Analytics in Human Capital Management

Pentaho

Exclusive Verizon Employee Webinar: Getting More From Your CDR DataPentaho

Predictive Analytics with Pentaho Data Mining - Análisis Predictivo con Penta...

Pentaho

This webinar is in Spanish - El uso de análisis predictivo o minería de datos está en auge. A nivel mundial, cada vez más, las empresas contratan servicios especializados de análisis de información que ayuden a marcar una diferencia con la competencia. Por otro lado, el volumen creciente de data así como su naturaleza cambiante y compleja, hacen inmanejable el proceso de análisis de forma tradicional y está siendo necesario incorporar tecnología y consultoría de punta, basada en el uso de modelos matemáticos avanzados. Pentaho Corporation y Matrix CPM Solutions los invita a participar en el seminario en línea “Análisis Predictivo con Pentaho Data Mining”, en donde se revisarán las grandes oportunidades que existen para su uso y aplicación.

Big Data Integration Webinar: Reducing Implementation Efforts of Hadoop, NoSQ...Pentaho

Big Data Integration Webinar: Getting Started With Hadoop Big DataPentaho

Pentaho Healthcare SolutionsPentaho

Pentaho Business Analytics for ISVs and SaaS providers in healthcarePentaho

Bay Area Hadoop User Group

Pentaho

More from Pentaho (11)

30 for 30: Quick Start Your Pentaho Evaluation

Embedded Analytics in CRM and Marketing

Embedded Analytics in Customer Success

Embedded Analytics in Human Capital Management

Exclusive Verizon Employee Webinar: Getting More From Your CDR Data

Predictive Analytics with Pentaho Data Mining - Análisis Predictivo con Penta...

Big Data Integration Webinar: Reducing Implementation Efforts of Hadoop, NoSQ...

Big Data Integration Webinar: Getting Started With Hadoop Big Data

Pentaho Healthcare Solutions

Pentaho Business Analytics for ISVs and SaaS providers in healthcare

Bay Area Hadoop User Group

Recently uploaded

2024 RoOUG Security model for the cloud.pptx

Georgi Kodinov

Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...

informapgpstrackings

Vitthal Shirke Microservices Resume Montevideo

Vitthal Shirke

Corporate Management | Session 3 of 3 | Tendenci AMS

Tendenci - The Open Source AMS (Association Management Software)

Experience our free, in-depth three-part Tendenci Platform Corporate Membership Management workshop series! In Session 1 on May 14th, 2024, we began with an Introduction and Setup, mastering the configuration of your Corporate Membership Module settings to establish membership types, applications, and more. Then, on May 16th, 2024, in Session 2, we focused on binding individual members to a Corporate Membership and Corporate Reps, teaching you how to add individual members and assign Corporate Representatives to manage dues, renewals, and associated members. Finally, on May 28th, 2024, in Session 3, we covered questions and concerns, addressing any queries or issues you may have. For more Tendenci AMS events, check out www.tendenci.com/events

Globus Compute Introduction - GlobusWorld 2024

Globus

Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...

Globus

The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.

WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation

WSO2

How Recreation Management Software Can Streamline Your Operations.pptx

wottaspaceseo

Recreation management software streamlines operations by automating key tasks such as scheduling, registration, and payment processing, reducing manual workload and errors. It provides centralized management of facilities, classes, and events, ensuring efficient resource allocation and facility usage. The software offers user-friendly online portals for easy access to bookings and program information, enhancing customer experience. Real-time reporting and data analytics deliver insights into attendance and preferences, aiding in strategic decision-making. Additionally, effective communication tools keep participants and staff informed with timely updates. Overall, recreation management software enhances efficiency, improves service delivery, and boosts customer satisfaction.

top nidhi software solution freedownload

vrstrong314

This presentation emphasizes the importance of data security and legal compliance for Nidhi companies in India. It highlights how online Nidhi software solutions, like Vector Nidhi Software, offer advanced features tailored to these needs. Key aspects include encryption, access controls, and audit trails to ensure data security. The software complies with regulatory guidelines from the MCA and RBI and adheres to Nidhi Rules, 2014. With customizable, user-friendly interfaces and real-time features, these Nidhi software solutions enhance efficiency, support growth, and provide exceptional member services. The presentation concludes with contact information for further inquiries.

Strategies for Successful Data Migration Tools.pptx

varshanayak241

Data migration is a complex but essential task for organizations aiming to modernize their IT infrastructure and leverage new technologies. By understanding common challenges and implementing these strategies, businesses can achieve a successful migration with minimal disruption. Data Migration Tool like Ask On Data play a pivotal role in this journey, offering features that streamline the process, ensure data integrity, and maintain security. With the right approach and tools, organizations can turn the challenge of data migration into an opportunity for growth and innovation.

Into the Box 2024 - Keynote Day 2 Slides.pdf

Ortus Solutions, Corp

GlobusWorld 2024 Opening Keynote session

Globus

Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL

Natan Silnitsky

In software engineering, the right architecture is essential for robust, scalable platforms. Wix has undergone a pivotal shift from event sourcing to a CRUD-based model for its microservices. This talk will chart the course of this pivotal journey. Event sourcing, which records state changes as immutable events, provided robust auditing and "time travel" debugging for Wix Stores' microservices. Despite its benefits, the complexity it introduced in state management slowed development. Wix responded by adopting a simpler, unified CRUD model. This talk will explore the challenges of event sourcing and the advantages of Wix's new "CRUD on steroids" approach, which streamlines API integration and domain event management while preserving data integrity and system resilience. Participants will gain valuable insights into Wix's strategies for ensuring atomicity in database updates and event production, as well as caching, materialization, and performance optimization techniques within a distributed system. Join us to discover how Wix has mastered the art of balancing simplicity and extensibility, and learn how the re-adoption of the modest CRUD has turbocharged their development velocity, resilience, and scalability in a high-growth environment.

Accelerate Enterprise Software Engineering with Platformless

WSO2

Key takeaways: Challenges of building platforms and the benefits of platformless. Key principles of platformless, including API-first, cloud-native middleware, platform engineering, and developer experience. How Choreo enables the platformless experience. How key concepts like application architecture, domain-driven design, zero trust, and cell-based architecture are inherently a part of Choreo. Demo of an end-to-end app built and deployed on Choreo.

A Comprehensive Look at Generative AI in Retail App Testing.pdf

kalichargn70th171

Advanced Flow Concepts Every Developer Should Know

Peter Caitens

Visitor Management System in India- Vizman.app

NaapbooksPrivateLimi

Your Digital Assistant. Making complex approach simple. Straightforward process saves time. No more waiting to connect with people that matter to you. Safety first is not a cliché - Securely protect information in cloud storage to prevent any third party from accessing data. Would you rather make your visitors feel burdened by making them wait? Or choose VizMan for a stress-free experience? VizMan is an automated visitor management system that works for any industries not limited to factories, societies, government institutes, and warehouses. A new age contactless way of logging information of visitors, employees, packages, and vehicles. VizMan is a digital logbook so it deters unnecessary use of paper or space since there is no requirement of bundles of registers that is left to collect dust in a corner of a room. Visitor’s essential details, helps in scheduling meetings for visitors and employees, and assists in supervising the attendance of the employees. With VizMan, visitors don’t need to wait for hours in long queues. VizMan handles visitors with the value they deserve because we know time is important to you. Feasible Features One Subscription, Four Modules – Admin, Employee, Receptionist, and Gatekeeper ensures confidentiality and prevents data from being manipulated User Friendly – can be easily used on Android, iOS, and Web Interface Multiple Accessibility – Log in through any device from any place at any time One app for all industries – a Visitor Management System that works for any organisation. Stress-free Sign-up Visitor is registered and checked-in by the Receptionist Host gets a notification, where they opt to Approve the meeting Host notifies the Receptionist of the end of the meeting Visitor is checked-out by the Receptionist Host enters notes and remarks of the meeting Customizable Components Scheduling Meetings – Host can invite visitors for meetings and also approve, reject and reschedule meetings Single/Bulk invites – Invitations can be sent individually to a visitor or collectively to many visitors VIP Visitors – Additional security of data for VIP visitors to avoid misuse of information Courier Management – Keeps a check on deliveries like commodities being delivered in and out of establishments Alerts & Notifications – Get notified on SMS, email, and application Parking Management – Manage availability of parking space Individual log-in – Every user has their own log-in id Visitor/Meeting Analytics – Evaluate notes and remarks of the meeting stored in the system Visitor Management System is a secure and user friendly database manager that records, filters, tracks the visitors to your organization. "Secure Your Premises with VizMan (VMS) – Get It Now"

Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...

Hivelance Technology

Cryptocurrency trading bots are computer programs designed to automate buying, selling, and managing cryptocurrency transactions. These bots utilize advanced algorithms and machine learning techniques to analyze market data, identify trading opportunities, and execute trades on behalf of their users. By automating the decision-making process, crypto trading bots can react to market changes faster than human traders Hivelance, a leading provider of cryptocurrency trading bot development services, stands out as the premier choice for crypto traders and developers. Hivelance boasts a team of seasoned cryptocurrency experts and software engineers who deeply understand the crypto market and the latest trends in automated trading, Hivelance leverages the latest technologies and tools in the industry, including advanced AI and machine learning algorithms, to create highly efficient and adaptable crypto trading bots

Lecture 1 Introduction to games development

abdulrafaychaudhry

Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...

Globus

Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.

Recently uploaded (20)

2024 RoOUG Security model for the cloud.pptx

Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...

Vitthal Shirke Microservices Resume Montevideo

Corporate Management | Session 3 of 3 | Tendenci AMS

Globus Compute Introduction - GlobusWorld 2024

Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...

WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation

How Recreation Management Software Can Streamline Your Operations.pptx

top nidhi software solution freedownload

Strategies for Successful Data Migration Tools.pptx

Into the Box 2024 - Keynote Day 2 Slides.pdf

GlobusWorld 2024 Opening Keynote session

Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL

Accelerate Enterprise Software Engineering with Platformless

A Comprehensive Look at Generative AI in Retail App Testing.pdf

Advanced Flow Concepts Every Developer Should Know

Visitor Management System in India- Vizman.app

Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...

Lecture 1 Introduction to games development

Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...

Filling the Data Lake - Strata + HadoopWorld San Jose 2016 Preview Presentation

1. Filling the Data Lake Chuck Yarbrough, Director, Pentaho Solutions Mark Burnette, Enterprise Sales Engineer Preview presentation for Strata + HadoopWorld San Jose 2016 session Thursday, March 31 at 11:50 am, room 230B

2. Hadoop is Hard… Empower team members to integrate and process Hadoop Data Establish a modern data on boarding process that is flexible and scalable Deliver governed analytic insights for large production use bases Things that can help ease the pain

3. Proper Care and Feeding of the Data Lake

4. How do we effectively scale data pipelines to accommodate exploding data sources, volumes, and complexity? More Data, More Problems Have you ever had the pleasure of… Migrating hundreds of tables between databases? Enabling business users to onboard a variety of data themselves? Ingesting hundreds of changing data sources into Hadoop?

5. More Data, More Problems Modern data onboarding is more than just “connecting” or “loading” – it includes: Managing a changing array of data sources Establishing repeatable processes at scale Maintaining control and governance

6. CSVCSV RDBM S Big Data On Boarding Ingest Procedures Hadoop AVRO RDBM S Disparate Data Sources Dynamic Integration Processes Dynamic Transformations RDBM S

7. Continuous Big Data On Boarding Blueprint Streamline data ingest from wide variety of source data Reduce dependence on hard coded data movement procedures Simplify regular data movement at scale into Hadoop

8. Visit Pentaho in the exhibit hall booth #1025 Attend the session Filling the Data Lake Thursday, March 31 11:50a-12:30p Room 230B http://conferences.oreilly.com/strata/hadoop-big-data- ca/public/schedule/detail/50677

Editor's Notes

Modern data onboarding challenges go beyond just ‘connecting’ to data sources or ‘ingesting’ data into a store of choice They introduce significant new challenges related to dealing with many more source of data that may changes over time They also require a flexible, efficient, and governed process to be fully successful…
ELT type process
Data Migration / Ingest

Filling the Data Lake - Strata + HadoopWorld San Jose 2016 Preview Presentation

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Filling the Data Lake - Strata + HadoopWorld San Jose 2016 Preview Presentation

Similar to Filling the Data Lake - Strata + HadoopWorld San Jose 2016 Preview Presentation (20)

More from Pentaho

More from Pentaho (11)

Recently uploaded

Recently uploaded (20)

Filling the Data Lake - Strata + HadoopWorld San Jose 2016 Preview Presentation

Editor's Notes