"Beyond the Data Lake", Matthias Korn, Technical Consultant at datavirtuality

•Download as PPTX, PDF•

0 likes•793 views

"Beyond the Data Lake", Matthias Korn, Technical Consultant at datavirtuality YouTube Link: https://www.youtube.com/watch?v=P5StySlZTzU Watch more from Data Natives 2015 here: http://bit.ly/1OVkK2J Visit the conference website to learn more: www.datanatives.io Follow Data Natives: https://www.facebook.com/DataNatives https://twitter.com/DataNativesConf Stay Connected to Data Natives by Email: Subscribe to our newsletter to get the news first about Data Natives 2016: http://bit.ly/1WMJAqS Presentation Slides: https://www.slideshare.net/secret/sis... About the Author: Matthias Korn, Technical Consultant at datavirtuality. His talk, "Beyond the Data Lake" will talk about the shift in digital era, which harnesses large amounts of data to make astute business decisions and improve operations, which is now an imperative. While our ability to generate data still far outstrips our ability to analyze it, we are making strides. Exciting new approaches are merging big data solutions with traditional enterprise data strategies. Logical data warehouses, in which there is no single data repository, hold enormous promise. By offering an ecosystem of multiple best-fit repositories, technologies, and tools, business can effectively analyze data for powerful insight.

US Office:
1355 Market Street, #488
San Francisco, CA 94103
German Office:
Katharinenstr. 15
04109 Leipzig, Germany
Beyond the Data Lake
Simplifying data integration for the modern age
Matthias Korn | Technical Consultant
matthias.korn@datavirtuality.de

The Challenge
Gartner 2014: “VARIETY
is the biggest
challenge.”
“When asked about the
dimensions of data
organizations struggle
with most, 49% answered
variety, while 35%
answered volume and
16% velocity.”

Integration using the Data Warehouse
Data is integrated by copying it into a central repository
Approach: ETL process
Structure is applied in the repository
BI users query Data Marts

Why do so many DWH projects fail
Inflexible; costly modifications
Labour-intensive setup and maintenance
77% failure rate*
Slow data-to-actionable-insights (6 to 9+ months)

Data Lake – getting data in is pretty easy…

…but making sense of it is the challenge
Business User

Approaches to data fishing
Situation improved with YARN
Apache Mahout, HBase, Hive, Pig and MapReduce
Data Marts are created
BI user‘s report tools query Data Marts
Wait, didn‘t they do this before already?

„Transform“ just changed its position: ETL -> ELT
Data Marts have to be created by Data Scientists
BI users can‘t do new things
No permission concept
A lot of the stored data is never used, eating up the low storage
costs

The Logical Data Warehouse
Introduced by Gartner in 2012
new data management architecture for analytics
Uses repositories just like the EDW
Adds distributed processes
Adds virtualization of data sources

What does the Logical Data Warehouse do?
LDW knows where the data is stored instead of copying it
Repositories are used for datasources that are too slow
Presents all data in a single virtual database
Quickly reacts to changes in data models of source systems

Advantages of the Logical Data Warehouse
Real time data available and ready for analysis
Immediately productive
Logical Data Model
Permission concept
Webservices
Write to connected systems

Example data flow in an LDW
Distributed query
BI frontend aware of all data sources -
creates SQL statement
Performance optimization engine replicates
data only if needed

Conclusion
Logical Warehouse holds enormous promise
Flexibility and real-time access give an advantage
Use Hadoop for batch jobs rather than integration
We dataconomy!

US Office:
1355 Market Street, #488
San Francisco, CA 94103
German Office:
Katharinenstr. 15
04109 Leipzig, Germany
DataVirtuality
Thanks for your attention!
Visit our stand in the exhibition area.
matthias.korn@datavirtuality.de

Watch full webinar here: https://bit.ly/3H5AYZf Implementing a logical data fabric as an architecture makes absolute sense when you have data spread across various sources in the cloud, including data warehouses, data lakes and even realtime data. In this session our customer will discuss the ways in which they implemented Denodo as a logical data fabric and how it helped them reduce risk and speed up time to access data.

Data Services and the Modern Data Ecosystem (Middle East)

Denodo

Watch full webinar here: https://bit.ly/3xdSTIU Digital Transformation has changed IT the way information services are delivered. The pace of business engagement, the rise of Digital IT (formerly known as “Shadow IT), has also increased demands on IT, especially in the area of Data Management. Data Services exploits widely adopted interoperability standards, providing a strong framework for information exchange but also has enabled a growth of robust systems of engagement that can now exploit information that was normally locked away in some internal silo. Join us for our upcoming Middle East Webinar series episode, “Data Services and the Modern Data Ecosystem,” presented by Chief Evangelist MEA, Alexey Sidorov. Tune-in as we explore how a business can easily support and manage a Data Service ecosystem, providing a more flexible approach for information sharing supporting an ever diverse community of consumers. Watch on-demand this webinar to learn: - Why Data Services are a critical part of a modern data ecosystem - How IT teams can manage Data Services and the increasing demand by businesses - How Digital IT can benefit from Data Services and how this can support the need for rapid prototyping allowing businesses to experiment with data and fail fast where necessary. - How a good Data Virtualization platform can encourage a culture of Data amongst business consumers (internally and externally)

Self Service Analytics enabled by Data Virtualization from Denodo

Denodo

Watch full webinar here: https://bit.ly/39U9qY8 Self-service Analytics BI is often quoted by many - ie, allow users to discover and access data without having to ask IT to create a data mart, or by allowing users to directly export/copy the data from the data sources themselves into their analytics tools and systems. The challenge is not just to provide access to the data – even from Excel this can be done - but to do this in real time without creating processing overhead, while getting trusted data, with the best response time possible, in a managed, governed and secure way in order for these users to trust the output of the analysis. Data Virtualization provides a data access platform that allows users to access the data they need from multiple data sources, when they need it, and with the best possible response time. In addition, a Data Marketplace built on top of this proven technology enables Self Service Analytics by exposing consistent and governed data sets to be discovered by users, providing the trusted foundation for a successful Self-Service Analytics initiative.

A Successful Data Strategy for Insurers in Volatile Times (EMEA)

Denodo

Watch full webinar here: https://bit.ly/34gVVzH To capitalize on all their data, insurers need a flexible and easily adaptable data integration technology that allows them to keep up with the ever-changing and growing data environment. Data virtualization is that modern data integration technology. It can support insurers not only on their journey to digitization, but also on their future infrastructure changes and innovations, adding agility, flexibility and efficiency to data architectures. Join this webinar to: - Find out why data virtualization should be a part of your enterprise data strategy - See how this technology can help you capitalize on your data - Hear how many of your peers are already leveraging the Denodo Platform for Data Virtualization and the benefits they’re observing

Watch this webinar in full here: https://buff.ly/2IxM8Iy Watch all webinars from the Denodo Packed Lunch webinar series here: https://buff.ly/2IR3q6w While big data initiatives have become necessary for any business to generate actionable insights, big data fabric has become a necessity for any successful big data initiative. The best of breed big data fabrics should deliver actionable insights to the business users with minimal effort, provide end-to-end security to the entire enterprise data platform and provide real-time data integration, while delivering self-service data platform to business users. Attend this session to learn how big data fabric enabled by data virtualization: • Provides lightning fast self-service data access to business users • Centralizes data security, governance and data privacy • Fulfills the promise of data lakes to provide actionable insights

The Virtualization of Clouds - The New Enterprise Data Architecture Opportunity

Denodo

Watch full webinar here: https://bit.ly/3x7xVuR Organizations worldwide are adopting a variety of the public cloud service providers (i.e. AWS, Google, Microsoft) and each have a portfolio of storage, compute, network, and security options. All of which create significant challenges in managing a hybrid and multi-cloud enterprise architecture. Even worse is the impact to the governance and integration of data from the clouds and physical infrastructure to support the broad array of analytics and operational requirements. Can one public cloud provider meet all your needs today and in the future? How do you manage across multiple public and private clouds you have today and where your data exists? And, how would you manage and operate your multi-cloud and on-premises systems to gain value from your data in any of them? The Chief Research Officer at Ventana Research, Mark Smith, will expound the challenges and path ahead for virtualization and integration of your data and the clouds, setting an architectural path for best success.

Denodo’s Data Catalog: Bridging the Gap between Data and Business

Denodo

Watch full webinar here: https://bit.ly/3rrE6rh Self service is a major goal of modern data strategists. Denodo’s data catalog is a key piece in Denodo’s portfolio to bridge the gap between the technical data infrastructure and business users. It provides documentation, search, governance and collaboration capabilities, and data exploration wizards. It’s the perfect companion for a virtual layer to fully empower those self service initiatives with minimal IT intervention. It provides business users with the tool to generate their own insights with proper security, governance and guardrails. In this session we will see: - The role of a virtual semantic layer in self service initiatives - What are the key capabilities of Denodo’s new Data Catalog - Best practices and advanced tips for a successful deployment - How customers are using the Denodo’s Data Catalog to enable self-service initiatives

Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...

Databricks

Future of Data Strategy (ASEAN)

Denodo

Watch full webinar here: https://bit.ly/3mdj9i7 You will often hear that "data is the new gold"? In this context, data management is one of the areas that has received more attention from the software community in recent years. From Artificial Intelligence and Machine Learning to new ways to store and process data, the landscape for data management is in constant evolution. From the privileged perspective of an enterprise middleware platform, we at Denodo have the advantage of seeing many of these changes happen. In this webinar, we will discuss the technology trends that will drive the enterprise data strategies in the years to come. Don't miss it if you want to keep yourself informed about how to convert your data to strategic assets in order to complete the data-driven transformation in your company. Watch this on-demand webinar as we cover: - The most interesting trends in data management - How to build a data fabric architecture? - How to manage your data integration strategy in the new hybrid world - Our predictions on how those trends will change the data management world - How can companies monetize the data through data-as-a-service infrastructure? - What is the role of voice computing in future data analytic

Big Data Fabric: A Recipe for Big Data Initiatives

Denodo

Big data fabric combines essential big data capabilities in a single platform to automate the many facets of data discovery, preparation, curation, orchestration, and integration across a multitude of data sources. Attend this session to learn how Big Data Fabric enabled by data virtualization constitutes a recipe for: • Enabling new actionable insights with minimal effort • Securing big data end-to-end • Addressing big data skillset scarcity • Providing easy access to data without having to decipher various data formats Agenda: • Big Data with Data Virtualization • Product Demonstration • Summary & Next Steps • Q&A Watch webinar on demand here: https://goo.gl/EpmIBx This webinar is part of the Data Virtualization Packed Lunch Webinar Series: https://goo.gl/W1BeCb

Data Virtualization: From Zero to Hero (Middle East)

Denodo

Watch full webinar here: https://bit.ly/3vPcmQ4 At the rate which enterprise data volume is increasing, replicating data to a central repository for analysis purposes is slow and expensive which might not even be a necessary part of the data integration process in many situations. With technologies such as data virtualization, companies can now place a single secure virtual layer between all disparate data sources (including both on-premise and in the cloud) on one side and various consuming applications on the other. Data replication for data integration is now an option and not a necessity. Join us for this webinar to become a "Data Virtualization Hero" inside your organization. In this session you will learn: - What data virtualization really is - How it differs from other enterprise data integration technologies - Why data virtualization is finding enterprise-wide deployment inside some of the largest organizations

A Logical Architecture is Always a Flexible Architecture (ASEAN)

Denodo

Watch full webinar here: https://bit.ly/3joZa0a The current data landscape is fragmented, not just in location but also in terms of processing paradigms: data lakes, IoT architectures, NoSQL, and graph data stores, SaaS applications, etc. are found coexisting with relational databases to fuel the needs of modern analytics, ML, and AI. The physical consolidation of enterprise data into a central repository, although possible, is both expensive and time-consuming. A logical data warehouse is a modern data architecture that allows organizations to leverage all of their data irrespective of where the data is stored, what format it is stored in, and what technologies or protocols are used to store and access the data. Watch this session to understand: - What is a logical data warehouse and how to architect one - The benefits of logical data warehouse – speed with agility - Customer use case depicting logical architecture implementation

How to Build the Data Mesh Foundation: A Principled Approach | Zhamak Dehghan...

HostedbyConfluent

Organizations have been chasing the dream of data democratization, unlocking and accessing data at scale to serve their customers and business, for over a half a century from early days of data warehousing. They have been trying to reach this dream through multiple generations of architectures, such as data warehouse and data lake, through a cambrian explosion of tools and a large amount of investments to build their next data platform. Despite the intention and the investments the results have been middling. In this keynote, Zhamak shares her observations on the failure modes of a centralized paradigm of a data lake, and its predecessor data warehouse. She introduces Data Mesh, a paradigm shift in big data management that draws from modern distributed architecture: considering domains as the first class concern, applying self-sovereignty to distribute the ownership of data, applying platform thinking to create self-serve data infrastructure, and treating data as a product. This talk introduces the principles underpinning data mesh and Zhamak's recent learnings in creating a path to bring data mesh to life in your organization.

Logical Data Warehouse: The Foundation of Modern Data and Analytics

Denodo

Watch full webinar here: https://buff.ly/2Vhew78 According to a leading analyst firm, the total spend in data and analytics is expected to reach $104 billion in 2019! Companies are investing in data warehouse modernization and data lake projects for descriptive and advanced analytics; however, for the analysis to be holistic, today’s architects weave disparate data streams together, not only from these analytical sources, but also from operational, third party, and streaming data sources. Logical data warehouse is a modern architectural methodology that virtually combines all the data across the enterprise and makes it available to analytical and visualization tools that facilitate timely, insightful, and impactful decisions throughout the enterprise. In this session, you will learn: *What is logical data warehouse and how to architect one *The benefits of logical data warehouse – speed with agility *The Insights Development Workbench – a framework in action

Analyst Keynote: Forrester: Data Fabric Strategy is Vital for Business Innova...

Denodo

Denodo DataFest 2016: Data Science: Operationalizing Analytical Models in Rea...

Denodo

Watch the full session: Denodo DataFest 2016 sessions: https://goo.gl/yVJnti Data virtualization starts with democratizing data access for business users, but goes well beyond to enable entire analytics life cycle. This session will discuss the critical role of data virtualization in the four key phases of big data analytics: Discovery of raw and enriched data, Analytic Exploration, Real-time Operationalization, and Predictive Intervention. In this session, you will learn: • Design of advanced analytics with view towards business goal realization • The role of data virtualization in enabling analytics through four key phases • How to exploit product capabilities relevant to each stage • Creating a system of governed self-service and collaborative analytics This session is part of the Denodo DataFest 2016 event. You can also watch more Denodo DataFest sessions on demand here: https://goo.gl/VXb6M6

Minimizing the Complexities of Machine Learning with Data Virtualization

Denodo

Watch full webinar here: https://buff.ly/309CZ1Y Advanced data science techniques, like machine learning, have proven an extremely useful tool to derive valuable insights from existing data. Platforms like Spark, and complex libraries for R, Python and Scala put advanced techniques at the fingertips of the data scientists. However, these data scientists spent most of their time looking for the right data and massaging it into a usable format. Data virtualization offers a new alternative to address these issues in a more efficient and agile way. Attend this webinar and learn: *How data virtualization can accelerate data acquisition and massaging, providing the data scientist with a powerful tool to complement their practice *How popular tools from the data science ecosystem: Spark, Python, Zeppelin, Jupyter, etc. integrate with Denodo *How you can use the Denodo Platform with large data volumes in an efficient way *About the success McCormick has had as a result of seasoning the Machine Learning and Blockchain Landscape with data virtualization

Maximizing Data Lake ROI with Data Virtualization: A Technical Demonstration

Denodo

Watch full webinar here: https://bit.ly/3ohtRqm Companies with corporate data lakes also need a strategy for how to best integrate them with their overall data fabric. To take full advantage of a data lake, data architects must determine what data belongs in the Lake vs. other sources, how end users are going to find and connect to the data they need as well as the best way to leverage the processing power of the data lake. This webinar will provide you with a deep dive look at how the Denodo Platform for data virtualization enables companies to maximize their investment in their corporate data lake. Watch on-demand this webinar to learn: - How to create a logical data fabric with Denodo - How to leverage the a data lake for MPP Acceleration and Summary Views - How to leverage Presto with Denodo for file based data lakes (ie. S3, ADLS, HDFS, etc.)

Why Data Virtualization? An Introduction.

Denodo

Watch full webinar here: https://bit.ly/3dhbZTK What started to evolve as the most agile and real-time enterprise data fabric, data virtualization is proving to go beyond its initial promise and is becoming one of the most important enterprise big data fabrics. Watch this session to learn: - What data virtualization really is. - How it differs from other enterprise data integration technologies. - Why data virtualization is finding enterprise-wide deployment inside some of the largest organizations.

What's new in Hortonworks DataFlow 3.0 by Andrew Psaltis

Data Con LA

Abstract:- Hortonworks DataFlow (HDF) is built with the vision of creating a platform that enables enterprises to build dataflow management and streaming analytics solutions that collect, curate, analyze and act on data in motion across the datacenter and cloud. Do you want to be able to provide a complete end-to-end streaming solution, from an IoT device all the way to a dashboard for your business users with no code? Come to this session to learn how this is now possible with HDF 3.0.

Simplifying Your Cloud Architecture with a Logical Data Fabric (APAC)

Denodo

Watch full webinar here: https://bit.ly/3dudL6u It's not if you move to the cloud, but when. Most organisations are well underway with migrating applications and data to the cloud. In fact, most organisations - whether they realise it or not - have a multi-cloud strategy. Single, hybrid, or multi-cloud…the potential benefits are huge - flexibility, agility, cost savings, scaling on-demand, etc. However, the challenges can be just as large and daunting. A poorly managed migration to the cloud can leave users frustrated at their inability to get to the data that they need and IT scrambling to cobble together a solution. In this session, we will look at the challenges facing data management teams as they migrate to cloud and multi-cloud architectures. We will show how the Denodo Platform can: - Reduce the risk and minimise the disruption of migrating to the cloud. - Make it easier and quicker for users to find the data that they need - wherever it is located. - Provide a uniform security layer that spans hybrid and multi-cloud environments.

Destroying Data Silos

Hellmar Becker

DAMA Webinar: Turn Grand Designs into a Reality with Data Virtualization

Denodo

Watch full webinar here: https://buff.ly/2HMdbUp What started to evolve as the most agile and real-time enterprise data fabric, data virtualization is proving to go beyond its initial promise and is becoming one of the most important enterprise big data fabrics. Attend this session to learn: • What data virtualization really is, • How it differs from other enterprise data integration technologies • Real-world examples of data virtualization in action from companies such as Logitech, Autodesk and Festo.

Representation Theoristscloudbouncer

Empower your Enterprise with language intelligence_Francisco Webber

Dataconomy Media

What's hot

Driving Datascience at scale using Postgresql, Greenplum and Dataiku - Greenp...

VMware Tanzu

Big Data Fabric: A Necessity For Any Successful Big Data Initiative

Denodo

The Virtualization of Clouds - The New Enterprise Data Architecture Opportunity

Denodo

Denodo’s Data Catalog: Bridging the Gap between Data and Business

Denodo

Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...

Databricks

Future of Data Strategy (ASEAN)

Denodo

Big Data Fabric: A Recipe for Big Data Initiatives

Denodo

Data Virtualization: From Zero to Hero (Middle East)

Denodo

A Logical Architecture is Always a Flexible Architecture (ASEAN)

Denodo

How to Build the Data Mesh Foundation: A Principled Approach | Zhamak Dehghan...

HostedbyConfluent

Logical Data Warehouse: The Foundation of Modern Data and Analytics

Denodo

Analyst Keynote: Forrester: Data Fabric Strategy is Vital for Business Innova...

Denodo

Denodo DataFest 2016: Data Science: Operationalizing Analytical Models in Rea...

Denodo

Minimizing the Complexities of Machine Learning with Data Virtualization

Denodo

Maximizing Data Lake ROI with Data Virtualization: A Technical Demonstration

Denodo

Why Data Virtualization? An Introduction.

Denodo

What's new in Hortonworks DataFlow 3.0 by Andrew Psaltis

Data Con LA

Simplifying Your Cloud Architecture with a Logical Data Fabric (APAC)

Denodo

Destroying Data Silos

Hellmar Becker

DAMA Webinar: Turn Grand Designs into a Reality with Data Virtualization

Denodo

What's hot (20)

Driving Datascience at scale using Postgresql, Greenplum and Dataiku - Greenp...

Big Data Fabric: A Necessity For Any Successful Big Data Initiative

The Virtualization of Clouds - The New Enterprise Data Architecture Opportunity

Denodo’s Data Catalog: Bridging the Gap between Data and Business

Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...

Future of Data Strategy (ASEAN)

Big Data Fabric: A Recipe for Big Data Initiatives

Data Virtualization: From Zero to Hero (Middle East)

A Logical Architecture is Always a Flexible Architecture (ASEAN)

How to Build the Data Mesh Foundation: A Principled Approach | Zhamak Dehghan...

Logical Data Warehouse: The Foundation of Modern Data and Analytics

Analyst Keynote: Forrester: Data Fabric Strategy is Vital for Business Innova...

Denodo DataFest 2016: Data Science: Operationalizing Analytical Models in Rea...

Minimizing the Complexities of Machine Learning with Data Virtualization

Maximizing Data Lake ROI with Data Virtualization: A Technical Demonstration

Why Data Virtualization? An Introduction.

What's new in Hortonworks DataFlow 3.0 by Andrew Psaltis

Simplifying Your Cloud Architecture with a Logical Data Fabric (APAC)

Destroying Data Silos

DAMA Webinar: Turn Grand Designs into a Reality with Data Virtualization

Viewers also liked

Representation Theoristscloudbouncer

Empower your Enterprise with language intelligence_Francisco Webber

Dataconomy Media

We'll Always Have Paris

Numenta

State of NuPIC

Numenta

Temporal memory in racket

Numenta

Beginner's Guide to NuPICNumenta

Numenta Anomaly Benchmark - SF Data Science Meetup

Numenta

Applications of Hierarchical Temporal Memory (HTM)

Numenta

Sparse Distributed Representations: Our Brain's Data Structure

Numenta

HTM Theory

Asfak Mahamud

Getting Started with Numenta Technology

Numenta

Real-Time Streaming Data Analysis with HTM

Numenta

What the Brain says about Machine Intelligence

Numenta

"Updates on Semantic Fingerprinting", Francisco Webber, Inventor and Co-Found...

Dataconomy Media

Francisco Webber is the CEO and Founder of Cortical.io, a company that develops Natural Language Processing solutions for Big Text Data. Francisco’s medical background in genetics combined with over two decade’s of experience in Information Technology, inspired him to create a groundbreaking technology, called Semantic Folding, which is based on the latest findings on the way the human neocortex processes information.

Principles of Hierarchical Temporal Memory - Foundations of Machine Intelligence

Numenta

～ワークスペーステクノロジー最前線～

Citrix Systems Japan

Science of Anomaly Detection

Numenta

Evaluating Real-Time Anomaly Detection: The Numenta Anomaly Benchmark

Numenta

Detecting Anomalies in Streaming Data

Numenta

Anomaly Detection Using the CLANumenta

Viewers also liked (20)

Representation Theorists

Empower your Enterprise with language intelligence_Francisco Webber

We'll Always Have Paris

State of NuPIC

Temporal memory in racket

Beginner's Guide to NuPIC

Numenta Anomaly Benchmark - SF Data Science Meetup

Applications of Hierarchical Temporal Memory (HTM)

Sparse Distributed Representations: Our Brain's Data Structure

HTM Theory

Getting Started with Numenta Technology

Real-Time Streaming Data Analysis with HTM

What the Brain says about Machine Intelligence

"Updates on Semantic Fingerprinting", Francisco Webber, Inventor and Co-Found...

Principles of Hierarchical Temporal Memory - Foundations of Machine Intelligence

～ワークスペーステクノロジー最前線～

Science of Anomaly Detection

Evaluating Real-Time Anomaly Detection: The Numenta Anomaly Benchmark

Detecting Anomalies in Streaming Data

Anomaly Detection Using the CLA

Similar to "Beyond the Data Lake", Matthias Korn, Technical Consultant at datavirtuality

Accelerating Big Data Analytics

Attunity

Logical Data Lakes: From Single Purpose to Multipurpose Data Lakes (APAC)

Denodo

Watch full webinar here: https://bit.ly/3aePFcF Historically data lakes have been created as a centralized physical data storage platform for data scientists to analyze data. But lately the explosion of big data, data privacy rules, departmental restrictions among many other things have made the centralized data repository approach less feasible. In this webinar, we will discuss why decentralized multipurpose data lakes are the future of data analysis for a broad range of business users. Attend this session to learn: - The restrictions of physical single purpose data lakes - How to build a logical multi purpose data lake for business users - The newer use cases that makes multi purpose data lakes a necessity

WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRobertsJane Roberts

From Single Purpose to Multi Purpose Data Lakes - Broadening End Users

Denodo

Watch full webinar here: https://buff.ly/2Mt555e Historically data lakes have been created as centralized physical data storage platform for data scientists to analyze data. But lately the explosion of big data, data privacy rules, departmental restrictions among many other things have made the centralized data repository approach less feasible. In his recent whitepaper, renowned analyst Rick F. Van Der Lans talks about why decentralized multi purpose data lakes are the future of data analysis for a broad range of business users. Please attend this session to learn: • The restrictions of physical single purpose data lakes • How to build a logical multi purpose data lake for business users • The newer use cases that makes multi purpose data lakes a necessity

DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization

Denodo

Watch here: https://bit.ly/2NGQD7R In an era increasingly dominated by advancements in cloud computing, AI and advanced analytics it may come as a shock that many organizations still rely on data architectures built before the turn of the century. But that scenario is rapidly changing with the increasing adoption of real-time data virtualization - a paradigm shift in the approach that organizations take towards accessing, integrating, and provisioning data required to meet business goals. As data analytics and data-driven intelligence takes centre stage in today’s digital economy, logical data integration across the widest variety of data sources, with proper security and governance structure in place has become mission-critical. Attend this session to learn: - Learn how you can meet cloud and data science challenges with data virtualization. - Why data virtualization is increasingly finding enterprise-wide adoption - Discover how customers are reducing costs and improving ROI with data virtualization

Building an Effective Data Warehouse Architecture

James Serra

Why use a data warehouse? What is the best methodology to use when creating a data warehouse? Should I use a normalized or dimensional approach? What is the difference between the Kimball and Inmon methodologies? Does the new Tabular model in SQL Server 2012 change things? What is the difference between a data warehouse and a data mart? Is there hardware that is optimized for a data warehouse? What if I have a ton of data? During this session James will help you to answer these questions.

Logical Data Lakes: From Single Purpose to Multipurpose Data Lakes (APAC)

Denodo

Watch full webinar here: https://bit.ly/3dmOHyQ Historically, data lakes have been created as a centralized physical data storage platform for data scientists to analyze data. But lately, the explosion of big data, data privacy rules, departmental restrictions among many other things have made the centralized data repository approach less feasible. In this webinar, we will discuss why decentralized multi-purpose data lakes are the future of data analysis for a broad range of business users. Watch on-demand this webinar to learn: - The restrictions of physical single-purpose data lakes - How to build a logical multi-purpose data lake for business users - The newer use cases that make multi-purpose data lakes a necessity

Exploring the Wider World of Big Data- Vasalis Kapsalis

NetAppUK

Every second of every day you hear about Electronic systems creating ever increasing quantities of data. Systems in markets such as finance, media, healthcare, government and scientific research feature strongly in the Big Data processing conversation. While extracting business value from Big Data is forecast to bring customer and competitive advantage and benefits. In this session hear Vas Kapsalis, NetApp Big Data Business Development Manager, discuss his views and experience on the wider world of Big Data.

Bridging the Last Mile: Getting Data to the People Who Need It

Denodo

Watch full webinar here: https://bit.ly/3cUA0Qi Many organizations are embarking on strategically important journeys to embrace data and analytics. The goal can be to improve internal efficiencies, improve the customer experience, drive new business models and revenue streams, or – in the public sector – provide better services. All of these goals require empowering employees to act on data and analytics and to make data-driven decisions. However, getting data – the right data at the right time – to these employees is a huge challenge and traditional technologies and data architectures are simply not up to this task. This webinar will look at how organizations are using Data Virtualization to quickly and efficiently get data to the people that need it. Attend this session to learn: - The challenges organizations face when trying to get data to the business users in a timely manner - How Data Virtualization can accelerate time-to-value for an organization’s data assets - Examples of leading companies that used data virtualization to get the right data to the users at the right time

Smarter Management for Your Data Growth

RainStor

Big Data Made Easy: A Simple, Scalable Solution for Getting Started with Hadoop

Precisely

With so many new, evolving frameworks, tools, and languages, a new big data project can lead to confusion and unwarranted risk. Many organizations have found Data Warehouse Optimization with Hadoop to be a good starting point on their Big Data journey. Offloading ETL workloads from the enterprise data warehouse (EDW) into Hadoop is a well-defined use case that produces tangible results for driving more insights while lowering costs. You gain significant business agility, avoid costly EDW upgrades, and free up EDW capacity for faster queries. This quick win builds credibility and generates savings to reinvest in more Big Data projects. A proven reference architecture that includes everything you need in a turnkey solution – the Hadoop distribution, data integration software, servers, networking and services – makes it even easier to get started.

Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...

Hortonworks

Many enterprises are turning to Apache Hadoop to enable Big Data Analytics and reduce the costs of traditional data warehousing. Yet, it is hard to succeed when 80% of the time is spent on moving data and only 20% on using it. It’s time to swap the 80/20! The Big Data experts at Attunity and Hortonworks have a solution for accelerating data movement into and out of Hadoop that enables faster time-to-value for Big Data projects and a more complete and trusted view of your business. Join us to learn how this solution can work for you.

Big Data and Oracle - 2013

Connor McDonald

Meet the experts dwo bde vds v7

mmathipra

Exploring the Wider World of Big Data

NetApp

Hadoop and the Data Warehouse: Point/Counter Point

Inside Analysis

Robin Bloor and Teradata Live Webcast on April 22, 2014 Watch the archive: https://bloorgroup.webex.com/bloorgroup/lsr.php?RCID=2e69345c0a6a4e5a8de6fc72652e3bc6 Can you replace the data warehouse with Hadoop? Is Hadoop an ideal ETL subsystem? And what is the real magic of Hadoop? Everyone is looking to capitalize on the insights that lie in the vast pools of big data. Generating the value of that data relies heavily on several factors, especially choosing the right solution for the right context. With so many options out there, how do organizations best integrate these new big data solutions with the existing data warehouse environment? Register for this episode of The Briefing Room to hear veteran analyst Dr. Robin Bloor as he explains where Hadoop fits into the information ecosystem. He’ll be briefed by Dan Graham of Teradata, who will offer perspective on how Hadoop can play a critical role in the analytic architecture. Bloor and Graham will interactively discuss big data in the big picture of the data center and will also seek to dispel several common misconceptions about Hadoop. Visit InsideAnlaysis.com for more information.

Data Ninja Webinar Series: Accelerating Business Value with Data Virtualizati...

Denodo

Watch the full webinar - Session one: Data Ninja Webinar Series by Denodo: https://goo.gl/yAdMpL The following presentation was used during the webinar entitled: "Accelerating Business Value with Data Virtualization Solutions". It discusses the role of data virtualization in delivering real business value from your new and existing data assets. This is session 1 of the Data Ninja Webinar Series organized by Denodo. If you want to learn more about some of the solutions enabled by data virtualization, click here to watch the entire series: https://goo.gl/8XFd1O

Building the Next-gen Digital Meter Platform for Fluvius

Databricks

Fluvius is the network operator for electricity and gas in Flanders, Belgium. Their goal is to modernize the way people look at energy consumption using a digital meter that captures consumption and injection data from any electrical installation in Flanders ranging from households to large companies. After full roll-out there will be roughly 7 million digital meters active in Flanders collecting up to terabytes of data per day. Combine this with regulation that Fluvius has to maintain a record of these reading for at least 3 years, we are talking petabyte scale. delaware BeLux was assigned by Fluvius to setup a modern data platform and did so on Azure using Databricks as the core component to collect, store, process and serve these volumes of data to every single consumer and beyond in Flanders. This enables the Belgian energy market to innovate and move forward. Maarten took up the role as project manager and solution architect.

Traditional BI vs. Business Data Lake – A Comparison

Capgemini

Traditional Business Intelligence (BI) systems provide various levels and kinds of analyses on structured data but they are not designed to handle unstructured data. For these systems Big Data brings big problems because the data that flows in may be either structured or unstructured. That makes them hugely limited when it comes to delivering Big Data benefits. The way forward is a complete rethink of the way we use BI - in terms of how the data is ingested, stored and analyzed. More information: http://www.capgemini.com/big-data-analytics/pivotal

the Data World Distilled

RTTS

The Data World Distilled Understanding how the data world works in the Big Data era I created this slide deck as a learning tool for new employees, I figured I would post it in case it can help others understand the data space. This slide deck covers: - Big Data - Data Warehouses - ETL/Data Integration - Business Intelligence and Analytics - Data Quality - Data Testing - Data Governance It provides a brief description along with key vendors in the space.

Similar to "Beyond the Data Lake", Matthias Korn, Technical Consultant at datavirtuality (20)

Accelerating Big Data Analytics

Logical Data Lakes: From Single Purpose to Multipurpose Data Lakes (APAC)

WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRoberts

From Single Purpose to Multi Purpose Data Lakes - Broadening End Users

DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization

Building an Effective Data Warehouse Architecture

Logical Data Lakes: From Single Purpose to Multipurpose Data Lakes (APAC)

Exploring the Wider World of Big Data- Vasalis Kapsalis

Bridging the Last Mile: Getting Data to the People Who Need It

Smarter Management for Your Data Growth

Big Data Made Easy: A Simple, Scalable Solution for Getting Started with Hadoop

Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...

Big Data and Oracle - 2013

Meet the experts dwo bde vds v7

Exploring the Wider World of Big Data

Hadoop and the Data Warehouse: Point/Counter Point

Data Ninja Webinar Series: Accelerating Business Value with Data Virtualizati...

Building the Next-gen Digital Meter Platform for Fluvius

Traditional BI vs. Business Data Lake – A Comparison

the Data World Distilled

More from Dataconomy Media

Data Natives Paris v 10.0 | "Blockchain in Healthcare" - Lea Dias & David An...

Dataconomy Media

Data Natives Frankfurt v 11.0 | "Competitive advantages with knowledge graphs...

Dataconomy Media

The challenges of increasing complexity of organizations, companies and projects are obvious and omnipresent. Everywhere there are connections and dependencies that are often not adequately managed or not considered at all because of a lack of technology or expertise to uncover and leverage the relationships in data and information. In his presentation, Axel Morgner talks about graph technology and knowledge graphs as indispensable building blocks for successful companies.

Data Natives Frankfurt v 11.0 | "Can we be responsible for misuse of data & a...

Dataconomy Media

Data Natives Munich v 12.0 | "How to be more productive with Autonomous Data ...

Dataconomy Media

Data Natives meets DataRobot | "Build and deploy an anti-money laundering mo...

Dataconomy Media

Compliance departments within banks and other financial institutions are turning to machine learning for improving their Anti Money Laundering compliance activities. Today, the systems that aim to detect potentially suspicious activity are commonly rule-based, and suffer from ultra-high false positive rates. DataRobot will discuss how their Automated Machine Learning platform was successfully used for a real use case to reduce their false positives and to enhance their Anti-Money Laundering activities.

Data Natives Munich v 12.0 | "Political Data Science: A tale of Fake News, So...

Dataconomy Media

Trump, Brexit, Cambridge Analytica... In the last few years, we have had to confront the consequences of the use and misuse of data science algorithms in manipulating public opinion through social media. The use of private data to microtarget individuals is a daily practice (and a trillion-dollar industry), which has serious side-effects when the selling product is your political ideology. How can we cope with this new scenario?

Data Natives Vienna v 7.0 | "Building Kubernetes Operators with KUDO for Dat...

Dataconomy Media

Data Natives Vienna v 7.0 | "The Ingredients of Data Innovation" - Robbert de...

Dataconomy Media

Data Natives Cologne v 4.0 | "The Data Lorax: Planting the Seeds of Fairness...

Dataconomy Media

What does it take to build a good data product or service? Data practitioners always think about the technology, user experience and commercial viability. But rarely do they think about the implications of the systems they build. This talk will shed light on the impact of AI systems and the unintended consequences of the use of data in different products. It will also discuss our role, as data practitioners, in planting the seeds of fairness in the systems we build.

Data Natives Cologne v 4.0 | "How People Analytics Can Reveal the Hidden Aspe...

Dataconomy Media

Data Natives Amsterdam v 9.0 | "Ten Little Servers: A Story of no Downtime" -...

Dataconomy Media

Cloud Infrastructure is a hostile environment: a power supply failure or a network outage leads to downtime and big losses. There is nothing we can trust: a single server, a server rack, even a whole datacenter can fail, and if an application is fragile by design, disruption is inevitable. We must distribute our application and diversify cloud data strategy to survive disturbances of any scale. Apache Cassandra is a cloud-native platform-agnostic database that stores data with a distributed redundancy so it easily survives any issue. What to know how Apple and Netflix handle petabytes of data, keeping it highly available? Join us and listen to a story of 10 little servers and no downtime!

Data Natives Amsterdam v 9.0 | "Point in Time Labeling at Scale" - Timothy Th...

Dataconomy Media

Data Natives Hamburg v 6.0 | "Interpersonal behavior: observing Alex to under...

Dataconomy Media

Data Natives Hamburg v 6.0 | "About Surfing, Failing & Scaling" - Florian Sch...

Dataconomy Media

Data NativesBerlin v 20.0 | "Serving A/B experimentation platform end-to-end"...

Dataconomy Media

Data Natives Berlin v 20.0 | "Ten Little Servers: A Story of no Downtime" - A...

Dataconomy Media

Big Data Frankfurt meets Thinkport | "The Cloud as a Driver of Innovation" - ...

Dataconomy Media

Creativity is the mental ability to create new ideas and designs. Innovation, on the other hand, Means developing useful solutions from new ideas. Creativity can be goal-oriented, Whereas innovation is always goal-oriented. This bedeutet, dass innovation aims to achieve defined goals. The use of cloud services and technologies promises enterprise users many benefits in terms of more flexible use of IT resources and faster access to innovative solutions. That’s why we want to examine the question in this talk, of what role cloud computing plays for innovation in companies.

Thinkport meets Frankfurt | "Financial Time Series Analysis using Wavelets" -...

Dataconomy Media

Big Data Helsinki v 3 | "Distributed Machine and Deep Learning at Scale with ...

Dataconomy Media

"With most machine learning (ML) and deep learning (DL) frameworks, it can take hours to move data for ETL, and hours to train models. It's also hard to scale, with data sets increasingly being larger than the capacity of any single server. The amount of the data also makes it hard to incrementally test and retrain models in near real-time. Learn how Apache Ignite and GridGain help to address limitations like ETL costs, scaling issues and Time-To-Market for the new models and help achieve near-real-time, continuous learning. Yuriy Babak, the head of ML/DL framework development at GridGain and Apache Ignite committer, will explain how ML/DL work with Apache Ignite, and how to get started. Topics include: — Overview of distributed ML/DL including architecture, implementation, usage patterns, pros and cons — Overview of Apache Ignite ML/DL, including built-in ML/DL algorithms, and how to implement your own — Model inference with Apache Ignite, including how to train models with other libraries, like Apache Spark, and deploy them in Ignite — How Apache Ignite and TensorFlow can be used together to build distributed DL model training and inference"

Big Data Helsinki v 3 | "Federated Learning and Privacy-preserving AI" - Oguz...

Dataconomy Media

"Machine learning algorithms require significant amounts of training data which has been centralized on one machine or in a datacenter so far. For numerous applications, such need of collecting data can be extremely privacy-invasive. Recent advancements in AI research approach this issue by a new paradigm of training AI models, i.e., Federated Learning. In federated learning, edge devices (phones, computers, cars etc.) collaboratively learn a shared AI model while keeping all the training data on device, decoupling the ability to do machine learning from the need to store the data in the cloud. From personal data perspective, this paradigm enables a way of training a model on the device without directly inspecting users’ data on a server. This talk will pinpoint several examples of AI applications benefiting from federated learning and the likely future of privacy-aware systems."

More from Dataconomy Media (20)