Watch full webinar here: https://bit.ly/3gSmtQY
Data lakes have been both praised and loathed. They can be incredibly useful to an organization, but it can also be the source of major headaches. Its ease to scale storage with minimal cost has opened the door to many new solutions, but also to a proliferation of runaway objects that have coined the term data swamp.
However, the addition of an MPP engine, based on Presto, to Denodo’s logical layer can change the way you think about the role of the data lake in your overall data strategy.
Watch on-demand this session to learn:
- The new MPP capabilities that Denodo includes
- How to use them to your advantage to improve security and governance of your lake
- New scenarios and solutions where your data fabric strategy can evolve
Delhi Call Girls Punjabi Bagh 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Shaping the Role of a Data Lake in a Modern Data Fabric Architecture
1. Shaping the Role of a
Data Lake in a Modern
Data Fabric Architecture
A S E A N W E B I N A R
Felix Liao
Product Management Director, APAC | Denodo
2. Agenda
1. The Role of the Data Lake Today and Tomorrow
2. Current Challenges Associated with Data Lake
3. Data Lake within a Data Fabric Architecture
3. 3
• Data Lakes have evolved from the distributed file storage
technologies of the Hadoop era (HDFS)
• Modern data lakes are often used as a cost effective and
flexible storage for structured, semi-structured and
unstructured data.
• Data lake can become “data swamps” due to lack of
process, governance and security
• The data lake needs to be integrated and leveraged in
broader analytics and BI use cases
The Evolving Roles of Data Lake
4. 4
• Object Storage is a form of storage for unstructured
data (objects) that eliminates scaling limitations of
traditional file storage technologies
• Amazon’s S3 (Simple Storage Service) and Azure’s
ADLS (Azure Data Lake Storage) are the most popular
object storage today
• Modern cloud-based data lakes now take advantage of
objective storage and file format such as Parquet to
offer limitless storage in a more cost-efficient way
Modern Data Lakes Powered by Object Storage
5. Let’s look at some examples
Increasing roles and
importance of
modern data lakes
6. 6
• Data science playground
• Cheap storage for backup or infrequently
used data
• 3rd party data sharing and exchange
• Non-critical query workloads
Emerging Use Cases for Modern Data Lakes
7. 7
• Lack of an integrated and scalable SQL query
engine
• Difficult to use for non-technical users
• Lack of fine-grained security and data access
control
• An integrated approach to all enterprise data
repositories
Shortfalls of Modern Data Lakes
Incorporating a modern data lake in your enterprise data
strategy will need additional capabilities
8. 8
DATA
CONSUMERS
Logical Layer Powered by Data Virtualization
LOGICAL
DATA
FABRIC
SDATA
OURCES
BI Tools Data Science Tools
Self-Service
Monitoring &
Auditing
Global
Security
Smart Query
Acceleration
Real Time
Access
Hybrid/
Multi-Cloud
Traditional
DB & DW
Cloud Excel
Lake Storage
(S3/ADLS)
MPP Engine
Data Lake through a Logical Data Fabric Layer
9. 9
How does it work? – Scalable Compute via MPP engine
• Efficient, scalable access to
content in the object
storage
• No need for an
additional external MPP
engine
• Integrated security and
management
• Out-of-the-box MPP
options for caching and
query acceleration
Logical Fabric
Layer
MPP
Coordinator
MPP worker
MPP worker
MPP worker
MPP worker
Object
Storage
10. 10
How does it work? – Integrated GUI
• Automated deployment
using Kubernetes and
Helm charts
• Integrated configuration
• Graphical browsing and
introspection of object
storage
Object Storage configuration
Object Storage browsing
11. 11
How does it work? - Putting in Context
Denodo
Virtualization
Server
Denodo
Data Catalog
Denodo
Web Services
On-prem
data
Other Apps
IdP
Denodo
MPP
Warehouse A
Warehouse B
AWS S3 bucket
AWS Aurora
12. 12
• Data scientist traditionally have relied on specialist
data science tool to access granular data stored in
data lake
• Denodo provides scalable and unified SQL access
to all data assets including data lake
• The semantic layer allows organisations to provide
controlled and governed access to granular data
when needed
Improved Use Case - Data Science Playground
13. 13
• Separation of compute and storage means that the
same data and queries can be supported by multiple
platforms and engines with minimal changes
• Denodo includes the tools to move and keep data
updated when needed
• A logical layer means that the change is transparent
for consumers
Improved Use Case - Non-Critical Query Workloads to Cheaper Systems
14. 14
• Object storage is a great option for data that is rarely
used but that need to be stored for backup or
compliance reasons
• These data can be exported into Parquet and moved
to the object storage
• Denodo can provide quick and easy access to
these data in a secure and governed way
Improved Use Case - Cheap Storage for Backup or Infrequently Used Data
15. 15
Improved use case - 3rd Party Data sharing and Exchange
• An object storage is often used to share and
exchange 3rd party data between partners today.
• Data can be in parquet, but also in JSON, CSV or
even Excel
• Denodo can automatically map and publish these
types of data for easy consumption
• Denodo can help integrate third party data with
existing corporate data repositories
16. 16
Key Takeaways
1. Object Storage technologies (S3, ADLS) are
powering modern data lakes and driving increasing
adoption and usages
2. Next generation MPP engines provide efficient
processing capabilities for data stored in an object
storage, making them more accessible by more users
3. A logical data fabric layer such as Denodo provides
the additional compute, security, governance and
integration capabilities need to include modern data
lake into your enterprise data strategy today
18. 18
Get Started Today
Try Denodo for a Test Drive with a 30-
day free trial in the cloud marketplaces
CHOICE
Under your cloud account
SUPPORT
Community forum AND remote sales
engineer
OPPORTUNITY
30 minutes free consultation with
Denodo Cloud specialist
denodo.link/TD2022
19. Key Takeaways 2022
A P A C W E B I N A R
Benjamin Henshall
Regional Vice President, ANZ | Denodo
REGISTER NOW
denodo.link/kt22
15 December | 11.30am – 12.30pm SGT