The document discusses the concept of data services marketplaces, highlighting their role in providing access to data for users and developers. It covers the challenges associated with data services, such as security, privacy, performance, and data governance, as well as outlining various technologies for implementing data services. Additionally, it provides customer success stories demonstrating the benefits of data services in real-world applications.

1. New York City
9th June, 2016
Logical Data Warehouse,
Data Lakes, and Data
Services Marketplaces

2. Agenda1.Introductions
2.Logical Data Warehouse and Data Lakes
3.Coffee Break
4.Data Services Marketplaces
5.Q&A

3. Data Services Marketplace
New York City
June 2016

4. Agenda1.Data Services Marketplace
2.Data Services Demo
3.Addressing the Challenges
4.Customer Success Stories
5.Q&A

5. Data, Data, Everywhere…
• Organizations are awash with data, but…
• How do I know what data is available?
• What’s its structure?
• How do I know how good it is?
• How do I access the data?
• Data Services Marketplaces address these
questions
• Provide a mechanism for end users and
developers to find and access data
• For reports, applications, analytics, etc.
And not a drop of it to read!
5

6. What is a Data Services Marketplace?
A single place where consumers of data –
developers or end users – can search for, find,
and access data, that is available to them, as a
service.
6

7. Data Services Marketplace
7
Enterprise Apps
SQL (JDBC/ODBC), RESTful Web Services, SOAP, JMS, etc.
Operational
Systems
Analytical
Systems
Big Data External/SaaS
Systems
Virtual
Data Marts Virtual ODS
Reusable
Data Services
Metadata Scheduling & Delivery Usage Stats
Enterprise Data
Service Registry
Data Services
Layer

8. Enterprise Data Service Registry
• Catalog of data available to consumers
• Metadata for data ‘services’
• Format and structure of data, description of data and attributes
• Data lineage information – where does the data come from?
• Access permissions for data services
• Enforcing privacy policies for PII
• Monitoring and auditing of data usage
• Monitoring and managing QoS/SLA
• Knowing who is access data, when and how…
8

9. Virtual Data Services Layer
A data access layer that abstracts underlying data sources and
exposes them as discrete services to form a ‘data API’
 Different users and developers across the enterprise can access data in a
secure and managed fashion and share a common data ‘model’
 Provides secure and managed access to data across the enterprise
 Provides consistency of data
 Hides complexity, format, and location of actual data sources
 Supports many consumption protocols and patterns
Example: Single data access layer for all development teams to avoid
‘hunting down and interpreting data differently by project’
9

10. Data Services Layer
10
Enterprise Apps
SQL (JDBC/ODBC), RESTful Web Services, SOAP, JMS, etc.
Operational
Systems
Analytical
Systems
Big Data External/SaaS
Systems

11. Benefits of Data Services
• Agility
• Rapid development, service reuse, quicker time-to-value
• Data Integration
• Combine data to provide data ‘as needed’ not ‘as stored’
• Aligned with logical data models
• Data Quality
• Data consistency, common ‘model’
• Single Point of Interaction
• Users don’t need direct access to data sources, better management and
security
11

12. Challenges of Data Services
• Security
• How secure is the data? How is access controlled?
• Privacy
• How is PII protected? How can you audit access compliance?
• Performance/QoS
• Does the data services layer ‘get in the way’? How does it impact
performance? And QoS/SLAs?
• Data Governance and Veracity
• How do you know that the data is ‘good’?
12

13. 13
Implementing Data Services
• Data services can be implemented using a
number of different technologies:
1. ESB/SOA
2. ETL
3. MDM
4. Data Virtualization
• Typically it will be one or more of the above
Different Technologies

14. 14
Data Services with Data Virtualization
• Optimized for data services
• Configuration and not coding
• Rapid development and time-to-value
• Supports multiple delivery styles
• Real-time/right-time, batch/file, etc.
• Multiple protocols – SQL (JDBC/ODBC), Web Services (REST/SOAP), …
• Complements other technologies
• MDM exposed as services through data virtualization
• Combined with an ESB for process flows
The Foundation for the Data Services Marketplace

15. Data Services Demo

16. Addressing the Challenges

17. Challenges of Data Services
• Security & Privacy
• How secure is the data? How is access controlled?
• How is PII protected? How can you audit access compliance?
• Performance & QoS
• Does the data services layer ‘get in the way’? How does it impact
performance?
• How can we control the resources to comply with SLAs?
• Data Governance & Veracity
• How do we know that the data is ‘good’?
17

18. Security & Privacy
Challenges of Data Services
18

19. 19
Security in Denodo
Overview
Authentication
• Pass-through authentication
• Kerberos and Windows SSO
• OAuth, SPNEGO
Authentication
• Standard JDBC/ODBC security
• Kerberos and Windows SSO
• Web Service security
LDAP
Active Directory
Role based Authentication
Guest, employee, corporate
Schema-wide Permissions
Data Specific Permissions
(Row, Column level, Masking)
Policy Based Security
Data in motion
• SSL/TLS
Data in motion
• SSL/TLS
Encrypted
data at rest
• Cache
• Swap

20. 20
Security in Denodo
Data in Motion – secure channels
 Using SSL/TLS
 Client-to-Denodo and Denodo-to-source
 Available for all protocols (JDBC, ODBC, ADO.NET and WS)
 WS security: Basic, Digest, SPNEGO (Kerberos), integration with LDAP
Data at Rest – secure storage
 Cache: third party database. Can leverage its own encryption mechanism
 Swapping to disk: serialized temporarily stored in a configurable folder that can be
encrypted by the OS
Encryption/Decryption
 Support for custom decryption for files and web services
 Transparent integration with RDBMs encryption
Securing data

21. 21
Security in Denodo
Authentication
 Native and LDAP/Active Directory based
 Support for Kerberos and Windows SSO
Authorization
 Virtual Database
 View
 Row and Column level authorization
 Masking
 Custom policies for specific security constrains and integration with external policy servers
Roles
 Integration with LDAP/AD groups
 Role hierarchies supported
Pass-through session credentials
 Leverage existing source privileges
Authentication and Authorization

22. Role-Based Granular Privileges
22
Security In Denodo

23. Advanced Selective Data Masking
23
Security In Denodo

24. Advanced Selective Data Masking
24
Security In Denodo

25. 25
Custom
Policy
Conditions satisfied
Security: applies custom security
policies
• If person accessing data has role of
'Supervisor' and location is 'New
York', then show compensation
information for employees in the
New York office only.
Enforcement: rejects/filters
queries by specified criteria like
user priority, cost, time of day etc.
• If the production batch window runs
from 3 am - 6 am, there is
increased load on production
servers at this time. So, all queries
on these servers can be blocked
during this time to prevent failure of
a process.
Data consuming users, Apps
Query
Accept / add filters
Reject
Security - Custom Policies
Interception of queries before they are executed

26. Performance & QoS
Challenges of Data Services
26

27. 27
Resource Manager
Apply resource restrictions based on a set of rules
 Rules classify sessions into groups
 By user, role, application, IP, time of the day, etc.
 E.g. Connections from application ‘app1’ coming from users with role
‘reporting’ are assigned to a group
 Apply restrictions for each group.
 Change priority, change concurrency settings, change max timeouts, etc
Controlled Resource Allocation

28. 28
Resource Manager
Controlled Resource Allocation
1 Defines a rule that will be
triggered for “app1” and users
with the role “reporting”
2 For those request that fulfill the rule, if the
CPU usage is greater than 85%, will apply the
following:
• Reduce thread priority
• Reduce the number of concurrent requests
• Limit the number of queued queries

29. 29
Performance Features
Data Provisioning Layer
Selective Materialization
Intelligent Caching of only the most relevant and often used
information
Streaming & pagination
Operate on data in streaming mode for a low memory
footprint. Paginate responses to control the size of datasets
Parallelism
Parallel access to disparate sources to minimize latency
NESTED JOINs for concurrent access to sources with
restricted query capabilities
Optimized Resource Management
Smart allocation of resources to handle high concurrency
Throttling to control and mitigate source impact
Resource plans based on rules

30. 30
Quality of Service in Real Scenarios
• Multinational insurance & reinsurance company
• Average response time of 80-100ms
• 200+ concurrent queries
• 2 nodes – 4 cores each
• Global semiconductor chip manufacturer
• Enterprise-wide data access layer
• 200+ developers trained in Denodo
• ~50 data sources, +90 data services published
• Response times under 120ms, well in compliance with their internal SLAs
(200-300ms)
• 128+ cores in production
Data Provisioning Layer

31. Data Governance & Veracity
Challenges of Data Services
31

32. 32
Enterprise Data Governance
Understand the “source of truth” and transformations of every piece of data in the
model
Data lineage

33. 33
Enterprise Data Governance
Understand the “source of truth” and transformations of every piece of data in the
model
Data lineage

34. Customer Success Stories

35. 35
DrillingInfo
• SaaS-based platform that provides business intelligence and
decision support technology
• Facilitates faster, smarter decisions for the oil and gas upstream
E&P industry
• HQs in Austin, Texas. More than 400 employees on 5 continents
• Services 3,000+ companies globally
Overview

36. 36
DrillingInfo
Architecture

37. 37
-Jay Heydt, Manager, Drillinginfo
As a data and business intelligence provider, one of our biggest
challenges is the need to rapidly sell the data that we acquire. The
Denodo Platform enables us to build and deliver data services to our
internal and external consumers within 3–4 hours instead of the 1–2
weeks that would take with ETL”

38. 40
Guardian Life
• Large mutual life insurer with $7.3 billion in capital and $1.5 billion in operating
income in 2015.
• Founded in 1860, the company has paid dividends to policyholders every year
since 1868.
• ~8,000 employees and a over 3,000 financial representatives in 70+ agencies
nationwide.
• Offerings:
• Life insurance
• Disability income insurance
• Annuities
• Investments to dental, vision, and 401(k) plans.
Overview

39. Enterprise Data Marketplace
41

40. Enterprise Data Marketplace
42

41. Enterprise Data Marketplace
43

42. Enterprise Data Marketplace
44

43. Q&A

44. Thanks!
www.denodo.com info@denodo.com
© Copyright Denodo Technologies. All rights reserved
Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical,
including photocopying and microfilm, without prior the written authorization from Denodo Technologies.