Best Practices for Protecting Sensitive Data Across the Big Data Platform

®
1© 2016 MapR Technologies | © 2016 Dataguise, Inc..© 2016 MapR Technologies | © 2016 Dataguise, Inc.
Best Practices for Protecting Sensitive Data
Across the Big Data Platform
Mitesh Shah
MapR | Product Manager
Security & Data Governance
®
Venkat Subramanian
CTO | Dataguise

®
2© 2016 MapR Technologies | © 2016 Dataguise, Inc..
Business Intelligence Trend for 2016 onwards…
IT-led, System-of-Record
• Limited access
• Glacial speed of response
Pervasive, Business-led, Self-service Analytics
• Near Real-time
• Agile BI & Analytics
• Deeper Insights into Diverse Data
Rita Sallam (Gartner)*

®
Big Data Paradox
Data is the Biggest Asset
Data is also the Biggest Vulnerability

®
Secure Business Execution
The ability of an Enterprise to safely and responsibly
leverage the value of all of their data assets to gain new
business insights, maximize competitive advantage,
and drive revenue growth.

®
MapR and Dataguise…
Enable SECURE BUSINESS EXECUTION
Through
Trusted Platform and Sensitive Data Management
…

®
Big Data Platform Needs to be Trusted (not just secure)
Can we properly identify users?
Can we authorize access
to data?
Can we plug in existing
enterprise systems?
Is my data highly available?
Is there a proper paper trail?
Have others done this before?
Is multi-tenancy supported?
Are apps supported across
geographies and data centers?
Is my data governed?
TRUSTED
SECURE
Questions to Ask of Your Big Data Vendor.
Verify the Platform is Trusted.

®
MapR Trust Model
Credibility
VulnMgmt
Detection
Response
Compliance
AA
DPA
Governance
Resilience
Four Pillars
of Security
Auditing
Authorization
Data
Protection
Authentication

®
What’s the (Big) Difference?
Flexibility
•Multiple execution engines:
Hive, Spark, MapReduce,
Drill…
Scale
•1000s of users, groups
and applications sharing
the same cluster
•100s data sources
•PBs of data
Multi-Structured Data
•Multiple data formats: Parquet,
JSON, CSV, MapR-DB tables

®
A
MapR Trust Model (Product Security)
Granular
Authorization
Ubiquitous
Data Protection
• Access Control Expressions (ACEs)
• Protect files, tables, column families,
columns,and managementobjects
• Extend to role-based access control
(RBAC) with custom role functions
• Drill Views
• Encryption for data in motion
• Within a cluster
• Between clusters
• Between clientand cluster
• Encryption for data at rest
• LUKS
• Self-encrypting disk
• Partners
• NSA-level cryptographic algorithms
• All events recorded immediately
in JSON log files, with minimal
performance impact
• Includes data access and
administrative actions
• Ad hoc queries and custom
reports on audit logs via SQL and
standard BI tools
• Ticket-based authentication for all
services in the cluster
• Integration with LDAP, Active Directory
and other third-party directory services
• Kerberos or username/password
authentication
AA
DPA
4
21
3
Flexible
Authentication
Robust
Auditing

®
Granular Authorization with MapR

®
The Problem with POSIX Permissions
-rw-rw---- bruce dev-team
POSIX Permissions
user
group
other
1.Change ownership of file to Sally.
2.Add Sally to dev-team group, even
if she’s not a developer.
3.Allow ‘others’ to read the file.
Scenario 1:
Sally needs to read the file.
Options:
???
Scenario 3:
All members that belong to both dev_team
and managers.
1.Allow ‘others’ to read the file.
2.Create a supergroup ‘Tech’, and
include all members from dev,
QA, and Support in that group.
chgrp Tech <filename>
Scenario 2:
Groups ‘QA’ and ‘Support’ need to read the file.
Options:
POSIX Permissions Are Limiting
AUTHORIZATION

®
POSIX ACLs vs ACEs
r : user:sally |
(group:dev_team & group:managers)
Access Control Lists
MapR Access Control Expressions
AUTHORIZATION
Which one is easier to set and understand?
Which one allows for higher granularity?

®
MapR Has ACEs for Files and MapR-DB Records
Example: user:mary | (group:admins & group:VP) & user:!bob
Permissions on files, tables, column families, columns, JSON documents and sub-documents
AUTHORIZATION
Use Access Control Expressions (ACEs) to set granular permissions.

®
File ACEs – Key Features
Intuitive
Inheritance
Subdirectories and
files inherit perms
from parent
directory
Whole-Volume
ACEs
Volume-level filter –
useful in multitenant
environments.
Roles
Arbitrary grouping of
users according to
your business needs
High Performance
No performance hit
Boolean Operators
Allowing for
ultra fine-grain
permissions
AUTHORIZATION

®
File ACEs: Whole Volume ACE Example
Whole-Volume ACE
r: group:finance
Jane grants read access to Bob.
File: /finance/final_report.csv
r: user:bob
Bob cannot read the file
/finance/final_report.csv because
the whole-volume ACE is set to
allow read-access to finance only.
Jane
(Finance)
Bob
(Developer)
Whole-Volume ACE
AUTHORIZATION

®
© 2015 MapR Technologies 16
ACEs for Streams Too
AUTHORIZATION

®
Robust Auditing with MapR

®
MapR Audits
• Who touched customer records outside of
business hours?
• What actions did users take in the days
before leaving the company?
• What operations were performed without
following change control?
• Are users accessing sensitive files from
protected/secured source IPs?
• Why do my reports look different, despite
sourcing from same underlying data?
Monitoring Incident
Response
Security
AUDITING
Serving Security Analysts

®
MapR Audits – Key Features
Data Access
• Files
• MapR-DB Tables
Cluster Operations
• Administrative Operations
• Maprcli commands
Authentication Requests
Secure High Performance
Flexible
• Retention Period
• Maxsize
• Coalesce Interval
• Selective Auditing
JSON Format
{"timestamp":"{$date=2015-06-
01T05:24:58.231Z}","operation":"GETATTR",
"user":"root","uid":"0","ipAddress":"10.10.x.x",
"nfsServer":"10.10.x.x","srcPath":"/dbtest.0/","
srcFid":"2147.16.2","VolumeName":“mktg_file
s","volumeId":“mktg_files","status":"0"}
AUDITING

®
Querying Audit Logs with SQL
Example: detect suspicious, failed commands

®
Auditing After-Hours Access

®
Data Protection with MapR

®
Encryption at Rest (Today)
SSN
Credit
Card #
Health Records
Name +
Age + Address
Sensitive Data
Volume
Self-
Encrypting
Disk
2
3
Use Partners for Masking, Tokenization,
Format Preserving Encryption
DATA PROTECTION
Many Options for Block-Level, Disk-Level,
and Field-Level Encryption
1

®
Sensitive Data Management with Dataguise

®
Cost of a Data Breach
“Hackers and criminal insiders cause the most data
breaches…malicious attacks can take an average of
256 days to identify…The most costly breaches
continue to occur in the US and Germany at $217 and
$211 per compromised record…If a healthcare
organization has a breach, the average cost could be
as high as $363.”
Time and Financial Impact on Organizations
Ponemon Institute’s 2015

®
Secure Environment
Perimeter Security
• Physical security, Firewalls, IDS/IPS…
Volume/File-level Encryption
• Control over data access
• Meeting regulatory compliance…
Aren’t these enough?
YOU NEED BOTH…AND *MORE

®
PHI: Guidance for Data De-Identification
Sensitive/Privacy Data
• Name
• Address
• Dates – Birth, Death...
• Telephone Numbers
• Device Identifiers and Serial Numbers
• Email Addresses
• SSN
• Medical Record Numbers
• Account Numbers
….
….

®
What Should We Do?
At a Granular (cell) Level:
• Precisely locate sensitive content across ALL repositories
• Protect those assets appropriately – masking, encryption
• Provide “controlled” access to data
• Enable employees, trusted partners to make data-driven decisions
RISKS
BREACH
SECURITY
COMPLIANCE
VALUE
REVENUE
DATA DRIVEN DECISIONS
BUSINESS INTELLIGENCE

®
DgSecure
DETECT
Where sensitive content is
present in structured,
unstruct. & semi-
structured data
AUDIT
Who has access to
which sensitive data &
identify misalignments
and risk factors
PROTECT
Sensitive data at the
element level –
encrypt/decrypt with RBAC
mask or redact
MONITOR
Based on alert policies,
track sensitive data
access through a 360°
dashboard

®
DgSecure
DETECT
unstruct. & semi-
structured data
AUDIT
Who has access to
and risk factors
PROTECT
element level –
mask or redact
MONITOR
dashboard
Across Hadoop, RDBMS,
Files, NoSQL DB

®
DgSecure
On Premise, in the
Cloud, or Hybrid
DETECT
unstruct. & semi-
structured data
AUDIT
Who has access to
and risk factors
PROTECT
element level –
mask or redact
MONITOR
dashboard
Across Hadoop, RDBMS,
Files, NoSQL DB

®
How do we do that in DgSecure?

®
Complex Sensitive Data Detection
SENSITIVE DATA DISCOVERY FOR COMPLEX
ENVIRONMENTS
Patterns in “Strings”
• Digit Patterns: 4451 3340 0023 1200 8/16 B7127157
Expires 04-19-15
Patterns in “Grammar”
• August Thomson vs
1240 AugustAve vs
12 August 1994
Patterns in Context (Dependent)
• Other data elements in horizontal or vertical vicinity
‘94538’near address elements
Patterns in Combination (Composite)
• CCN & Name, CCN, Name, Expiry not just CCN
Patterns in Knowledge
• Ontologies HL7 Encoding, Financial Market Data
DISCOVERY FOR:
Data at Rest
• Hadoop (HDFS)
• DBMS
• Teradata
• Files
• SharePoint
Data in Motion
• Flume (into HDFS)
• FTP (into HDFS or between file
systems)
• Scoop (into HDFS)
• Kafka (Q3 2016)

®
Sensitive Data Protection
Masking
• Obfuscation, one-way operation
• Multiple options in DgSecure – fictitious but realistic values, X’ing out part of the content…
• Consistent masking to retain statistical distribution of data
Encrpytion
• Encrypted cell/row
• Accessible by authorized users only – Hive, bulk, via App
• Granular protection
Redaction
• X’ing out entire sensitive data cell
• Nullifying
Masking & Encryption in Hadoop

®
Data Masking multiple Options - Examples
Masking Option Applied Original Value Masked Value
Telephone – Random
- Realistic, fictitious -
(508) 850-0058 (325) 418-0131
Telephone – Character
- Hide digits -
(508) 850-0058 XXX-XXX-0058
Telephone – Intellimask
- Replace first 3 digits -
(508) 850-0058 (451) 850-0058
Telephone – FPM - Format Preserving
- Replace char & Digits with same type -
(508) 850-0058
508 850 0058
508-850-0058
(729) 432-9647
729 432 9647
729-432-9647
Telephone – Static Masking
- Replace all with (111) 222-3333 -
(508) 850-0058 (111) 222-3333

®
Unstructured Data – Any Sensitive Elements?
RAW DATA

®
Masking Data in Hadoop (Cell Level)
RAW DATA

®
MASKED DATA

®
Encrypting Data in Hadoop (Cell Level)
MASKED DATAENCRYPTED DATA

®

®
Decryption through Hive Queries
User WITHOUT access privileges for Names and SSN

®
Decryption through Hive Queries
User WITH access privileges for Names and SSN

®
BI Use Cases and Sensitive Elements
Brand Sentiment
Log Analysis
Customer Retention
Clinical Trial Analysis
Payments Risk Mgmt.
Trading System Perf.
Risk Modeling
Supply Chain Optimization
Smart Metering
Insurance Premiums
Process Efficiency
Person of Interest Discovery
Dynamic Pricing
IT Security Intelligence
Real-time Upsell
Monitoring Sensors
Analytic
Transactional
Name
Address
Email Address
Customer Lifetime Value
IP Address
URL
Medical Record Number
Social Security Number
Telephone Number
Date of Birth (DOB)
IP Addresses
Credit Card Number
Credit Limit
Purchase Amount
VIN
Device ID

®
Protection Policy: Encryption, Masking
Brand Sentiment
Log Analysis
Customer Retention
Payments Risk Mgmt.
Risk Modeling
Supply Chain Otimization
Smart Metering
Insurance Premiums
Process Efficiency
Dynamic Pricing
Real-time Upsell
Monitoring Sensors
Analytic
Transactional
Name
Address
Email Address
IP Address
URL
Telephone Number
Date of Birth (DOB)
Medical Test Results
Credit Card Number
Credit Limit
Purchase Amount
VIN
Device ID
Transaction Date
Mask
Encrypt

®
DgSecure Solution Workflow

®
DgSecure for Hadoop: Policy
DETECT AUDIT PROTECT REPORT
• Policy
• Per Data Feed?
• Protection Options
• Custom Elements
• Singleton
• Composite
• Dependent
• Domain Definition
• Key Management

®
DgSecure for Hadoop: Detection
In-Flight
Within HDFS
Full vs. Incremental
Structured, Semi,
Unstructured
Quick Scan
Element Count

®
DgSecure for Hadoop: Access Audit
In-Flight
Within HDFS
Structured, Semi,
Unstructured
Quick Scan
Element Count
Files/Directories
- Sensitive Elements
- Protected?
- Who has access?
Users
- What can they
access?

®
DgSecure for Hadoop: Protection
In-Flight
Within HDFS
Structured, Semi,
Unstructured
Quick Scan
Element Count
Files/Directories
- Protected?
- Who has access?
Users
- What can they
access?
Domain Based
Masking
Redaction
Encryption
- Field or Record
- AES or FPE

®
DgSecure for Hadoop: Reports
In-Flight
Within HDFS
Structured, Semi,
Unstructured
Quick Scan
Element Count
Files/Directories
- Protected?
- Who has access?
Users
- What can they
access?
Domain Based
Masking
Redaction
Encryption
- Field or Record
- AES or FPE
Job Level
- Sensitive elements
- Directories & Files
- Remediation applied
Dashboard
- Directory or by policy
- Drill-down
Audit report
- User actions
Notifications

®
DgSecure Monitor

®
DgSecure Monitor
Precisely Focused on Monitoring Sensitive Data
• Where are the sensitive content and how many (density)
• How is it protected
• What data is accessed
• Who is accessing it
Across All Enterprise Repositories
• Hadoop and Cassandra
• Cloud support (AWS S3 and Azure Blob)
Continuous, Near-real-time Anomaly Behavior Detection
• Using maching learning to build user profile
• Complex event processing to detect breach
“Out of the Box” Templates

®
DgSecure Monitor
NoSQL
ON PREMISE
Sensitive Info
RDBMS
Hadoop
DgSECURE
CLOUD
DATASTORES
S3
RDBMS
BlobStorage
Hadoop
DgSecure
Repository
Monitoring
Metadata
Monitoring Metadata Manager
Detection
Data Access Information
Monitoring Engine

®
Secure Business Workflow
Enterprise Data Marketplace Use Case

®
Data Marketplace End-to-End Workflow
Multiple Data Feeds with their own Policies
Data Asset Marketplace: Data Assets (Indexed)
Access Granted upon Request per policy & compliance
1SOURCES LANDING ZONE DATA PROCESS
COLLECT METADATA ANNOTATE & PUBLISH BROWSE & REQUEST APPROVE ANALYZE & REPORT
Data Admin Data Scientist Data Admin Data Scientist
Set policy per feed
Data Lake
Data Feed 1
Data Feed 2
Data Feed 3
Data Feed 4
Set Access Control
Metadata Repository
Q1 Region 1 Data Set 1
Access Given
Access Denied
WORKFLOW
SECURE BUSINESS EXECUTION
1 2 3
4 5 6 7 8

®
Set policy per feed
Data Lake
Data Feed 1
Data Feed 2
Data Feed 3
Data Feed 4
Set Access Control
Metadata Repository
Access Given
Access Denied
WORKFLOW
1 2 3
4 5 6 7 8
CISO/CPO:
Set policy per data
feed type

®
Set policy per feed
Data Lake
Data Feed 1
Data Feed 2
Data Feed 3
Data Feed 4
Set Access Control
Metadata Repository
Access Given
Access Denied
WORKFLOW
1 2 3
4 5 6 7 8
Data Asset Owner:
Provenance
metadata

®
Set policy per feed
Data Lake
Data Feed 1
Data Feed 2
Data Feed 3
Data Feed 4
Set Access Control
Metadata Repository
Access Given
Access Denied
WORKFLOW
1 2 3
4 5 6 7 8
Run Discovery to
detect sensitive data
Metadata to
repository
Mask/Encrypt to protect
sensitive data
Metadata incl. lineage
to repository

®
Set policy per feed
Data Lake
Data Feed 1
Data Feed 2
Data Feed 3
Data Feed 4
Set Access Control
Metadata Repository
Access Given
Access Denied
WORKFLOW
1 2 3
4 5 6 7 8
IT/Set Process:
Use Metadata to set
access control

®
Set policy per feed
Data Lake
Data Feed 1
Data Feed 2
Data Feed 3
Data Feed 4
Set Access Control
Metadata Repository
Access Given
Access Denied
WORKFLOW
1 2 3
4 5 6 7 8
Data Asset owner
adds annotations &
adds to Data Asset
Index

®
Set policy per feed
Data Lake
Data Feed 1
Data Feed 2
Data Feed 3
Data Feed 4
Set Access Control
Metadata Repository
Access Given
Access Denied
WORKFLOW
1 2 3
4 5 6 7 8
Data Scientist
browses available
data sets and makes
access request

®
Set policy per feed
Data Lake
Data Feed 1
Data Feed 2
Data Feed 3
Data Feed 4
Set Access Control
Metadata Repository
Access Given
Access Denied
WORKFLOW
1 2 3
4 5 6 7 8
Data owner
approves request
Sets access control
in Ranger

®
Set policy per feed
Data Lake
Data Feed 1
Data Feed 2
Data Feed 3
Data Feed 4
Set Access Control
Metadata Repository
Access Given
Access Denied
WORKFLOW
1 2 3
4 5 6 7 8
Data Scientist runs
data
mining/BI/Analytics

®
Set policy per feed
Data Lake
Data Feed 1
Data Feed 2
Data Feed 3
Data Feed 4
Set Access Control
Metadata Repository
Access Given
Access Denied
WORKFLOW
1 2 3
4 5 6 7 8
Data Scientist runs
data
mining/BI/Analytics
Other Data Sources

®
Set policy per feed
Data Lake
Data Feed 1
Data Feed 2
Data Feed 3
Data Feed 4
Set Access Control
Metadata Repository
Access Given
Access Denied
WORKFLOW
1 2 3
4 5 6 7 8
Other Data Sources

®
MapR + Dataguise: Comprehensive Data Security
Active
Directory
Disk
Auditing
Incident
Response
Authentication
Authorization
Data Protection
Data Protection
Compliance
Vulnerability Management

Best Practices for Protecting Sensitive Data Across the Big Data Platform

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (18)

Similar to Best Practices for Protecting Sensitive Data Across the Big Data Platform

Similar to Best Practices for Protecting Sensitive Data Across the Big Data Platform (20)

More from MapR Technologies

More from MapR Technologies (18)

Recently uploaded

Recently uploaded (20)

Best Practices for Protecting Sensitive Data Across the Big Data Platform