Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Intel boubker el mouttahid
Similar to Intel boubker el mouttahid (20)
More from BigDataExpo
More from BigDataExpo (20)
Recently uploaded
Recently uploaded (20)
Intel boubker el mouttahid
- 1. McAfee Confidential Securing Big Data Boubker El Mouttahid | Enterprise Architect
- 2. McAfee Confidential The risks associated with Big Data technologies New Technology New Risks Any technology that is not well understood will introduce new vulnerabilities. Typically include open source code Luck of security best practices Security User authentication and access to data from multiple locations may not be sufficiently controlled There is significant opportunity for malicious data input and inadequate data validation Attack surface of the nodes in a cluster may not have been reviewed and servers adequately hardened. Data Privacy & Compliancy Regulatory requirements may not be fulfilled, with access to logs and audit trails problematic Data Security: Hadoop store the data as it is without encryption to improve efficiency Granular access control: What types of personal information can be deemed sharable and with whom 2
- 3. McAfee Confidential Emerging Hadoop Security Maturity Model 3
- 4. McAfee Confidential Security Requirements of the Enterprise • Network Security Authentication Authorization Data Protection Visibility & Monitoring Secure Configurations for Hardware and Software
- 5. McAfee Confidential Comprehensive, Security & Compliance-Ready Securing the underlying operating system Authentication, Authorization, Audit, and Compliance Perimeter Guarding access to the cluster itself InfoSec Concept: Authentication Access Defining what users and applications can do with data InfoSec Concept: Authorization Visibility Reporting on where data came from and how it’s being used InfoSec Concept: Audit Data Protecting data in the cluster from unauthorized visibility InfoSec Concept: Compliance Network Security Secure Configuration Real Time Monitoring
- 6. McAfee Confidential Platform Security Requirements Network Security Secure Configuration Real Time Monitoring Defense, resilience Deep packet inspections Securing the underlying operating system and the applications installed on the system Log correlation and analysis to rapidly identify anomalies
- 7. McAfee Confidential Perimeter Security Requirements Preserve user choice of the right Hadoop service (e.g. Impala, Spark) Conform to centrally managed authentication policies Implement with existing standard systems: Active Directory and Kerberos Perimeter Guarding access to the cluster itself InfoSec Concept: Authentication
- 8. McAfee Confidential • Contributed by Intel in 2013 • Blueprint for enterprise-grade security Cloudera and Intel Project Rhino Rhino Goal: Unified Authorization Engineers at Intel and Cloudera (together with Oracle and IBM) are now jointly contributing to Apache Sentry Rhino Goal: Encryption and Key Management Framework Cloudera and Intel engineers are now contributing HDFS encryption capabilities that can plug into enterprise key managers
- 9. McAfee Confidential Right Solution • Provides maximum flexibility • Delivers centrally managed authentication • Automates configuration while leveraging existing infrastructure Watch for other solutions that… • Require setup of additional Kerberos server and cross-realm trust • Lead to manual and error-prone Kerberos config on individual nodes • Offer limited support for username/password authentication against AD Business Impact • DELAY: Must seek InfoSec sign-off for cross-realm trust establishment • SET-UP COST: Must procure & configure Kerberos server & nodes • ONGOING MAINTENANCE: Additional task for each new Kerberos user Authentication and Identity ✓ ✗ “Integrate With Existing Enterprise Authentication Mechanisms for Hadoop Identity and Access Management.” Gartner – Best Practices for Securing Hadoop
- 10. McAfee Confidential Access Security Requirements Provide users access to data needed to do their job Centrally manage access policies Leverage a role-based access control model built on AD Access Defining what users and applications can do with data InfoSec Concept: Authorization
- 11. McAfee Confidential Manage data access by role, instead of by individual user • Fraud Analyst Role has read access on ALL transaction data • Branch Teller Role has read / write access on very limited set of data • Relationships between users and roles are established via groups An RBAC policy is then uniformly enforced for all Hadoop services • Provides unified authorization controls • As opposed to tools for managing numerous, service specific policies RBAC and Centralized Authorization
- 12. McAfee Confidential Sentry provides unified authorization via fine-grained RBAC for Impala, Hive, Search, MapReduce, Pig, HDFS… Unified Authorization with Apache Sentry Sentry Perm. Read Access to ALL Transaction Data Sentry Role Fraud Analyst Role Group Fraud Analysts Sam Smith
- 13. McAfee Confidential • Sentry can be configured to use AD to determine a user’s group assignments • Group assignment changes in AD are automatically picked up, resulting in updated Sentry role assignments Sentry and Active Directory Groups Sentry Perm. Read Access to ALL Transaction Data Sentry Role Fraud Analyst Role AD Group Fraud Analysts Sam Smith
- 14. McAfee Confidential Sentry enforces each rule across Hadoop components 16 Hive Server 2 Enforcemen tcode Impala MapReduce , Pig, HDFS* Apps: Datameer, Platfora, etc* Permissions rules Common enforcement code for consistency. Rule 1: Allow fraud analysts read access to the transaction table Permissions specified by administrators (top-level and delegated) Enforcemen tcode Enforcemen tcode Enforcemen tcode
- 15. McAfee Confidential Sentry – The Open Standard Broad Contributions • Cloudera • IBM • Intel • Oracle Multi-Vendor Support • Cloudera • IBM • MapR • Oracle Wide Industry Adoption • Banking • Healthcare • Insurance • Pharma • Telco Third-Party Integrations • Oracle Endeca • Platfora
- 16. McAfee Confidential Right solution • User access determined via group assignments • Unified authorization via granular RBAC policy • Leverages existing AD infrastructure Watch for other solutions that… • Require redundant security policies; one for every access path • Do not offer an RBAC policy model • Cannot meet InfoSec requirement for centrally managed authorization • Depend on manual mirroring of AD group assignments Business Impact • DELAY: InfoSec approval for data access assignments outside of AD • ONGOING COSTS: Authorization policies have to be reviewed and tested for each access path • ONGOING MAINTENANCE: Mirroring of directory group assignments Authorization and Access ✓ ✗ “A must-have for enterprise access scenarios, the most prominent solution here is Apache Sentry” Gigaom– Hadoop Security: Solutions Emerge
- 17. McAfee Confidential Visibility Security Requirements Understand where report data came from and discover more data like it Comply with policies for audit, data classification, and lineage Centralize the audit repository; perform discovery; automate lineage Visibility Reporting on where data came from and how it’s being used InfoSec Concept: Audit
- 18. McAfee Confidential Right solution • Users can easily discover data and examine lineage • Complies with requirements for audit, classification, lineage • No additional IT burden: audit logs are centralized, lineage is automatic, users can self help Watch for other solutions that… • Offer no unified audit trail for point in time user access • Limited lineage, available only at file level, no visualization • Third-party tools required for data discovery Business Impact • DELAY: InfoSec testing, validation, and approval of third-party tools • COMPLIANCE RISK: Inability to respond quickly if point-in-time user access history needed. Inability to meet core data governance requirements without column level lineage Audit and Governance ✓ ✗ “Cloudera Navigator is… one pane of glass for all Hadoop metadata and events including security” Gartner – Protecting Big Data In Hadoop
- 19. McAfee Confidential ©2014 Cloudera, Inc. All rights reserved. Data Security Requirements Perform analytics on regulated data Encrypt data, conform to key management policies, protect from root Integrate with existing HSM as part of key management infrastructure Data Protecting data in the cluster from unauthorized visibility InfoSec Concept: Compliance
- 20. McAfee Confidential Right solution • Brings the power of pervasive analytics to regulated data • Delivers compliant protection with encryption, key management • Provides separation of duties between system and data admins • Conforms to existing policies regarding HSM based key management Watch for other solutions that… • Run risk of data breach or theft by storing data in clear text • Require 3rd party encryption and key management solutions • Offer no protection against privileged user access • May not integrate with corporate HSM’s Business Impact • EXPOSURE: theft or breach of data – fines, damage to brand • DELAY: InfoSec testing, validation and approval of 3rd party solutions (COST) • DELAY: InfoSec approval for key management independent of HSM • COMPLIANCE RISK: Potential PCI issues Encryption and Key Management ✓ ✗ “Cloudera… recognized that data in Hadoop must be protected both at rest and in transit… and have made data encryption part of their products.” Gartner – Protecting Big Data In Hadoop
- 21. McAfee Confidential 23 Recommendation Enterprise data hub with built-in security Comprehensive and integrated security Compliance Ready
- 22. Intel & McAfee Confidential 24
Editor's Notes
- For Hadoop operators in finance, government, healthcare, and other highly-regulated industries to enable access to sensitive data under proper compliance, each of the four functional requirements must be achieved: Perimeter Security: Guarding access to the cluster through network security, firewalls, and, ultimately, authentication to confirm user identities Data Security: Protecting the data in the cluster from unauthorized visibility through masking and encryption, both at rest and in transit Access Security: Defining what authenticated users and applications can do with the data in the cluster through filesystem ACLs and fine-grained authorization Visibility: Reporting on the origins of data and on data usage through centralized auditing and lineage capabilities
- For Hadoop operators in finance, government, healthcare, and other highly-regulated industries to enable access to sensitive data under proper compliance, each of the four functional requirements must be achieved: Perimeter Security: Guarding access to the cluster through network security, firewalls, and, ultimately, authentication to confirm user identities Data Security: Protecting the data in the cluster from unauthorized visibility through masking and encryption, both at rest and in transit Access Security: Defining what authenticated users and applications can do with the data in the cluster through filesystem ACLs and fine-grained authorization Visibility: Reporting on the origins of data and on data usage through centralized auditing and lineage capabilities
- Directory services and Kerberos
- “We currently manage all user authentication and service access through a combination of Active Directory and Kerberos. We have ‘audited’ procedures based around these technologies. Help me understand how your cluster will fit into these paradigms. Also, my cousin said I will have to stand up an additional KDC and put Kerberos clients on every desktop. I really hope that’s not the case, Kerberos configuration is a pain in the a**”
- There are many aspects to security - and it's all too easy for other vendors to claim their platforms are "secure" because they cover one or more of these pillars. To achieve comprehensive security, we offer all four pillars of security: Perimeter, Access, Visibility, and Data. Cloudera Enterprise achieves all of these and is compliance-ready out-of-the-box to ensure you’re protected
- In trying to solve the data access problem in Impala – we need to introduce a very important concept. Role Based Access Controls. This is very similar to the idea of Active Directory. With role based access control - I am a user in a group, and that group is assigned to some role, that role has a set of privileges that define what data the role can access and the actions that can be performed. This relationship user-group-role-privileges defines the users access and privileges.
- Recall: AD Group membership in conjunction with Kerberos is used to control access to SERVICES e.g. Impala
- Open standard for unified authorization in Hadoop Ensures compatibility with future developments Integrates with larger enterprise posture - How many companies are contributing to it - How many competitors ship it - How many supporting tools work with it - The business benefits of that openness
- There are many aspects to security - and it's all too easy for other vendors to claim their platforms are "secure" because they cover one or more of these pillars. To achieve comprehensive security, we offer all four pillars of security: Perimeter, Access, Visibility, and Data. Cloudera Enterprise achieves all of these and is compliance-ready out-of-the-box to ensure you’re protected
- Simplifies and automates the configuration reducing the administrative burden allowing processes to scale and reducing human error Introduced a unified, pluggable authorization model that is seeing adoption across distros and in 3rd party products Comprehensive capabilities = Superior value through inclusion of many capabilities that require 3rd party purchases in other distributions Takes the friction out of the go-live process by providing full coverage against the InfoSec checklist Security “Product” extends well beyond Hadoop security controls
- There are many aspects to security - and it's all too easy for other vendors to claim their platforms are "secure" because they cover one or more of these pillars. To achieve comprehensive security, we offer all four pillars of security: Perimeter, Access, Visibility, and Data. Cloudera Enterprise achieves all of these and is compliance-ready out-of-the-box to ensure you’re protected