Wadoop is an extensible security framework for Hadoop developed by Wipro that aims to address common security issues in big data administration. It delivers authentication, authorization, and auditing capabilities and provides a unified view of user access and reporting across Hadoop components. The architecture of Wadoop includes components for user management, access control, data zoning for logical data grouping, and dashboards for security reporting and resource utilization.

1. Wadoop
Xtensible Security
Framework for Hadoop
Vivek Shrivastava, Murlidhar Iyer
BigDataCamp Los Angeles
June 14th, 2014

2. Agenda
• Brief overview of data security in Hadoop
• BDAs are the new DBAs
• Data Security Problems in Big Data
Administration
• Introducing Wadoop
• Raffle for some data science books
www.wipro.com

3. > whoami
• Architect at Wipro Technology
• Responsible for Banking and Insurance Clients
• Previously worked at Yahoo!, Shopzilla, HRL
www.wipro.com

4. A bit about Wipro
• Indian multinational information
technology (IT), consulting and
service company
• 147,000 employees serving over
900 clients with a presence in 57
countries
• Multiple ongoing bigdata projects
in Hi-Technology, energy and
financial area
www.wipro.com

5. Why Security is Important
• Information Asset
Protection
• Regulatory Compliance
• Data Sharing
• Regulatory
• BIG Data
www.wipro.com

6. Threats
• Hackers – data breaches
• Access by privileged
users
• Application releases
• Faster changing
landscape of
applications
www.wipro.com

7. Components for Security
• Isolation
• Access Control
• Strong Authentication
– LDAP
– Kerberos
• Logging – Audit
• Encryption
– Network
– Disk
www.wipro.com

8. BDAs are the new DBAs
• Emerging role of Big Data Administrator
– Administrator
• Administration
• Optimal utilization
• Space management
– Developer
• Fast changing software landscape
– Data Analysts
• New tools
• Interaction with data
– Data Stewards
• Space allocation and management
• Directory ownership
• Data movement
• Data lineage
www.wipro.com

9. Problems with Big Data Administration
• Gartner predicts that, through 2016, more than 80 percent of
organizations will fail to develop a consolidated data security policy
• Missing a unified platform for big data management
• Most of the tools are focused towards operational reporting or data
computation
• Businesses have traditionally managed data within structured and
unstructured silos
• Need to collaborate and manage an enterprise data security
• Information security and identity and access management departments
don’t always work together to reduce the risks that lead to breaches
caused by insiders
Ref : http://www.indiainfoline.com/Markets/News/Gartner/5939357116
Ref: https://in.finance.yahoo.com/news/insight-senior-executives-top-security-130000034.html
www.wipro.com

10. Introducing Wadoop
• Framework focused for data management
• Xtensible to work with future products
• Delivers the “AAA of security”:
• Authentication
• Authorization and
• Auditing
• Non intrusive setup and installation
– Active mode
– Passive mode
• Rich set of security features
• Distribution independent
• Near realtime reporting for critical functions
www.wipro.com

11. Architecture of Wadoop
www.wipro.com

12. User Manager – Brings all the users
• User manager collects
all the users
• Search users by any of
the attributes
www.wipro.com

13. Unified Access View
• One place to view
access to different
software and Hadoop
proxy user ACL
• Access list with each
component ( e.g. HDFS,
Hive,HBase, ACL)
www.wipro.com

14. Unified Report of Access
• One place see access
report for different
softwares and whether
it should have been
allowed and disallowed
• Conflict resolution is in
the roadmap
www.wipro.com

15. Data Zone – Logical Grouping
• Provides logical
management without
affecting physical
directory layout
• Easier to maintain
ownership
• Simple Space
management and
chargeback
www.wipro.com

16. Dashboard – It has reporting too
• Visual report of
sensitive and public
data
• Space utilization
• Heatmap of overall
usage
• Resource utilization
www.wipro.com

17. Dashboard - It has reporting too
• Visual report of
sensitive and public
data
• Space utilization
• Heatmap of overall
usage
• Resource utilization
www.wipro.com

18. Common Questions
• How old Wadoop is?
• Can I touch it?
• How can I contribute?
• By the way, Who is
using it?
www.wipro.com

19. Thank You
vivek.shrivastava2@wipro.com
@vivshrivastava
#Wadoop