This document discusses security measures for Hadoop environments. It outlines existing issues with Hadoop's architecture like distributed nodes and lack of security. It proposes solutions like using Kerberos and encryption. It also discusses how some corporate security solutions like IBM and Solera Networks use big data analytics to detect abnormalities and insider threats in real-time. The conclusion states that traditional security solutions are insufficient for big data and real-time analytics is needed to derive actionable security intelligence from data streams.

1. Big Data in term of
security measure For Hadoop
Environments
Yaakub Bin Idris
Advanced Informatics School
Malaysia University of Technology
Kuala Lumpur, Malaysia
yaakub.idris@gmail.com

2. INTERNET SECURITY THREAT REPORT

3. DATA BREACH CONTRIBUTION
 Employees mishandling
information.
 Violations of industry
and government regulations.
 Lack of system controls.
 IT or business process
failure.
 Insiders greatly contribute
to data breaches and that
loss can be costly to organizations.

4. FACTORS THAT INFLUENCE THE COST OF
DATA BREACH
 The company had an incident management plan.
 The company had a relatively strong security posture at the time of
the incident.
 Chief information security officer (or equivalent) responsible for data
protection.
 Data was lost due to third party error.
 The company notified data breach victims quickly.
 Consultants were engaged to help remediate the data breach.

5. WHY HADOOP???

6. EXISTING HADOOP ARCHITECTURE
• Fault tolerance by detecting faults
and applying quick, automatic
recovery.
• Data access via MapReduce streaming
Simple and robust coherency model.
• Portability across varied commodity
hardware and operating systems.
• Scalability to reliably store and
process large amounts of data.
• Reliability by automatically
maintaining multiple copies of data
and automatically redeploying
processing logic in the event of
failures.

7. CURRENT ARCHITECTURAL ISSUES
 Distributed nodes.
 ‘Sharded’ data.
 Data access/ownership.
 Inter-node communication.
 Client interaction.
 NoSecurity.

8. PROPOSED SOLUTIONS
 Use Kerberos.
 Use file/OS layer encryption.
 Use key/certificate management.
 Validate nodes during deployment.
 Log transactions, anomalies, and administrative activity.
 Use SSL or TLS network security.

9. MAKING HADOOP FRAMEWORK AS A ‘STACK’

10. SOME OF CORPORATE SECURITY SOLUTIONS
IBM Security Intelligence
 Real-time abnormality
detection.
 High-speed querying of security
intelligence data.
 Flexible BDA across all types of
data sources.
 Interactive visualization
dashboards for exploring BDA
outputs.
 Network forensics.
Solera Networks
 Root Cause Analysis.
 Pathway Analysis.
 Application Discovery.
 Data Leakage Discovery.
 Insider Threat Analysis.
**Big Data Analytics (BDA)

11. CONCLUSIONS
 Traditional security solutions are not capable anymore of
encompassing the real-time big data
 Using security of Big Data techniques to derive actionable intelligence
and insights from streams in real-time is rapidly becoming a need.
 Current adoption of security measure solutions is by no means
revolutionary but the awareness of adoption is increasing rapidly.
 Different types of big data solution, the primary components of a
security model, some examples of security outputs, steps for
implementing a security solution, and finally, the corporates/product
which are offering solutions for cybersecurity along with their
features.