0
Geo Intelligence India
13-14 Jun 2013
New Delhi
Do lafzon ki hai DATA ki kahani...............
Ek hai ZERO....duja hai ONE.....
2
Big Spatial Data
Security
WELCOME
3
BIG SPATIAL DATA has been with us for ages in
various forms…but pretty invisible!!
4
5
Ancient Egypt
River nile
Engineers used
to try data
analysis to
predict crop
yields
6695 Km long
Basic Intro
Concepts
Perceptions
Challenges
…the 15 min route to THANK YOU slide
6
An English professor wrote the words :
“A Woman without her man is
nothing”
On the chalk board and asked his students to p...
A greater scope of Geo Int info
New kinds of Geo data and analysis
Real time Geo information
Data influx from new technolo...
Spatial data sets exceeding capacity of
current computing systems……
….to manage, process or analyze
the data with reasonab...
10
DATA is Exploding in
Volume Velocity VARIETY
While decreasing in
Veracity
DEFINING BIG SPATIAL DATA
BIG SPATIAL DATA
Finding
actionable info
in Massive
volumes of both
structured and
unstructured
...
90% of data in
the world was
created in the
last 2 years
2.5 EB of
data is
created
every day
U.S. drone aircraft
sent back...
* Estimated revenue FY 2013
growth of geospatial data is outpacing
both software and services and is set
to become a major...
100% security is a myth
No one has said this!!!
But it remains a fact
14
Increasing attack
surface
The technology is
ready….
But are we ready
?
15
16
16
DISASTER RELIEF
FINANCIAL
FRAUD DETECTION
CALL CENTER REQUESTS
DISEASE SURVEILLANCE
INSURANCE
RETAIL
TELECOMMUNICATI...
The other
of the
side
story
17
Security challenges before we adopt
Big spatial data
18
Distributed programming frameworks
Ek
19
Distributed programming frameworks
Input file
Map Intermediate
Combining Shuffle
Output File
Local
Reduce
Reduce
Mapper pe...
MAP REDUCE
FRAMEWORK
 Splits the input data-set into
independent chunks which are
processed
 in a completely parallel ma...
So challenge is not storage but it is I/O speed
One Machine
4 i/o Channels
Each channel : 100 MB/s
10 Machine’s
4 i/o Chan...
Untrusted Mappers
Securing the data in the
presence of an untrusted
mapper
Distributed programming frameworks
23
NO SQL ISSUES
TWO
24
25
First off : the name
NoSQL is not “NEVER SQL”
NoSQL is not “No To SQL “
26
NoSQL
Is simply
Not Only SQL!!!!!
MongoDB
Redis
27
NoSQL DB are still
evolving with
respect to security
infrastructure
Data storage & transaction logs
28
STORAGE TIERS
- Multi-tiered storage media
- Necessitated by scalable size
- Different categories of data
- Different type...
Lower tier means reduced
security, loose access
controls
Keeping track of data
location
Data storage & transaction logs
30
INPUT
VALIDATION/FILTERING
31
How can we trust data ?
Validating data when source
of input data is not reliable?
Filtering malicious data @
BYOD
Input v...
REAL TIME
MONITORING
33
Humongous number of
alerts!!!!
False positives
Filtering malicious data @
BYOD
REAL TIME MONITORING
34
Secure communication
35
End to end security ?
Data encryption : attribute based encryption!!!to be
made richer
Secure communication
36
Granular audits
37
New attacks will keep
happening…and to find
out we need detailed
audit logs
Missed true positives
Granular audits
38
PRIVACY ISSUES
39
EG : How a retailer was
able to identify that a
teenager was pregnant
before her father knew
40
PRIVACY ISSUES
In the worl...
And...
We
Also Have cloud with us?
41
At 1.4% in 2011-12
Cloud was a very small
percentage of the total IT spend
42
Pace of Big Spatial Data adoption has been
Sluggish
43
44
There is unlikely to be
a day soon in near
future when we have a
“FIND
TERRORIST”
BUTTON
45
We have mostly
been reactive till
date…..
46
USE KERBEROS FOR NODE AUTHENTICATION
– (BUT WE KNOW IT’S A PAIN TO SET UP)
STRINGENT POLICIES
STANDARD TO INTRA COUNTRY...
47
Upcoming SlideShare
Loading in...5
×

BIG DATA AND SECURITY CHALLENGES

438

Published on

Big Data is getting bigger and bigger but at the same time before adopting it seriously and exploiting it we should also take care of the security shortcomings it comes up with....from a forensics and security point of view....we need to understand the vulnerabilities they come up with before blindly adopting them!!!!

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
438
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
52
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "BIG DATA AND SECURITY CHALLENGES"

  1. 1. Geo Intelligence India 13-14 Jun 2013 New Delhi
  2. 2. Do lafzon ki hai DATA ki kahani............... Ek hai ZERO....duja hai ONE..... 2
  3. 3. Big Spatial Data Security WELCOME 3
  4. 4. BIG SPATIAL DATA has been with us for ages in various forms…but pretty invisible!! 4
  5. 5. 5 Ancient Egypt River nile Engineers used to try data analysis to predict crop yields 6695 Km long
  6. 6. Basic Intro Concepts Perceptions Challenges …the 15 min route to THANK YOU slide 6
  7. 7. An English professor wrote the words : “A Woman without her man is nothing” On the chalk board and asked his students to punctuate it correctly…. “A Woman,without her man,is nothing” “A Woman: Without her, man is nothing” 7
  8. 8. A greater scope of Geo Int info New kinds of Geo data and analysis Real time Geo information Data influx from new technologies Non traditional forms of Geo data Large volumes of Geo data The latest buzzword Social media data 0 2 4 6 8 10 12 14 16 18 20 Series 1 DEFINING BIG SPATIAL DATA 8 How we understand it ?
  9. 9. Spatial data sets exceeding capacity of current computing systems…… ….to manage, process or analyze the data with reasonable effort due to Volume, Velocity, Variety and Veracity DEFINING BIG SPATIAL DATA BIG SPATIAL DATA 9
  10. 10. 10 DATA is Exploding in Volume Velocity VARIETY While decreasing in Veracity
  11. 11. DEFINING BIG SPATIAL DATA BIG SPATIAL DATA Finding actionable info in Massive volumes of both structured and unstructured geo data that is so large and complex that it’s difficult to process with traditional database and software techniques…… Volume Velocity VARIETY VERACITY Data at rest Data in Motion Data in Many forms Data in Doubt 11
  12. 12. 90% of data in the world was created in the last 2 years 2.5 EB of data is created every day U.S. drone aircraft sent back 24 years worth of video footage in 2009 Gigabyte (GB) - 1,024MB Terabyte (TB) - 1,024GB Petabyte (PB) - 1,024TB Exabyte (EB) - 1,024PB
  13. 13. * Estimated revenue FY 2013 growth of geospatial data is outpacing both software and services and is set to become a major contributor to the overall growth of the industry 13
  14. 14. 100% security is a myth No one has said this!!! But it remains a fact 14 Increasing attack surface
  15. 15. The technology is ready…. But are we ready ? 15
  16. 16. 16 16 DISASTER RELIEF FINANCIAL FRAUD DETECTION CALL CENTER REQUESTS DISEASE SURVEILLANCE INSURANCE RETAIL TELECOMMUNICATIONS UTILITIES ECO-ROUTING
  17. 17. The other of the side story 17
  18. 18. Security challenges before we adopt Big spatial data 18
  19. 19. Distributed programming frameworks Ek 19
  20. 20. Distributed programming frameworks Input file Map Intermediate Combining Shuffle Output File Local Reduce Reduce Mapper performs computation & outputs a key/value pairs 20 Reducer combines the values belonging to each distict key and outputs the result Utilise parallilism in computation & storage to process massive amounts of data
  21. 21. MAP REDUCE FRAMEWORK  Splits the input data-set into independent chunks which are processed  in a completely parallel manner  Aggregate results from map phase  performs a summary operation  Schedules and re-runs tasks  Splits the input  Moves map outputs to reduce inputs  Receive the results Distributed programming frameworks 21
  22. 22. So challenge is not storage but it is I/O speed One Machine 4 i/o Channels Each channel : 100 MB/s 10 Machine’s 4 i/o Channels Each channel : 100 MB/s Read 1 TB 45 Min 4.5 Min
  23. 23. Untrusted Mappers Securing the data in the presence of an untrusted mapper Distributed programming frameworks 23
  24. 24. NO SQL ISSUES TWO 24
  25. 25. 25 First off : the name NoSQL is not “NEVER SQL” NoSQL is not “No To SQL “
  26. 26. 26 NoSQL Is simply Not Only SQL!!!!!
  27. 27. MongoDB Redis 27 NoSQL DB are still evolving with respect to security infrastructure
  28. 28. Data storage & transaction logs 28
  29. 29. STORAGE TIERS - Multi-tiered storage media - Necessitated by scalable size - Different categories of data - Different types of storage Data storage & transaction logs 29
  30. 30. Lower tier means reduced security, loose access controls Keeping track of data location Data storage & transaction logs 30
  31. 31. INPUT VALIDATION/FILTERING 31
  32. 32. How can we trust data ? Validating data when source of input data is not reliable? Filtering malicious data @ BYOD Input validation/filtering 32
  33. 33. REAL TIME MONITORING 33
  34. 34. Humongous number of alerts!!!! False positives Filtering malicious data @ BYOD REAL TIME MONITORING 34
  35. 35. Secure communication 35
  36. 36. End to end security ? Data encryption : attribute based encryption!!!to be made richer Secure communication 36
  37. 37. Granular audits 37
  38. 38. New attacks will keep happening…and to find out we need detailed audit logs Missed true positives Granular audits 38
  39. 39. PRIVACY ISSUES 39
  40. 40. EG : How a retailer was able to identify that a teenager was pregnant before her father knew 40 PRIVACY ISSUES In the world of big data,privacy invasion is a business model
  41. 41. And... We Also Have cloud with us? 41
  42. 42. At 1.4% in 2011-12 Cloud was a very small percentage of the total IT spend 42
  43. 43. Pace of Big Spatial Data adoption has been Sluggish 43
  44. 44. 44 There is unlikely to be a day soon in near future when we have a “FIND TERRORIST” BUTTON
  45. 45. 45 We have mostly been reactive till date…..
  46. 46. 46 USE KERBEROS FOR NODE AUTHENTICATION – (BUT WE KNOW IT’S A PAIN TO SET UP) STRINGENT POLICIES STANDARD TO INTRA COUNTRY LAWS EXHAUSTIVE LOGS SECURE COMMUNICATION STRINGENT POLICIES
  47. 47. 47
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×