Big data and security intelligence are the two hot security topics in 2012. We are collecting more and more information from both the infrastructure, but increasingly also directly from our applications. Some companies are moving away from traditional log management and SIEM tools and are deploying big data products. But what is this big data craze all about? Why is it that we have more and more data to look at? And is big data the right approach or what is missing? The presentation takes the audience on a journey through big data tools and show that analytical tools are needed to make use of these infrastructures. How can visualization be used to fill in the gap in analytics to move into gaining situational awareness and building up security intelligence.

1. Visual Analytics and
Security Intelligence
Big Data in Action
Nordic Security Conference - August 2012
Raffael Marty
pixlcloud | creating big data stories copyright (c) 2012

2. Doushuai's Three Barriers
‘You make your way through the darkness of abandoned grasses in a
search for meaning. As you do, where is the meaning?'
47th case of'The Gateless Barrier'
a collection of Zen koans

3. Raffael Marty
13 years in the log analysis and information visualization space
• Founder and CEO @ pixlcloud
• Founder and COO @ Loggly
• Chief Security Strategist and Product Manager @ Splunk
• Manager Solutions @ ArcSight
• Intrusion Detection Research @ IBM Research
• IT Security Consultant @ PriceWaterhouse Coopers
pixlcloud | turning data into actionable insights copyright © 2012

4. Security Intelligence
• Where We Wanna Be
• SIEM, log management
• Changing IT
• Did SIEM keep pace?
• What’s still missing?
• Security Intelligence and Big Data
pixlcloud | creating actionable data insights copyright (c) 2012

5. Oblong Industries
pixlcloud | creating big data stories copyright (c) 2012

6. http://www.agi.com
pixlcloud | creating big data stories copyright (c) 2012

7. Text
http://www.agi.com
pixlcloud | creating big data stories copyright (c) 2012

8. How do we map this
to cyber space?
pixlcloud | creating big data stories copyright (c) 2012

9. Security Intelligence Goals
‣ situational awareness
‣ uncover new / previously unknown attacks
‣monitor behavior
‣catch issues before everyone else and before signatures are available
‣ prioritized list of issues / attacks
‣ understand the data that is collected
‣ forensic support (having all the data)
‣ multi sensor fusion (possibly contradicting input)
pixlcloud | turning data into actionable insights copyright © 2012

10. Let’s Take Inventory
pixlcloud | creating big data stories copyright (c) 2012

11. Log Management and SIEM
log management
???
SIEM
pixlcloud | creating actionable data insights copyright (c) 2012

12. What’s Working
‣Log management
‣collecting large amount of logs for forensics
‣mandatory data retention
‣Security Information and Event Management
‣Solving specific, known use-cases for sets of known data sources,
e.g.,
‣ monitoring privileged access to financial servers
‣ generating compliance reports
pixlcloud | turning data into actionable insights copyright © 2012

13. What’s Not Working
‣ We use the wrong sources to answer our questions
‣ We don’t understand the data
‣ We don’t have enough context to understand the data
‣ Parsing and normalization is broken
‣ No working way of prioritizing data
‣ SIEMs don’t scale to data volumes
‣ No good way to deal with app-layer data
pixlcloud | turning data into actionable insights copyright © 2012

14. How Are We Tracking?
‣ situational awareness
‣ uncover new / previously unknown attacks
‣ prioritized list of issues / attacks
‣ understand the data that is shown
‣ forensic support (having all the data)
‣ multi sensor fusion (possibly contradicting input)
pixlcloud | turning data into actionable insights copyright © 2012

15. A New IT Landscape...

16. IT Has Been Changing
“memory has become the new hard
disk, hard disks are the tapes of
years ago” -- unknown source
pixlcloud | creating actionable data insights copyright (c) 2012

17. IT Has Been Changing
• Cloud
- on-demand compute resources
- on-demand, limitless storage
- on-demand ‘applications’ (MR, DB, ...)
• New, free search engines
• New data stores and paradigms
• New processing capabilities
pixlcloud | creating actionable data insights copyright (c) 2012

18. IT has changed
security ...

19. Collect it ALL!
‣ Storage has become cheap - we can afford to record more
for longer
‣ IT / development has started collecting application data
‣ Compliance has forced us to collect and keep more data
‣ Security can become a profit center!
‣leverage collected data for ‣fraud detection
‣insights into marketing
‣support product analytics, etc.
pixlcloud | turning data into actionable insights copyright © 2012

20. SIEMs Are Taking Note
• Start to utilize new paradigms (dynamic schema, better scale)
• More in the cloud - hands-off
• Tracking objects (users, machines) --> building models
pixlcloud | creating actionable data insights copyright (c) 2012

21. Has Big Data Helped?
‣ situational awareness
‣ uncover new / previously unknown attacks
‣ prioritized list of issues / attacks
‣ understand the data that is shown
‣ forensic support (having all the data)
‣ multi sensor fusion (possibly contradicting input)
pixlcloud | turning data into actionable insights copyright © 2012

22. What Now?