IT Data Visualization - Sumit 2008


Published on

IT data visualization for
- Perimeter Threat
- Insider Threat

More on security visualization at

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

IT Data Visualization - Sumit 2008

  1. 1. IT Data VisualizationRaffael Marty, GCIA, CISSPChief Security Strategist @ Splunk>SUMIT, Michigan - October ‘08
  2. 2. Raffael Marty• Chief Security Strategist @ Splunk>• Looked at logs/IT data for over 10 years - IBM Research - Conference boards / committees• Presenting around the world on SecViz• Passion for Visualization Applied Security Visualization - Paperback: 552 pages Publisher: Addison Wesley (August, 2008) - ISBN: 0321510100
  3. 3. Agenda• IT Data Visualization - Security Visualization Dichotomy - Research Dichotomy Visualization is a more effective• IT Data Management way of IT data management and analysis. - A shifted crime landscape• Perimeter Threat• Insider Threat• Security Visualization Community 3
  4. 4. Visualization Questions• Who analyzes logs?• Who uses visualization for log analysis?• Who has used DAVIX?• Have you heard of• What tools are you using for log analysis? 4
  5. 5. IT Data Visualization Applied Security Visualization, Chapter 3
  6. 6. What is Visualization? Generate a picture from IT data A picture is worth a thousand log records.Explore and Inspire Discover Answer a Pose a New Increase Communicate Support Question Question Efficiency Information Decisions 6
  7. 7. Information Visualization Process Capture Process Visualize 7
  8. 8. The 1st DichotomySecurity Visualization• security data • types of data• networking protocols • perception two domains• routing protocols (the Internet) • optics• security impact • color theory Security & Visualization• security policy • depth cue theory• jargon • interaction theory• use-cases • types of graphs• are the end-users • human computer interaction 8
  9. 9. The Failure - New Graphs9
  10. 10. The Right Thing - Reuse Graphs10
  11. 11. The Failure - The Wrong Graph11
  12. 12. The Right Thing - Adequate Graphs12
  13. 13. The Failure - The Wrong Integration /usr/share/man/man5/launchd.plist.5 <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "">• Using proprietary data format <plist version="1.0"> <dict> <key>_name</key>• Provide parsers for various data formats <dict> <key>_isColumn</key> <string>YES</string> <key>_isOutlineColumn</key> • does not scale <string>YES</string> <key>_order</key> <string>0</string> </dict> • is probably buggy / incomplete <key>bsd_name</key> <dict> <key>_order</key> <string>62</string>• Use wrong data access paradigm </dict> <key>detachable_drive</key> <dict> • complex configuration <key>_order</key> <string>59</string> </dict> e.g., needs an SSH connection <key>device_manufacturer</key> <dict> <key>_order</key> <string>41</string> </dict> <key>device_model</key> <dict> <key>_order</key> <string>42</string> </dict> <key>device_revision</key> 13
  14. 14. The Right Thing - KISS /usr/share/man/man5/launchd.plist.5 <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "">• Keep It Simple Stupid <plist version="1.0"> <dict> <key>_name</key> <dict>• Use CSV input <key>_isColumn</key> <string>YES</string> <key>_isOutlineColumn</key> <string>YES</string>• Use files as input <key>_order</key> <string>0</string> </dict> <key>bsd_name</key> # Using node sizes:• Offload to other tools <dict> <key>_order</key> <string>62</string> size.source=1; </dict> • parsers <key>detachable_drive</key> <dict> <key>_order</key> <string>59</string> maxNodeSize=0.2 • data conversions </dict> <key>device_manufacturer</key> <dict> <key>_order</key> <string>41</string> </dict> <key>device_model</key> <dict> <key>_order</key> <string>42</string> </dict> <key>device_revision</key> 14
  15. 15. The Failure - Unnecessary Ink15
  16. 16. The Right Thing - Apply Good Visualization Practices• Dont use graphics to decorate a few numbers• Reduce data ink ratio• Visualization principles 16
  17. 17. The 2nd Dichotomy Some comments are based on paper reviews from RAID 2007/08, VizSec 2007/08Industry Academia• don’t understand the real impact • don’t know what’s been done in industry• get the 70% solution • don’t understand the use-cases two worlds• don’t think big • don’t understand the environments / data / domain• no time/money for real research Industry & Academia• can’t scale • • work on simulated data construct their own problems• work based off of a few • use overly complicated, impractical customer’s input solutions • use graphs / visualization where it is not needed 17
  18. 18. The Way Forward• Building a secviz discipline• Bridging the gap Security Visualization• Learning the “other” discipline• More academia / industry collaboration SecViz 18
  19. 19. My Focus Areas• Use-case oriented visualization• IT data management• Perimeter Threat• Governance Risk Compliance (GRC)• Insider Threat• IT data visualization• SecViz.Org• DAVIX 19
  20. 20. IT Data Management
  21. 21. A Shifted Crime Landscape• Crimes are moving up the stack• Insider crime Application Layer• Large-scale spread of many small attacks Transport Layer Questions are not known in advance! Network Layer• Are you prepared? Have the data when you need it! Link Layer• Are you monitoring enough? Physical Layer 21
  22. 22. What Is IT Data? /var/log/messags multi-line files Logs /opt/log/* /etc/syslog.conf entire filesConfigurations /etc/hosts multi-line structuresTraps & Alerts iso. org. dod. internet. mgmt. mib-2. host. hrDevice. hrProcessorTable. hrProcessorEntry. hrProcessorLoad ps multi-line table formatScripts & Code netstat File system changes hooks into the OSChange Events Windows Registry The IT Search Company
  23. 23. Perimeter Threat Applied Security Visualization, Chapter 6
  24. 24. Sparklines• "Data-intense, design-simple, word-sized graphics". Edward Tufte (2006). Beautiful Evidence. Graphics Press. Average } Standard Deviation• Examples: • Java Script Implementation: - stock price over a day - access to port 80 over the last week 24
  25. 25. Port Sparklines Source IP Destination IP25
  26. 26. Insider Threat Applied Security Visualization, Chapter 8
  27. 27. Three Types of Insider Threats Information Fraud Leak Sabotage27
  28. 28. Example - Insider Threat Visualization• More and other data sources than for • The questions are not known in advance! the traditional security use-cases • Visualization provokes questions and• Insiders often have legitimate access helps find answers to machines and data. You need to log • Dynamic nature of fraud more than the exceptions • Problem for static algorithms• Insider crimes are often executed on • Bandits quickly adapt to fixed threshold- the application layer. You need based detection systems transaction data and chatty • Looking for any unusual patterns application logs 28
  29. 29. User ActivityColor indicatesfailed logins High ratio of failed logins 29
  30. 30. 30
  31. 31. Security Visualization Community
  32. 32. SecViz - Security VisualizationThis is a place to share, discuss, challenge, and learn about security visualization.
  33. 33. V D XData Analysis and Visualization Linux
  34. 34. ToolsCapture Processing Visualization- Network tools - Shell tools - Network Traffic ‣ Argus ‣ awk, grep, sed ‣ EtherApe - Graphic preprocessing ‣ InetVis ‣ Snort ‣ tnv ‣ Wireshark ‣ Afterglow - Generic- Logging ‣ LGL ‣ Afterglow ‣ syslog-ng - Date enrichment ‣ Treemap- Fetching data ‣ geoiplookup ‣ Mondrian ‣ wget ‣ whois/gwhois ‣ R Project ‣ ftp ‣ scp * Non-concluding list of tools
  35. 35. Thank You! raffy @ splunk . com