Solving Compliance for Big Data


Published on

EQALIS describe how Splunk can help solve your Compliance problems with Big Data. Presentation given at Infosec 2012.

Published in: Technology
1 Comment
1 Like
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Transition to Slide:So how does this effect us in the Security and Compliance space?Animation 1: Looking at some of these examples, we can see that Security & Compliance is already a challengeAnimation 2: Here are some facts you not be be aware of;GCHQ – persitent attacks form China etc.Conclusion: Throw in Big Data and we have a tidal wave of data to cope with, so “Don’t get washed away!”Data breach cost UK firms£1.9mTrust & reputation
  • What do we mean by Big: Machine Data is pervasive. Think about all the data coming from phones, social media, web servers, firewalls, network devices etc.According to IBM, Every day we create 2.5 quintillion bytes of data — so much that 90% of the data in the world today has been created in the last two years alone. Data Volumes will only ever increaseSo What is this data? Most, in fact around 90% is Unstructured data making it difficult to index and then report on. (ref. IBM So if this data is difficult to Index using traditional DBs, then it follows it is difficult to search and report.
  • So you need a product that can cope with the 3 V’s:For Reference the following is directly from Gartner:Volume: The increase in data volumes within enterprise systems is caused by transaction volumes and other traditional data types, as well as by new types of data. Too much volume is a storage issue, but too much data is also a massive analysis issue.Variety: IT leaders have always had an issue translating large volumes of transactional information into decisions — now there are more types of information to analyze — mainly coming from social media and mobile (context-aware). Variety includes tabular data (databases), hierarchical data, documents, e-mail, metering data, video, still images, audio, stock ticker data, financial transactions and more.Velocity: This involves streams of data, structured record creation, and availability for access and delivery. Velocity means both how fast data is being produced and how fast the data must be processed to meet demand.
  • Universal Indexing - means you can teach Splunk how to interpret new data formats without expensive consultancy.Unstructured Data – is now accessible because Splunk understands and learns about new unstructured data.Unlock the potential – Log data is the exhaust of your devices and applications. It contains a wealth of information. We can add embellish your data with tags, eventtypes, alerts, reports, graphs, tables etc.Ultimately Scalable – due to use of Map Reduce, Bloom Filters etc.
  • Splunk is a data engine for your IT data. It gives you real-time visibility and intelligence into what’s happening across your IT infrastructure – whether it’s physical, virtual or in the cloud. Everybody now recognizes the value of this data, the problem up to now has been getting to it. Having no predefined schemas, means you can point Splunk at any of your data, regardless of format, source or location.There is no need to build custom parsers or connectors, there’s no traditional RDBMS, there’s no need to filter and forward.[Point to Slide] Here we see just a sample of the kinds of data Splunk can ‘eat’.Reminder – what’s the ‘big deal’ about machine-generated IT data? It holds a categorical record of the following:User transactionsCustomer behaviorMachine behaviorSecurity threatsFraudulent activityYou can imagine that a single user transaction can span many systems and sources of this data, or a single service relies on many underlying systems. Splunk gives you one place to search, report on, analyze and visualize all this data.
  • Both GPG13 & PCI DSS aremandatory.Both suffer from challenge of having many different technologies which in turn have different log formats which have different reportable fields. This makes it difficult to generate generic reports across these different log types.Protective Monitoring Controls (PMC) is all about monitoring the security tools we have in place. Example: It’s no good installed an IDS and never checking to see if it has the current security rule set.PCI focuses on Daily Log Review. Someone actually needs to be looking at the millions of events generated everyday and checking to see if there are any issues.
  • GPG13 – Maps to the 12 PMCs of GPG13.Provides canned reports and alerts for the common data security log formats, plus the ability to configure customer-specific log formats, reports and alertsPCI DSS App – Maps to the 12 requirements of PCI. Focuses heavily on Section 10, “Track and monitor all access to network resources and cardholder data”
  • SIEM – Good at raising a red flag when they see a problem but what do you do then? Splunk allows you to drill down forensically.Appliances – Don’t scale well. Splunk allows you to choose any commodity hardwareTurn Key – there is no such thing as a turn key solution since how do we know which Operating Systems you are using or which make of Firewall, There’s no point having lots of Windows reports if you are a unix-only house right.
  • Now that I’ve fixed Compliance, what else can Splunk do for me?
  • Splunkbase is the home for our Splunk Apps. There, you'll find cool and useful downloads to extend Splunk. You can share what you make, from simple add-ons with a useful search, script, or report to full-fledged apps with multiple views. You’ll also find Apps from Splunk and our partners.Apps are being created all the time, so bookmark the site and check in frequently.Examples on this page include Apps for Cisco, F5, Twitter sentiment, external ‘WHOIS’ lookups, license usage, and more.
  • EQALIS offer Real-Time Operational IntelligenceWhat do we mean, Using tools like Splunk and AppDynamics EQALIS is able to
  • Solving Compliance for Big Data

    1. 1. Solving Compliance for BIG DATAAndrew Walley – Sales DirectorIan Tinney – Technical DirectorInfoSecurity Europe 24-26 April 2012
    2. 2. SECURITY & COMPLIANCE with BIG DATA Cyber attacks increasing Regulations tighten Skills shortage - By 2018, 190,000 too few people with analytical skills [McKinsey] Tools, like SIEMs, don’t scale; inflexible; expensive
    3. 3. What is BIG DATA? BIG… …DATA Daily log volumes >=petabytes  90% is Unstructured data 90% of data created in last 2 years  Beyond capabilities of traditional Database technologies Data is pervasive  email, pda, web-access, financial  Difficult to Search and report transactions, systems access, network devices.
    4. 4. HOLDING BACK THE FLOOD OF BIG DATA Volume  Need to be able to cope with massive amounts of data Variety  Need to cope with unstructured data Velocity  Need to scale beyond today!
    5. 5. YOU WANT SPLUNK ON YOUR SIDE Universal Indexing  ability to add new, unstructured data sources SPLUNK Unstructured Data THINKS  now accessible, usable, valuable… LIKE A Unlocks the potential CRIMINA L  expose a hidden treasure chest of information Ultimately Scalable  horizontally scalable
    6. 6. Collects and Indexes ANY Machine DataCustomerFacing Data •Any amount, any location, any Outside the Datacenter Click-stream data Shopping cart data source No upfront schema Manufacturing, logistics… CDRs & IPDRs Online transaction data No custom connectors Power consumption RFID data Logfiles Configs Messages Traps Metrics Scripts Changes Tickets GPS data No RDBMS Alerts No need to filter/forwardWindows Linux/Unix Virtualization Applications Databases Networking Registry Configurations & Cloud Web logs Configurations Configurations Event logs syslog Hypervisor Log4J, JMS, JMX Audit/query syslog File system File system Guest OS, Apps .NET events logs SNMP sysinternals ps, iostat, top Cloud Code and scripts Tables netflow Schemas
    7. 7. COMPLIANCE – ChallengesGPG13 PCI – DSSGood Practice Guide 13 Payment Card Industry Data Security Standard Mandatory for anyone working  Mandatory – for company with the Government on the processing CC payments GCSX network Different technologies, logs and  Myriad log formats fields Need to monitor the monitoring  Daily Log Review is labour- tools intensive
    8. 8. WHAT DOES EQALIS PROVIDE?Eqalis GPG13 app: Eqalis PCI app:
    9. 9. Why Splunk? What’s wrong with my SIEM?  Good at raising a Red Flag but then what? Why not an appliance-based solution?  Can your appliance scale? Can you improve performance? Is it a Turn-key solution?  Do you all use the same OSs and make of Firewall? Can it do anything else?  Splunk can be used for many things…
    10. 10. BONUS FEATURES Splunk for Enterprise …So what else canExchange for me? Splunk forfor Splunk do Splunk for VMware Splunk Cisco Security SecurityCollects performance metrics, tasks, events, logs to provide completevisibility into virtual environments• Collects and persists data directly from VMware vCenter Server hosts (to avoid the VC bottleneck)• Integrates data with VC inventory information VMware vSphere• Collects and persists tasks & events from VC to maintain complete picture• Initial set of views/dashboards as a starting point
    11. 11. A Growing Family of Splunk AppsSecurity IronPort WSA
    12. 12. Real Time Operational  HQ in Bracknell, Berkshire Intelligence  Founded 2008Network Complianc Security Web BI Ops e  Largest Splunk VAR in UK  Premier Splunk Partner EMEA 2009  Splunk Partner of the Year 2011 Professional Services  10 Employees, UK and EMEA Splunk Authorised Training focus  120+ Customers
    13. 13. Thank youQUESTIONS? www.eqalis.comDEMONSTRATION? 0845 643 9180 EQALIS Ltd, Lily Hill Road, Bracknell, Berkshire, RG12 2SJ