Big data security
•Download as PPTX, PDF•
1 like•544 views
Use Big Data to make you more resilient to Cyber Attacks
Recommended
Recommended
More Related Content
What's hot
What's hot (19)
Similar to Big data security
Similar to Big data security (20)
Recently uploaded
Recently uploaded (20)
Big data security
- 1. Secure your Enterprise with Big Data © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
- 2. Today’s agenda Why mix big data and security? Catching Hackers Building the Anti-Fragile Enterprise © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
- 3. AntiFragile © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
- 4. Vision: The Anti-Fragile Enterprise © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Andy Bell Nassim Taleb
- 5. Innovation -> More Data -> New Challenges eBay Dragon Diction NetReach Google SmugMug Pandora Scribd. AppFog buzzd NetDocuments LimeLight DCC Quality Control Lifestyle 23,148 apps downloaded 400,710 ad requests Burroughs Hitachi Yahoo! SCM CyberShift ERP HCM Bull Fijitsu Time and Expense Costing Fixed Assets Games Navigation Social Networking DocuSign Hootsuite Workbrain Workday Khan Academy SuccessFactors Mozy HP ePrint iSchedule © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. New technology & Business Models Too much data Too many security solutions No integrated solution Client/server The Internet Mobile, social, big data & the cloud CRM SCM HCM MRM Amazon Web Services IBM GoGrid Joyent Hosting.com Tata Communications Datapipe Alterian Hyland OpenText HP EMC Qvidian Sage salesforce.com Xactly Zoho Ariba CCC Cost Management Order Entry Product Configurator Bills of Material Engineering Inventory Manufacturing Projects Education Music Reference Sport Travel Every 60 seconds 2000 lyrics played on Tunewiki 1500 pings sent on PingMe 34,597 people using Zinio 208,333minutes of Angry Birds played Unisys NEC Taleo Workscape Cornerstone onDemand OpSource PPM PaperHost Xerox Microsoft SLI Systems IntraLinks SugarCRM Volusion Adobe Avid Corel Microsoft Serif Yahoo CyberShift Saba Softscape Sonar6 Quadrem Elemica Kinaxis ADP VirtualEdge Kenexa Saba Softscape Sonar6 Exact Online FinancialForce.com Intacct NetSuite SAP NetSuite Plex Systems Database PLM Claim Processing Cash Management Accounts Receivable Billing Activity Management Payroll Training Time & Attendance Rostering Sales tracking & Marketing Commissions Service Data Warehousing 98,000 tweets Finance box.net Facebook LinkedIn Pinterest Atlassian iHandy Amazon PingMe Snapfish Urban Bromium Splunk kaggle Parse ScaleXtreme SolidFire Quickbooks Foursquare SuperCam UPS Mobile Scanner Pro Rackspace Jive Software Paint.NET Business Entertainment News Photo & Video Productivity Utilities TripIt Zynga Zynga Baidu Twitter Twitter Yammer Atlassian SmugMug MobilieIron Atlassian Amazon PingMe Associatedcontent Flickr YouTube Answers.com Tumblr. MobileFrame.com Mixi CYworld Qzone Renren Xing Yandex Yandex Heroku RightScale New Relic CloudSigma cloudability nebula Zillabyte dotCloud BeyondCore Viber Fring Toggl MailChimp Fed Ex Mobile BrainPOP myHomework Cookie Doodle Ah! Fasion Girl Mainframe
- 6. Disruptive technology trends DEVICES,DATA & INFRASTRUCTURE CLOUD Public, Private, Adoption MOBILE Data Everywhere © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. CONSUMERIZATION Mobility, Device & Social Media COLLABORATIVE OPEN & EXTENDED FORTRESS Reactive Perimeter Security
- 7. Detecting Data Exfiltration Acquire target, sneak in, hop around (Perimeter doesn’t stop attacks) Get privileged access to critical assets (Impact takes time) Collect data (Early detection matters) Exfiltrate data (Final stage before data loss) Detect Data Loss © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
- 8. Hackers Leave Bread Crumbs too! © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
- 9. Obstacles of a Successful Information Security Program So Much Data (How do I store, process, collect?) What should I be looking at? How do I prioritize the events? How do I become situationallyaware? What does all this data mean? © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
- 10. Move from data to security intelligence Data Collection Integration Analytics Monitor & Respond A methodical, iterative approach © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. See Understand Act
- 11. Correlation: Making sense of your Data Where From Millions of Events to the One that Matters Who © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. What When Correlation Engine How
- 12. Correlation taken to Extremes Collect Enrich * * * *** Rules © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Consolidate Correlated Filters Priority formula Interactive Events Discovery Prioritize Pattern Discovery Raw Events Data monitors
- 13. Detecting Hackers through Abnormal Behavior © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
- 14. Visibility & Situational Awareness Privileged User Role Asset Anomaly Session Transactions IP Address © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Location History Action
- 15. Transforming Big Data into Intelligence Collect Collect anything from anywhere Enrich Normalize and categorize large volumes of machine data with metadata Consolidate Unify Machine Data for compression for big data Search Apply Rules and Indexes for Easy Prioritization Correlate Real-time, user-centric, and cross-device correlation of all events Detect Act on suspicious activities ? © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
- 16. Plan Ahead Identify Attack Deploy Defenses Detect Data Loss Re-evaluate Anti-Fragile Plan For Information Security Develop a response plan with your information security incident response team Make contacts within your upsteam ISP, they may offer DDoS mitigation services Know your network topology and the location of critical or exposed systems Have firewall and IPS deployed between zones of different trust levels Identifying the attack method will help in mounting the most appropriate defense Identification of tool used may prove more valuable that identifying the source Leverage multiple methods of detection throughout environment taking a defense in depth approach Multiple methods of defense can offer a gauntlet to an attacker and increase the ability to stop their actions Many defenses require infrastructure in place ahead of time Defenses must be agile , able to change in response to shifts in attacker techniques © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. A defensive strategy must be part of your overall plan Direct and indirect monitoring should be used. Your environment is constantly changing because of business drivers Your environment is constantly changing because of new vulnerabilities The attack landscape is constantly changing
- 17. Keys to Security Intelligence WHO (User Roles) WHAT (Logs) WHERE (Flows) Better visibility Superior threat detection Sophisticated correlation technologies and big data processing Pattern recognition: anomaly detection to identify modern known and unknown threats Adaptive: The more you collect, the smarter it gets © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
- 18. Strive to Become AntiFragile © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
- 19. Secure your Enterprise with Big Data simple, intelligent, efficient & manageable katherine.lam@hp.com @kathlam © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Editor's Notes
- We are going to take a look at why Information Security is so difficult, and go over some general IT trends. The goal of today’s presentation will be answering the question of how we can use big data to help your enterprise become more resilient. Big data can be used to understand what hackers are after in your organization, as well as to sniff out these actors. Finally, the crux of the presentation will then focus on how you can take this knowledge to help your organization take a more proactive stance In your battle against cyber crime.
- We will pause here for a second to define ‘anti-fragile’. Imagine you are about to ship some packages containing something breakable like glass. The clerk puts a red fragile sticker on, so that their carriers know not to handle your box in a rough manner. But, we’ve all seen the videos of the delivery guy tossing the box containing a monitor over a gate, or doing other some horrendous things like stealing your package. Now what if, instead of breaking, the more you shake and drop the box, the stronger the contents become. And the more you attempt to steal the information, the harder it is for the information to be stolen? In a way, it’s much like Kelly Clarkson’s song “What doesn’t kill you makes you stronger”. And if you think about that for a second humans are some of the most anti-fragile beings. What we want to do now, is to create a security solution for an enterprise that is antifragile and adaptive.
- Some examples of antifragile include Silicon Valley, since not only did the area survive the crash, we came back rearing. Imaging your company actually adapting and responding to attacks by not getting bogged down, but actually learning from the attacks and using that knowledge to gain strength. Much like Magneto in X-men, who absorbs the radiation and harnesses it. Your organization can use the information gained by attacks happening to you today to make your security systems more robust. Just to reiterate, an egg is fragile. You don’t want to be the egg. You want to be more like Hydra or Magneto, where the more they try to cut your head off, you just sprout another 2 in its place. Fragile: Egg Robust: Bouncy Ball AntiFragile: Silicon Valley
- Now back to our program…What is the current state of big data right now? For most of us, we are huge adopters of the information generation. We are No longer tied to large mainframe computers, and we can access information via applications on mobile devices, and laptops to make decisions in real-time data. It is because information is so pervasive that businesses want to capture this data and use it as a competitive differentiator. One example of this is Walmart being able to predict how many boxes of cereal will be purchased on a given day of the week, and to make sure that their shelves contain enough stock . For enterprises, the rise of social media, mobile computing means that data is being generated And while the human information gives an opportunity for large companies to establish a more personalized relationship with their customers, it also means this data has to be processed. There are now more than 1 Billion users of smartphones, and the resultant traffic combine to create a proliferation of data that is being created with incredible volume, velocity, and variety. As a result, organizations need a way to protect, utilize, and gain real-time insight from “big data”. This intelligence is not only valuable to businesses and consumers, but also to hackers. Robust information marketplaces have arisen for hackers to sell credit card information, account usernames, passwords, national secrets (wikileaks), as well as intellectual property. This data is both Structured data and unstructured, And as the volume of the data increases, it becomes more difficult to sift through this data and extract intelligence from it. If you have a system, and you have started to organize this data, it still tends to be organized within silos. We need to look at the bigger picture of who, what, where and when events are occurring on the network, and what are the implications.
- In parallel we are also seeing some significant and disruptive trends emerge. In most enterprise, there’s an increased drive to adopt new technologies, around devices and data. These trends also mean that the traditional corporate perimeter, with clearly identifiable boundaries, has diminished. In the past when the network was straightforward, controlling access to data was much simpler. If your secrets rested within the company network, all you had to do to keep the data safe was to make sure you had a strong firewall in place. Now, as data is being made available through the Internet, mobile devices, and the cloud having a firewall is simply not enough. By making the perimeter more permeable, The notion of inside and outside also changes when parts of the business are outsourced, where trusted outsiders access internal systems. Another change is simply a change in how and where the data is used. More traffic comes from more users accessing data from different devices both inside the data center, and in the cloud. This increased volume makes it more difficult to keep track of who is doing what. But first, let’s take a look at the anatomy of a hack so that we have an understanding of what we are looking for..
- Typically, data theft is done in five stages. First, hackers “research” their target in order to find a way to enter the network. They may run a port scan to see what systems the company is running, and based on that knowledge they will try various methods. The simple analogy to this, might be turning a few door knobs to see if any of them will open. Once a weak link is found, either by sending malware to employees to capture their system passwords, the hacker next “infiltrates” the network. They may install an agent to lie dormant and gather information until they “discover” where the payload are hosted, and how to “acquire” it. Because hackers are typically no longer the run of the mill script kiddies, but groups of people who have an agenda, they are usually very patient. They may be hired to steal your companys IP, to observe email interactions to learn more about any upcoming M&A activity, or they may simply want to steal information about your customer’s profile and credit card information. Once they have secured their target, the next step is to “exfiltrate” the information out of the network. Data ex-filtration can occur not only from outside hackers but also disgruntled or malicious insiders. And since they already have access to your systems through privileged and default accounts, they may be harder to catch. Most advanced attacks progress through these five stages, and having this understanding gives you clues on whether an attack is taking place in your environment, and how to stop the attacker from reaching their target. The key to determining what logs to collect are to focus on records where an actor is accessing information or systems. Most importantly these logs must be secure themselves, since a smart hacker will not only take your data out, but eliminate traces that they were on your system.
- So, how do we catch these criminals? Well believe it or not, more than likely they are already on your system. So, their access have either manifested itself in a configuration log change, or a modification to a users profile so that they have higher administrative access. Remember our goal is to become a resilient organization. You may want to do what many security research firms or universities have done, and place a few honeypots. This gives you the ability to not only see where you have failures in the system, but to observe patterns of behavior from your attackers. While cyber security is unlike physical security, where you won’t get a fat winney the pooh who is stuck in the wall, or a bear that is caught in a trap, you can find more information about your attacker, where they may be coming from (source ips), what type of data they are after. This may be useful in case you seek to prosecute these offenders, having this information, and storing it for the authorities is key to dealing with a data loss. More importantly, having information about these patterns make your security posture much more adaptive. Even if you don’ t have time to create and place honeypots, By observing your attackers, you can isolate the critical assets and use policies and control systems to protect them from unwary eyes. If it is a database, just observing that someone is running a select all in order to get the contents of this data will give you a huge clue that you are being hacked. So, in a way hunting from hackers is a lot like catching mice. You set traps, and observe carefully what systems have been touched, and what data it is they are after. In order to do so, you need to start by creating an information security program. This gives you the tools you need to be able to understand what is happening in your environment.
- So, Let’s go over some reasons on why it is difficult to create a successful information security program. First of all, it is difficult to manage all of these disparate devices. If you are like more organizations, you probably have many security systems in place. IDS/IPS, Firewalls, but since each event logs gathered by each device they are stored , in a non consistent manner. And you can’t simply go to one system to query everything. Also, if you do manage to consolidate it into one system, how do manage all that information , and determine which information is important. finally, how do you use this information to become more situationally aware. Being situationally aware could be getting a good lay of the land in army battle ground speak. But, personally for me it may just be as simple as no longer being blind. When I was first learning to swim, it was a lot easier, when I put on my goggles, and could actually see. Having a good information security program is like having a good maps of your surroundings and knowing where your troops are in relation to where your enemies are located.
- Your first job is to aggregate all the information from every relevant device into one place. This means collecting information from cloud, virtual, and real appliances: network devices, applications, servers, databases, desktops, and security devices. With higher utilization of saas applications, it is important to collect logs from usage of those applications since they may contain HR information, or Customer information, such as in the case of salesforce. Collecting this information gives you complete visibility into your company’s data, who is accessing the information, and when and how often this access is occurring. The goal is to capture usage patterns and look for signs of malicious behavior. Data integration Once the data is collected, the data needs to be parsed, so that you can derive intelligence from cryptic log messages. Automation and rule-based processing is needed because having you simply can’t have a person review logs manually. It would make the problem of finding an attacker more difficult than finding a needle in a haystack, since there you can simply use a strong magnet. The solution is to normalize machine logs so that queries can pull context-aware information from log data. For example, HP ArcSight’s solution normalizes and categorizes log data into over 400 meta fields. Logs that have been normalized become much more powerful because you no longer need an expert on a particular device to interpret the log. The metadata gives you so much more information because you can then query failed login attempts from different systems and you won’t need to use ‘system specific syntax’ to understand your logs. Understanding who did what, can be a matter of a simple text search. Next, you want to analyze your data.
- The goal of a security analytics engine is to combine logs into a single pane of glass, so that you can answer the who, what, when and where for an organization. It collects log activity from ANY and ALL data sources, consolidates the information for maximum storage efficiency, and more relevant event searches, and correlates the events using multiple dimensions including, identity, vulnerability, asset, the time it occurred, statistical calculations, pattern, and other events to detect the advanced threats that organizations face.
- Data analytics Normalized logs are indexed and categorized to make it easy for a correlation engine to process and identify patterns based on heuristics and security rules. It is here where the art of combining logs from multiple sources and correlating events together help to create real-time alerts. This pre-processing also speeds up correlation but also makes event logs vendor-agnostic which gives analysts the ability to build reports and filters with simple English queries. In order to handle big data, you also need a system that will aggregate the information, compressing it so, that you can significantly increase the data that can be monitored and stored on a single appliance. As attacks gestate over longer periods of time, this functionality is critical to helping you uncover attacks. Finally, using advanced correlation techniques, you can use the data to detect modern cybercrime. Our analytics engine include a patter discovery process, which is used to pattern match and detect anomalous activities. This can be used to find very subtle and sophisticated threats including zero-day outbreaks and fraud. It includes correlating user roles and trends to determine who is violating policies and putting the business at risk. By correlating the WHO-WHAT-WHERE, using data gathered in your logs and flows, you can start to understand not only what’s happening but if it’s really a problem. And the best part is that the more info you collect and store, the smarter the system gets The net result is that you can detect and therefore prevent not only the basic stuff, but especially the attacks that you can’t predict.
- These tools can be used to find repeating event patterns, because it takes detail event information that can help analysts separate benign patterns from malicious ones. The activity profiling engine can churn through events to find relationships that aren’t readily apparent to the human eye. And then create visualizations of these patterns to draw attention, so that analysts can focus on the patterns that stand out. This machine learning can discover subtle relationships and risks across variables. By feeding this intelligence back into your SIEM as new process rules, you immediately add a new dimension to your security monitoring program. As an example, a low-and-slow attack takes place when an attacker purposely lowers the threshold on their attack to avoid detection. Such an evasive technique might be when the attacker is using a dictionary attack to guess a user’s password. They would not try to brute-force the authentication system all at once, as most systems would lock out the user’s account after a series of unsuccessful login attempts. So the attacker who uses a scripted stealth method of only attempting to login twice while trying to guess the password, then sleeps for five minutes and continues to invoke two attempts every five minutes. This means there would be 576 unsuccessful login attempts daily, but since most correlation rules look for brute-force methods, only a routine that would mine through historical data would be able to match this pattern. Pattern discovery would detect this attack and then allow customers to introduce new rules that would block the attacker going forward. The key in making use of BIG DATA is The ability to uncover hidden patterns across various parameters such as: Pairs of source and destination IP addresses Ports Event behavior Event outcomes
- A great correlation engine delivers accurate and automated prioritization of security risks and compliance violations in a business relevant context. Real-time alerts show administrators the most critical security events occurring in the environment, along with the context necessary to further analyze and mitigate a breach. With built-in network asset and user models, your Security Intelligence system can help you understand who is on the network, what data they are seeing, and which actions they are taking with that data. Using this multidimensional correlation engine combines real time, in memory event log data with asset awareness, asset vulnerability, and identity correlation to assist operating teams with immediate detection of threats. The powerful correlation engine allows you to maintain a state of situational awareness by processing millions of log events in real time. We help to prioritize critical events so that your security administrator can review only those events that need specialized attention. Going back to the kill-chain that we saw earlier, having an advanced pattern recognition system, means that when you uncover one activity that is suspicious, you can tie the whole chain of events to see how an attacker gained access, how long they have been sitting on your systems, and what data they are after.
- So, here’s a quick summary: What problems must we solve in order to gain control and use this valuable data? We must collect information from all relevant sources We need a way to quickly parse the data, and in order to do so,…enriching the meta-data makes event information easily searchable and actionable. Next, you need to be able to consolidate a large data set, so that your analytics can take more information into context. You also need an advanced correlation engine, so that you can deliver accurate and automated prioritization of security risks in a business relevant context. Real-time alerts show administrators the most critical security events occurring in the environment, along with the context necessary to further analyze and mitigate a breach. Using CORR-Engine, administrators and analysts are able to: In one example, one of our communications customers was able to reduce the 40 million security events we receive each day down to just 45 critical events Imagine the cost savings, and the ability to wrap your hands around a problem, when you can isolate the events that matter as opposed to anything that triggers a suspicion.
- So, what’s the next step? You can take the insights from your Security Intelligence solution to give you a 5 step iterative plan that will help you become more resilient. Using this data, you can determine what are you critical assets and where they are located, so you can develop a response plan. Different types of attacks mean that you need different mitigation strategies that focus on limiting the affects of an attack. Identifying the attack method using your system logs will help in mounting the most appropriate defense. Knowing which hacking tool is being used in an attack may prove more valuable than identifying the source of the attack itself. With this knowledge, you can rapidly mount an most effective defense. Rarely does a situation arise where more information is less helpful when performing this type of analysis. Defenses could include a next Generation Intrusion Prevention System (NGIPS) acts as an enforcement point, inspecting traffic in real time, identifying “known bad” traffic. You can enforce policies on traffic to or from hosts based on their reputation score, which can be provided via a Reputation service subscription. Combing reputation with a NextGeneration IPS, you can tracks millions of IPv4 and IPv6 IP addresses and DNS names to see if attacks are coming from these IP addresses, or data is leaving your organization for one of its destinations. To detect whether information was leaked after a breach has occurred, various tactics can be employed. The primary tactics are to detect the content of the sensitive data traversing the network or connections to known bad destinations. Another method that has been effective to detect data exfiltration has been the salting of sensitive data stores. Within a given database, tables or rows can be added with salted data. This data would not be the result of normal queries in daily activities and would be apparent with our security intelligence pattern discovery. The analogy of the bear trap. Finally, once you have deployed such a system, create a red team, and have them attack your systems, and see how your defenses stood up. By focusing protecting your key assets, and adapting your infrastructure, you can become more immune to security attacks.
- Analyze Intelligent Threat & Risk Detection Any log from any system, understand them in a business and security aware context Be able to mine attack patterns from your log data, and using that information to reinforce your defense mechanisms By combining knowledge of the WHO, WHAT and WHERE, you can use big data to protect your enterprise.
- In today’s business environment, having access to the right information means making the right decision critical to surviving. It’s time to use this data for more that business analytics, and use it to the advantage of security professionals. You already have the data Now Act on It