The document describes the evolution of Incapsula's big data system over four generations from 2010 to 2015. Each generation improved on processing throughput, read performance, and scalability by simplifying the data model and moving to distributed processing across multiple points of presence. Key changes included moving from a centralized SQL database to NoSQL storage, implementing multi-threaded processing, and distributing workloads across data centers.
Presentation on topics beyond the conventional ethical hacking , discusses job factors and scope in the security field :) this was presented in LPU (Lovely Professional University) as a Seminar with attendees over 200. Meet m e at FB if u want it fb/nipun.jaswal
Hijacking Softwares for fun and profitNipun Jaswal
Presentation for my talk at Global Infosec Summit, LPU (11 Nov 2017). The Presentation demonstrates risk of using outdated and cracked software. Additionally, demonstrates the hand-on approach to finding DLL search order hijacking vulnerabilities. The Presentation is for educational purposes only.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
The presentation demonstrates basics of antivirus evasion on the payloads created using metasploit. The aim of this presentation is to aid penetration testers during a professional VAPT and is for educational purposes only.
Presentation on topics beyond the conventional ethical hacking , discusses job factors and scope in the security field :) this was presented in LPU (Lovely Professional University) as a Seminar with attendees over 200. Meet m e at FB if u want it fb/nipun.jaswal
Hijacking Softwares for fun and profitNipun Jaswal
Presentation for my talk at Global Infosec Summit, LPU (11 Nov 2017). The Presentation demonstrates risk of using outdated and cracked software. Additionally, demonstrates the hand-on approach to finding DLL search order hijacking vulnerabilities. The Presentation is for educational purposes only.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
The presentation demonstrates basics of antivirus evasion on the payloads created using metasploit. The aim of this presentation is to aid penetration testers during a professional VAPT and is for educational purposes only.
SANS @Night Talk: SQL Injection ExploitedMicah Hoffman
This presentation was given at the SANS Rpcky Mountain Conference in Denver, CO June 2014. The presentation had a rather large portion that was demo. That is not captured here. Sorry.
Pro Tips for Power Users – Palo Alto Networks Live Community and Fuel User Gr...PaloAltoNetworks
Palo Alto Networks Live Community Senior Engineers Tom and Joe present best security practices at the Fuel Spark event in London. For more details, please visit: https://live.paloaltonetworks.com/t5/Community-Blog/Live-Community-team-at-Spark-User-Summit-London/ba-p/153182
The presentation focus on some known and unknown methods of android pentetration testing. I have taken help from many resources which I have mentioned in PPT.
Abusing bleeding edge web standards for appsec gloryPriyanka Aash
"Through cooperation between browser vendors and standards bodies in the recent past, numerous standards have been created to enforce stronger client-side control for web applications. As web appsec practitioners continue to shift from mitigating vulnerabilities to implementing proactive controls, each new standard adds another layer of defense for attack patterns previously accepted as risks. With the most basic controls complete, attention is shifting toward mitigating more complex threats. As a result of the drive to control for these threats client-side, standards such as SubResource Integrity (SRI), Content Security Policy (CSP), and HTTP Public Key Pinning (HPKP) carry larger implementation risks than others such as HTTP Strict Transport Security (HSTS). Builders supporting legacy applications actively make trade-offs between implementing the latest standards versus accepting risks simply because of the increased risks newer web standards pose.
In this talk, we'll strictly explore the risks posed by SRI, CSP, and HPKP; demonstrate effective mitigation strategies and compromises which may make these standards more accessible to builders and defenders supporting legacy applications; as well as examine emergent properties of standards such as HPKP to cover previously unforeseen scenarios. As a bonus for the breakers, we'll explore and demonstrate exploitations of the emergent risks in these more volatile standards, to include multiple vulnerabilities uncovered quite literally during our research for this talk (which will hopefully be mitigated by d-day)."
(Source: Black Hat USA 2016, Las Vegas)
DevSecCon Tel Aviv 2018 - Serverless SecurityAvi Shulman
Serverless architectures enable organizations to build and deploy software and services without having to maintain or provision any physical or virtual servers. Applications built using serverless architectures are suitable for a wide range of services, and can scale elastically as cloud workloads grow. From a software development perspective, organisations adopting serverless can focus on core product functionality, and completely disregard the underlying operating system, application server or software runtime environment. In essence, when you develop applications using serverless, you relieve yourself from the daunting task of having to constantly apply security patches for the underlying operating system and application servers – these tasks are now the responsibility of the serverless architecture provider.
However, the comfort and elegance of serverless architectures is not without its drawbacks – serverless architectures introduce a new set of security concerns that must be taken into consideration when coming to secure such applications. In this talk, we will present an overview of serverless architectures, the challenge of securing serverless applications, and an overview of the top 10 most common security concerns that developers, DevSecOps and architects should consider when designing and developing such applications. We will also demonstrate a unique CI/CD tool for hardening serverless projects during deployment time.
Migrating from Akamai to Incapsula: What You Need to KnowImperva Incapsula
The webinar gives an overview of and compares the two platforms: Incapsula and Akamai. In addition to the benefits of migrating to Incapsula, it covers planning, transitioning, configuring Incapsula and lessons learned from the field.
Web Application Security And Getting Into Bug Bountieskunwaratul hax0r
This PPT is focused on how to begin into bug bounty programs, what approach you should follow and what are the major things you should look before begin.
Incapsula: How to Increase SaaS Websites’ Uptime and Accelerate PerformanceImperva Incapsula
All too often, online threats such as DDoS attacks, scrapers, or traffic that consumes too much bandwidth are disrupting or slowing down SaaS websites. It is now more important than ever to keep website traffic flowing quickly without service interruptions.
Tempus Technologies’ president, Jason Sweitzer, talks about the technological challenges his company faced and the solutions his team adopted to increase website acceleration and uptime.
Join us for Incapsula’s free 30-minute webinar to learn how you can increase your website’s uptime and enhance its performance. We’ll be discussing opportunities SaaS companies can explore through WAF protection, frontend SSL, failover ISPs, and against DDoS attacks and using Incapsula solutions.
Based on "The Mobile Application Hacker's Handbook 1st Edition", by Dominic Chell
Instructor: Sam Bowne
More info: https://samsclass.info/128/128_S19.shtml
OWASP ATL - Social Engineering Technical Controls PresentationOWASP Atlanta
Meetup July 16th, 2015
User awareness training will always fail to prevent 100% of social engineering attacks. However, consistent and reliable technical controls drastically mitigate an organization’s risk and increase the difficulty for malicious actors to launch successful attacks.
This talk describes social engineering from the perspectives of an attacker and a defender. The presentation will cover techniques designed to help organizations develop an ideal incident response plan crafted specifically for social engineering attacks. It will explain technical controls that are designed to inhibit attackers, as well as procedures that allow an incident response team to quickly identify successful attacks and eradicate their presence.
Bishop Fox conducted new research into the state of email spoofing defenses and identified organizations that are most commonly targeted for brand spoofing. This research will show that 99.9% of the top million domains are vulnerable to email spoofing and provide recommendations for avoiding attack.
This presentation covers attacks and defenses for dangerous social engineering activities, including:
· Email spoofing
· Domain hijacks
· Typo-squatting
· Client-side attacks
· Watering hole attacks
Fruit vs Zombies: Defeat Non-jailbroken iOS Malware by Claud XiaoShakacon
Since 2014, fifteen new malware or riskware families successfully attacked non-jailbroken iOS devices (e.g., WireLurker, Oneclickfraud, XcodeGhost, InstaAgent, ZergHelper, AceDeceiver), affected thousands of iOS apps and tens of millions users around the world. Ten of them even bypassed Apple’s code vetting and occurred at App Store. In this presentation, we will systematically study how could these malware, riskware and some Proof-of-Concepts infect non-jailbroken devices via practical vectors and approaches including abusing development certificates, bypassing code review by obfuscation, performing FairPlay MITM attack, abusing MDM solution, abusing private APIs, exploiting design flaws or app level vulnerabilities, and stealing privacy data. For each topic, we will introduce its implementation, explore real world cases, analyze its risky and consequences, explain Apple’s countermeasures, and discuss why some problems will still exist in near future. We will also share some stories of how we discovered those interesting iOS malware. Through this topic, audiences could make more effective policies to protect iOS devices in their organizations, build their own systems/tools to evaluate security risks in iOS apps, and hunt more iOS malware in the future.
DerbyCon 2016
Nick Landers @monoxgas
External mail via Exchange is one of the most common services offered by organizations today. The Microsoft Office suite is even more prevalent making Outlook the most common mail client around. This talk focuses on the abuse of these two products for the purpose of gaining code execution inside remote networks. Subjects include E-Mail and password scraping, OWA/EWS brute forcing techniques, and new research into abusing Outlook mail rules for remote code execution. Learn about the capabilities of client side rules, the underlying Windows APIs, and how to modify these rule objects to make phishing attacks obsolete. Security Consultant at Silent Break Security. Professional Hacker for 2 years. Current work involves writing custom malware and researching unique attack vectors that abuse functionality in windows environments.
DevOops Redux Ken Johnson Chris Gates - AppSec USA 2016Chris Gates
In a follow-up to the duo’s offensive focused talk “DevOops, How I hacked you”, they discuss defensive countermeasures and real experiences in preventing attacks that target flaws in your DevOps environments. In this talk, Chris and Ken describe common ways in which DevOps environments fall prey to malicious actors with a focus on preventative steps. The team will present their recommended approach to hardening for teams using AWS, Continuous Integration, GitHub, and common DevOps tools and processes. More specifically, the following items will be demonstrated:
-AWS Hardening
-AWS Monitoring
-AWS Disaster Recovery
-GitHub Monitoring
-OPINT
-Software Development Practices/Processes
-Secure use of Jenkins/Hudson
-Developer laptop hardening (OS X)
Is the Cloud Going to Kill Traditional Application Delivery?Imperva Incapsula
Application delivery controllers provide load balancing, acceleration, traffic shaping and other services that improve the performance, availability and security of web applications. But with more and more web application developers hosting their applications in the cloud, using application delivery hardware is often a non-starter.
This presentation discusses the architecture of a new type of service called the Application Delivery Cloud. This new cloud service not only offers critical performance, availability and security capabilities to web application vendors, it goes beyond its hardware analog to deliver new capabilities that today’s applications require, including regional content policies and up-to-the-minute security intelligence.
You’ve seen the headlines—"[Well-Known Company] Falls Victim To Hackers".
These data breaches result in the theft of millions of names, passwords, credit card numbers, and other personal data. Imagine if such a breach lead to the theft of your application's data. . .
If multi-national companies with dedicated security teams and expansive budgets aren’t immune to the impact of hackers, how can you adequately prepare yourself to defeat this threat?
This presentation will explore the web application threat landscape. It will zero in on some of the most common attacks wreaking havoc on the internet, teaching you how to defend your online assets from them.
This presentation will discuss:
• The major security breaches of 2014
• Web application threats and common attack types
• How to defend against today’s common attacks
• Automated tools to help simplify website security
SANS @Night Talk: SQL Injection ExploitedMicah Hoffman
This presentation was given at the SANS Rpcky Mountain Conference in Denver, CO June 2014. The presentation had a rather large portion that was demo. That is not captured here. Sorry.
Pro Tips for Power Users – Palo Alto Networks Live Community and Fuel User Gr...PaloAltoNetworks
Palo Alto Networks Live Community Senior Engineers Tom and Joe present best security practices at the Fuel Spark event in London. For more details, please visit: https://live.paloaltonetworks.com/t5/Community-Blog/Live-Community-team-at-Spark-User-Summit-London/ba-p/153182
The presentation focus on some known and unknown methods of android pentetration testing. I have taken help from many resources which I have mentioned in PPT.
Abusing bleeding edge web standards for appsec gloryPriyanka Aash
"Through cooperation between browser vendors and standards bodies in the recent past, numerous standards have been created to enforce stronger client-side control for web applications. As web appsec practitioners continue to shift from mitigating vulnerabilities to implementing proactive controls, each new standard adds another layer of defense for attack patterns previously accepted as risks. With the most basic controls complete, attention is shifting toward mitigating more complex threats. As a result of the drive to control for these threats client-side, standards such as SubResource Integrity (SRI), Content Security Policy (CSP), and HTTP Public Key Pinning (HPKP) carry larger implementation risks than others such as HTTP Strict Transport Security (HSTS). Builders supporting legacy applications actively make trade-offs between implementing the latest standards versus accepting risks simply because of the increased risks newer web standards pose.
In this talk, we'll strictly explore the risks posed by SRI, CSP, and HPKP; demonstrate effective mitigation strategies and compromises which may make these standards more accessible to builders and defenders supporting legacy applications; as well as examine emergent properties of standards such as HPKP to cover previously unforeseen scenarios. As a bonus for the breakers, we'll explore and demonstrate exploitations of the emergent risks in these more volatile standards, to include multiple vulnerabilities uncovered quite literally during our research for this talk (which will hopefully be mitigated by d-day)."
(Source: Black Hat USA 2016, Las Vegas)
DevSecCon Tel Aviv 2018 - Serverless SecurityAvi Shulman
Serverless architectures enable organizations to build and deploy software and services without having to maintain or provision any physical or virtual servers. Applications built using serverless architectures are suitable for a wide range of services, and can scale elastically as cloud workloads grow. From a software development perspective, organisations adopting serverless can focus on core product functionality, and completely disregard the underlying operating system, application server or software runtime environment. In essence, when you develop applications using serverless, you relieve yourself from the daunting task of having to constantly apply security patches for the underlying operating system and application servers – these tasks are now the responsibility of the serverless architecture provider.
However, the comfort and elegance of serverless architectures is not without its drawbacks – serverless architectures introduce a new set of security concerns that must be taken into consideration when coming to secure such applications. In this talk, we will present an overview of serverless architectures, the challenge of securing serverless applications, and an overview of the top 10 most common security concerns that developers, DevSecOps and architects should consider when designing and developing such applications. We will also demonstrate a unique CI/CD tool for hardening serverless projects during deployment time.
Migrating from Akamai to Incapsula: What You Need to KnowImperva Incapsula
The webinar gives an overview of and compares the two platforms: Incapsula and Akamai. In addition to the benefits of migrating to Incapsula, it covers planning, transitioning, configuring Incapsula and lessons learned from the field.
Web Application Security And Getting Into Bug Bountieskunwaratul hax0r
This PPT is focused on how to begin into bug bounty programs, what approach you should follow and what are the major things you should look before begin.
Incapsula: How to Increase SaaS Websites’ Uptime and Accelerate PerformanceImperva Incapsula
All too often, online threats such as DDoS attacks, scrapers, or traffic that consumes too much bandwidth are disrupting or slowing down SaaS websites. It is now more important than ever to keep website traffic flowing quickly without service interruptions.
Tempus Technologies’ president, Jason Sweitzer, talks about the technological challenges his company faced and the solutions his team adopted to increase website acceleration and uptime.
Join us for Incapsula’s free 30-minute webinar to learn how you can increase your website’s uptime and enhance its performance. We’ll be discussing opportunities SaaS companies can explore through WAF protection, frontend SSL, failover ISPs, and against DDoS attacks and using Incapsula solutions.
Based on "The Mobile Application Hacker's Handbook 1st Edition", by Dominic Chell
Instructor: Sam Bowne
More info: https://samsclass.info/128/128_S19.shtml
OWASP ATL - Social Engineering Technical Controls PresentationOWASP Atlanta
Meetup July 16th, 2015
User awareness training will always fail to prevent 100% of social engineering attacks. However, consistent and reliable technical controls drastically mitigate an organization’s risk and increase the difficulty for malicious actors to launch successful attacks.
This talk describes social engineering from the perspectives of an attacker and a defender. The presentation will cover techniques designed to help organizations develop an ideal incident response plan crafted specifically for social engineering attacks. It will explain technical controls that are designed to inhibit attackers, as well as procedures that allow an incident response team to quickly identify successful attacks and eradicate their presence.
Bishop Fox conducted new research into the state of email spoofing defenses and identified organizations that are most commonly targeted for brand spoofing. This research will show that 99.9% of the top million domains are vulnerable to email spoofing and provide recommendations for avoiding attack.
This presentation covers attacks and defenses for dangerous social engineering activities, including:
· Email spoofing
· Domain hijacks
· Typo-squatting
· Client-side attacks
· Watering hole attacks
Fruit vs Zombies: Defeat Non-jailbroken iOS Malware by Claud XiaoShakacon
Since 2014, fifteen new malware or riskware families successfully attacked non-jailbroken iOS devices (e.g., WireLurker, Oneclickfraud, XcodeGhost, InstaAgent, ZergHelper, AceDeceiver), affected thousands of iOS apps and tens of millions users around the world. Ten of them even bypassed Apple’s code vetting and occurred at App Store. In this presentation, we will systematically study how could these malware, riskware and some Proof-of-Concepts infect non-jailbroken devices via practical vectors and approaches including abusing development certificates, bypassing code review by obfuscation, performing FairPlay MITM attack, abusing MDM solution, abusing private APIs, exploiting design flaws or app level vulnerabilities, and stealing privacy data. For each topic, we will introduce its implementation, explore real world cases, analyze its risky and consequences, explain Apple’s countermeasures, and discuss why some problems will still exist in near future. We will also share some stories of how we discovered those interesting iOS malware. Through this topic, audiences could make more effective policies to protect iOS devices in their organizations, build their own systems/tools to evaluate security risks in iOS apps, and hunt more iOS malware in the future.
DerbyCon 2016
Nick Landers @monoxgas
External mail via Exchange is one of the most common services offered by organizations today. The Microsoft Office suite is even more prevalent making Outlook the most common mail client around. This talk focuses on the abuse of these two products for the purpose of gaining code execution inside remote networks. Subjects include E-Mail and password scraping, OWA/EWS brute forcing techniques, and new research into abusing Outlook mail rules for remote code execution. Learn about the capabilities of client side rules, the underlying Windows APIs, and how to modify these rule objects to make phishing attacks obsolete. Security Consultant at Silent Break Security. Professional Hacker for 2 years. Current work involves writing custom malware and researching unique attack vectors that abuse functionality in windows environments.
DevOops Redux Ken Johnson Chris Gates - AppSec USA 2016Chris Gates
In a follow-up to the duo’s offensive focused talk “DevOops, How I hacked you”, they discuss defensive countermeasures and real experiences in preventing attacks that target flaws in your DevOps environments. In this talk, Chris and Ken describe common ways in which DevOps environments fall prey to malicious actors with a focus on preventative steps. The team will present their recommended approach to hardening for teams using AWS, Continuous Integration, GitHub, and common DevOps tools and processes. More specifically, the following items will be demonstrated:
-AWS Hardening
-AWS Monitoring
-AWS Disaster Recovery
-GitHub Monitoring
-OPINT
-Software Development Practices/Processes
-Secure use of Jenkins/Hudson
-Developer laptop hardening (OS X)
Is the Cloud Going to Kill Traditional Application Delivery?Imperva Incapsula
Application delivery controllers provide load balancing, acceleration, traffic shaping and other services that improve the performance, availability and security of web applications. But with more and more web application developers hosting their applications in the cloud, using application delivery hardware is often a non-starter.
This presentation discusses the architecture of a new type of service called the Application Delivery Cloud. This new cloud service not only offers critical performance, availability and security capabilities to web application vendors, it goes beyond its hardware analog to deliver new capabilities that today’s applications require, including regional content policies and up-to-the-minute security intelligence.
You’ve seen the headlines—"[Well-Known Company] Falls Victim To Hackers".
These data breaches result in the theft of millions of names, passwords, credit card numbers, and other personal data. Imagine if such a breach lead to the theft of your application's data. . .
If multi-national companies with dedicated security teams and expansive budgets aren’t immune to the impact of hackers, how can you adequately prepare yourself to defeat this threat?
This presentation will explore the web application threat landscape. It will zero in on some of the most common attacks wreaking havoc on the internet, teaching you how to defend your online assets from them.
This presentation will discuss:
• The major security breaches of 2014
• Web application threats and common attack types
• How to defend against today’s common attacks
• Automated tools to help simplify website security
Cloudflare speeds up and protects millions of websites, APIs, SaaS services, and other properties connected to the Internet. Our Anycast technology enables our benefits to scale with every server we add to our growing footprint of data centers.
Overview of SSL: choose the option that's right for youCloudflare
Keeping communication between your visitors and your website secure and confidential has never been more important. Data can be vulnerable to theft as it’s transferred to and from your website. One simple solution to this security threat is to encrypt your traffic with SSL (Secure Sockets Layer).
SSL encryption ensures the data transferred between your visitors and your site is safe from data theft, and having SSL enabled can also boost your Google search rankings.
CloudFlare has made it simple and easy to add SSL to your site: you don’t have to purchase a separate certificate or install anything. In this webinar CloudFlare’s solution engineer Peter Griffin explains the key features of SSL, and walks you through the simple process of getting SSL running on your site.
Running a Robust DNS Infrastructure with CloudFlare Virtual DNSCloudflare
CloudFlare is excited to announce the release of Virtual DNS. Virtual DNS protects and accelerates any organization’s DNS infrastructure through robust DDoS mitigation, lightning-fast DNS lookups, and caching at 31 locations around the globe.
Hardening Microservices Security: Building a Layered Defense StrategyCloudflare
Microservices architecture is forcing developers to not only rethink how they design and develop applications, but also common security assumptions and practices.
With the decomposition of traditional applications, each microservice instance represents a unique network endpoint, creating a distributed attack surface that is no longer limited to a few isolated servers or IP addresses.
In this presention, we will review:
-How microservices differ from SOA or monolithic architectures
-Best practices for adopting and deploying secure microservices for production use
-Avoiding continuous delivery of new vulnerabilities
-Limiting attack vectors on a growing number of API endpoints
-Protecting Internet-facing services from resource exhaustion
Latest Trends in Web Application SecurityCloudflare
Hear the talk on YouTube: https://www.youtube.com/watch?v=lp4dQTSH130
Web Application Firewall security is evolving. Join John Graham-Cumming, CTO of CloudFlare, as he shares the latest trends and changes in Web Application Security. This talk will give details of the big trends in web application security seen in 2015, and how to defend against these threats and talk about the evolving web application security landscape.
Netflix Open Source Meetup Season 4 Episode 2aspyker
In this episode, we will take a close look at 2 different approaches to high-throughput/low-latency data stores, developed by Netflix.
The first, EVCache, is a battle-tested distributed memcached-backed data store, optimized for the cloud. You will also hear about the road ahead for EVCache it evolves into an L1/L2 cache over RAM and SSDs.
The second, Dynomite, is a framework to make any non-distributed data-store, distributed. Netflix's first implementation of Dynomite is based on Redis.
Come learn about the products' features and hear from Thomson and Reuters, Diego Pacheco from Ilegra and other third party speakers, internal and external to Netflix, on how these products fit in their stack and roadmap.
Enterprise Cloud Databases are fully managed and clustered databases tailored for production needs.
OVH takes care of all the infrastructure setup, you end up with you SQL access and are able to focus on your business.
Scality CTO Giorgio Regni and Software Engineer Lauren Spiegel talk about the open source S3 clone, written in Node.js. This presentation was given at a meetup on September 1, 2016 in San Francisco.
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2lGNybu.
Stefan Krawczyk discusses how his team at StitchFix use the cloud to enable over 80 data scientists to be productive. He also talks about prototyping ideas, algorithms and analyses, how they set up & keep schemas in sync between Hive, Presto, Redshift & Spark and make access easy for their data scientists, etc. Filmed at qconsf.com..
Stefan Krawczyk is Algo Dev Platform Lead at StitchFix, where he’s leading development of the algorithm development platform. He spent formative years at Stanford, LinkedIn, Nextdoor & Idibon, working on everything from growth engineering, product engineering, data engineering, to recommendation systems, NLP, data science and business intelligence.
3 Things to Learn About:
-How Kudu is able to fill the analytic gap between HDFS and Apache HBase
-The trade-offs between real-time transactional access and fast analytic performance
-How Kudu provides an option to achieve fast scans and random access from a single API
Building a data pipeline to ingest data into Hadoop in minutes using Streamse...Guglielmo Iozzia
Slides from my talk at the Hadoop User Group Ireland meetup on June 13th 2016: building a data pipeline to ingest data from sources of different nature into Hadoop in minutes (and no coding at all) using the Open Source Streamsets Data Collector tool.
Logging at OVHcloud :
Logs Data platform est la plateforme de collecte, d'analyse et de gestion centralisée de logs d'OVHcloud. Cette plateforme a pour but de répondre aux challenges que constitue l'indexation de plus de 4000 milliards de logs par une entreprise comme OVHcloud. Cette présentation vous décrira l'architecture générale de Logs Data Platform autour de ses composants centraux Elasticsearch et Graylog et vous décrira les différentes problématiques de scalabilité, disponibilité, performance et d'évolutivité qui sont le quotidien de l'équipe Observability à OVHcloud.
Galaxy Big Data with MariaDB 10 by Bernard Garros, Sandrine Chirokoff and Stéphane Varoqui.
Presented 26.6.2014 at the MariaDB Roadshow in Paris, France.
This is the presentation I delivered on Hadoop User Group Ireland meetup in Dublin on Nov 28 2015. It covers at glance the architecture of GPDB and most important its features. Sorry for the colors - Slideshare is crappy with PDFs
Advanced Administration, Monitoring and BackupMongoDB
Sailthru has been using MongoDB for 4 years, pushing the system to scale. Maintaining a high degree of client-side customizability while growing aggressively has posed unique challenges to our infrastructure. We have maintained high uptime and performance by using monitoring that covers expected use patterns as well as monitoring that catches edge cases for new and unexpected access to the database. In this session, we will talk about Sailthru's use of MongoDB Management Service (MMS), as well as areas in which we have implemented custom monitoring and alerting tools. I will also discuss our transition from a hybrid backup solution using on-premise hardware and AWS snapshots, to using backups with MMS, and how this has benefited Sailthru.
OS for AI: Elastic Microservices & the Next Gen of MLNordic APIs
AI has been a hot topic lately, with advances being made constantly in what is possible, there has not been as much discussion of the infrastructure and scaling challenges that come with it. How do you support dozens of different languages and frameworks, and make them interoperate invisibly? How do you scale to run abstract code from thousands of different developers, simultaneously and elastically, while maintaining less than 15ms of overhead?
At Algorithmia, we’ve built, deployed, and scaled thousands of algorithms and machine learning models, using every kind of framework (from scikit-learn to tensorflow). We’ve seen many of the challenges faced in this area, and in this talk I’ll share some insights into the problems you’re likely to face, and how to approach solving them.
In brief, we’ll examine the need for, and implementations of, a complete “Operating System for AI” – a common interface for different algorithms to be used and combined, and a general architecture for serverless machine learning which is discoverable, versioned, scalable and sharable.
Scylla Summit 2022: Stream Processing with ScyllaDBScyllaDB
Palo Alto Networks processes terabytes of events each day. One of their many challenges is to understand which of those events (which might come from various different sensors) actually describe the same story but from many different viewpoints.
Traditionally, such a system would need some sort of a database to store the events, and a message queue to notify consumers about new events that arrived into the system. They wanted to mitigate the cost and operational overhead of deploying yet another stateful component to their system, and designed a solution that uses ScyllaDB as the database for the events *and* as a message queue that allows our consumers to consume the correct events each time. Join this talk with Daniel Belenky, Principal Software Engineer, Palo Alto Networks where he will walk you through their process.
To watch all of the recordings hosted during Scylla Summit 2022 visit our website here: https://www.scylladb.com/summit.
Learn everything from the Imperva resources you can count on when you need help, to how you can bolster your security and performance by working with the Incapsula support organization.
Get an inside look at Incapsula Security, straight from the Security Research Team. Plus, get your vulnerability management strategy on track by assessing the automated threats you face and learn about the new security features we’re working on to keep you protected.
D3TLV17- The Incapsula WAF: Your Best Line of Denfense Against Application La...Imperva Incapsula
Learn about the most important aspects of a Web Application Firewall your organization needs to have in place to protect against the most critical web application security risks. Plus, see how we’re evolving to ensure you’re protected against new attack campaigns.
In this presentation, we cover advanced mitigation techniques used by Behemoth 2 – our latest mitigation platform – as well as real-life examples of different DDoS attack vectors and traffic samples. Plus, learn how we utilize a network of 4.7 Tbps to handle complex high throughput attacks and get a heads up on the latest trends we’re seeing in DDoS attacks.
Scott Helme, renowned security researcher and international speaker, shares his unique perspective on content security policy and how security has evolved.
Phil Williams, Principal Cloud Solutions Architect, explains how to evaluate your exposure to DDoS attack and how to best shape your defenses to budget requirements.
D3NY17- Customizing Incapsula to Accommodate Single Sign-OnImperva Incapsula
In this session, learn how the Greek Orthodox Archdiocese of America was able to customize their Incapsula service to accommodate a single sign-on solution.
In this session, learn how The Economist approached migrating to the cloud and moving economist.com from legacy datacenters to Amazon Web Services (AWS).
IncapRules are an integral method to customize Incapsula for your specific applications and environment. However, we find that our enterprise clients may have questions on building advanced rules or need help understanding how to write them for complex scenarios. In this session, Jeff Serota, Technical Account Manager, discusses the interface, some of the most common filters and actions, and how a large client collaborated with our security team to thwart credential stuffing on their client self-service portal.
D3SF17- Using Incap Rules to Customize Your Security and Access ControlImperva Incapsula
IncapRules are an integral method to customize Incapsula for your specific applications and environment. However, we find that our enterprise clients may have questions on building advanced rules or need help understanding how to write them for complex scenarios. In this session, Peter Klimek, Principal Security Engineer, discusses the interface, some of the most common filters and actions, and how a large client collaborated with our security team to thwart credential stuffing on their client self-service portal.
D3SF17- Boost Your Website Performance with Application Delivery RulesImperva Incapsula
Incapsula introduced Application Delivery Rules (ADR) in October of 2016, but many clients have not tapped into their powerful abilities. In this session, Jeff Serota, Technical Account manager, provides an overview of ADR, discusses how they differ from IncapRules, and teaches you how to leverage them in your own Incapsula deployment.
D3SF17- A Single Source of Truth for Security Issues- Pushing Siem Logs to Cl...Imperva Incapsula
It can be challenging for security teams to cut through the clutter of SIEM logs in order to analyze security information and alerts. In this session, Bryan Jones, Senior Security Engineer, walks you through the 5 major configuration steps needed to help you better manage security issues across your entire tech stack.
In this session, David Ting, VP of Engineering at DataVisor, explores the latency challenges associated with a global client base and what can be learned when implementing a performance-improving solution.
D3SF17- Migrating to the Cloud 5- Years' Worth of Lessons LearnedImperva Incapsula
Moving your critical applications from on-premises servers to the cloud can be a daunting prospect — but it doesn't need to be. Drawing on over 5 years of experience bringing some of the largest CMS sites on the Web into the cloud, Vasken Hauri, VP of Engineering at 10up, covers the key aspects you'll need to consider to ensure a smooth and successful migration. He also touches on some best practices you can apply post-migration to keep your sites secure, performant, and worry-free in an era where our toasters can launch DDoS attacks.
Keynote presentation by Dvir Shapira, Director of Product Management. Opening remarks include a look at where we’ve been in terms of the Internet as a whole and Internet security and performance, as well as where we’re going.
E-commerce Optimization: Using Load Balancing and CDN to Improve Website Perf...Imperva Incapsula
As more people shop online, it’s critical that your website meets—and even exceeds—their expectations. Online shoppers want sites that are easy to use and don’t waste their time.
According to a recent Imperva Incapsula survey, more than 60% of users said they wouldn’t wait more than five seconds for a site to load. And almost 70% said that poor website performance would cause them to leave a site and never return.
If you’re serious about reaping the benefits of the significant growth in online shopping, it’s time to get your web “house” in order. And a new free webinar from Imperva can help.
A secure web server isn’t really secure if the infrastructure supporting it remains vulnerable. Unless you implement infrastructure protection, your non-HTTP assets are vulnerable and you may not be as protected as you think you are.
You may be like others who need to get better DDoS protection but haven’t been able to or had to settle for an imperfect solution because of deployment limitations such as protocol dependencies and BGP restrictions. Incapsula IP Protection has now overcome these barriers — and we are the only service that can do it.
At this webinar our product experts will discuss how Incapsula customers are adopting IP Protection and bringing their DDoS protection to the next level. We’ll also have a discussion with Imperva CISO Shahar Ben-Hador who will share insights on how we use IP Protection and real-world lessons learned.
You need to protect more than just your web servers from DDoS attacks. We’ll address these questions:
Why do you need to protect more than just your web servers?
What were the limitations others ran into when they tried to do it?
How did Incapsula help them overcome the limitations?
...and much more!
[Webinar] DDoS Pentester Reveals: How Hackers Find Your Website’s Weak Points...Imperva Incapsula
DDoS attacks are bigger and more sophisticated than ever before. Odds are your business is going to be attacked – and without an effective mitigation strategy, you don't stand a chance.
In this webinar Andrew Shoemaker a DDoS simulation expert from NimbusDDOS gives you a rare glimpse into how hackers find the weak points in your defenses and exploit them to level devastating DDoS attacks. You'll see real world examples of the tactics and methods used to create tailored DDoS attacks that can bring down a targeted network or application, and learn how best to defend them.
An Inside Look at a Sophisticated Multi-Vector DDoS AttackImperva Incapsula
By Nabeel Saeed
This presentation explores the current DDoS attack landscape, it covers the basics of DDoS attacks, current trends including the most recent results from the newly published 2015 Imperva Incapsula DDoS Report. It also discusses a detailed analysis of one of today’s modern, multi-vector DDoS attacks. While dissecting this DDoS attack, this presentation explores the anatomy and timeline of the attack, as well as the steps used to mitigate each phase of the assault. This session will close with a review of the aspects of effective DDoS protection solutions used to combat these sophisticated denial of service attacks.
An Inside Look at a Sophisticated Multi-Vector DDoS Attack
From 1000/day to 1000/sec: The Evolution of Incapsula's BIG DATA System [Surge2014]
1. From 1000/day to 1000/sec
The evolution of our big data system
Yoav Cohen
VP Engineering
2. This Talk
A walk-through of how we built our big-data system
Incapsula, Inc. / Proprietary and Confidential. 2 All Rights Reserved.
3. About Incapsula
Vendor of a cloud-based Application Delivery Controller
Web Application
Firewall
Incapsula, Inc. / Proprietary and Confidential. 3 All Rights Reserved.
Load-
Balancing
CDN &
Optimizer
DDoS
Protection
4. How does it work?
Incapsula, Inc. / Proprietary and Confidential. 4 All Rights Reserved.
5. Modeling Web-Traffic
1. First request to a website starts a new session
2. Subsequent requests are part of the same session
3. After being idle for 30 minutes the session ends
Session 1 starts 10:03:01 GET www.incapsula.com/
Session 1 request 1 10:03:10 GET www.incapsula.com/ddos
Session 1 request 2 10:03:12 GET www.incapsula.com/cdn
… …
Session 1 ends
Session 2 starts 10.35:05 GET www.incapsula.com/signup
Incapsula, Inc. / Proprietary and Confidential. 5 All Rights Reserved.
6. The Data
A stream of messages in Google Protobuf format
msgTid: 144021710000000001
ype: SESSION_MESSAGE_CREATE
siteID: 7
startTime: 1409578192017
clientIP: ******
countryCode: "US"
entryUrlID: 5544402418256865164
visitorID: "7e59c804-f663-4595-a0df-35d9b02eb747"
userAgent: "Incapsula Site Monitor - OPS"
visitorClAppId: 209
…
Incapsula, Inc. / Proprietary and Confidential. 6 All Rights Reserved.
requestStartTime: 1410004769258
responseStartTime: 1410004769258
responseEndTime: 1410004769261
sessionID: 151009030147748952
urlID: 5544402418256865164
request_id: 567472919066130553
queryString: ""
postBody: ""
statusCode: 200
serialNumber: 1
content_length: 6350
protocol: HTTP
requestResult: REQ_CACHED_FRESH
...
7. The Problem
Transforming the stream of messages to readable data
• Processing throughput
• Read performance
• Scalability
Incapsula, Inc. / Proprietary and Confidential. 7 All Rights Reserved.
?
Session 1 starts
Session 1 request 1
Session 1 request 2
…
Session 1 ends
Session 2 starts
…
9. Gen 1
2010 – 2011
Incapsula, Inc. / Proprietary and Confidential. 9 All Rights Reserved.
Gen 2
2011 – 2013
Gen 3
2013
Gen 4
2015
System Evolution
10. Gen 1: Code Name “rtproc”
Incapsula, Inc. / Proprietary and Confidential. 10 All Rights Reserved.
11. Gen 1: OLAP Cube
• A text book solution
• Time x IP x Country x … # requests, # attacks, …
• Slice and dice to answer any question (how many attack from
Germany in Jan-2010?)
Incapsula, Inc. / Proprietary and Confidential. 11 All Rights Reserved.
dimensions counters
select sum(number_of_attacks) from Attacks where
site_id=140 and country_code=‘DE’ and time > ‘20100100’
and time < ‘20100200’
12. Gen 1: OLAP Cube
• Loading data for individual attacks requires joins:
Incapsula, Inc. / Proprietary and Confidential. 12 All Rights Reserved.
13. Gen 1: Analysis
• Generic solution
• Very big tables
• Overly complex (lots of moving parts)
Processing
Read
Scalability
Incapsula, Inc. / Proprietary and Confidential. 13 All Rights Reserved.
14. Gen 1
2010 – 2011
Incapsula, Inc. / Proprietary and Confidential. 14 All Rights Reserved.
Gen 2
2011 – 2013
Gen 3
2013
Gen 4
2015
System Evolution
15. Gen 2: Code Name “rtprocng”
• Main problems to solve:
> Read Performance
> Simplify
• New approach:
> Count things on the edge instead of centrally
> NoSQL model to improve read performance (no joins)
Incapsula, Inc. / Proprietary and Confidential. 15 All Rights Reserved.
16. Gen 2: Simpler Design
Incapsula, Inc. / Proprietary and Confidential. 16 All Rights Reserved.
17. Gen 2: Stats NoSQL Storage
• One document per day, containing
all the data to build the charts
• Read performance improved (one
lookup for all charts)
• Can even load parts of the data
(MongoDB feature)
Incapsula, Inc. / Proprietary and Confidential. 17 All Rights Reserved.
{"_id" : "7_09-04-2014",
"pageviews" : [
NumberLong(2369),
NumberLong(2380),
NumberLong(2520),
NumberLong(5651),
NumberLong(2912),
NumberLong(3357),
NumberLong(3723),
NumberLong(3301),
NumberLong(3092),
NumberLong(2984),
NumberLong(3791),
NumberLong(3069)
],
"humsess" : [
NumberLong(213),
NumberLong(258),
NumberLong(298),
…
18. Gen 2: Events NoSQL Storage
• One document per session, containing
all its actions
• Lookups are easy (no joins)
• Searches use MongoDB indexes (OK
but not great)
Incapsula, Inc. / Proprietary and Confidential. 18 All Rights Reserved.
{
"_id": 226000330131098770,
"start": {
"$date": "2014-09-09T10:19:00Z"
},
"cc": ["CA"],
"securityFlags": ["rid4"],
"badbot": true,
"prxy": [226],
"clappt": 1,
"actns": [
{
"reqRes": 10,
"u": "www.incapsula.com/",
"attack": [
{
"loc": 1,
"acode": 0,
"act": 7,
"rid": 4,
"more": 0,
"atype": 314,
"hidden": false,
"match": "",
"pval": ""
}
...
19. Gen 2: Python Processor
• Batch process:
> Process the files in the directory for up to X minutes
> Flush to storage and exit
• How to achieve good processing throughput?
> Cache objects in memory
> When processing messages, update object in memory
> When process finishes, flush all the objects from memory to
storage
Incapsula, Inc. / Proprietary and Confidential. 19 All Rights Reserved.
20. Gen 2 Storage Bottleneck
• Single DB for all sessions
• Reality check:
> MongoDB coarse-grained locking (lock per DB server)
> When batch process flushes, UIs are stuck (lock prefers writes)
> Dropping old data impossible
> Fragmentation caused excessive disk usage
Incapsula, Inc. / Proprietary and Confidential. 20 All Rights Reserved.
21. Gen 2 Storage Re-Factoring
• Single DB DB per day
> Drop DBs that are X days old
• Live sessions Live DB
“Dead” sessions per-day DB
> 0% fragmentation in per-day DBs
> Daily maintenance of Live DB (but it’s relatively small)
• DB locking not resolved (later MongoDB versions
have lock per DB)
Incapsula, Inc. / Proprietary and Confidential. 21 All Rights Reserved.
22. Gen 2: Analysis
• Simple and scalable
• MongoDB is easy to get started with
> Over time TCO increases
• Reached batch processing limits
Processing
Read
Scalability
Incapsula, Inc. / Proprietary and Confidential. 22 All Rights Reserved.
23. Gen 1
2010 – 2011
Incapsula, Inc. / Proprietary and Confidential. 23 All Rights Reserved.
Gen 2
2011 – 2013
Gen 3
2013
Gen 4
2015
System Evolution
24. Gen 3: Code Name “Graceland”
• Main problems to solve:
> Faster, online processing
> Better search capabilities
• New approach:
> Multi-threaded Java-based processor:
- Faster protobuf library than python
- Keep objects in memory for longer periods of time and reduce flushes
to storage
> Lucene for search
> A DB we can understand and control
Incapsula, Inc. / Proprietary and Confidential. 24 All Rights Reserved.
25. Gen 3: Design
Incapsula, Inc. / Proprietary and Confidential. 25 All Rights Reserved.
26. Gen 3: Multi-Threaded Java Processor
• One reader thread reads the
files and distributes the data
between the workers
• Workers process the data
> Load object from cache
> If not in cache, load from
storage
> Update object
> Flush to storage
- Periodically
- On certain events
Incapsula, Inc. / Proprietary and Confidential. 26 All Rights Reserved.
27. Gen 3: Cache Design
• Design goal: large cache, but not all in JVM heap
• Layered LRU cache (extends LinkedHashMap)
• One layer is the map, backing layer on tmpfs or disk
Incapsula, Inc. / Proprietary and Confidential. 27 All Rights Reserved.
28. Gen 3 Stats Storage (“Segmented Storage”)
• Binary file per day
• Keep recent files separate, archive older files
2014-02-03 2014-02-03.pbz 0 14325654845
2014-02-02 2014-02-02.pbz 0 14326542128
2014-02-01 2014-02-03.pbz 0 14325654845
2014-01-31 archive.pbz 76515 14325654845
...
2014-01-01 archive.pbz 0 14365428845
Incapsula, Inc. / Proprietary and Confidential. 28 All Rights Reserved.
29. Gen 3 Stats Storage (Segmented Storage)
• Files are served via nginx
• Clients keep cache
Incapsula, Inc. / Proprietary and Confidential. 29 All Rights Reserved.
30. Gen 3 Events Storage
• Tried different DBs:
> LevelDB, KyotoCabinet
- Storing the raw session data inside the lucene index
- Index memory footprint grew (all the session data got
memory-mapped)
> LevelDB, KyotoCabinet
- Couldn’t get these to work reliably
> Cassandra
- Rule of thumb: if your DB has its own conference, you
need a DBA
- We felt it’s easier to write our own than read the docs
Incapsula, Inc. / Proprietary and Confidential. 30 All Rights Reserved.
31. Gen 3 Events Storage (“Indexing Partition”)
• A partition (directory) per-day, containing:
> Lucene index of sessions
> Big file with sessions in it
• Same approach as in Gen 2 for live sessions:
> Live sessions Live partition
> Dead sessions per-day partitions
> 0% fragmentation
> Complicates searching a bit
> Live partitions require cleanup
or re-building
Incapsula, Inc. / Proprietary and Confidential. 31 All Rights Reserved.
32. Gen 3 Events Storage (“Indexing Partition”)
• Searches are more efficient:
> Search requests are served directly from index
> Session data is loaded only on-demand, and via nginx using HTTP
Range header
Incapsula, Inc. / Proprietary and Confidential. 32 All Rights Reserved.
33. Gen 3: Analysis
• Good processing throughput
• Good read performance
• Reaching JVM issues (big heap)
Processing
Read
Scalability
Incapsula, Inc. / Proprietary and Confidential. 33 All Rights Reserved.
34. Gen 1
2010 – 2011
Incapsula, Inc. / Proprietary and Confidential. 34 All Rights Reserved.
Gen 2
2011 – 2013
Gen 3
2013
Gen 4
2015
System Evolution
35. Gen 4: 2015
• Based on Gen 3
• Distribute work to more than one system
> One data server in each POP (> 20 POPs)
> Each POP processes and stores its own data
> Upload processed outputs to central servers or search on all POP
servers
Incapsula, Inc. / Proprietary and Confidential. 35 All Rights Reserved.
36. Summary
• It is equally important to understand how your system works
as it is to understand every other aspect of your business
• At some point we realized it’s better for us to build our
software from scratch than use off the shelves products as
black-boxes:
> We need to find people who know the products
- Which is crazy since we tried tons of them over the last 4 years
> We usually have less requirements
- Who needs multi-DC replication since day 1?
> We prefer coding it than reading documentations and
stackoverflows
- Then we can hack it in the middle of the night if needed
- It’s way more fun (at least for the developers…)
Incapsula, Inc. / Proprietary and Confidential. 36 All Rights Reserved.
38. Types of Data
Statistics – just numbers, used for charts, billing, etc.
Incapsula, Inc. / Proprietary and Confidential. 38 All Rights Reserved.
39. Types of Data
Events – in-depth information, used for forensics and research
Incapsula, Inc. / Proprietary and Confidential. 39 All Rights Reserved.
Editor's Notes
Click to edit Master text styles
Second level
Third level
Fourth level
Fifth level
Click to edit Master text styles
Second level
Third level
Fourth level
Fifth level
Click to edit Master text styles
Second level
Third level
Fourth level
Fifth level
Click to edit Master text styles
Second level
Third level
Fourth level
Fifth level
Click to edit Master text styles
Second level
Third level
Fourth level
Fifth level
Click to edit Master text styles
Second level
Third level
Fourth level
Fifth level
Click to edit Master text styles
Second level
Third level
Fourth level
Fifth level
Click to edit Master text styles
Second level
Third level
Fourth level
Fifth level
Click to edit Master text styles
Second level
Third level
Fourth level
Fifth level