This project set out to compare WAFs and RASP by building a prototype real-world environment and testing a RASP-based IMMUNIO solution against two well-regarded WAFs – CloudFlare and Incapsula.
Building AI-Driven Apps Using Semantic Kernel.pptx
IMMUNIO: RASP vs. WAF
1. IMMUNIO vs. Incapsula vs.
CloudFlare
RASP vs. WAF
Complete Test Results are available in the Case Study: “Selecting a Web Protection Solution”
2. The Project
Dojo College is a leading online provider of a multi tenant hosted Learning
Management System. This software product written in Ruby on Rails is designed to
allow other educational and corporate customers the ability to host their own Learning
Management systems within the larger Dojo College system. The software is all
developed in house with a small team of developers that make use of the AGILE
methodology. The development team is really focused on getting functionality out to
their customers, and would rather security be something that they can entrust to a 3rd
party solution.
While the project itself was dealing with a fictitious company, the scenario itself
was based off a real life problem that resonates with companies everywhere.
3. Client and Expectations
● Online provider of a multi tenant hosted
LMS (Learning Management System)
● Ruby on Rails
● Limited resources, looking for outside
help to armoring the application
About the client
● CloudFlare
● Incapsula
● IMMUNIO
Solutions Selected for Evaluation
● Must not impact normal operation
● Functionality and performance must say
well below standard response times
● Must provide actionable near real-time
intelligence and reports to allow to
respond to threats
● Must address runtime portion of OWASP
Top 10
● Does not require specialized skills for
installation and maintenance in regular
operation
Requirements
4. Test Setup and Methodology
The Network
CloudFlare and
Incapsula operate
through Content Delivery
Networks (CDN)
Baseline testing
Conclusions / Takeaways:
● 55ms response is within the “norm”
● CPU overhead at about 2%
● Memory footprint ~300MB
5. Test Results (Key Functional Areas)
IMMUNIO Incapsula CloudFlare
Ease of
Installation
“IMMUNIO: Installation was as simple as creating
an account and deploying a Gem file.”
“Incapsula: Arguably tied with IMMUNIO for
ease of install. After setting up an account a
wizard walks you through creating a DNS
record for the newly installed service. Though,
without administrator access to install DNS
records, you would have to rely on another
team to get the correct record installed.”
“CloudFlare: by far the most difficult of the 3.
This install required moving the entire DNS
infrastructure to CloudFlare’s servers.
Normally this is something that would require
a lot of planning and consideration within an
organization, and may be met with a lot of
resistance”
Reporting “Clear concise reporting that is actionable is the
backbone of a good security program. In this test
we wanted to see what kind of reporting was
available during an attack and was it easily
understandable.”
“Incapsula: This solution surprised us during
the tests, what we figured would be the clear
winner. Unfortunately, this solution could not
provide updates in real time.”
“CloudFlare: While not the most
comprehensive reporting at all, this solution
presented the information in an easy to
understand format.”
Attack Body of
Knowledge
“IMMUNIO: the only solution here that had a
component that was dynamic in nature. Certain
functions within the service required priming and
sending good traffic through, so IMMUNIO could
“learn”. This proved to be the one thing that set this
product heads and tails over the others in the final
tests.”
Incapsula: Has a wide array of signatures, and
detected most of them. Incapsula has extra
support for DDOS as well as IP reputation,
which we found, would be a bonus when going
into the product selection phase.
“CloudFlare: It almost seemed like security
was an afterthought. Very small amount of
configuration ability. Most of it is based on
Mod_Security rulesets, and not nearly as
comprehensive as any of the other two
solutions.”
Completeness of
the Solution
“IMMUNIO: Detection rates were interesting. The
solution found some items that the Dojo.College team
was not even expecting. During our testing we
uncovered a couple of bugs in some well known Gems
that were being used within the application, and
uncovered some design issues within the Dojo.College
application that needed to be addressed. Overall the
IMMUNIO service performed above and beyond our
expectations.”
Incapsula: Detection rates were very low.
When it did detect, it would block, but most
rules required explicit tuning, and would
require that tuning on each update. There is
also no way to know if the app was actually
vulnerable to an attack, or where it should be
fixed.
“CloudFlare: Is this thing on? Even at the
“highest” setting the solution could only
provide a very weak, to modest
implementation of Mod_Security and no way
to manually update rules or create new rules.”
Performance Measured response time: 5.47 ms
Measured Throughput: 614 rpm
Measured response time: 9.97 ms
Measured Throughput: 484 rpm
Measured response time: 14ms
Measured Throughput: 58rpm
Complete Test Results are available in the Case Study: “Selecting a Web Protection Solution”
6. Test Results (Runtime Portion of OWASP Top 10)
IMMUNIO Incapsula CloudFlare
Injection (SQL injection,
LDAP injection and
Command Injection)
100% 60% < 20%
Session Management 100% 100% 100%
Cross-Site Scripting
(Stored XSS, DOM XSS,
Script Execution, HTML
Injection)
100%, low FP rate > 90%, high FP rate < 10%, had to be turned
off due to noise
Direct Object Access
and Function Level
Access Control
RFI: 100%
LFI: 100%
RFI: 100%
LFI: 0%
RFI 100%
LFI: 0%
CSRF 100% 100% 100%
Complete Test Results are available in the Case Study: “Selecting a Web Protection Solution”