Watch on-demand webinar: https://info.avinetworks.com/webinars/web-application-security-continuous-delivery-pipelines Applications today have evolved into containers and microservices deployed in fully automated and distributed environments across data centers and clouds. Application services such as load balancing, security, and analytics become critical for continuous delivery. To secure modern web applications, security policies including SSL/TLS, ACLs, IP Reputation, and WAF need to be applied quickly. We will share a reference implementation from Avi Networks. Join this webinar to learn: - CI/CD in the web application security context - Challenges and solutions integrating a modern web application firewall (WAF) into the application development pipeline - How to create processes that support both security and development requirements

1. Web Application security
for continuous delivery
pipelines
Christian Treutler, Avi Networks

2. AppSec DefenderEngineer at heart
Christian Treutler
Interested in
(sometimes scared by)
Security Automation,
IoT security, Big Data
security, AI security
implications.
Security evangelist

3. Agenda
1. Mission statement
2. CICD Introduction
3. Demo: Rolling out a new version
4. How to build application security into a CICD
pipeline from a WAF perspective?
5. Demo: Pipeline with enabled WAF
6. Demo: Security with ”infra as code”

4. Mission Statement:
Including AppSec into CICD environments
can be achieved with the right tools.
Teams must work together closely to allow this kind of automation to succeed.

5. Continuous Delivery Introduction
“Continuous Delivery is a software development discipline where you build software in such a way that the software can be
released to production at any time.” – Martin Fowler
“Our goal is to make deployments—whether of a large-scale distributed system, a complex production environment, an
embedded system, or an app—predictable, routine affairs that can be performed on demand.” - Jez Humble
Continuous Delivery involves automation and integration of all stages of software
development including – development, build, test, acceptance, configuration management,
infrastructure, provisioning and ultimately release!

6. Running a successful CICD
The very short version. For detailed information, please see references at the end of the slide deck.
Goals
Predictable, reliable, fast and better quality!
Business – Sales, PM and Customers love this!
Operations
Organizational
Success metrics
How to express the Pass / Fail decision?
Use Normalized metrics
Metrics measuring Quality is more important than Quantity
Benchmark your application!
Process Develop
Build
Test
Deploy
Monitor /
Validate
In-Service /
Out-of-service
Feedback delay
as small as
possible
Deployment
Application config
as code
Check
Apply
Updates
Validate &
Record
Continuous
Integration
Software Build
Integration
Tests
Sandbox Production
Deployment Automation

7. Demo Setup
Environment Walkthrough

8. Demo Environment
Application: vulnerable application Hackazon
Application lifecycle includes security scanning and iWAF Protection
All parts use “infrastructure as code”
Kubernetes: a cluster of 3 nodes + dashboard
Service Mesh: Avi with iWAF module
CICD: Concourse
Gitea: local GIT repository
Docker registry
OWASP ZAP scanner
All services run on Kubernetes in Docker containers

9. BARE METAL VIRTUALIZED CONTAINERSON PREMISES PUBLIC CLOUDVIRTUALIZED CONTAINERS
Modern, Scalable, Multi-Cloud Architecture
CONTROLLER
(SaaS / Self-Managed)
SERVICE
ENGINE
SEPARATE CONTROL
& DATA PLANE
ELASTICITY
INTELLIGENCE AUTOMATIONMULTI-CLOUD
Copyright © 2018 Avi Networks

10. K8S Node
K8S Master
DC: app1
K8S Node
Pod 4Pod 3
K8S Node
Pod 2Pod 1
K8S Node
Avi Controller Avi Service
Engine
Proxy Functions:
• Full-Featured Service Proxy (Distributed Load Balancing: L4-7)
• Service Discovery (DNS, IPAM)
• App Maps, Service Performance Monitoring, Connection log
search and analytics
• Security Rules, Web App Firewall, and Traffic Encryption
• Application Auto-Scaling
Avi’s Container Services for OpenShift/Kubernetes
Distributed Proxy for both North-South (Ingress) and East-West Traffic

11. Demo 1
CICD in an application security context

12. CICD in an application security context
Which parts of AppSec can be applied?
DAST, SAST, Vulnerability management
(Black and white box testing)
Trusted solutions
Usually automatable in test/staging through APIs
Success metric: vulnerabilities within expected numbers
Misc.
DDoS Protection – part of ADC/LB requirements
Transport Security – TLS encryption – ADC
Virus/Malware scanning – application or agent based
Success metric: depends on requirements
RASP
Runtime Application Self-Protection
Built into the application / tool chain
Essentially part of the application
Success metric: application test is green
WAF
Application protection (usually proxy based)
WAF policy needs to be part of application or infra code
Must deployed in all environments (dev, test ,production)
Success metric: application test is green

13. Challenges of AppSec vs Dev
High level view
Infrastructure does not support
security requirements
Vendor/product X does not support security
requirement foo.
Security is a first class feature
Should be part of company DNA
Security education
Every employee needs to be educated about the
implications if security best practices are not met.
Features vs. Security
When planned and designed correctly security is
part of a feature and not something that competes
for the same resources.
Team Silos
Security is a team effort and will fail it not all
teams take their share of the responsibility.

14. Demo 2
Designing a CICD pipeline with Web Application Firewall

15. Legacy WAF challenges with CICD
Elasticity, Scalability, Programmability, Automation, Management as code
PROPRIETARY
HARDWARE
Costly Refresh Cycle
Tied to Environment
MANAGE EACH DEVICE
No central management
No central monitoring
NO AUTOMATION
Incomplete REST API
No plug-n-play automation
NO TELEMETRY
LBs are black boxes
No analytics or insights
STATIC CAPACITY
Requires overprovisioning
No cloud like elasticity

16. CICD example: WAF – properties for a working solution
An overview
Mindset
• Application developer, security networking all work together
• Application developer accepts WAF as a additional security layer to protect the application
• Application developer uses WAF analytics for application behavior analysis
• Security team defines security requirements and risk assessments
• Security team provides oversight and guidance
Infrastructure
• Pure software solution that supports deployment anywhere with the application
• Integration into a Load Balancer or Web Server
• Automation (APIs, SDK, integrations)
• Orchestration integration (Cloud, Container, Scalable)
• Insights, metrics
Process
• WAF is part of all stages of the development process
• WAF policy is owned by application developer and kept with the application source code
• WAF policy changes are tracked within the source code repository with ticket management
• Security audit process: security team can monitor and approve policy changes if needed (pull requests)
• Every application uses the default security policy (for that specific application type, risk profile)

17. CICD workflow with WAF – development
Sample integration of a WAF into a CICD workflow:
Develop
Build
Test
Deploy
Monitor /
Validate
In-Service /
Out-of-service
Develop
WAF is part of the development process.
If needed WAF policy is updated by the developer.
Security team approves the changes to the WAF
policy.
WAF policy is part of the application code and will
be rolled out with application build.

18. CICD workflow with WAF – build
Sample integration of a WAF into a CICD workflow:
Develop
Build
Test
Deploy
Monitor /
Validate
In-Service /
Out-of-service
Build
WAF has no part in the build process, since it does
not directly influence the application code.

19. CICD workflow with WAF – test
Sample integration of a WAF into a CICD workflow:
Develop
Build
Test
Deploy
Monitor /
Validate
In-Service /
Out-of-service
Test
All tests are run with the new build.
Test environment uses stored WAF policy.
Any test failures or unexpected WAF findings can
trigger a new ticket for the developer to investigate
and fix.
Tests include unit, functional and ideally end-to-
end tests.
Caveat: Simulated test traffic is usually inferior to
real world traffic.

20. CICD workflow with WAF – deploy
Sample integration of a WAF into a CICD workflow:
Develop
Build
Test
Deploy
Monitor /
Validate
In-Service /
Out-of-service
Deploy
WAF gets deployed alongside application code.

21. CICD workflow with WAF – monitor / validate
Sample integration of a WAF into a CICD workflow:
Develop
Build
Test
Deploy
Monitor /
Validate
In-Service /
Out-of-service
Monitor / Validate
WAF metrics will provide necessary insight into the
health and expected WAF behavior to measure.

22. CICD workflow with WAF – in-service / out-of-service
Sample integration of a WAF into a CICD workflow:
Develop
Build
Test
Deploy
Monitor /
Validate
In-Service /
Out-of-service
In-Service / Out-of-service
According to the success criteria the new
application code is put in-service and the old
application code is taken out-of-service.
WAF metrics can have a direct impact on the cut
over if they degrade. It is one part of the success
criteria.

23. Demo 3
Security with ”infrastructure as code”

24. ●1. Continuous Delivery transcends technical, cultural,
process and business aspects of any team.
●2. Deployment decisions can be automated –
provided metrics are wisely used.
●3. If possible make your applications elastic and set
it up such that it can scale-out and scale-in during the
deployment evaluation.
●4. Modern tools for CI, configuration management,
deployment pipeline automation, ONF (SDN), public
and private cloud programmability allows complex
applications to be easily deployed.
●1. As Continuous Delivery is introduced the security
strategy of a company is a key factor for success.
●2. Application security process is required.
●3. Any WAFs that supports the infrastructure
requirements can be used for CICD.
●4. A WAF needs to provide metrics (via API) for
evaluation of successful deployment and operations.
●5. Change management is crucial and auditing must be
possible (trace of configuration changes).
Summary: Continuous Delivery with and without Application Security focus

25. Thank You!
Christian Treutler, Avi Networks
christian@avinetworks.com
Next Steps
1. Learn about GDPR and WAF
https://info.avinetworks.com/webinars/secure-web-
applications-and-achieve-compliance
2. Schedule a demo
https://info.avinetworks.com/schedule-a-demo
3. Get your hands “dirty”
https://github.com/avinetworks/devops/tree/master/an
sible/deployment_pipeline

26. References
Jez Humble, https://continuousdelivery.com
Martin Fowler, https://martinfowler.com/delivery.html
Chen, Lianping, Paddy Power. Continuous Delivery: Huge Benefits, but Challenges Too. IEEE Software. 32.
10.1109/MS.2015.27
Demo code is available at github
Download Avi Solution from www.avinetworks.com
https://github.com/avinetworks/devops/tree/master/ansible/deployment_pipeline