The document discusses a three-layered firewall approach for securing services within an Istio service mesh. Layer 1 involves using AWS WAF for basic rate limiting of traffic at the load balancer level. Layer 2 proposes using a Coraza WASM filter within Istio ingress gateways for loose DevOps-managed firewall rules. Layer 3 suggests enabling fine-grained, developer-managed firewall policies specific to each service deployment using Coraza WASM filters within individual pods.

1. MEETUP
Firewalling a Service Mesh with WebAssembly: Pick low hanging Fruits
for secure and performant self-servicing
SDA DevOps Team | 29.11.2023

2. SDA SE
SDA SE
• What we have now
• The Plan: 3 Layered Firewall
• Layer 1: IP-Based Rate-Limiting
• Layer 2: DevOps managed loose Istio Firewall
• Layer 3: Developer managed fine-grained Istio Firewall
• Closing words and questions
AGENDA
Thanks for attending
30.11.23 Firewalling a Service Mesh with WebAssembly 2

3. SDA SE
SDA SE
CLUSTER CONFIGURATION
11/30/23 3
Tools Cluster
Here we find all the tools we need to develop software based on industry best practices.
Includes the following tools, among others: Applications Cluster (Testing)
Software will be deployed over GitOps, by using ArgoCD
from the Tools Cluster
Applications Cluster (Integration)
Software will be deployed over GitOps, by using ArgoCD
from the Tools Cluster, after passing all tests in the application
Cluster (Testing)
Applications Cluster (Prod)
Software will be deployed over GitOps, by using ArgoCD
from the Tools Cluster, after passing all tests in the application
Cluster (Staging)
Log
Cluster Analysis Platt-
form
Security
Monitor
Cluster
Image
Scanner
Requirements/
Supporting Tools
We believe in IaC (Infrastructure as Code), these tools help us
deploy our Code/Infrastructure
Tools & Application Cluster (4 separated kubernetes cluster)
Cluster
Monitor
Log
DBs
Cluster
Monitor
Log
DBs
Cluster
Monitor
Log
DBs
Firewalling a Service Mesh with WebAssembly
What we have now

4. SDA SE
SDA SE
CLUSTER CONFIGURATION
11/30/23 4
Firewalling a Service Mesh with WebAssembly
What we have now
Key Points to take away
• Many Clusters
• Like to pull Cloud-Features into k8s
• Plattform Tools
• Business-Logic Software
• DevOps doesn’t know upfront what will be deployed
• Terraform everything

5. SDA SE
SDA SE
CLUSTER CONFIGURATION
Private Clusters Networking
30.11.23 5
What we have now
AWS Cloud
AWS account
VPC - 10.x.x.x/18
Private – tools-meetup-a - 10.x.x.x/22
Private – tools-meetup-b - 10.x.x.x/22
Private – tools-meetup-c - 10.x.x.x/22
Public – tools-meetup-a - 10.x.x.x/24
Public – tools-meetup-b - 10.x.x.x/24
Public – tools-meetup-c - 10.x.x.x/24
Private – dev-meetup-a - 10.x.x.x/22
Private – dev-meetup-b - 10.x.x.x/22
Private – dev-meetup-c - 10.x.x.x/22
Public – dev-meetup-a - 10.x.x.x/24
Public – dev-meetup-b - 10.x.x.x/24
Public – dev-meetup-c - 10.x.x.x/24
Firewalling a Service Mesh with WebAssembly

6. SDA SE
SDA SE
CLUSTER CONFIGURATION
Private Clusters Networking
30.11.23 Firewalling a Service Mesh with WebAssembly 6
What we have now

7. SDA SE
SDA SE
CLUSTER CONFIGURATION
Private Clusters Networking
30.11.23 Firewalling a Service Mesh with WebAssembly 7
What we have now
External-DNS needs
svc.Status.LoadBalancer.Ingress
for Route53 record content

8. SDA SE
SDA SE
CLUSTER CONFIGURATION
11/30/23 8
Firewalling a Service Mesh with WebAssembly
What we have now
Key Points to take away
• Using Istio Service-Mesh
• Currently no Firewall
• Always Public/Private capabilities
• Manage Infrastructure from inside k8s
• External-DNS needs service status

9. SDA SE
SDA SE
3 LAYERS OF FIREWALL
30.11.23 Firewalling a Service Mesh with WebAssembly 9
The Plan
1. Point of first contact
• Implement Rate-Limit to keep malicious traffic out of the cluster
• Swap NLB for ALB
• Attach AWS WAF Rules to said ALB
2. Last DevOps managed point of contact
• Implement Software Firewall at Ingress-Gateways
• Use loose ruling to fit common use-cases
• Implement lower coverage of OWASP CRS
3. Deployment specific point of contact
• Implement Software Firewall as part of Deployment
• Implement, at best, all of OWASP CRS

10. SDA SE
SDA SE
IP-BASED RATE LIMIT
30.11.23 Firewalling a Service Mesh with WebAssembly 10
Layer 1
• Guard from large number of requests from recent IPs
• Required for both Public/Private – so no CDN
• Istio is distributed, counting requests is hard
• AWS WAF is cheap, request agnostic

11. SDA SE
SDA SE
IP-BASED RATE LIMIT
30.11.23 Firewalling a Service Mesh with WebAssembly 11
Layer 1
From this:

12. SDA SE
SDA SE
IP-BASED RATE LIMIT
30.11.23 Firewalling a Service Mesh with WebAssembly 12
Layer 1
To this:

13. SDA SE
SDA SE
IP-BASED RATE LIMIT
30.11.23 Firewalling a Service Mesh with WebAssembly 13
Layer 1
But:
• AWS LoadBalancer Controller only creates ALB for Ingress type resources
• AWS LoadBalancer Controller doesn’t create Service Status for ALBs
• Can’t manually set Service Status using Terraform

14. SDA SE
SDA SE
IP-BASED RATE LIMIT
30.11.23 Firewalling a Service Mesh with WebAssembly 14
Layer 1
So:
• Create ALB using Terraform
• Allocate public EIP and cut private subnet IP from subnets
• Set IPs to externalIPs field of service

15. SDA SE
SDA SE
IP-BASED RATE LIMIT
30.11.23 Firewalling a Service Mesh with WebAssembly 15
Layer 1
But:
• ALBs don’t support choosing Ips
• Only NLBs support attaching EIPs and private IPs

16. SDA SE
SDA SE
IP-BASED RATE LIMIT
30.11.23 Firewalling a Service Mesh with WebAssembly 16
Layer 1
So:

17. SDA SE
SDA SE
IP-BASED RATE LIMIT
30.11.23 Firewalling a Service Mesh with WebAssembly 17
Layer 1
But:
• External-DNS VirtualService does only read service status
and not externalIPs

18. SDA SE
SDA SE
IP-BASED RATE LIMIT
30.11.23 Firewalling a Service Mesh with WebAssembly 18
Layer 1
So:
• We need to add this via Open-Source PR
• Layer 1 plan has failed so far

19. SDA SE
SDA SE
DEVOPS MANAGED SERVICE-MESH FIREWALL
Service-Mesh
30.11.23 Firewalling a Service Mesh with WebAssembly 19
Layer 2
• dedicated infrastructure layer
• communication between microservices
• manage and control the interactions
• We use Istio – biggest player

20. SDA SE
SDA SE
DEVOPS MANAGED SERVICE-MESH FIREWALL
Coraza WAF
30.11.23 Firewalling a Service Mesh with WebAssembly 20
Layer 2
• Web Application Firewall (WAF)
• 100% compatible with OWASP CRS v4
• written in Go - is a library at its core
• replace ModSecurity Engine
• Itself an Engine – useless without rules

21. SDA SE
SDA SE
DEVOPS MANAGED SERVICE-MESH FIREWALL
OWASP CRS
30.11.23 Firewalling a Service Mesh with WebAssembly 21
Layer 2
• Generic attack detection rules, like OWASP Top Ten
• Can be loaded into compatible WAFs
• Customizable – using paranoia levels, exclusions etc.

22. SDA SE
SDA SE
DEVOPS MANAGED SERVICE-MESH FIREWALL
ModSecurity policy
30.11.23 Firewalling a Service Mesh with WebAssembly 22
Layer 2
SecRule REQUEST_URI "@streq /admin" "id:101,phase:1,t:lowercase,deny"
• SecRule = Keyword
• REQUEST_URI = Variable e.g “/admin”
• “@streq /admin” = compare variable to string
• Id = id of the rule – they need to be unique
• phase:1 = request headers and body processing
• deny = action

23. SDA SE
SDA SE
DEVOPS MANAGED SERVICE-MESH FIREWALL
Coraza Proxy WASM
30.11.23 Firewalling a Service Mesh with WebAssembly 23
Layer 2
• WAF WASM filter
• Implementing the proxy-wasm Application Binary Interface specification
• Event-driven streaming APIs
• In binary format and run in a sandbox (module)
• WASM modules gain adoption in container ecosystems like containerd and docker registries
• Can be loaded into Istio-Proxy (envoy)

24. SDA SE
SDA SE
DEVOPS MANAGED SERVICE-MESH FIREWALL
Istio Ingress-Gateway WAF
30.11.23 Firewalling a Service Mesh with WebAssembly 24
Layer 2

25. SDA SE
SDA SE
DEVOPS MANAGED SERVICE-MESH FIREWALL
Istio Ingress-Gateway WAF
30.11.23 Firewalling a Service Mesh with WebAssembly 25
Layer 2
• WAF implemented at Gateway level
• ALL services of whole Gateway now secured by the rules
• Activating full OWASP CRS at this level is nearly impossible in our case

26. SDA SE
SDA SE
DEVELOPER MANAGER FINE-GRAINED ISTIO FIREWALL
Service specific WAF
30.11.23 Firewalling a Service Mesh with WebAssembly 26
Layer 3
• As close as possible to service
• No host matching
• Each pod can get own WAF and own paranoia levels
• As close as possible to full OWASP CRS coverage
• Can even be used for service-to-service
• WAF can be tested in Pull-Requests

27. SDA SE
SDA SE
DEVELOPER MANAGER FINE-GRAINED ISTIO FIREWALL
Service specific WAF
30.11.23 Firewalling a Service Mesh with WebAssembly 27
Layer 3

28. SDA SE
SDA SE
DEVELOPER MANAGER FINE-GRAINED ISTIO FIREWALL
Service specific WAF
30.11.23 Firewalling a Service Mesh with WebAssembly 28
Layer 3

29. SDA SE
SDA SE
DEVELOPER MANAGER FINE-GRAINED ISTIO FIREWALL
Take aways
30.11.23 Firewalling a Service Mesh with WebAssembly 29
Layer 3
• Layer 1 for generic rate-limiting
• Private endpoints can’t have CDN
• Possible in Istio but have to manage Redis Cluster
• Cloud Solutions scale great and are not too expensive
• Layer 2 for generic DevOps managed WAF
• Enforced least amount of OWASP CRS coverage
• Great if deployed services are unknown
• Enforcing great coverage in Layer 3 is hard
• Layer 3 for close at service WAF
• Very specific maximum OWASP CRS coverage
• Requires a lot of testing
• Is part of deployment and can be tested in PRs

30. SDA SE
SDA SE
DEVELOPER MANAGER FINE-GRAINED ISTIO FIREWALL
Closing words
30.11.23 Firewalling a Service Mesh with WebAssembly 30
Layer 3

31. SDA SE
SDA SE
Contact us at devops@sda-se.com for further questions or specific code snippet
FINISHED
ANY QUESTIONS?
30.11.23 Titel der Präsentation 31