Download as PDF, PPTX
Uploaded byModern Data Stack France
HUGFR : Une infrastructure Kafka & Storm pour lutter contre les attaques DDoS en temps-réel par Steven Le Roux de la société OVH
The document discusses the architecture and functionality of an anti-DDoS solution employed by OVH, utilizing Kafka and Storm for real-time data processing and threat detection. It highlights the system's performance metrics, configurations, and challenges such as false positives and user behavior anomalies. Potential solutions for application-centric issues are also mentioned, along with resources for further learning.
More Related Content
![CETH for XDP [Linux Meetup Santa Clara | July 2016]](https://cdn.slidesharecdn.com/ss_thumbnails/ceth5overview1-160801192921-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to HUGFR : Une infrastructure Kafka & Storm pour lutter contre les attaques DDoS en temps-réel par Steven Le Roux de la société OVH
HUGFR : Une infrastructure Kafka & Storm pour lutter contre les attaques DDoS en temps-réel par Steven Le Roux de la société OVH
- 1. AntiDDoS : ThreatDetection Steven Le Roux Infrastructure Engineer
- 6.
- 8.
- 9.
- 14. 3 Tbps 17Datacenters 32 PoPs
- 19.
- 23.
- 25. Clients Producers Consumers Brokers Topics Partitions Replicas / kafka
- 26. / kafka /topic
- 27. / kafka /topic / replicas
- 28. / kafka /topic / replicas / factor / 3
- 29. / kafka /topics
- 30.
- 31.
- 32. / kafka /producers
- 33.
- 35. Topology (DAG) Spouts Bolts Cluster Nimbus Supervisors Workers / storm
- 36. / storm /topology
- 37. / storm /topology / antiddos
- 39.
- 40. / storm ShuffleGrouping Field Grouping Direct Grouping Other Grouping
- 41. Attacks Router Grouping Scans IP src Grouping / storm
- 42. Attacks ≈ 1s Scoring Filters Burst Scans IP Proto / storm
- 43. Indexing Prooving Producing / storm / event
- 44.
- 45.
- 46. Nice speech… …so what ?
- 47. #issues False positives Strange behaviours from customers e.g. DB sync without connection pool Application centric i.e. UDP protocols
- 48. #solutions Add othersources Application Anti-DDoS Game Half Life/Source CS:GO TeamSpeak / Mumble GTA SA:MP … More to come (any special need ?)
- 50.
- 52. #hardware Nodes -Hardware CPU 16c/32t RAM 256GB Disks : OS : Raid 1 Data : 10 disks per node 200 MB/s ~ 1,5-2 Gbps
- 53. Kafka I/O bound Bench (1node) 1M+ msg/s No compression No ackers 80MB/s Tuning num.io.thread num.network.thread socket.*.buffer.* Storm #config CPU/RAM bound M+ tuples/s No ackers Break SRP Minimal workers Avoid transfer buffer
- 54.
- 55. Clément Sciascia -@csciasci Magnus Edenhill - @edenhillm https://github.com/edenhill/librdkafka LinkedIn - Apache Kafka Nathan Marz - Apache Storm #Thanks Storm basic training – Mickael G. Noll #more http://fr.slideshare.net/miguno/apache-storm-09-basic-training-verisign Kafka documentation
- 56. Steven LE ROUX @StevenLeRoux steven.le-roux@ovh.net Thanks