To defend against attacks, think like a hacker. But does that mean you need to be a DevOps expert? Security researchers today need to discover new attack techniques. However, much of their focus is diverged to backend coding. We share how to build an infrastructure for researchers that allows them concentrate on business logic and writing hacker “tasks”. Using Docker and Kubernetes on Google Cloud, these tasks can then be performed in parallel and without a lot of DevOps hassle. Our technique removes two common barriers: first, long and risky deployment processes and second, low transparency within the production system. Promise to share the stupid things too.

1. Hacking for Fun & Profit
The Kubernetes Way
Demi Ben-Ari - VP R&D @ Panorays

3. About Me
Demi Ben-Ari, Co-Founder & VP R&D @ Panorays
" Google Developer Expert
" Co-Founder of Communities:
○ “Big Things” - Big Data, Data Science, DevOps
○ Google Developer Group Cloud
○ Ofek Alumni Association
In the Past:
" Sr. Data Engineer - Windward
" Team Leader & Sr. Java Software Engineer,
Missile defence and Alert System - “Ofek” – IAF

4. Some important things
● What I’m not: A Docker / Kubernetes Expert
● What you won’t be after this talk: A Docker / Kubernetes Expert
● What you will be after this talk?
● Happier people (Because I’ve stopped talking)
● You’ll know what was our problem and our way of solution
● You’ll know where to search and learn more things
● The answer to the “What’s the meaning of life?” (42)

5. Mapping the World’s Cyber Posture

6. - Confidential -
It’s Not Only Your IT Vendors
6
“We’re seeing third party risk management show up as one of the top
three board agenda items”
• T.R. Kane, cybersecurity and privacy partner at PwC, 2016
Financial platforms Payroll services Law firms
3rd Party vendors flow
data into company’s
systems
Providers hold information
about customers /
employees
Consultants hold sensitive
information of the company

7. So Basically what do we do? (Previous Situation)
● Every VM running would imitate the whole reconnaissance phase of the hackers
lifecycle.
● Parallelizm is being done through firing up more VMs.
● Built an internal orchestration system to launch all of the scans.
● All of the servers are running on Google Cloud Platform.

8. The Problem

9. What’s the biggest problem in Software Engineering?

10. Naming Things

11. What’s the biggest problem in Software Engineering?
● Naming Things
https://www.pinterest.com/pin/52424783138601042/

13. Step #1 - Appoint a CNO
● Chief Naming Officer - your go to guy for all of the hardest problems

14. Step #2 - Define the problem and abstractions
● Parallelizm happening in the manner of a company (VMs being launched).
● Scan and evaluation process is not transparent.
● Server utilization is low.
● Had to build an internal orchestration system via Cron & Bash.
● (Think how fun is that…)
● How do you monitor all of this?
● Need to control it all via an easy API

15. https://www.coscale.com/hs-fs/hubfs/Blog_Pictures/2016_06/monolithic_vs_microservices.jpg

16. We’ve created a “Microlith”

17. the Transporter

18. the Transporter
" the transporter will transport anything,
no questions asked, always on time,
and he is known as the best in the business.
" He strictly follows three rules when transporting:
" 1: "Don't change the deal”
" 2: "No names” - (We kind of broke that rule…sorry)
" 3: "Never open the package"

19. the Transporter
● Distributed task queue
● Supports controlling the workers in Celery.
● Every queue has its own a "Job" - Not a company.
● All of the workers are running tasks that are blocking
● Abstractions:
1. Job - Everything that has a run method.
2. Phase - has the definition of if it's sequential or parallel.
3. Workflow - A list of phases.
● Has the ability also to run partial workflows.

20. A bit about Kubernetes
● Greek for “Helmsman”; also the root of the
words “governor” and “cybernetic”.
● Manages container clusters
● Inspired and informed by Google’s
experience and an internal system (Borg)
● Supports multiple cloud and bare-metal
environments
● 100% Open source, written in Go
● Manage applications, not Machines

21. Cattle vs. Pets
CattlePet
• Has a name
• Is unique or rare
• Personal Attention
• If it gets ill, you make it better
• Has a number
• One is much like any other
• Run as a group
• If it gets ill, you make hamburgers

22. Community
Top 0.01% of all
GitHub projects
1,200+ external
projects based on
k8s
Companies
Contributing
Companies
Using
1,000+
unique contributors
15,000+ people
signed up for k8s
meetups

23. Kubernetes (Openhub)
https://www.openhub.net/p/kubernetes

24. Kubernetes Terminology
● Deployment
● Service
● ReplicaSet
● Pod
● Volume
● Label
● Selector
● ConfigMap
● Secret
● DaemonSet
● Stateful Set
● Job
● Liveness Probe
● Readiness Probe

25. What’s not spoken about a lot in K8s?

26. Kubernetes - Job
● Run to Completion, as opposed to run-forever
● Express parallelism vs. required completions
● Workflow: restart on failure.
● Build / Test: don’t restart on failure.
Aggregates success / failure counts
● Built for batch and Big Data work

27. Monitoring

28. (Celery) Flower
● Flower is a web based tool for monitoring and administrating Celery clusters.
● Abilities
● Real-time monitoring using Celery Events Remote Control
● Broker monitoring
● HTTP API
● Downside:
● Monitors only the execution of Celery and not “the Transporter”

29. (Celery) Flower

30. Monitoring - Current Status
● Monitoring K8s via StackDriver
● Monitoring Celery - via Flower (doesn’t give us all of the functionality)
● Monitoring end points created on “the Transporter” - CLI API
● What we’re missing?
● GUI tool that will provide a dashboard
● Proper alerting and automatic error handling
(Currently handling things via “notifications-api”)

31. General Architecture
Flow Execution K8s
Managed K8s on GCP
the Transporter
Celery Tasks
Also Running on K8s
Flower UI
CLI Monitoring

32. Architecture flow
Phase 1
Phase 2
Phase 3
Phase 4

33. Summary & Conclusions
● Kubernetes can also run Jobs and not only long living services
● If you can avoid distributed systems, please do so :)
● “Perfect” is the nemesis of “Done” / “Working and giving value”

34. Questions?

35. Resources
Learn More
● Links
○ http://kubernetes.io
○ Kubernetes documentation
○ Kubernetes user guide and troubleshooting guide
○ Kubernetes Github Repository
● Videos:
○ Kubernetes Cluster Federation Sneak Peek: https://www.youtube.com/watch?v=86jZdmAjWns
Ask Questions
● Kubernetes on Stackoverflow: http://stackoverflow.com/questions/tagged/kubernetes
Keep in Touch
● @kubernetesio: https://twitter.com/kubernetesio
● #kubernetes-users on Slack: http://slack.kubernetes.io
● Kubernetes-dev Google Group: https://groups.google.com/forum/#!forum/kubernetes-dev

36. " LinkedIn
" Twitter: @demibenari
" Blog: http://
progexc.blogspot.com/
" demi.benari@gmail.com
" “Big Things” Community
Meetup, YouTube, Facebook, Twitter
" GDG Cloud