SlideShare a Scribd company logo
1 of 39
Hacker vs Company,
Cyber Security Automated with
Kubernetes
Demi Ben-Ari – VP R&D @ Panorays
About Me
Demi Ben-Ari, Co-Founder & VP R&D @ Panorays
● Google Developer Expert
● Co-Founder of Communities:
○ “Big Things” - Big Data, Data Science, DevOps
○ Google Developer Group Cloud
○ Ofek Alumni Association
In the Past:
● Sr. Data Engineer - Windward
● Team Leader & Sr. Java Software Engineer,
Missile defence and Alert System - “Ofek” – IAF
Some important things
● What I’m not: Docker / Kubernetes / Security Expert
● What you won’t be after this talk: Docker / Kubernetes / Security Expert
● What you will be after this talk?
● Happier people (Because I’ve stopped talking)
● You’ll know what was our problem and our way of solution
● You’ll know where to search and learn more things
● The answer to the “What’s the meaning of life?” (42)
What is (Cyber) security?
https://www.pinterest.com/mechlite/security-fails/ http://www.techeblog.com/index.php/tech-gadget/funny-security-cameras https://www.pinterest.com/pin/449585975275324132/
What is “Security”?
● Security is a Technology
● Security is an Action
● Security is a Process
● Risk Assessment Methods
● https://www.iso.org/standard/56742.html
● https://www.securityforum.org/
Resources
Security
VS
Assets, Security, Threats, Adversaries
Risk = (Vulnerabilities X Threats X Consequences)
PrivacyContent
Identity
Files
Assets
Tor VPN
HTTPS
Encryption
2FA
HTTP Filter
Patching
WAFFirewall
SSL / TLS
SSH
Security Threats Adversaries
Vishing
Phishsing
Spying
Adware
Backdoors
Adware
Exploit Kits
Spyware
Viruses
Malware
Mass Surveillance
Spies Nation-States
Hackers
Hackers Groups
Crackers
Colleagues
Cyber Criminals
Law Enforcement
Ex-Partner
Governments
Pseudo
Anonymity
Anonymity
What do we do at Panorays?
How Do Hackers Get to Third Parties?
Supplier
Employees
IT & Network Application
- Confidential -12p.
52% by the
Human Factor
The Problem
What’s the hardest problem in Software Engineering?
Naming Things
What’s the biggest problem in Software Engineering?
● Naming Things
https://www.pinterest.com/pin/52424783138601042
/
Step #1 - Appoint a CNO
● Chief Naming Officer - your go to guy for all of the hardest problems
Step #2 - Define the problem and abstractions
● Parallelizm happening in the manner of a company (VMs being launched).
● Scan and evaluation process is not transparent.
● Server utilization is low.
● Had to build an internal orchestration system via Cron & Bash.
● (Think how fun is that…)
● How do you monitor all of this?
● Need to control it all via an easy API
Step #3 – Understand the people involved
● We had software engineers and security researchers on the team
● How do you integrate Dev – Sec – Ops?
● Creating a CI / CD pipeline?
● How do you make not software engineers deploy code to production?
https://www.coscale.com/hs-fs/hubfs/Blog_Pictures/2016_06/monolithic_vs_microservices.jpg
We’ve created a “Microlith”
In the beginning...
#!/bin/bash
In the beginning...
In the beginning...
1st Job
2nd Job
n-1 Job
nth Job
.
.
.
The Transporter
a Dynamic Workflow Engine
Built for Running Kubernetes Jobs According to a Predefined Workflow.
The Transporter
● Flexible and Efficient
● Parallel
● Automated
{ }
A bit about Kubernetes
● Greek for “Helmsman”; also the root of the
words “governor” and “cybernetic”.
● Manages container clusters
● Inspired and informed by Google’s
experience and an internal system (Borg)
● Supports multiple cloud and bare-metal
environments
● 100% Open source, written in Go
● Manage applications, not Machines
Cattle vs. Pets
CattlePet
• Has a name
• Is unique or rare
• Personal Attention
• If it gets ill, you make it better
• Has a number
• One is much like any other
• Run as a group
• If it gets ill, you make hamburgers
Kubernetes Terminology
● Deployment
● Service
● ReplicaSet
● Pod
● Volume
● Label
● Selector
● ConfigMap
● Secret
● DaemonSet
● Stateful Set
● Job
● Liveness Probe
● Readiness Probe
● Jobs
● Phases
● Workflows
The Deal is The Deal
discover
map
fetch
rate
saveaggregate
detect
Parallel
Sequential Sequential
Never Make a Promise You Can’t Keep
● Retries
● Schedule
● Timeouts
The Transporter
Q1
Worker1 ... Workern
Task1
...
Taskn
Q2
Almost Never Make a Promise You Can’t Keep
● Failure
1
● Notifications
No Names
● UUID
(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')
Max Chars: 63
CompanyName JobName WorkflowId● Labels, Labels everywhere
website__bot
So What’s On Our Cluster?
● The Transporter Service
● Workers Deployments
● Redis
● KubernetesJobs triggered by the
transporter
What’s Next?
● Workflows Monitoring (Grafana and Dash dashboards)
● ConfigMap for Versions
● Asset level Parallelization~O(n_assets * jobslowest) ~O(jobslowest)x100-x1,000,000
Conclusions
● If you have a possibility -> Don’t implement distributed systems
● Kubernetes is a great container orchestration tool
● Installing it on bare metal is not that fun - but also possible
● “Perfect” is the enemy of “Working” / “Giving Value”
● Making non developers deploy to production seamlessly
Questions
Thank You

More Related Content

What's hot

Dev secops opsec, devsec, devops ?
Dev secops opsec, devsec, devops ?Dev secops opsec, devsec, devops ?
Dev secops opsec, devsec, devops ?Kris Buytaert
 
Devops is a Security Requirement
Devops is a Security RequirementDevops is a Security Requirement
Devops is a Security RequirementKris Buytaert
 
Is there a Future for devops ?
Is there a Future for devops   ? Is there a Future for devops   ?
Is there a Future for devops ? Kris Buytaert
 
Devopsdays Amsterdam 2017 Keynote, looking back at 5 years of AMS
Devopsdays Amsterdam 2017 Keynote, looking back at 5 years of AMSDevopsdays Amsterdam 2017 Keynote, looking back at 5 years of AMS
Devopsdays Amsterdam 2017 Keynote, looking back at 5 years of AMSKris Buytaert
 
ADDO 2019: Looking back at over 10 years of Devops
ADDO 2019:    Looking back at over 10 years of DevopsADDO 2019:    Looking back at over 10 years of Devops
ADDO 2019: Looking back at over 10 years of DevopsKris Buytaert
 
Devops is dead, Long Live Devops
Devops is dead, Long Live DevopsDevops is dead, Long Live Devops
Devops is dead, Long Live DevopsKris Buytaert
 
Is there a future for devops ?
Is there a future for devops ?Is there a future for devops ?
Is there a future for devops ?Kris Buytaert
 
Nightmare on Docker street
Nightmare on Docker streetNightmare on Docker street
Nightmare on Docker streetKris Buytaert
 
Run stuff, Deploy Stuff
Run stuff, Deploy StuffRun stuff, Deploy Stuff
Run stuff, Deploy StuffKris Buytaert
 
From MonitoringSucks to Monitoring Love , 2016 Edition
From MonitoringSucks to Monitoring Love , 2016 EditionFrom MonitoringSucks to Monitoring Love , 2016 Edition
From MonitoringSucks to Monitoring Love , 2016 EditionKris Buytaert
 
Pipeline as code for your infrastructure as Code
Pipeline as code for your infrastructure as CodePipeline as code for your infrastructure as Code
Pipeline as code for your infrastructure as CodeKris Buytaert
 
Looking back at 7.5 years of Devopsdays , DOd PDX
Looking back at 7.5 years of Devopsdays , DOd PDXLooking back at 7.5 years of Devopsdays , DOd PDX
Looking back at 7.5 years of Devopsdays , DOd PDXKris Buytaert
 
The Return of the Dull Stack Engineer
The Return of the Dull Stack EngineerThe Return of the Dull Stack Engineer
The Return of the Dull Stack EngineerKris Buytaert
 
Moby is killing your devops efforts
Moby is killing your devops effortsMoby is killing your devops efforts
Moby is killing your devops effortsKris Buytaert
 
Looking back at 5 years of #cfgmgmtcamp
Looking back at 5 years of #cfgmgmtcampLooking back at 5 years of #cfgmgmtcamp
Looking back at 5 years of #cfgmgmtcampKris Buytaert
 
Elite Bug Squashing
Elite Bug SquashingElite Bug Squashing
Elite Bug SquashingTony Brown
 

What's hot (20)

Dev secops opsec, devsec, devops ?
Dev secops opsec, devsec, devops ?Dev secops opsec, devsec, devops ?
Dev secops opsec, devsec, devops ?
 
Devops is a Security Requirement
Devops is a Security RequirementDevops is a Security Requirement
Devops is a Security Requirement
 
Is there a Future for devops ?
Is there a Future for devops   ? Is there a Future for devops   ?
Is there a Future for devops ?
 
Devopsdays Amsterdam 2017 Keynote, looking back at 5 years of AMS
Devopsdays Amsterdam 2017 Keynote, looking back at 5 years of AMSDevopsdays Amsterdam 2017 Keynote, looking back at 5 years of AMS
Devopsdays Amsterdam 2017 Keynote, looking back at 5 years of AMS
 
ADDO 2019: Looking back at over 10 years of Devops
ADDO 2019:    Looking back at over 10 years of DevopsADDO 2019:    Looking back at over 10 years of Devops
ADDO 2019: Looking back at over 10 years of Devops
 
Devops is dead, Long Live Devops
Devops is dead, Long Live DevopsDevops is dead, Long Live Devops
Devops is dead, Long Live Devops
 
Is there a future for devops ?
Is there a future for devops ?Is there a future for devops ?
Is there a future for devops ?
 
Nightmare on Docker street
Nightmare on Docker streetNightmare on Docker street
Nightmare on Docker street
 
Pipeline as Code
Pipeline as CodePipeline as Code
Pipeline as Code
 
Run stuff, Deploy Stuff
Run stuff, Deploy StuffRun stuff, Deploy Stuff
Run stuff, Deploy Stuff
 
From MonitoringSucks to Monitoring Love , 2016 Edition
From MonitoringSucks to Monitoring Love , 2016 EditionFrom MonitoringSucks to Monitoring Love , 2016 Edition
From MonitoringSucks to Monitoring Love , 2016 Edition
 
Pipeline as code for your infrastructure as Code
Pipeline as code for your infrastructure as CodePipeline as code for your infrastructure as Code
Pipeline as code for your infrastructure as Code
 
devops is a reorg
devops is a reorgdevops is a reorg
devops is a reorg
 
Looking back at 7.5 years of Devopsdays , DOd PDX
Looking back at 7.5 years of Devopsdays , DOd PDXLooking back at 7.5 years of Devopsdays , DOd PDX
Looking back at 7.5 years of Devopsdays , DOd PDX
 
The Return of the Dull Stack Engineer
The Return of the Dull Stack EngineerThe Return of the Dull Stack Engineer
The Return of the Dull Stack Engineer
 
Dod is not done
Dod is not doneDod is not done
Dod is not done
 
Moby is killing your devops efforts
Moby is killing your devops effortsMoby is killing your devops efforts
Moby is killing your devops efforts
 
Sprint 0 projects
Sprint 0 projectsSprint 0 projects
Sprint 0 projects
 
Looking back at 5 years of #cfgmgmtcamp
Looking back at 5 years of #cfgmgmtcampLooking back at 5 years of #cfgmgmtcamp
Looking back at 5 years of #cfgmgmtcamp
 
Elite Bug Squashing
Elite Bug SquashingElite Bug Squashing
Elite Bug Squashing
 

Similar to Hacker vs company, Cloud Cyber Security Automated with Kubernetes - Demi Ben-Ari at Panorays

Hacking for fun & profit - The Kubernetes Way - Demi Ben-Ari - Panorays
Hacking for fun & profit - The Kubernetes Way - Demi Ben-Ari - PanoraysHacking for fun & profit - The Kubernetes Way - Demi Ben-Ari - Panorays
Hacking for fun & profit - The Kubernetes Way - Demi Ben-Ari - PanoraysDemi Ben-Ari
 
Thinking DevOps in the Era of the Cloud - Demi Ben-Ari
Thinking DevOps in the Era of the Cloud - Demi Ben-AriThinking DevOps in the Era of the Cloud - Demi Ben-Ari
Thinking DevOps in the Era of the Cloud - Demi Ben-AriDemi Ben-Ari
 
Kubernetes, Toolbox to fail or succeed for beginners - Demi Ben-Ari, VP R&D @...
Kubernetes, Toolbox to fail or succeed for beginners - Demi Ben-Ari, VP R&D @...Kubernetes, Toolbox to fail or succeed for beginners - Demi Ben-Ari, VP R&D @...
Kubernetes, Toolbox to fail or succeed for beginners - Demi Ben-Ari, VP R&D @...Demi Ben-Ari
 
Understanding and implementing website security
Understanding and implementing website securityUnderstanding and implementing website security
Understanding and implementing website securityDrew Gorton
 
Devops, Secops, Opsec, DevSec *ops *.* ?
Devops, Secops, Opsec, DevSec *ops *.* ?Devops, Secops, Opsec, DevSec *ops *.* ?
Devops, Secops, Opsec, DevSec *ops *.* ?Kris Buytaert
 
Devops Devops Devops, at Froscon
Devops Devops Devops, at FrosconDevops Devops Devops, at Froscon
Devops Devops Devops, at FrosconKris Buytaert
 
Moving to tdd bdd
Moving to tdd bddMoving to tdd bdd
Moving to tdd bddKim Carter
 
Pentester++
Pentester++Pentester++
Pentester++CTruncer
 
The Final Frontier, Automating Dynamic Security Testing
The Final Frontier, Automating Dynamic Security TestingThe Final Frontier, Automating Dynamic Security Testing
The Final Frontier, Automating Dynamic Security TestingMatt Tesauro
 
Drupal and Devops , the Survey Results
Drupal and Devops , the Survey ResultsDrupal and Devops , the Survey Results
Drupal and Devops , the Survey ResultsKris Buytaert
 
Services, tools & practices for a software house
Services, tools & practices for a software houseServices, tools & practices for a software house
Services, tools & practices for a software houseParis Apostolopoulos
 
Devops, the future is here, it's just not evenly distributed yet.
Devops, the future is here, it's just not evenly distributed yet.Devops, the future is here, it's just not evenly distributed yet.
Devops, the future is here, it's just not evenly distributed yet.Kris Buytaert
 
Break Up the Monolith- Testing Microservices by Marcus Merrell
Break Up the Monolith- Testing Microservices by Marcus MerrellBreak Up the Monolith- Testing Microservices by Marcus Merrell
Break Up the Monolith- Testing Microservices by Marcus MerrellSauce Labs
 
Tooling Matters - Development tools
Tooling Matters - Development toolsTooling Matters - Development tools
Tooling Matters - Development toolsSimon Dittlmann
 
Continuous Infrastructure First Ignite Edition
Continuous Infrastructure First  Ignite EditionContinuous Infrastructure First  Ignite Edition
Continuous Infrastructure First Ignite EditionKris Buytaert
 
Devops, the future is here it's not evenly distributed yet
Devops, the future is here it's not evenly distributed yetDevops, the future is here it's not evenly distributed yet
Devops, the future is here it's not evenly distributed yetKris Buytaert
 
Pen Testing Development
Pen Testing DevelopmentPen Testing Development
Pen Testing DevelopmentCTruncer
 
Google Associate Cloud Engineer Certification Tips
Google Associate Cloud Engineer Certification TipsGoogle Associate Cloud Engineer Certification Tips
Google Associate Cloud Engineer Certification TipsDaniel Zivkovic
 
Using and Building Open Source in Google Corporate Engineering - Justin McWil...
Using and Building Open Source in Google Corporate Engineering - Justin McWil...Using and Building Open Source in Google Corporate Engineering - Justin McWil...
Using and Building Open Source in Google Corporate Engineering - Justin McWil...OSCON Byrum
 

Similar to Hacker vs company, Cloud Cyber Security Automated with Kubernetes - Demi Ben-Ari at Panorays (20)

Hacking for fun & profit - The Kubernetes Way - Demi Ben-Ari - Panorays
Hacking for fun & profit - The Kubernetes Way - Demi Ben-Ari - PanoraysHacking for fun & profit - The Kubernetes Way - Demi Ben-Ari - Panorays
Hacking for fun & profit - The Kubernetes Way - Demi Ben-Ari - Panorays
 
Thinking DevOps in the Era of the Cloud - Demi Ben-Ari
Thinking DevOps in the Era of the Cloud - Demi Ben-AriThinking DevOps in the Era of the Cloud - Demi Ben-Ari
Thinking DevOps in the Era of the Cloud - Demi Ben-Ari
 
Kubernetes, Toolbox to fail or succeed for beginners - Demi Ben-Ari, VP R&D @...
Kubernetes, Toolbox to fail or succeed for beginners - Demi Ben-Ari, VP R&D @...Kubernetes, Toolbox to fail or succeed for beginners - Demi Ben-Ari, VP R&D @...
Kubernetes, Toolbox to fail or succeed for beginners - Demi Ben-Ari, VP R&D @...
 
Understanding and implementing website security
Understanding and implementing website securityUnderstanding and implementing website security
Understanding and implementing website security
 
Devops For Drupal
Devops  For DrupalDevops  For Drupal
Devops For Drupal
 
Devops, Secops, Opsec, DevSec *ops *.* ?
Devops, Secops, Opsec, DevSec *ops *.* ?Devops, Secops, Opsec, DevSec *ops *.* ?
Devops, Secops, Opsec, DevSec *ops *.* ?
 
Devops Devops Devops, at Froscon
Devops Devops Devops, at FrosconDevops Devops Devops, at Froscon
Devops Devops Devops, at Froscon
 
Moving to tdd bdd
Moving to tdd bddMoving to tdd bdd
Moving to tdd bdd
 
Pentester++
Pentester++Pentester++
Pentester++
 
The Final Frontier, Automating Dynamic Security Testing
The Final Frontier, Automating Dynamic Security TestingThe Final Frontier, Automating Dynamic Security Testing
The Final Frontier, Automating Dynamic Security Testing
 
Drupal and Devops , the Survey Results
Drupal and Devops , the Survey ResultsDrupal and Devops , the Survey Results
Drupal and Devops , the Survey Results
 
Services, tools & practices for a software house
Services, tools & practices for a software houseServices, tools & practices for a software house
Services, tools & practices for a software house
 
Devops, the future is here, it's just not evenly distributed yet.
Devops, the future is here, it's just not evenly distributed yet.Devops, the future is here, it's just not evenly distributed yet.
Devops, the future is here, it's just not evenly distributed yet.
 
Break Up the Monolith- Testing Microservices by Marcus Merrell
Break Up the Monolith- Testing Microservices by Marcus MerrellBreak Up the Monolith- Testing Microservices by Marcus Merrell
Break Up the Monolith- Testing Microservices by Marcus Merrell
 
Tooling Matters - Development tools
Tooling Matters - Development toolsTooling Matters - Development tools
Tooling Matters - Development tools
 
Continuous Infrastructure First Ignite Edition
Continuous Infrastructure First  Ignite EditionContinuous Infrastructure First  Ignite Edition
Continuous Infrastructure First Ignite Edition
 
Devops, the future is here it's not evenly distributed yet
Devops, the future is here it's not evenly distributed yetDevops, the future is here it's not evenly distributed yet
Devops, the future is here it's not evenly distributed yet
 
Pen Testing Development
Pen Testing DevelopmentPen Testing Development
Pen Testing Development
 
Google Associate Cloud Engineer Certification Tips
Google Associate Cloud Engineer Certification TipsGoogle Associate Cloud Engineer Certification Tips
Google Associate Cloud Engineer Certification Tips
 
Using and Building Open Source in Google Corporate Engineering - Justin McWil...
Using and Building Open Source in Google Corporate Engineering - Justin McWil...Using and Building Open Source in Google Corporate Engineering - Justin McWil...
Using and Building Open Source in Google Corporate Engineering - Justin McWil...
 

More from Demi Ben-Ari

CTO Management Tool Box - Demi Ben-Ari at Panorays
CTO Management Tool Box - Demi Ben-Ari at PanoraysCTO Management Tool Box - Demi Ben-Ari at Panorays
CTO Management Tool Box - Demi Ben-Ari at PanoraysDemi Ben-Ari
 
CTO Management ToolBox - Demi Ben-Ari -- Panorays
CTO Management ToolBox - Demi Ben-Ari -- PanoraysCTO Management ToolBox - Demi Ben-Ari -- Panorays
CTO Management ToolBox - Demi Ben-Ari -- PanoraysDemi Ben-Ari
 
All I Wanted Is to Found a Startup - Demi Ben-Ari - Panorays
All I Wanted Is to Found a Startup - Demi Ben-Ari - PanoraysAll I Wanted Is to Found a Startup - Demi Ben-Ari - Panorays
All I Wanted Is to Found a Startup - Demi Ben-Ari - PanoraysDemi Ben-Ari
 
Community, Unifying the Geeks to Create Value - Demi Ben-Ari
Community, Unifying the Geeks to Create Value - Demi Ben-AriCommunity, Unifying the Geeks to Create Value - Demi Ben-Ari
Community, Unifying the Geeks to Create Value - Demi Ben-AriDemi Ben-Ari
 
Apache Spark 101 - Demi Ben-Ari - Panorays
Apache Spark 101 - Demi Ben-Ari - PanoraysApache Spark 101 - Demi Ben-Ari - Panorays
Apache Spark 101 - Demi Ben-Ari - PanoraysDemi Ben-Ari
 
Know the Startup World - Demi Ben-Ari - Ofek Alumni
Know the Startup World - Demi Ben-Ari - Ofek AlumniKnow the Startup World - Demi Ben-Ari - Ofek Alumni
Know the Startup World - Demi Ben-Ari - Ofek AlumniDemi Ben-Ari
 
Big Data made easy in the era of the Cloud - Demi Ben-Ari
Big Data made easy in the era of the Cloud - Demi Ben-AriBig Data made easy in the era of the Cloud - Demi Ben-Ari
Big Data made easy in the era of the Cloud - Demi Ben-AriDemi Ben-Ari
 
Know the Startup World - Demi Ben Ari - Ofek Alumni
Know the Startup World - Demi Ben Ari - Ofek AlumniKnow the Startup World - Demi Ben Ari - Ofek Alumni
Know the Startup World - Demi Ben Ari - Ofek AlumniDemi Ben-Ari
 
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017 - D...
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017 - D...Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017 - D...
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017 - D...Demi Ben-Ari
 
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Berlin 2017
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Berlin 2017Monitoring Big Data Systems Done "The Simple Way" - Codemotion Berlin 2017
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Berlin 2017Demi Ben-Ari
 
Monitoring Big Data Systems "Done the simple way" - Demi Ben-Ari - Codemotion...
Monitoring Big Data Systems "Done the simple way" - Demi Ben-Ari - Codemotion...Monitoring Big Data Systems "Done the simple way" - Demi Ben-Ari - Codemotion...
Monitoring Big Data Systems "Done the simple way" - Demi Ben-Ari - Codemotion...Demi Ben-Ari
 
Thinking DevOps in the era of the Cloud - Demi Ben-Ari
Thinking DevOps in the era of the Cloud - Demi Ben-AriThinking DevOps in the era of the Cloud - Demi Ben-Ari
Thinking DevOps in the era of the Cloud - Demi Ben-AriDemi Ben-Ari
 
Bootstrapping a Tech Community - Demi Ben-Ari
Bootstrapping a Tech Community - Demi Ben-AriBootstrapping a Tech Community - Demi Ben-Ari
Bootstrapping a Tech Community - Demi Ben-AriDemi Ben-Ari
 
Apache Spark 101 - Demi Ben-Ari
Apache Spark 101 - Demi Ben-AriApache Spark 101 - Demi Ben-Ari
Apache Spark 101 - Demi Ben-AriDemi Ben-Ari
 
Quick dive into the big data pool without drowning - Demi Ben-Ari @ Panorays
Quick dive into the big data pool without drowning - Demi Ben-Ari @ PanoraysQuick dive into the big data pool without drowning - Demi Ben-Ari @ Panorays
Quick dive into the big data pool without drowning - Demi Ben-Ari @ PanoraysDemi Ben-Ari
 
Monitoring Big Data Systems - "The Simple Way"
Monitoring Big Data Systems - "The Simple Way"Monitoring Big Data Systems - "The Simple Way"
Monitoring Big Data Systems - "The Simple Way"Demi Ben-Ari
 
Scala like distributed collections - dumping time-series data with apache spark
Scala like distributed collections - dumping time-series data with apache sparkScala like distributed collections - dumping time-series data with apache spark
Scala like distributed collections - dumping time-series data with apache sparkDemi Ben-Ari
 
S3 cassandra or outer space? dumping time series data using spark
S3 cassandra or outer space? dumping time series data using sparkS3 cassandra or outer space? dumping time series data using spark
S3 cassandra or outer space? dumping time series data using sparkDemi Ben-Ari
 
Spark 101 – First Steps To Distributed Computing - Demi Ben-Ari @ Ofek Alumni
Spark 101 – First Steps To Distributed Computing - Demi Ben-Ari @ Ofek AlumniSpark 101 – First Steps To Distributed Computing - Demi Ben-Ari @ Ofek Alumni
Spark 101 – First Steps To Distributed Computing - Demi Ben-Ari @ Ofek AlumniDemi Ben-Ari
 
Migrating Data Pipeline from MongoDB to Cassandra
Migrating Data Pipeline from MongoDB to CassandraMigrating Data Pipeline from MongoDB to Cassandra
Migrating Data Pipeline from MongoDB to CassandraDemi Ben-Ari
 

More from Demi Ben-Ari (20)

CTO Management Tool Box - Demi Ben-Ari at Panorays
CTO Management Tool Box - Demi Ben-Ari at PanoraysCTO Management Tool Box - Demi Ben-Ari at Panorays
CTO Management Tool Box - Demi Ben-Ari at Panorays
 
CTO Management ToolBox - Demi Ben-Ari -- Panorays
CTO Management ToolBox - Demi Ben-Ari -- PanoraysCTO Management ToolBox - Demi Ben-Ari -- Panorays
CTO Management ToolBox - Demi Ben-Ari -- Panorays
 
All I Wanted Is to Found a Startup - Demi Ben-Ari - Panorays
All I Wanted Is to Found a Startup - Demi Ben-Ari - PanoraysAll I Wanted Is to Found a Startup - Demi Ben-Ari - Panorays
All I Wanted Is to Found a Startup - Demi Ben-Ari - Panorays
 
Community, Unifying the Geeks to Create Value - Demi Ben-Ari
Community, Unifying the Geeks to Create Value - Demi Ben-AriCommunity, Unifying the Geeks to Create Value - Demi Ben-Ari
Community, Unifying the Geeks to Create Value - Demi Ben-Ari
 
Apache Spark 101 - Demi Ben-Ari - Panorays
Apache Spark 101 - Demi Ben-Ari - PanoraysApache Spark 101 - Demi Ben-Ari - Panorays
Apache Spark 101 - Demi Ben-Ari - Panorays
 
Know the Startup World - Demi Ben-Ari - Ofek Alumni
Know the Startup World - Demi Ben-Ari - Ofek AlumniKnow the Startup World - Demi Ben-Ari - Ofek Alumni
Know the Startup World - Demi Ben-Ari - Ofek Alumni
 
Big Data made easy in the era of the Cloud - Demi Ben-Ari
Big Data made easy in the era of the Cloud - Demi Ben-AriBig Data made easy in the era of the Cloud - Demi Ben-Ari
Big Data made easy in the era of the Cloud - Demi Ben-Ari
 
Know the Startup World - Demi Ben Ari - Ofek Alumni
Know the Startup World - Demi Ben Ari - Ofek AlumniKnow the Startup World - Demi Ben Ari - Ofek Alumni
Know the Startup World - Demi Ben Ari - Ofek Alumni
 
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017 - D...
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017 - D...Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017 - D...
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017 - D...
 
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Berlin 2017
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Berlin 2017Monitoring Big Data Systems Done "The Simple Way" - Codemotion Berlin 2017
Monitoring Big Data Systems Done "The Simple Way" - Codemotion Berlin 2017
 
Monitoring Big Data Systems "Done the simple way" - Demi Ben-Ari - Codemotion...
Monitoring Big Data Systems "Done the simple way" - Demi Ben-Ari - Codemotion...Monitoring Big Data Systems "Done the simple way" - Demi Ben-Ari - Codemotion...
Monitoring Big Data Systems "Done the simple way" - Demi Ben-Ari - Codemotion...
 
Thinking DevOps in the era of the Cloud - Demi Ben-Ari
Thinking DevOps in the era of the Cloud - Demi Ben-AriThinking DevOps in the era of the Cloud - Demi Ben-Ari
Thinking DevOps in the era of the Cloud - Demi Ben-Ari
 
Bootstrapping a Tech Community - Demi Ben-Ari
Bootstrapping a Tech Community - Demi Ben-AriBootstrapping a Tech Community - Demi Ben-Ari
Bootstrapping a Tech Community - Demi Ben-Ari
 
Apache Spark 101 - Demi Ben-Ari
Apache Spark 101 - Demi Ben-AriApache Spark 101 - Demi Ben-Ari
Apache Spark 101 - Demi Ben-Ari
 
Quick dive into the big data pool without drowning - Demi Ben-Ari @ Panorays
Quick dive into the big data pool without drowning - Demi Ben-Ari @ PanoraysQuick dive into the big data pool without drowning - Demi Ben-Ari @ Panorays
Quick dive into the big data pool without drowning - Demi Ben-Ari @ Panorays
 
Monitoring Big Data Systems - "The Simple Way"
Monitoring Big Data Systems - "The Simple Way"Monitoring Big Data Systems - "The Simple Way"
Monitoring Big Data Systems - "The Simple Way"
 
Scala like distributed collections - dumping time-series data with apache spark
Scala like distributed collections - dumping time-series data with apache sparkScala like distributed collections - dumping time-series data with apache spark
Scala like distributed collections - dumping time-series data with apache spark
 
S3 cassandra or outer space? dumping time series data using spark
S3 cassandra or outer space? dumping time series data using sparkS3 cassandra or outer space? dumping time series data using spark
S3 cassandra or outer space? dumping time series data using spark
 
Spark 101 – First Steps To Distributed Computing - Demi Ben-Ari @ Ofek Alumni
Spark 101 – First Steps To Distributed Computing - Demi Ben-Ari @ Ofek AlumniSpark 101 – First Steps To Distributed Computing - Demi Ben-Ari @ Ofek Alumni
Spark 101 – First Steps To Distributed Computing - Demi Ben-Ari @ Ofek Alumni
 
Migrating Data Pipeline from MongoDB to Cassandra
Migrating Data Pipeline from MongoDB to CassandraMigrating Data Pipeline from MongoDB to Cassandra
Migrating Data Pipeline from MongoDB to Cassandra
 

Recently uploaded

Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...
Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...
Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...OnePlan Solutions
 
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Matt Ray
 
What is Advanced Excel and what are some best practices for designing and cre...
What is Advanced Excel and what are some best practices for designing and cre...What is Advanced Excel and what are some best practices for designing and cre...
What is Advanced Excel and what are some best practices for designing and cre...Technogeeks
 
Unveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML DiagramsUnveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML DiagramsAhmed Mohamed
 
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Angel Borroy López
 
Software Project Health Check: Best Practices and Techniques for Your Product...
Software Project Health Check: Best Practices and Techniques for Your Product...Software Project Health Check: Best Practices and Techniques for Your Product...
Software Project Health Check: Best Practices and Techniques for Your Product...Velvetech LLC
 
PREDICTING RIVER WATER QUALITY ppt presentation
PREDICTING  RIVER  WATER QUALITY  ppt presentationPREDICTING  RIVER  WATER QUALITY  ppt presentation
PREDICTING RIVER WATER QUALITY ppt presentationvaddepallysandeep122
 
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...OnePlan Solutions
 
英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作qr0udbr0
 
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024StefanoLambiase
 
Recruitment Management Software Benefits (Infographic)
Recruitment Management Software Benefits (Infographic)Recruitment Management Software Benefits (Infographic)
Recruitment Management Software Benefits (Infographic)Hr365.us smith
 
Odoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 EnterpriseOdoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 Enterprisepreethippts
 
How to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationHow to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationBradBedford3
 
React Server Component in Next.js by Hanief Utama
React Server Component in Next.js by Hanief UtamaReact Server Component in Next.js by Hanief Utama
React Server Component in Next.js by Hanief UtamaHanief Utama
 
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanySuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanyChristoph Pohl
 
Sending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdfSending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdf31events.com
 
Precise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive GoalPrecise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive GoalLionel Briand
 
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfGOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfAlina Yurenko
 

Recently uploaded (20)

Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...
Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...
Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...
 
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort ServiceHot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
 
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
 
What is Advanced Excel and what are some best practices for designing and cre...
What is Advanced Excel and what are some best practices for designing and cre...What is Advanced Excel and what are some best practices for designing and cre...
What is Advanced Excel and what are some best practices for designing and cre...
 
Unveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML DiagramsUnveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML Diagrams
 
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
 
Software Project Health Check: Best Practices and Techniques for Your Product...
Software Project Health Check: Best Practices and Techniques for Your Product...Software Project Health Check: Best Practices and Techniques for Your Product...
Software Project Health Check: Best Practices and Techniques for Your Product...
 
PREDICTING RIVER WATER QUALITY ppt presentation
PREDICTING  RIVER  WATER QUALITY  ppt presentationPREDICTING  RIVER  WATER QUALITY  ppt presentation
PREDICTING RIVER WATER QUALITY ppt presentation
 
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
 
英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作
 
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
 
Advantages of Odoo ERP 17 for Your Business
Advantages of Odoo ERP 17 for Your BusinessAdvantages of Odoo ERP 17 for Your Business
Advantages of Odoo ERP 17 for Your Business
 
Recruitment Management Software Benefits (Infographic)
Recruitment Management Software Benefits (Infographic)Recruitment Management Software Benefits (Infographic)
Recruitment Management Software Benefits (Infographic)
 
Odoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 EnterpriseOdoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 Enterprise
 
How to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationHow to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion Application
 
React Server Component in Next.js by Hanief Utama
React Server Component in Next.js by Hanief UtamaReact Server Component in Next.js by Hanief Utama
React Server Component in Next.js by Hanief Utama
 
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanySuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
 
Sending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdfSending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdf
 
Precise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive GoalPrecise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive Goal
 
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfGOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
 

Hacker vs company, Cloud Cyber Security Automated with Kubernetes - Demi Ben-Ari at Panorays

  • 1. Hacker vs Company, Cyber Security Automated with Kubernetes Demi Ben-Ari – VP R&D @ Panorays
  • 2.
  • 3. About Me Demi Ben-Ari, Co-Founder & VP R&D @ Panorays ● Google Developer Expert ● Co-Founder of Communities: ○ “Big Things” - Big Data, Data Science, DevOps ○ Google Developer Group Cloud ○ Ofek Alumni Association In the Past: ● Sr. Data Engineer - Windward ● Team Leader & Sr. Java Software Engineer, Missile defence and Alert System - “Ofek” – IAF
  • 4. Some important things ● What I’m not: Docker / Kubernetes / Security Expert ● What you won’t be after this talk: Docker / Kubernetes / Security Expert ● What you will be after this talk? ● Happier people (Because I’ve stopped talking) ● You’ll know what was our problem and our way of solution ● You’ll know where to search and learn more things ● The answer to the “What’s the meaning of life?” (42)
  • 5. What is (Cyber) security? https://www.pinterest.com/mechlite/security-fails/ http://www.techeblog.com/index.php/tech-gadget/funny-security-cameras https://www.pinterest.com/pin/449585975275324132/
  • 6. What is “Security”? ● Security is a Technology ● Security is an Action ● Security is a Process ● Risk Assessment Methods ● https://www.iso.org/standard/56742.html ● https://www.securityforum.org/
  • 8.
  • 9. Assets, Security, Threats, Adversaries Risk = (Vulnerabilities X Threats X Consequences) PrivacyContent Identity Files Assets Tor VPN HTTPS Encryption 2FA HTTP Filter Patching WAFFirewall SSL / TLS SSH Security Threats Adversaries Vishing Phishsing Spying Adware Backdoors Adware Exploit Kits Spyware Viruses Malware Mass Surveillance Spies Nation-States Hackers Hackers Groups Crackers Colleagues Cyber Criminals Law Enforcement Ex-Partner Governments Pseudo Anonymity Anonymity
  • 10. What do we do at Panorays?
  • 11. How Do Hackers Get to Third Parties? Supplier Employees IT & Network Application - Confidential -12p. 52% by the Human Factor
  • 13. What’s the hardest problem in Software Engineering?
  • 15. What’s the biggest problem in Software Engineering? ● Naming Things https://www.pinterest.com/pin/52424783138601042 /
  • 16.
  • 17. Step #1 - Appoint a CNO ● Chief Naming Officer - your go to guy for all of the hardest problems
  • 18. Step #2 - Define the problem and abstractions ● Parallelizm happening in the manner of a company (VMs being launched). ● Scan and evaluation process is not transparent. ● Server utilization is low. ● Had to build an internal orchestration system via Cron & Bash. ● (Think how fun is that…) ● How do you monitor all of this? ● Need to control it all via an easy API
  • 19. Step #3 – Understand the people involved ● We had software engineers and security researchers on the team ● How do you integrate Dev – Sec – Ops? ● Creating a CI / CD pipeline? ● How do you make not software engineers deploy code to production?
  • 21. We’ve created a “Microlith”
  • 24. In the beginning... 1st Job 2nd Job n-1 Job nth Job . . .
  • 25. The Transporter a Dynamic Workflow Engine Built for Running Kubernetes Jobs According to a Predefined Workflow.
  • 26. The Transporter ● Flexible and Efficient ● Parallel ● Automated { }
  • 27. A bit about Kubernetes ● Greek for “Helmsman”; also the root of the words “governor” and “cybernetic”. ● Manages container clusters ● Inspired and informed by Google’s experience and an internal system (Borg) ● Supports multiple cloud and bare-metal environments ● 100% Open source, written in Go ● Manage applications, not Machines
  • 28. Cattle vs. Pets CattlePet • Has a name • Is unique or rare • Personal Attention • If it gets ill, you make it better • Has a number • One is much like any other • Run as a group • If it gets ill, you make hamburgers
  • 29. Kubernetes Terminology ● Deployment ● Service ● ReplicaSet ● Pod ● Volume ● Label ● Selector ● ConfigMap ● Secret ● DaemonSet ● Stateful Set ● Job ● Liveness Probe ● Readiness Probe
  • 30.
  • 31. ● Jobs ● Phases ● Workflows The Deal is The Deal discover map fetch rate saveaggregate detect Parallel Sequential Sequential
  • 32. Never Make a Promise You Can’t Keep ● Retries ● Schedule ● Timeouts The Transporter Q1 Worker1 ... Workern Task1 ... Taskn Q2
  • 33. Almost Never Make a Promise You Can’t Keep ● Failure 1 ● Notifications
  • 34. No Names ● UUID (([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?') Max Chars: 63 CompanyName JobName WorkflowId● Labels, Labels everywhere website__bot
  • 35. So What’s On Our Cluster? ● The Transporter Service ● Workers Deployments ● Redis ● KubernetesJobs triggered by the transporter
  • 36. What’s Next? ● Workflows Monitoring (Grafana and Dash dashboards) ● ConfigMap for Versions ● Asset level Parallelization~O(n_assets * jobslowest) ~O(jobslowest)x100-x1,000,000
  • 37. Conclusions ● If you have a possibility -> Don’t implement distributed systems ● Kubernetes is a great container orchestration tool ● Installing it on bare metal is not that fun - but also possible ● “Perfect” is the enemy of “Working” / “Giving Value” ● Making non developers deploy to production seamlessly