Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
AWS AND SOFTSERVE
CONTAINERIZATION:
THE DEVOPS REVOLUTION
Tuesday, April 12, 2016
8 AM PST/4 PM BST/5 PM CEST
webinar webi...
SPEAKERS
Valentyn Kropov
Solutions Architect,
SoftServe
Craig Williams
Solutions Architect,
Amazon Web Services
webinar
WHY CONTAINERS ARE SUCH A BIG DEAL
webinar webinar@softserveinc.com
• Own Process Space
• Can run commands as root
• Can install ad-hoc packages
• Can run Services
• You can get a shell on i...
• It uses hosts kernel
• Can’t boot different OS
• Can’t have it’s own kernel modules
• From host OS perspective Container...
webinar
CONTAINERS vs VMs
• Resource Metering and Limiting
• CPU and CPUSet
• Memory
• Network
• Block I/O
• /dev/*
CGROUPS (CONTROL GROUPS):
webinar
• Provides containers with their own view of the system
• Limits what you can see (and use)
• Multiple namespaces: pid, ne...
• Create new container instantly instead of
copying whole system
• Storage keeps tracking of what has change
(AUFS, ZFS, e...
• LXC
• systemd-nspawn
• Docker Engine
• rkt/runC
• OpenVZ
• Jails (FreeBSD), Zones (Solaris)
CONTAINER RUNTIMES:
webinar
• Uses the same kernel features => Performance will be the same
• What matters is:
Design
Features
Ecosystem (e.g. 100.000...
THE STORY OF SUCCESS
webinar webinar@softserveinc.com
PROBLEM & OPPORTUNITY
• Rapid innovation in computing and application development services
• No single service is optimal ...
DATACENTER AND SOLUTION TODAY
VM7 VM8
VM4 VM5 VM6
VM1 VM2 VM3
VM1 VM2 VM2
Visualization Service
Data Ingestion
Service
Ana...
WHAT DO WE WANT TO DO?
Data
Ingestion
Service
Analytics
Service
Visualization
Service
….to maximize utilization
...to shar...
SHARED CLUSTER
AWS
VM1 VM2 VM3 VM4 VM5
webinar
WHAT IS IN IT FOR CUSTOMERS?
Maximize utilization
Deliver more services with smaller footprint
Shared clusters for all ser...
HOW DOES THIS WORK?
Mesos Master
Mesos Slave
Spark Task Executor Mesos Executor
Task #1 Task #2 ./python XYZ
Mesos Slave
D...
HOW DOES THIS WORK?
Mesos provides fine grained resource isolation
webinar
Mesos Slave Process
Spark Task Executor Mesos E...
HOW DOES THIS WORK?
Mesos provides scalability
Mesos Slave Process
./python XYZ
COMPUTE NODE
Python executor finished,
mor...
Mesos has no single point of failure Services keep running if VM fails!
webinar
HOW DOES THIS WORK?
Mesos MasterMesos Mast...
webinar
HOW DOES THIS WORK?
Mesos MasterMesos Master
Mesos Master
VM1 VM2 VM3 VM4 VM5
!
Master node can failover Services ...
HOW DOES THIS WORK?
Mesos Slave Process
./python XYZ
COMPUTE NODE
Python executor finished,
more available resources
more ...
Can deploy in many environments
Get orchestrated by Openstack, Ansible
(scripts), Cloudbreak
True Hybrid Cloud deployment:...
CONTAINERS:
Service Product
Cloud/Virtualization AWS/CIS/vSphere/Metacloud/UCS…
Provisioning Terraform
Automation Ansible
...
ISSUES
• Service Discovering
• Networking for Containers
• Persistent Storage
• Docker Performance
webinar
MORE DETAILS
https://mantl.io
webinar
Upcoming SlideShare
Loading in …5
×

Containerization: The DevOps Revolution

432 views

Published on

We are now witnessing a new wave of IT revolution and its effect is very similar to the Cloud and Virtualization revolutions that started in the last decade. This new wave, called Containerization, is related to technologies such as Docker and Kubernetes, which now fuel large scale solutions including Big Data and IoT.

Learn about:

- Typical DevOps challenges and modern solutions
- Using Docker as Amazon EC2 Container Service Evolution of Enterprise Architecture (Containers, IoT, Machine Learning and technologies of tomorrow)
- Business value of using advances DevOps technologies with real-life case study

Published in: Software
  • Be the first to comment

Containerization: The DevOps Revolution

  1. 1. AWS AND SOFTSERVE CONTAINERIZATION: THE DEVOPS REVOLUTION Tuesday, April 12, 2016 8 AM PST/4 PM BST/5 PM CEST webinar webinar@softserveinc.com
  2. 2. SPEAKERS Valentyn Kropov Solutions Architect, SoftServe Craig Williams Solutions Architect, Amazon Web Services webinar
  3. 3. WHY CONTAINERS ARE SUCH A BIG DEAL webinar webinar@softserveinc.com
  4. 4. • Own Process Space • Can run commands as root • Can install ad-hoc packages • Can run Services • You can get a shell on it (e.g. through SSH) CONTAINER IS LIKE A LIGHTWEIGHT VM: webinar
  5. 5. • It uses hosts kernel • Can’t boot different OS • Can’t have it’s own kernel modules • From host OS perspective Containers are the bunch of processes DIFFERENCES TO VM: webinar
  6. 6. webinar CONTAINERS vs VMs
  7. 7. • Resource Metering and Limiting • CPU and CPUSet • Memory • Network • Block I/O • /dev/* CGROUPS (CONTROL GROUPS): webinar
  8. 8. • Provides containers with their own view of the system • Limits what you can see (and use) • Multiple namespaces: pid, net, mnt, uts, ipc, user NAMESPACE: webinar
  9. 9. • Create new container instantly instead of copying whole system • Storage keeps tracking of what has change (AUFS, ZFS, etc) • Reduces footprint and overhead • Decreases boot time COPY-ON-WRITE STORAGE: webinar
  10. 10. • LXC • systemd-nspawn • Docker Engine • rkt/runC • OpenVZ • Jails (FreeBSD), Zones (Solaris) CONTAINER RUNTIMES: webinar
  11. 11. • Uses the same kernel features => Performance will be the same • What matters is: Design Features Ecosystem (e.g. 100.000+ apps in Docker Hub) Support WHAT’S THE DIFFERENCE BETWEEN THEM? webinar
  12. 12. THE STORY OF SUCCESS webinar webinar@softserveinc.com
  13. 13. PROBLEM & OPPORTUNITY • Rapid innovation in computing and application development services • No single service is optimal for all solutions • Customers want to run multiple services in a single cluster and run multiple clusters in Intercloud environment ...to maximize utilization ...to share data between services webinar
  14. 14. DATACENTER AND SOLUTION TODAY VM7 VM8 VM4 VM5 VM6 VM1 VM2 VM3 VM1 VM2 VM2 Visualization Service Data Ingestion Service Analytics Service • Configuration and management of 3 separate clusters • Resources stay idle if service is not active • Need to move data between clusters for each service webinar
  15. 15. WHAT DO WE WANT TO DO? Data Ingestion Service Analytics Service Visualization Service ….to maximize utilization ...to share data between services Shared cluster Multiple clusters webinar
  16. 16. SHARED CLUSTER AWS VM1 VM2 VM3 VM4 VM5 webinar
  17. 17. WHAT IS IN IT FOR CUSTOMERS? Maximize utilization Deliver more services with smaller footprint Shared clusters for all services Easier deployment and management with unified service platform Shared data between services Faster and more competitive services and solutions webinar
  18. 18. HOW DOES THIS WORK? Mesos Master Mesos Slave Spark Task Executor Mesos Executor Task #1 Task #2 ./python XYZ Mesos Slave Docker Executor Docker Executor java -jar XYZ.jar ./xyz Mesos Master Mesos Master Spark Service Scheduler Marathon Service Scheduler Zookeeper quorum webinar
  19. 19. HOW DOES THIS WORK? Mesos provides fine grained resource isolation webinar Mesos Slave Process Spark Task Executor Mesos Executor Task #1 Task #2 ./python XYZ COMPUTE NODE Executor Container (cgroups)
  20. 20. HOW DOES THIS WORK? Mesos provides scalability Mesos Slave Process ./python XYZ COMPUTE NODE Python executor finished, more available resources more Spark Container (cgroups) webinar Spark Task Executor Task #1 Task #2 Task #3 Task #4
  21. 21. Mesos has no single point of failure Services keep running if VM fails! webinar HOW DOES THIS WORK? Mesos MasterMesos Master Mesos Master VM1 VM2 VM3 VM4 VM5 !
  22. 22. webinar HOW DOES THIS WORK? Mesos MasterMesos Master Mesos Master VM1 VM2 VM3 VM4 VM5 ! Master node can failover Services keep running if Mesos Master fails!
  23. 23. HOW DOES THIS WORK? Mesos Slave Process ./python XYZ COMPUTE NODE Python executor finished, more available resources more Spark Container (cgroups) webinar Spark Task Executor Task #1 Task #2 Task #3 Task #4 ! Slave process can failover Tasks keep running if Mesos Slave Process fails!
  24. 24. Can deploy in many environments Get orchestrated by Openstack, Ansible (scripts), Cloudbreak True Hybrid Cloud deployment: AWS, CIS, UCS, vSphere, other AWS VM5VM1 VM2 VM3 VM4 Terraform REST API (policy, auto-scaling) REST API (direct provisioning) Scripted provisioning webinar HOW DOES THIS WORK?
  25. 25. CONTAINERS: Service Product Cloud/Virtualization AWS/CIS/vSphere/Metacloud/UCS… Provisioning Terraform Automation Ansible Clustering & Resource Management Mesos, Marathon, Docker Load Balancing Avi Networks ETL & Data Shaping StreamSets Log Data Gathering Logstash Metrics Gathering CollectD, Avi Networks Messaging Kafka, Solace Data Storing (Batch) HDFS Data Storing (OLTP/Real-time) Cassandra Data Storing (Indexing) Elastic search Data Processing Apache Spark Visualization Zoomdata *Subset example webinar
  26. 26. ISSUES • Service Discovering • Networking for Containers • Persistent Storage • Docker Performance webinar
  27. 27. MORE DETAILS https://mantl.io webinar

×