Adform Cloud Infrastructure
•
2 likes•243 views
Matas Tvarijonas presented on Adform's cloud infrastructure. The key points are: 1) Adform has a global infrastructure across 8 datacenters with over 4,600 OpenStack VMs, 2,300 Kubernetes Pods, and 1,700 physical servers handling large amounts of network traffic and data storage. 2) Over the last 5 years, Adform has transitioned from virtualization to private cloud, growing its use of OpenStack VMs, Kubernetes Pods, and reducing other virtualization technologies. 3) Adform utilizes practices like ChatOps, observability tools, storage QoS, provisioning automation, and efficiency to operate its private cloud infrastructure.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Adform Cloud Infrastructure
Similar to Adform Cloud Infrastructure (20)
Recently uploaded
Recently uploaded (20)
Adform Cloud Infrastructure
- 1. ADFORM CLOUD INFRASTRUCTURE Matas Tvarijonas, Cloud Architect Bridge to Cloud 1 2019 10 08
- 2. #whoami Name: Matas First IT job 2007 First virtualization host 2009 First VMWare cluster 2010 First successful exit from vi 2015 First commit to git 2016 First working program 2018 First OSS PR merged 2019 Support engineer -> Hardware engineer -> System Architect -> Product owner -> Technical product manager -> Cloud Architect -> Team lead 2
- 3. Agenda Slide • Intro to Adform • Adform global infrastructure • Practices and solutions 3
- 4. Platform Combines Data, Creativity and Trading for Enhanced Results The Most Complete Independent Advertising Technology Platform 4 DEMAND Advertiser Edge SUPPLY Publisher Edge DMP Automated Guaranteed Demand Side Platform Supply Side Platform Ad Server Publisher Ad Server
- 5. Global Infrastructure 5 x3 2+ Mil Kafka Messages per Second 160+ Gb/s Incoming Network Traffic 8 Datacenters 2,6+ Mil Aerospike TPS 4800+ OpenStack VMs 1700+ Physical Servers 5+ PB Hadoop Storage 2300+ Kubernetes PODs 730+ Gb/s Traffic via core 4,6+ Mil Nginx QPS
- 6. Do we need private cloud at all? 6
- 7. in the Last 5 Years Virtualization to Cloud 7 600 650 0 0 0 400 600 750 850 0 200 400 2000 3600 4800 0 0 0 400 2300 0 1000 2000 3000 4000 5000 6000 2015 2016 2017 2018 2019 Compute instances (count) Proxmox VMs Hyper-V VMs OpenStack VMs Kubernetes PODs
- 8. Building Blocks – Private Cloud 8 OpenStack Services Self-service Clouds Trove Octavia Keystone Horizon Neutron Cinder Glance Nova Watcher Freezer Self-service IT DEVCLOUD PRECLOUD Core Products Orchestration and automation PROCLOUD
- 9. Building Blocks - DevOps 9 Self-service Key Services ChatOps/Self-Service MaaS LaaS Core Products Orchestration and automation
- 10. Agenda Slide • Intro to Adform • Adform global infrastructure • Practices and solutions 10
- 11. Observability - Internal monitoring solutions(MaaS +LaaS): - Prometheus - Grafana - Thanos - Karma - ELK stack - External solutions: - Pingdom - NewRelic - Catchpoint 11
- 12. 12 Observability
- 13. ChatOps 13
- 14. 14
- 15. Operations: 15 Agile operations: - Scrum and Jira - IaaC - Slack - Standardization - Hardware - Operating systems - Server names - Tools - Technolgies
- 16. Storage QoS: Defending IO: - VM and storage QoS (static): - OS drive 300 iops and 80 MBps - SSD volume: 5000 iops and 500 MBps - HDD volume: 500 iops and 200 MBps - Object and Block storage - S3 compatible storage - HDD and SSD volumes - Next: QoS per GB 16
- 17. Provisioning:
- 19. Efficiency: 19
- 20. Efficiency: 20
- 22. @wizardmatas
Editor's Notes
- Hello everyone my name is Matas and I am going to talk about how we are running huge private cloud powered by openstack.
- During my presentation I will cover some details on Adform as a company and how adform infrastructure looks today. Later I will talk about how we built our private cloud and what challenges we faced.
- Adform is company founded in 2002 in Copenhaged and their primary business was third party ad serving. Today Adform have offices in 19 countries and more than 800 employees. If someone would ask what we are doing today, I would describe Adform as advertinment market place where we are connecting advertisers with publishers in our intelligent and powerfull platform. And that requires huge infrastructure. SO ….
- Adform is running in 9 datacenters all around the world. We manage more than one thousand of physical servers and more thant 4k of virtual machines. Every second platform is quered for more than 3 millions requests external requests.
- We still need hardware for: High load services Databases Legacy applications We still want to keep data on-premises
- In the last few years Adform virtualization platform is on a shift to private cloud solution. In 2017 we managed to get rid of proxmox virtualization and now we are moving out from hyper-v. In the last few years we also noticed increased demand for virtual resources, and I was lucky to find something what may explain what is happening.
- Q: how many of you are familiar with Opestack and its services ? Or aws ? In our platform we support these main openstack services and I am going shortly introduce each of these. Keystone is identity service responsible for authorization and token issueing. Horizon is running Openstack dashboard. Neutron with openvirtual switch is covering networking part. Cinder and glance relly on CEPH for block and object storage. Nova is computing service which is powered by Centos and KVM virtualization. All our services require monitoring and alerting so prometheus is responsible for that and we use grafana for very nice dashboards ( you can check it on our booth). Ha proxy and consul helps us to have multicontinent solution and provides API availability. For configuration management and task automation we use salt stack and Jenkins. Kubernetes helps us to run some important supporting services like Self-Service portals, metrics exporters. From openstack servies we have watcher and freezer on our future vision, and also it is important to mention the didn’t managed to successfully deploy trove dbaas and octacia lbaas they were limited to single region deployments.
- Q: how many of you are familiar with Opestack and its services ? Or aws ? In our platform we support these main openstack services and I am going shortly introduce each of these. Keystone is identity service responsible for authorization and token issueing. Horizon is running Openstack dashboard. Neutron with openvirtual switch is covering networking part. Cinder and glance relly on CEPH for block and object storage. Nova is computing service which is powered by Centos and KVM virtualization. All our services require monitoring and alerting so prometheus is responsible for that and we use grafana for very nice dashboards ( you can check it on our booth). Ha proxy and consul helps us to have multicontinent solution and provides API availability. For configuration management and task automation we use salt stack and Jenkins. Kubernetes helps us to run some important supporting services like Self-Service portals, metrics exporters. From openstack servies we have watcher and freezer on our future vision, and also it is important to mention the didn’t managed to successfully deploy trove dbaas and octacia lbaas they were limited to single region deployments.
- During my presentation I will cover some details on Adform as a company and how adform infrastructure looks today. Later I will talk about how we built our private cloud and what challenges we faced.
- Our team is responsible for Storage platform Private CND solution and high performance load balancing We also provide service discovery services based on Consul and Private cloud powered by Openstack.