The document discusses an auto-scaling solution for services at eBay Turkey, focusing on scaling strategies without using public cloud services. It outlines the use of various technologies such as Docker Swarm, Prometheus, and Slack for monitoring and automating scaling operations based on certain thresholds. The solution includes a Python bot that handles scaling up and down of services based on alert notifications from Prometheus.

1. Your Auto-Scaling Bot
Volkan Tufekci
DevOps Eng.,
eBay Turkey - GittiGidiyor

2. • DevOps Eng 2+ years
• Backend Dev. 6+ years
Who am I?

3. • 15M Subscriber
• 38M Visitor/Month
• In every 2 secs a product
is sold
eBay TR -
GittiGidiyor

4. • When we hit the thresholds
• When there is a push notification
• When we employ more services
Problem
We need to scale

5. • Container scaling
• Host(Node) scaling*
Problem
What kind of scaling?
* We can’t use public cloud services

6. We can dedicate someone to monitor the load on our services and
perform ”docker scale”
or
We can use a bot to do the stuff automatically for us
Solution

7. Ingredients
• Docker Swarm Mode
• Node Exporter
• cAdvisor
• Prometheus
• Alert Manager
• Slack + Python Bot
• Grafana(not mandatory but nice to have)
Recipe

8. Official Prometheus exporter
for node metrics
Node Exporter

9. Container resource usage and
performance metrics
cAdvisor

10. Time series data based
monitoring and alerting
solution
Prometheus

11. Prometheus
• scrape_interval
• evaluation_interval
• states:
• inactive
• pending
• firing
Under the Hood

12. For deduplicating, grouping,
and routing alarms to the
correct receiver integration
such as Slack
Alert Manager

13. Under the Hood
Alert Manager
• group_by
• group_wait
• group_interval

14. Open source software for
time series analytics.
Graphical visualization
solution.
Grafana

15. Big Picture
PROMETHEUS
BOT

16. • Parse the notification
• If it is ”FIRING” alarm from a service
• Get current # of tasks
• Check if we are below upper
threshold
• If so, increase scale
• If not, notify
Algorithm
Scaling up the service

17. • Parse the notification
• If it is ”RESOLVED” notification
• For each task
• Get current # of tasks
• Check if we are above
lower threshold
• If so, decrease scale
Algorithm
Scaling down the services

18. Simplified Code

19. Thank You!
@volkantufekci
@docker
#dockercon