© 2014 Nippon Telegraph and Telephone Corporation
AUTOSCALING
CLOUD FOUNDRY WITH BOSH
Yudai Iwasaki
NTT Service Innovation...
© 2014 Nippon Telegraph and Telephone Corporation
•  Core developer of Cloudn PaaS
•  Working on Cloud Foundry since 2012
...
© 2014 Nippon Telegraph and Telephone Corporation
Agenda
•  Problem
– System load is not fixed
•  Solution
– BOSH AutoScale...
© 2014 Nippon Telegraph and Telephone Corporation
Problem:
System Load is
Not Fixed
© 2014 Nippon Telegraph and Telephone Corporation
Daily, Weekly, Monthly, and Yearly Peaks
© 2014 Nippon Telegraph and Telephone Corporation
Daily, Weekly, Monthly, and Yearly Peaks
© 2014 Nippon Telegraph and Telephone Corporation
Manually Scaling?
Always monitor load?
© 2014 Nippon Telegraph and Telephone Corporation
Impossible
No one wants to do such a boring job
© 2014 Nippon Telegraph and Telephone Corporation
Solution:
BOSH AutoScaler
© 2014 Nippon Telegraph and Telephone Corporation
BOSH AutoScaler
•  Extension for BOSH
•  Flexible scaling policies in de...
© 2014 Nippon Telegraph and Telephone Corporation
Architecture
Overview
© 2014 Nippon Telegraph and Telephone Corporation
Director
BOSH
BOSH Client
BOSH Scaler (Listener)
NATS
BOSH NATS Collecto...
© 2014 Nippon Telegraph and Telephone Corporation
Director
BOSH
BOSH Client
BOSH Scaler (Listener)
NATS
BOSH NATS Collecto...
© 2014 Nippon Telegraph and Telephone Corporation
Director
BOSH Client
BOSH Scaler (Listener)
NATS
BOSH NATS Collector
Aut...
© 2014 Nippon Telegraph and Telephone Corporation
Director
BOSH Client
BOSH Scaler (Listener)
NATS
BOSH NATS Collector
Aut...
© 2014 Nippon Telegraph and Telephone Corporation
Policy Definitions
© 2014 Nippon Telegraph and Telephone Corporation
Defining policies
•  Adding “scale” section
•  Defining conditions for
eac...
© 2014 Nippon Telegraph and Telephone Corporation
Conditions: BOSH Heartbeat
•  CpuAverage
–  Average CPU percentage for d...
© 2014 Nippon Telegraph and Telephone Corporation
Conditions: CF Plugin
•  CFVarzAverage
–  Average Varz value for duratio...
© 2014 Nippon Telegraph and Telephone Corporation
Web UI
© 2014 Nippon Telegraph and Telephone Corporation
© 2014 Nippon Telegraph and Telephone Corporation
© 2014 Nippon Telegraph and Telephone Corporation
© 2014 Nippon Telegraph and Telephone Corporation
Getting Started
It’s easy to plug in
© 2014 Nippon Telegraph and Telephone Corporation
https://github.com/nttlabs/bosh-scaler
© 2014 Nippon Telegraph and Telephone Corporation
-­‐-­‐-­‐
#	
  BOSH	
  NATS	
  settings	
  
nats:	
  &nats	
  
	
  	
  u...
© 2014 Nippon Telegraph and Telephone Corporation
...	
  
properties:	
  
	
  	
  collector:	
  
	
  	
  	
  	
  use_tsdb:...
© 2014 Nippon Telegraph and Telephone Corporation
Optimize
Running Costs with
BOSH AutoScaler
Upcoming SlideShare
Loading in …5
×

Autoscaling Cloud Foundry with BOSH (Cloud Foundry Summit 2014)

1,422 views

Published on

Technical track presented Yudai Iwasaki, Lead Engineer, Software Innovation Center at NTT.

BOSH AutoScaler is an extension for BOSH to automatically scale-in/out your deployments according to your preferred rules. You can optimize the resource consumption of your deployments with BOSH AutoScaler by dynamically changing the number of virtual machines. This talk will cover how to get started with BOSH AutoScaler to auto-scale your Cloud Foundry deployment

Published in: Technology

Autoscaling Cloud Foundry with BOSH (Cloud Foundry Summit 2014)

  1. 1. © 2014 Nippon Telegraph and Telephone Corporation AUTOSCALING CLOUD FOUNDRY WITH BOSH Yudai Iwasaki NTT Service Innovation Laboratory Group CF Summit 2014
  2. 2. © 2014 Nippon Telegraph and Telephone Corporation •  Core developer of Cloudn PaaS •  Working on Cloud Foundry since 2012 –  Nise BOSH, BOSH CloudStack CPI and BOSH AutoScaler •  Twitter: @i_yudai
  3. 3. © 2014 Nippon Telegraph and Telephone Corporation Agenda •  Problem – System load is not fixed •  Solution – BOSH AutoScaler •  Getting Started
  4. 4. © 2014 Nippon Telegraph and Telephone Corporation Problem: System Load is Not Fixed
  5. 5. © 2014 Nippon Telegraph and Telephone Corporation Daily, Weekly, Monthly, and Yearly Peaks
  6. 6. © 2014 Nippon Telegraph and Telephone Corporation Daily, Weekly, Monthly, and Yearly Peaks
  7. 7. © 2014 Nippon Telegraph and Telephone Corporation Manually Scaling? Always monitor load?
  8. 8. © 2014 Nippon Telegraph and Telephone Corporation Impossible No one wants to do such a boring job
  9. 9. © 2014 Nippon Telegraph and Telephone Corporation Solution: BOSH AutoScaler
  10. 10. © 2014 Nippon Telegraph and Telephone Corporation BOSH AutoScaler •  Extension for BOSH •  Flexible scaling policies in deployment manifest files •  Special support for Cloud Foundry
  11. 11. © 2014 Nippon Telegraph and Telephone Corporation Architecture Overview
  12. 12. © 2014 Nippon Telegraph and Telephone Corporation Director BOSH BOSH Client BOSH Scaler (Listener) NATS BOSH NATS Collector AutoScaler (with CF Plugin) Heartbeat Heartbeat (subscribe) process (BOSH::Monitor::Event::Heartbeat) Event Processor process (BOSH::Monitor::Event::Base) Deploy (PUT /deployments) matching rules against log periodically Logging metrics(@buffers)run() Agent Agent Agent Agent Agent CF Varz Collector process (Scaler::CfVarzMetric) CF CF Component CF Component CF Component Collector TSDB Varz Load policies (GET /deployments)
  13. 13. © 2014 Nippon Telegraph and Telephone Corporation Director BOSH BOSH Client BOSH Scaler (Listener) NATS BOSH NATS Collector AutoScaler (with CF Plugin) Heartbeat Heartbeat (subscribe) process (BOSH::Monitor::Event::Heartbeat) Event Processor process (BOSH::Monitor::Event::Base) Deploy (PUT /deployments) matching rules against log periodically Logging metrics(@buffers)run() Agent Agent Agent Agent Agent CF Varz Collector process (Scaler::CfVarzMetric) CF CF Component CF Component CF Component Collector TSDB Varz Load policies (GET /deployments)
  14. 14. © 2014 Nippon Telegraph and Telephone Corporation Director BOSH Client BOSH Scaler (Listener) NATS BOSH NATS Collector AutoScaler (with CF Plugin) Heartbeat (subscribe) process (BOSH::Monitor::Event::Heartbeat) Event Processor process (BOSH::Monitor::Event::Base) Deploy (PUT /deployments) matching rules against log periodically Logging metrics(@buffers)run() Agent CF Varz Collector process (Scaler::CfVarzMetric) Collector TSDBLoad policies (GET /deployments)
  15. 15. © 2014 Nippon Telegraph and Telephone Corporation Director BOSH Client BOSH Scaler (Listener) NATS BOSH NATS Collector AutoScaler (with CF Plugin) Heartbeat (subscribe) process (BOSH::Monitor::Event::Heartbeat) Event Processor process (BOSH::Monitor::Event::Base) Deploy (PUT /deployments) matching rules against log periodically Logging metrics(@buffers)run() Agent CF Varz Collector process (Scaler::CfVarzMetric) Collector TSDBLoad policies (GET /deployments)
  16. 16. © 2014 Nippon Telegraph and Telephone Corporation Policy Definitions
  17. 17. © 2014 Nippon Telegraph and Telephone Corporation Defining policies •  Adding “scale” section •  Defining conditions for each job •  Various condition classes to define flexible policies -­‐-­‐-­‐   name:  cf     ....     scale:      jobs:          -­‐  name:  router              cooldown:  300              out:                  limit:  10                  unit:  2                  conditions:                      -­‐  class:  CpuAverage                          larger_than:  80                          duration:  300                      -­‐  class:  MemoryAverage                          larger_than:  90                          duration:  300              in:                  limit:  3                  conditions:                      -­‐  class:  CpuAverage                          smaller_than:  10                          duration:  300                      -­‐  class:  MemoryAverage                          smaller_than:  20                          duration:  300  
  18. 18. © 2014 Nippon Telegraph and Telephone Corporation Conditions: BOSH Heartbeat •  CpuAverage –  Average CPU percentage for duration •  MemoryAverage –  Average memory percentage for duration •  LoadAverage1 –  Latest Load Average in 1 minute •  LoadAverage5 –  Latest Load Average in 5 minutes •  LoadAverage15 –  Latest Load Average in 15 minutes
  19. 19. © 2014 Nippon Telegraph and Telephone Corporation Conditions: CF Plugin •  CFVarzAverage –  Average Varz value for duration DEA available_memory_ratio available_disk_ratio Router total_routes latency.1m HM9000 NumberOfRunningInstances NumberOfCrashedInstances etcd SendingRequestRate ReceivingRequestRate Loggregator Server receivedMessageCount numberOfWebsocketSinks Cloud Controller connection_count threadqueue.num_waiting
  20. 20. © 2014 Nippon Telegraph and Telephone Corporation Web UI
  21. 21. © 2014 Nippon Telegraph and Telephone Corporation
  22. 22. © 2014 Nippon Telegraph and Telephone Corporation
  23. 23. © 2014 Nippon Telegraph and Telephone Corporation
  24. 24. © 2014 Nippon Telegraph and Telephone Corporation Getting Started It’s easy to plug in
  25. 25. © 2014 Nippon Telegraph and Telephone Corporation https://github.com/nttlabs/bosh-scaler
  26. 26. © 2014 Nippon Telegraph and Telephone Corporation -­‐-­‐-­‐ #  BOSH  NATS  settings   nats:  &nats      uri:  mbus://192.168.50.4:21084      user:  nats      pass:  nats     #  BOSH  Director  REST  API  settings   rest:  &rest      endpoint_uri:  https://192.168.50.4:25555      user:  scaler      password:  scaler      disable_verify_certification:  true   collectors:      -­‐  class:  BoshNatsCollector          bosh_nats:  *nats          bosh_rest:  *rest      -­‐  class:  CfVarzTsdbCollector          port:  4567     listeners:      -­‐  class:  BoshScaler          bosh_rest:  *rest          interval:  60  #  seconds          buffer_size:  1000          ui:              enable:  true              port:  8888   Configuration
  27. 27. © 2014 Nippon Telegraph and Telephone Corporation ...   properties:      collector:          use_tsdb:  true          deployment_name:  cf      opentsdb:          #  your  AutoScaler  address          address:  192.168.15.139          port:  4567                 Cloud Foundry Manifest
  28. 28. © 2014 Nippon Telegraph and Telephone Corporation Optimize Running Costs with BOSH AutoScaler

×