Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2jZgs1F. Rob Harrop talks about the increasing automated field of operations and explores what the future might hold when machine learning and AI techniques are brought to bear on the problem of systems operations. He also shows some real examples of what is possible with today’s toolchains and compute power. Filmed at qconsf.com. Rob Harrop is CEO at Skipjaq where he leads a team working on the cutting edge of machine-driven performance optimization. Rob is well known as a co-founder of SpringSource, the software company behind the wildly-successful Spring Framework. At SpringSource he was a core contributor to the Spring Framework and led the team that built dm Server (now Eclipse Virgo).

1. Automating Operations
with Machine Intelligence
Rob Harrop

2. InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
microservices-ml-ai

3. Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
Presented at QCon San Francisco
www.qconsf.com

4. CEO @ Skipjaq
Co-founder @ SpringSource
Automated performance management

5. Why automate operations?
Why now?
What does automated operations look like?
How do we build for automation?
Solving a real problem…

6. Why automate operations?

7. More Complexity

8. Monolith -> Microservices
Strong -> Eventual Consistency
Assume reliability -> Assume failure

9. More Deployments

10. Very end of 2009
Today
30
20
10
40
Credit: Mike Brittain, Engineering Director @ Easy

11. Less time to identify fixes
Rollbacks more likely
Tiny window for human intervention

12. Harder
Faster

13. Why now?

14. We have to

15. We can

16. Cloud
Containers
Observability
Microservices
ML/AI
Trends

17. Current trends provide the impetus
and tools for automation by AI

18. Automated Operations

19. Move 37

21. Move 78 - God’s Touch

23. AI Human

24. Wholly performed by human
Wholly performed by AI
Co-operation between human and AI
Actionable insight
Types of Operation Actions

25. Data is not insight
Gathering metrics is not automating operations
But, metrics are critical to automating operations
On Metrics

26. Human ≠ Manual

27. Testing
Deployment
Provisioning
Actions by Human

28. Anomaly alerting
Rollback broken builds
Dependency upgrade
Cooperative Actions

29. Predictive auto scaling
Workload placement
Automatic rollback
Performance optimisation?
Security?
Actions by AI

30. Actions
and
Actionable Insights

31. Building for Automation

32. Visible metrics and logs
Ability to start/stop/restart/move workload
Ability to change configuration
Ability to modify dependencies
Ability to wire/rewire external services
Requirements for Operations

33. Self-contained package
Disposable processes
Externally-configurable
Externally-observable
Externalised dependencies
Externalised service wiring

34. 12+1 Factor

35. Metrics as event streams
Standard metrics
- CPU usage, memory usage, …
Service-specific metrics
- Leads received, items sold, …
13th Factor - Observability

36. Detecting Anomalous DB CPU
Case Study

37. Background
Consumer-facing web application running Rails against PostgreSQL on AWS
RDS
Mix of transactional and batch workloads running against the same database
Question: when is the DB unusually overloaded?

39. Detecting Anomalies
Policy-based
Statistical model
Predictive model
Classification model

40. Policy Based
Fixed threshold alerting
How well does this work?

41. Not Very

42. Statistical Model
Twitter AnomalyDetection package
- Seasonal Hybrid ESD
Is this point unexpected in our distribution?
- With seasonal and trend effects removed

46. Statistical Model
Stream
Metrics
Sliding window of observations
(1 month, 1 year?)
Each new observation
run model (S - H - ESD)
Is the new point an outlier?

47. Predictive Model
Train a model to predict values in the time series
Prediction error > critical value => outlier

48. x1
x2
x3
+1 +1
Layer L1 Layer L2
Layer L3
hW,b(x)
a2
(2)
a3
(2)
a1
(2)

49. From: http://colah.github.io/posts/2015-08-Understanding-LSTMs/
h0 h1 h2 h3 h4
x0 x1 x2 x3 x4
A A A A A

53. Predictive Model
Metrics Stream
Prediction
Training set
?? last month
Model
Re-Train
(Nightly, weekly?)
Is prediction error
an outlier???

54. Handling Anomalies
Actionable alerts
- Confidence in predictions
No alerts for pointless things

55. Handling Anomalies
Taking action
- Rewiring services to read-replica?
- Kill long-running queries?

56. Handling Anomalies
Confidence in the model leads
to confidence in automation

57. Summary
Increasing complexity and deployment speed make operational
automation a must
We must build services that are ready for automation
Simple models can often beat complex ones
Cheap compute and storage makes large-scale ML available to
everyone

58. Thank You

59. Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
microservices-ml-ai