This document provides an overview of the Prometheus monitoring system. It discusses that Prometheus was originally created by frustrated engineers at SoundCloud to address their monitoring needs. It gained popularity and was later maintained as an open source project. Major milestones included Prometheus 1.0 and 2.0 releases, joining the CNCF, and graduating as a CNCF project. The document outlines ongoing work to improve areas like data integrity, security, and community involvement through initiatives like OpenMetrics. The future of Prometheus is focused on further developing the project and monitoring ecosystem through an expanding team and community.

1. Prometheus, the future!
Goutham Veeramachaneni
Grafana Labs

2. About Me
● Prometheus maintainer
● Cortex maintainer
● Loki maintainer
● BIKING

4. Prometheus: The monitoring system
● Made by frustrated Xooglers at Soundcloud
● Was really good
● People loved it

5. Prometheus: The monitoring system
● Made by frustrated Xooglers at Soundcloud
● Was really good
● People loved it

6. Prometheus: The monitoring system
● Made by frustrated Xooglers at Soundcloud
● Was really good
● People loved it

7. Prometheus: The monitoring system
● Made by frustrated Xooglers at Soundcloud
● Was really good
● People loved it

8. Prometheus: The monitoring system
Nov 18
We decided to focus on actual problems
Beer Brew monitoring

9. Prometheus: The monitoring system
Jan 2015: First blog post! After 2+ years of dev.

10. Prometheus: The monitoring system
May 2016: Join the CNCF

11. Prometheus: The monitoring system
July 2016: Prometheus 1.0

12. Prometheus: The monitoring system
Nov 2017: Prometheus 2.0

13. Prometheus: The monitoring system
Aug 2018: Prometheus is a CNCF graduated project

17. Major Milestone: Prometheus 2.0
● Completely re-thought a lot of things, re-wrote the storage engine.
● 2-10x improvement across the board.
● Fix-ing EVERYTHING.
● 2.0 was Nov 2017

18. 3 more team-members past year!
Expanding the team

19. ● 1.0:
○ We did it when someone wanted to do it.
○ Worked really well until it didn’t.
● No testing, huge set of changes
● 2.4+: Every 6 weeks
● We fixed it, maybe a little too well ;)
Release Process

20. Release Process
Github: @sipian
● 1.0:
○ We did it when someone wanted to do it.
○ Worked really well until it didn’t.
● No testing, huge set of changes
● 2.4+: Every 6 weeks
● We fixed it, maybe a little too well ;)

21. Data Integrity
● 1.0: We made no promises about your data
● 1.0: Still really, really good!

22. Data Integrity
● 1.0: We made no promises about your data
● 1.0: Still really, really good!
● 2.0: Backups and Restore!

23. Data Integrity
● 1.0: We made no promises about your data
● 1.0: Still really, really good!
● 2.0: Backups and Restore!
● Coming soon to a Prometheus near you: Backfilling

24. Backfilling
● TONS OF REQUESTS! No good answer.
● In master, will be in 2.8

25. Backfilling
mutable prometheuswrite
t0 t1 t2 t3 now
query
merge

26. Backfilling
mutable prometheuswrite
t0 t1 t2 t3 now
query
merge

27. Backfilling
mutable prometheuswrite
t0 t1 t2 t3 now
query
merge

28. Data Integrity
● Still a single node system

29. Long Term Storage
Via the remote_write and remote_read

30. Remote Write
● snappy-compressed protocol buffer encoding over HTTP
● Uses an in-memory sample buffer in-case remote is down
● OOMs when remote is unreachable
● Dataloss when remote is down for more than few mins

31. Remote Write

32. Aaaaand more!
● Alerts unit testing
● Metadata API
● New shiny UI (almost there)
● Even MOAR faster
● Subqueries!

33. The future

34. Security
● 1.x: Secure it yourself
● Security Audit: Everything is ~great, but give a default way to secure things

35. Security
● 1.x: Secure it yourself
● Security Audit: Everything is ~great, but give a default way to secure things

36. BIGGEST THING

37. BIGGEST THING
Community

38. ● We changed the world
● Now, we need to CONQUER it!
Community

39. OpenMetrics
● The Prometheus exposition format is pretty f***** good!
● Let’s make it a standard!

40. OpenMetrics
● The Prometheus exposition format is pretty f***** good!
● Let’s make it a standard!

41. Why? Cuz it’s simple and naaice!

42. Why? Cuz it’s simple and naaice!

43. Why?
● Explosion of compatible metric endpoints
● Avoid lock-in
● Kill SNMP ;)

44. Status
● CNCF Project
● Spec shelled out, now being written
● Prometheus 2.5 has support for it
● The prometheus python client is first client that can expose OM
● Google, Uber will write another reference parser

45. First commitments
Cloudflare
CNCF at large
GitLab
Google
Grafana
InfluxData
Prometheus!
RobustPerception
SpaceNet
Uber
….. Too many to list ;)

46. Community: Enable it
● MOAR exporters
● MOAR language support
● https://github.com/prometheus-community

47. THANKS!
Have any questions about our conquest of the world?