Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Monitoring @ scale over diverse data sources @ PayPal - Druid, TSDB, Hadoop
Similar to Monitoring @ scale over diverse data sources @ PayPal - Druid, TSDB, Hadoop (20)
Recently uploaded
Recently uploaded (20)
Monitoring @ scale over diverse data sources @ PayPal - Druid, TSDB, Hadoop
- 1. Monitoring at Scale over diverse data sources @ PayPal – Druid, TSDB, Hadoop
- 2. Agenda ©2016 PayPal Inc. Confidential and proprietary. • Introduction • Monitoring Platform Architecture • Grafana Druid Plugin • Q & A
- 3. PayPal Command Center Powered by Monitoring Team 3
- 4. PayPal Monitoring Ecosystem - Sherlock 4 APIs Logs Processing Metric Processing Insights Engine Alerts Visualization Events Stream Metrics Stream Topology Logs Stream Pools & Hosts PaaS IaaS Network Databases CAL (Logging) Remediation Orchestration Configuration Sherlock is a unified monitoring, alerting and logging platform that exposes real time data and insights, enabled by an extensible visualization platform. Developers Analysts Command Center SRE
- 5. Tools & Technologies ©2016 PayPal Inc. Confidential and proprietary. Storage Processing Visualization
- 6. Ingest Queue Process Store Query PayPal Monitoring Platform - Architecture 6 MessageBus App Logs AppApp S e r v i c e s DBLB NO SQ L Metrics Stream processing MetricsLogs Log Views Hadoop Reports Custom Visualizations Alerts
- 7. Agenda ©2016 PayPal Inc. Confidential and proprietary. • Introduction • Monitoring Platform Architecture • Grafana Druid Plugin • Q & A
- 8. Grafana Druid Plugin https://grafana.net/plugins/abhisant-druid-datasource/installation ©2016 PayPal Inc. Confidential and proprietary.
- 9. Grafana - Adding Druid Data source ©2016 PayPal Inc. Confidential and proprietary.
- 10. Grafana Druid Plugin – Query Editor ©2016 PayPal Inc. Confidential and proprietary.
- 11. Druid Query Types ©2016 PayPal Inc. Confidential and proprietary. Time Series Top N Group by Select
- 12. Multiple Data sources ©2016 PayPal Inc. Confidential and proprietary. Time Series Top N Group by Select TSDB Druid Hadoop Reports
- 13. Thank You
Editor's Notes
- https://www.paypal.com/stories/us/a-behind-the-scenes-tour-of-the-paypal-command-center http://www.mercurynews.com/business/ci_29276113/q-paypals-command-center-guru-sri-shivananda?source=email Paypal processd $8773 in payments every second in Q3 2015 In 2015, 28% of the 4.9 billion payments we processed were made on a mobile device. With our 192 million active customer accounts, PayPal is a truly global payments platform that is available to people in more than 200 markets, allowing customers to get paid in more than 100 currencies, withdraw funds to their bank accounts in 56 currencies and hold balances in their PayPal accounts in 25 currencies. For more information on PayPal, visit https://www.paypal.com/about. For PYPL financial information, visit https://investor.paypal-corp.com. From http://www.businesswire.com/news/home/20161026005468/en/Binge-Watching-Big-Buys-Bed-PayPal-Predicts-Holiday PayPal currently operates in 200 countries around the world, with 192 million people using its platform. Its goal is to help small businesses grow by exposing them to the global consumer
- TTD, TTR https://github.com/paypal/squbs - https://www.paypal-engineering.com/2016/05/11/squbs-a-new-reactive-way-for-paypal-to-build-applications/ Actor model (message based system) of computation – Akka and spary as the http library