SlideShare a Scribd company logo
1 of 8
Real-time Application monitoring
Amit Gupta
Central App Monitoring Platform
• Graphite
– Carbon
– Whisper
– Webapp
• Simple time series data
• Highly scalable, optimized for I/O
• Queuing/Caching
• Capability to club non-persisted metrics with persisted
while returning data for monitoring (Real-time)
• Carbon-relay -> Clustering
• Works fine for lacs of metrics coming every second
• Can work with any client (statd daemon, jmxtrans, yammer
etc. or we can write our own easily) because of simple spec
Central App Monitoring Platform
• Metrics names recommended to be given in reverse dns format
– Easy to aggregate metrics for multiple servers of a system (eg.
com.snapdeal.cams.*)
• Supports both TCP & UDP
• Lots of mathematical functions to derive, aggregate, join multiple
metrics
• Grafana used for visualization (though graphite also provides UI)
• Grafana
– Cutting edge graphs and very easy configuration through GUI
– Easily integrates with Graphite APIs
– Integrated with elastic search for faster response on graphs
– Many more calculations (average, max, min etc.) possible on UI itself
Application Metrics – Yammer
• Simple Java library
• Different kind of Metrics
– Gauge
– Counter
– Meter
– Histogram
– Timer
– HealthCheck
• Supports both push (configurable frequency) or pull (jmx or http)
• Also sends JVM Info by default
• Very less overhead on application
– Just instantiate POJOs in between your existing code
– All processing in app’s control
– No need to write separate code
– Pushing via separate thread pool (no impact on app even if server is down)
• Yammer metrics can be exposed as MBeans (for JMX) as well.
Application Metrics – Jmxtrans
• What if you want to monitor your existing
Mbeans via central monitoring platform (like
graphite) or there are 3rd party frameworks
already exposing data using JMX
• Use Jmxtrans to integrate with Graphite server
• Frequency of pushing can be configured
• A separate centralized Jmxtrans App can suffice
for multiple applications (though running as
agent in same process also possible)
Alerting - Nagios
• Little crude, but flexible & powerful way to
alert.
• Client/Server Architecture
Architecture Diagram
Thank You

More Related Content

What's hot

Desarrollo de una plataforma de observabilidad centralizada
Desarrollo de una plataforma de observabilidad centralizadaDesarrollo de una plataforma de observabilidad centralizada
Desarrollo de una plataforma de observabilidad centralizadaElasticsearch
 
Windows azure traffic manager
Windows azure traffic managerWindows azure traffic manager
Windows azure traffic managerK.Mohamed Faizal
 
What's New Overview for IBM Streams V4.3
What's New Overview for IBM Streams V4.3 What's New Overview for IBM Streams V4.3
What's New Overview for IBM Streams V4.3 lisanl
 
Olympus pesentation2
Olympus pesentation2Olympus pesentation2
Olympus pesentation2mskmoorthy
 
Load Balancing in Cloud Computing Thesis Research Help
Load Balancing in Cloud Computing Thesis Research HelpLoad Balancing in Cloud Computing Thesis Research Help
Load Balancing in Cloud Computing Thesis Research HelpPhdtopiccom
 
Op Manager Review
Op Manager ReviewOp Manager Review
Op Manager Reviewmatis_x
 
Event Driven Architectures with Apache Kafka
Event Driven Architectures with Apache KafkaEvent Driven Architectures with Apache Kafka
Event Driven Architectures with Apache KafkaMatt Masuda
 
Monitoreo sencillo de la infraestructura, de la ingesta a la visualización
Monitoreo sencillo de la infraestructura, de la ingesta a la visualizaciónMonitoreo sencillo de la infraestructura, de la ingesta a la visualización
Monitoreo sencillo de la infraestructura, de la ingesta a la visualizaciónElasticsearch
 
New Relic_Heroku_Presentation_Dreamforce11
New Relic_Heroku_Presentation_Dreamforce11New Relic_Heroku_Presentation_Dreamforce11
New Relic_Heroku_Presentation_Dreamforce11New Relic
 
SCU Asia System Center: Beyond orchestrator
SCU Asia System Center: Beyond orchestratorSCU Asia System Center: Beyond orchestrator
SCU Asia System Center: Beyond orchestratorAlexandre Verkinderen
 
[Webinar] AWS Monitoring with Site24x7
[Webinar] AWS Monitoring with Site24x7[Webinar] AWS Monitoring with Site24x7
[Webinar] AWS Monitoring with Site24x7Site24x7
 
IBM Maximo Performance Tuning
IBM Maximo Performance TuningIBM Maximo Performance Tuning
IBM Maximo Performance TuningFMMUG
 
Simulation Projects in Matlab
Simulation Projects in MatlabSimulation Projects in Matlab
Simulation Projects in MatlabPhdtopiccom
 
An introduction to Zen 6 - integrated network and customer experience analytics
An introduction to Zen 6 - integrated network and customer experience analyticsAn introduction to Zen 6 - integrated network and customer experience analytics
An introduction to Zen 6 - integrated network and customer experience analyticsSysMech
 
Choosing the Right Transformer for Your Data Challenge
Choosing the Right Transformer for Your Data ChallengeChoosing the Right Transformer for Your Data Challenge
Choosing the Right Transformer for Your Data ChallengeSafe Software
 

What's hot (18)

Desarrollo de una plataforma de observabilidad centralizada
Desarrollo de una plataforma de observabilidad centralizadaDesarrollo de una plataforma de observabilidad centralizada
Desarrollo de una plataforma de observabilidad centralizada
 
ADSL ppt
ADSL pptADSL ppt
ADSL ppt
 
Windows azure traffic manager
Windows azure traffic managerWindows azure traffic manager
Windows azure traffic manager
 
What's New Overview for IBM Streams V4.3
What's New Overview for IBM Streams V4.3 What's New Overview for IBM Streams V4.3
What's New Overview for IBM Streams V4.3
 
Olympus pesentation2
Olympus pesentation2Olympus pesentation2
Olympus pesentation2
 
Load Balancing in Cloud Computing Thesis Research Help
Load Balancing in Cloud Computing Thesis Research HelpLoad Balancing in Cloud Computing Thesis Research Help
Load Balancing in Cloud Computing Thesis Research Help
 
Op Manager Review
Op Manager ReviewOp Manager Review
Op Manager Review
 
Event Driven Architectures with Apache Kafka
Event Driven Architectures with Apache KafkaEvent Driven Architectures with Apache Kafka
Event Driven Architectures with Apache Kafka
 
Monitoreo sencillo de la infraestructura, de la ingesta a la visualización
Monitoreo sencillo de la infraestructura, de la ingesta a la visualizaciónMonitoreo sencillo de la infraestructura, de la ingesta a la visualización
Monitoreo sencillo de la infraestructura, de la ingesta a la visualización
 
A Short Presentation on Kafka
A Short Presentation on KafkaA Short Presentation on Kafka
A Short Presentation on Kafka
 
New Relic_Heroku_Presentation_Dreamforce11
New Relic_Heroku_Presentation_Dreamforce11New Relic_Heroku_Presentation_Dreamforce11
New Relic_Heroku_Presentation_Dreamforce11
 
SCU Asia System Center: Beyond orchestrator
SCU Asia System Center: Beyond orchestratorSCU Asia System Center: Beyond orchestrator
SCU Asia System Center: Beyond orchestrator
 
[Webinar] AWS Monitoring with Site24x7
[Webinar] AWS Monitoring with Site24x7[Webinar] AWS Monitoring with Site24x7
[Webinar] AWS Monitoring with Site24x7
 
IBM Maximo Performance Tuning
IBM Maximo Performance TuningIBM Maximo Performance Tuning
IBM Maximo Performance Tuning
 
Simulation Projects in Matlab
Simulation Projects in MatlabSimulation Projects in Matlab
Simulation Projects in Matlab
 
An introduction to Zen 6 - integrated network and customer experience analytics
An introduction to Zen 6 - integrated network and customer experience analyticsAn introduction to Zen 6 - integrated network and customer experience analytics
An introduction to Zen 6 - integrated network and customer experience analytics
 
Choosing the Right Transformer for Your Data Challenge
Choosing the Right Transformer for Your Data ChallengeChoosing the Right Transformer for Your Data Challenge
Choosing the Right Transformer for Your Data Challenge
 
Brazil
BrazilBrazil
Brazil
 

Viewers also liked

Managing and Monitoring Application Performance
Managing and Monitoring Application PerformanceManaging and Monitoring Application Performance
Managing and Monitoring Application PerformanceSebastian Marek
 
Nagios Conference 2011 - Tony Roman - Cacti Workshop
Nagios Conference 2011 - Tony Roman - Cacti WorkshopNagios Conference 2011 - Tony Roman - Cacti Workshop
Nagios Conference 2011 - Tony Roman - Cacti WorkshopNagios
 
Multi-Platform Application Monitoring
Multi-Platform Application Monitoring Multi-Platform Application Monitoring
Multi-Platform Application Monitoring HelpSystems
 
Application Monitoring Tools
Application Monitoring ToolsApplication Monitoring Tools
Application Monitoring ToolsAdtech Global
 
Application Monitoring using Datadog
Application Monitoring using DatadogApplication Monitoring using Datadog
Application Monitoring using DatadogMukta Aphale
 
Conference on Nagios: Reinhard Scheck on Cacti
Conference on Nagios: Reinhard Scheck on CactiConference on Nagios: Reinhard Scheck on Cacti
Conference on Nagios: Reinhard Scheck on CactiWürth Phoenix
 
Infrastructure - Monitoring - Cacti
Infrastructure - Monitoring - CactiInfrastructure - Monitoring - Cacti
Infrastructure - Monitoring - CactiFrédéric FAURE
 
Network Monitoring System and Alert System
Network Monitoring System and Alert SystemNetwork Monitoring System and Alert System
Network Monitoring System and Alert SystemSupawit Wannapila
 
DevOps@Work 2017 - Application insights more control, more power
DevOps@Work 2017 - Application insights more control, more powerDevOps@Work 2017 - Application insights more control, more power
DevOps@Work 2017 - Application insights more control, more powerRoberto Albano
 
Building Resilient Log Aggregation Pipeline with Elasticsearch & Kafka
Building Resilient Log Aggregation Pipeline with Elasticsearch & KafkaBuilding Resilient Log Aggregation Pipeline with Elasticsearch & Kafka
Building Resilient Log Aggregation Pipeline with Elasticsearch & KafkaSematext Group, Inc.
 
Monitoring Kafka w/ Prometheus
Monitoring Kafka w/ PrometheusMonitoring Kafka w/ Prometheus
Monitoring Kafka w/ Prometheuskawamuray
 
NoSQL - "simple" web monitoring
NoSQL - "simple" web monitoringNoSQL - "simple" web monitoring
NoSQL - "simple" web monitoringSamir Siqueira
 
TEDx Manchester: AI & The Future of Work
TEDx Manchester: AI & The Future of WorkTEDx Manchester: AI & The Future of Work
TEDx Manchester: AI & The Future of WorkVolker Hirsch
 
How to Become a Thought Leader in Your Niche
How to Become a Thought Leader in Your NicheHow to Become a Thought Leader in Your Niche
How to Become a Thought Leader in Your NicheLeslie Samuel
 

Viewers also liked (19)

Managing and Monitoring Application Performance
Managing and Monitoring Application PerformanceManaging and Monitoring Application Performance
Managing and Monitoring Application Performance
 
PEARS Systems
PEARS SystemsPEARS Systems
PEARS Systems
 
coms_pdf
coms_pdfcoms_pdf
coms_pdf
 
Cacti
CactiCacti
Cacti
 
Nagios Conference 2011 - Tony Roman - Cacti Workshop
Nagios Conference 2011 - Tony Roman - Cacti WorkshopNagios Conference 2011 - Tony Roman - Cacti Workshop
Nagios Conference 2011 - Tony Roman - Cacti Workshop
 
Snapdeal
SnapdealSnapdeal
Snapdeal
 
Multi-Platform Application Monitoring
Multi-Platform Application Monitoring Multi-Platform Application Monitoring
Multi-Platform Application Monitoring
 
Application Monitoring Tools
Application Monitoring ToolsApplication Monitoring Tools
Application Monitoring Tools
 
Application Monitoring using Datadog
Application Monitoring using DatadogApplication Monitoring using Datadog
Application Monitoring using Datadog
 
Conference on Nagios: Reinhard Scheck on Cacti
Conference on Nagios: Reinhard Scheck on CactiConference on Nagios: Reinhard Scheck on Cacti
Conference on Nagios: Reinhard Scheck on Cacti
 
Infrastructure - Monitoring - Cacti
Infrastructure - Monitoring - CactiInfrastructure - Monitoring - Cacti
Infrastructure - Monitoring - Cacti
 
Docker From Scratch
Docker From ScratchDocker From Scratch
Docker From Scratch
 
Network Monitoring System and Alert System
Network Monitoring System and Alert SystemNetwork Monitoring System and Alert System
Network Monitoring System and Alert System
 
DevOps@Work 2017 - Application insights more control, more power
DevOps@Work 2017 - Application insights more control, more powerDevOps@Work 2017 - Application insights more control, more power
DevOps@Work 2017 - Application insights more control, more power
 
Building Resilient Log Aggregation Pipeline with Elasticsearch & Kafka
Building Resilient Log Aggregation Pipeline with Elasticsearch & KafkaBuilding Resilient Log Aggregation Pipeline with Elasticsearch & Kafka
Building Resilient Log Aggregation Pipeline with Elasticsearch & Kafka
 
Monitoring Kafka w/ Prometheus
Monitoring Kafka w/ PrometheusMonitoring Kafka w/ Prometheus
Monitoring Kafka w/ Prometheus
 
NoSQL - "simple" web monitoring
NoSQL - "simple" web monitoringNoSQL - "simple" web monitoring
NoSQL - "simple" web monitoring
 
TEDx Manchester: AI & The Future of Work
TEDx Manchester: AI & The Future of WorkTEDx Manchester: AI & The Future of Work
TEDx Manchester: AI & The Future of Work
 
How to Become a Thought Leader in Your Niche
How to Become a Thought Leader in Your NicheHow to Become a Thought Leader in Your Niche
How to Become a Thought Leader in Your Niche
 

Similar to Real-time Application Monitoring

Running an erlang based messaging system on AWS
Running an erlang based messaging system on AWSRunning an erlang based messaging system on AWS
Running an erlang based messaging system on AWSLahav Savir
 
Monitoring Node.js Microservices on CloudFoundry with Open Source Tools and a...
Monitoring Node.js Microservices on CloudFoundry with Open Source Tools and a...Monitoring Node.js Microservices on CloudFoundry with Open Source Tools and a...
Monitoring Node.js Microservices on CloudFoundry with Open Source Tools and a...Tony Erwin
 
Infrastructure monitoring made easy, from ingest to insight
 Infrastructure monitoring made easy, from ingest to insight Infrastructure monitoring made easy, from ingest to insight
Infrastructure monitoring made easy, from ingest to insightElasticsearch
 
SaaS Enablement of your existing application (Cloud Slam 2010)
SaaS Enablement of your existing application (Cloud Slam 2010)SaaS Enablement of your existing application (Cloud Slam 2010)
SaaS Enablement of your existing application (Cloud Slam 2010)Nati Shalom
 
Using AWS To Build A Scalable Machine Data Analytics Service
Using AWS To Build A Scalable Machine Data Analytics ServiceUsing AWS To Build A Scalable Machine Data Analytics Service
Using AWS To Build A Scalable Machine Data Analytics ServiceChristian Beedgen
 
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...In-Memory Computing Summit
 
Architectures, Frameworks and Infrastructure
Architectures, Frameworks and InfrastructureArchitectures, Frameworks and Infrastructure
Architectures, Frameworks and Infrastructureharendra_pathak
 
Stream Processing with Apache Apex
Stream Processing with Apache ApexStream Processing with Apache Apex
Stream Processing with Apache ApexPramod Immaneni
 
Application performance monitoring with Applications Manager
Application performance monitoring with Applications ManagerApplication performance monitoring with Applications Manager
Application performance monitoring with Applications ManagerManageEngine, Zoho Corporation
 
Re-define network visibility for capacity planning & forecasting with Grafana
Re-define network visibility for capacity planning & forecasting with GrafanaRe-define network visibility for capacity planning & forecasting with Grafana
Re-define network visibility for capacity planning & forecasting with GrafanaBangladesh Network Operators Group
 
A Low-Cost IoT Application for the Urban Traffic of Vehicles, Based on Wirele...
A Low-Cost IoT Application for the Urban Traffic of Vehicles, Based on Wirele...A Low-Cost IoT Application for the Urban Traffic of Vehicles, Based on Wirele...
A Low-Cost IoT Application for the Urban Traffic of Vehicles, Based on Wirele...Fatima Qayyum
 
IoT Ingestion & Analytics using Apache Apex - A Native Hadoop Platform
 IoT Ingestion & Analytics using Apache Apex - A Native Hadoop Platform IoT Ingestion & Analytics using Apache Apex - A Native Hadoop Platform
IoT Ingestion & Analytics using Apache Apex - A Native Hadoop PlatformApache Apex
 
November 2013 HUG: Real-time analytics with in-memory grid
November 2013 HUG: Real-time analytics with in-memory gridNovember 2013 HUG: Real-time analytics with in-memory grid
November 2013 HUG: Real-time analytics with in-memory gridYahoo Developer Network
 
Big Stream Processing Systems, Big Graphs
Big Stream Processing Systems, Big GraphsBig Stream Processing Systems, Big Graphs
Big Stream Processing Systems, Big GraphsPetr Novotný
 
Big Data Berlin v8.0 Stream Processing with Apache Apex
Big Data Berlin v8.0 Stream Processing with Apache Apex Big Data Berlin v8.0 Stream Processing with Apache Apex
Big Data Berlin v8.0 Stream Processing with Apache Apex Apache Apex
 
Thomas Weise, Apache Apex PMC Member and Architect/Co-Founder, DataTorrent - ...
Thomas Weise, Apache Apex PMC Member and Architect/Co-Founder, DataTorrent - ...Thomas Weise, Apache Apex PMC Member and Architect/Co-Founder, DataTorrent - ...
Thomas Weise, Apache Apex PMC Member and Architect/Co-Founder, DataTorrent - ...Dataconomy Media
 
PEARC17: Live Integrated Visualization Environment: An Experiment in General...
PEARC17: Live Integrated Visualization Environment: An Experiment in General...PEARC17: Live Integrated Visualization Environment: An Experiment in General...
PEARC17: Live Integrated Visualization Environment: An Experiment in General...moneyjh
 

Similar to Real-time Application Monitoring (20)

Running an erlang based messaging system on AWS
Running an erlang based messaging system on AWSRunning an erlang based messaging system on AWS
Running an erlang based messaging system on AWS
 
Monitoring Node.js Microservices on CloudFoundry with Open Source Tools and a...
Monitoring Node.js Microservices on CloudFoundry with Open Source Tools and a...Monitoring Node.js Microservices on CloudFoundry with Open Source Tools and a...
Monitoring Node.js Microservices on CloudFoundry with Open Source Tools and a...
 
Infrastructure monitoring made easy, from ingest to insight
 Infrastructure monitoring made easy, from ingest to insight Infrastructure monitoring made easy, from ingest to insight
Infrastructure monitoring made easy, from ingest to insight
 
SaaS Enablement of your existing application (Cloud Slam 2010)
SaaS Enablement of your existing application (Cloud Slam 2010)SaaS Enablement of your existing application (Cloud Slam 2010)
SaaS Enablement of your existing application (Cloud Slam 2010)
 
Using AWS To Build A Scalable Machine Data Analytics Service
Using AWS To Build A Scalable Machine Data Analytics ServiceUsing AWS To Build A Scalable Machine Data Analytics Service
Using AWS To Build A Scalable Machine Data Analytics Service
 
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
 
Architectures, Frameworks and Infrastructure
Architectures, Frameworks and InfrastructureArchitectures, Frameworks and Infrastructure
Architectures, Frameworks and Infrastructure
 
Stream Processing with Apache Apex
Stream Processing with Apache ApexStream Processing with Apache Apex
Stream Processing with Apache Apex
 
Application performance monitoring with Applications Manager
Application performance monitoring with Applications ManagerApplication performance monitoring with Applications Manager
Application performance monitoring with Applications Manager
 
Re-define network visibility for capacity planning & forecasting with Grafana
Re-define network visibility for capacity planning & forecasting with GrafanaRe-define network visibility for capacity planning & forecasting with Grafana
Re-define network visibility for capacity planning & forecasting with Grafana
 
A Low-Cost IoT Application for the Urban Traffic of Vehicles, Based on Wirele...
A Low-Cost IoT Application for the Urban Traffic of Vehicles, Based on Wirele...A Low-Cost IoT Application for the Urban Traffic of Vehicles, Based on Wirele...
A Low-Cost IoT Application for the Urban Traffic of Vehicles, Based on Wirele...
 
VNM_3.PPT
VNM_3.PPTVNM_3.PPT
VNM_3.PPT
 
IoT Ingestion & Analytics using Apache Apex - A Native Hadoop Platform
 IoT Ingestion & Analytics using Apache Apex - A Native Hadoop Platform IoT Ingestion & Analytics using Apache Apex - A Native Hadoop Platform
IoT Ingestion & Analytics using Apache Apex - A Native Hadoop Platform
 
Algorithmic Trading
Algorithmic TradingAlgorithmic Trading
Algorithmic Trading
 
Prometheus with Grafana - AddWeb Solution
Prometheus with Grafana - AddWeb SolutionPrometheus with Grafana - AddWeb Solution
Prometheus with Grafana - AddWeb Solution
 
November 2013 HUG: Real-time analytics with in-memory grid
November 2013 HUG: Real-time analytics with in-memory gridNovember 2013 HUG: Real-time analytics with in-memory grid
November 2013 HUG: Real-time analytics with in-memory grid
 
Big Stream Processing Systems, Big Graphs
Big Stream Processing Systems, Big GraphsBig Stream Processing Systems, Big Graphs
Big Stream Processing Systems, Big Graphs
 
Big Data Berlin v8.0 Stream Processing with Apache Apex
Big Data Berlin v8.0 Stream Processing with Apache Apex Big Data Berlin v8.0 Stream Processing with Apache Apex
Big Data Berlin v8.0 Stream Processing with Apache Apex
 
Thomas Weise, Apache Apex PMC Member and Architect/Co-Founder, DataTorrent - ...
Thomas Weise, Apache Apex PMC Member and Architect/Co-Founder, DataTorrent - ...Thomas Weise, Apache Apex PMC Member and Architect/Co-Founder, DataTorrent - ...
Thomas Weise, Apache Apex PMC Member and Architect/Co-Founder, DataTorrent - ...
 
PEARC17: Live Integrated Visualization Environment: An Experiment in General...
PEARC17: Live Integrated Visualization Environment: An Experiment in General...PEARC17: Live Integrated Visualization Environment: An Experiment in General...
PEARC17: Live Integrated Visualization Environment: An Experiment in General...
 

Recently uploaded

What is Binary Language? Computer Number Systems
What is Binary Language?  Computer Number SystemsWhat is Binary Language?  Computer Number Systems
What is Binary Language? Computer Number SystemsJheuzeDellosa
 
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time ApplicationsUnveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time ApplicationsAlberto González Trastoy
 
chapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptchapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptkotipi9215
 
why an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfwhy an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfjoe51371421
 
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideBuilding Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideChristina Lin
 
Hand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptxHand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptxbodapatigopi8531
 
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...Christina Lin
 
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer DataAdobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer DataBradBedford3
 
Unit 1.1 Excite Part 1, class 9, cbse...
Unit 1.1 Excite Part 1, class 9, cbse...Unit 1.1 Excite Part 1, class 9, cbse...
Unit 1.1 Excite Part 1, class 9, cbse...aditisharan08
 
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASEBATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASEOrtus Solutions, Corp
 
TECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service providerTECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service providermohitmore19
 
DNT_Corporate presentation know about us
DNT_Corporate presentation know about usDNT_Corporate presentation know about us
DNT_Corporate presentation know about usDynamic Netsoft
 
A Secure and Reliable Document Management System is Essential.docx
A Secure and Reliable Document Management System is Essential.docxA Secure and Reliable Document Management System is Essential.docx
A Secure and Reliable Document Management System is Essential.docxComplianceQuest1
 
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...MyIntelliSource, Inc.
 
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...harshavardhanraghave
 
Asset Management Software - Infographic
Asset Management Software - InfographicAsset Management Software - Infographic
Asset Management Software - InfographicHr365.us smith
 
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdfLearn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdfkalichargn70th171
 
5 Signs You Need a Fashion PLM Software.pdf
5 Signs You Need a Fashion PLM Software.pdf5 Signs You Need a Fashion PLM Software.pdf
5 Signs You Need a Fashion PLM Software.pdfWave PLM
 
Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)OPEN KNOWLEDGE GmbH
 

Recently uploaded (20)

What is Binary Language? Computer Number Systems
What is Binary Language?  Computer Number SystemsWhat is Binary Language?  Computer Number Systems
What is Binary Language? Computer Number Systems
 
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time ApplicationsUnveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
 
chapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptchapter--4-software-project-planning.ppt
chapter--4-software-project-planning.ppt
 
why an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfwhy an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdf
 
Call Girls In Mukherjee Nagar 📱 9999965857 🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...
Call Girls In Mukherjee Nagar 📱  9999965857  🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...Call Girls In Mukherjee Nagar 📱  9999965857  🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...
Call Girls In Mukherjee Nagar 📱 9999965857 🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...
 
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideBuilding Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
 
Hand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptxHand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptx
 
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
 
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer DataAdobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
 
Unit 1.1 Excite Part 1, class 9, cbse...
Unit 1.1 Excite Part 1, class 9, cbse...Unit 1.1 Excite Part 1, class 9, cbse...
Unit 1.1 Excite Part 1, class 9, cbse...
 
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASEBATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
 
TECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service providerTECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service provider
 
DNT_Corporate presentation know about us
DNT_Corporate presentation know about usDNT_Corporate presentation know about us
DNT_Corporate presentation know about us
 
A Secure and Reliable Document Management System is Essential.docx
A Secure and Reliable Document Management System is Essential.docxA Secure and Reliable Document Management System is Essential.docx
A Secure and Reliable Document Management System is Essential.docx
 
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
 
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
 
Asset Management Software - Infographic
Asset Management Software - InfographicAsset Management Software - Infographic
Asset Management Software - Infographic
 
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdfLearn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
 
5 Signs You Need a Fashion PLM Software.pdf
5 Signs You Need a Fashion PLM Software.pdf5 Signs You Need a Fashion PLM Software.pdf
5 Signs You Need a Fashion PLM Software.pdf
 
Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)
 

Real-time Application Monitoring

  • 2. Central App Monitoring Platform • Graphite – Carbon – Whisper – Webapp • Simple time series data • Highly scalable, optimized for I/O • Queuing/Caching • Capability to club non-persisted metrics with persisted while returning data for monitoring (Real-time) • Carbon-relay -> Clustering • Works fine for lacs of metrics coming every second • Can work with any client (statd daemon, jmxtrans, yammer etc. or we can write our own easily) because of simple spec
  • 3. Central App Monitoring Platform • Metrics names recommended to be given in reverse dns format – Easy to aggregate metrics for multiple servers of a system (eg. com.snapdeal.cams.*) • Supports both TCP & UDP • Lots of mathematical functions to derive, aggregate, join multiple metrics • Grafana used for visualization (though graphite also provides UI) • Grafana – Cutting edge graphs and very easy configuration through GUI – Easily integrates with Graphite APIs – Integrated with elastic search for faster response on graphs – Many more calculations (average, max, min etc.) possible on UI itself
  • 4. Application Metrics – Yammer • Simple Java library • Different kind of Metrics – Gauge – Counter – Meter – Histogram – Timer – HealthCheck • Supports both push (configurable frequency) or pull (jmx or http) • Also sends JVM Info by default • Very less overhead on application – Just instantiate POJOs in between your existing code – All processing in app’s control – No need to write separate code – Pushing via separate thread pool (no impact on app even if server is down) • Yammer metrics can be exposed as MBeans (for JMX) as well.
  • 5. Application Metrics – Jmxtrans • What if you want to monitor your existing Mbeans via central monitoring platform (like graphite) or there are 3rd party frameworks already exposing data using JMX • Use Jmxtrans to integrate with Graphite server • Frequency of pushing can be configured • A separate centralized Jmxtrans App can suffice for multiple applications (though running as agent in same process also possible)
  • 6. Alerting - Nagios • Little crude, but flexible & powerful way to alert. • Client/Server Architecture