Mario Mann ist Consultant bei der NovaTec Consulting GmbH. Seit seinem erfolgreichen Studium der Informatik ist er als Performance Engineer bei einer Großbank im Einsatz. Parallel dazu entwickelt er an inspectIT mit und ist an Themen rund um APM - Application Performance Management - engagiert.

1. Application Performance Management
with
Open Source Tooling

2. Disruptive Änderung im Nutzerverhalten
2
Ellen DeGeneres‘ Oscars selfie crashes Twitter
[Abbildung: Ellen DeGeneres]
Zusammenfassung
 2.6 Millionen Re-Tweets inerhalb 2 Stunden
 20 Minuten nicht verfügbar
[Abbildung : Twitter.com]
OSMC - Open Source Monitoring Conference

3. 3
Das HealthCare.gov Fiasko
Zusammenfassung
[Kevin Surace, fastcompany.com; Byron Wolf, CNN]
 Eines der schlimmsten Software Projekte
 Hunderte Millionen Dollar wurden investiert
 Ausfälle und langsame Anfragen waren normal
 US-Regierung musste sich entschuldigen
 Google & RedHat mussten aushelfen
Vor dem Start wurde Skalierbarkeit und Performance nie getestet!
[Abbildung: healthcare.gov]
OSMC - Open Source Monitoring Conference

4. Otto.de
OSMC - Open Source Monitoring Conference4
 Johannes Mainusch
 Performance >3s
(2011)
 Performance <2s
(2016)
[video | https://youtu.be/luy1lL8P0KE]

5. Take Away
[Abbildung: http://content.hollywire.com/sites/default/files/Success-Kid.jpg]
5 OSMC - Open Source Monitoring Conference

6. Referenten – Mario Mann
Hintergrund
 Seit 2013 Mitarbeiter der NovaTec Consulting GmbH
 In APM Projekten im Finance- und Automotive-Bereich
Tätigkeitsschwerpunkt
 (Werkzeug-)unabhängige APM Beratung
 Freie / Open Source Alternativen für APM
 Freie / Open Source Alternativen für Lasttests
 Engagement im inspectIT Open-Source Projekt
Mario Mann
@mawtourde
http://www.linkin.com/mario-mann
OSMC - Open Source Monitoring Conference6
mario.mann@novatec-gmbh.de

7. Das NovaTec APM Team
Führende unabhängige APM Beratung
 Aktuell 15 APM Berater
 Kunden: mittelständische bis große Konzerne
(Automobil, Finanzen, Versicherungen)
Unbefangenheit durch Multi-Partner Ansatz
 Partnerschaft mit den führenden APM
Software-Anbietern
 Zusammenarbeit mit neuen Innovatoren
Fördern Open-Source im APM-Bereich
 Eigenes open-source APM Werkzeug
 Bewertung und Kombination existierender
open-source Werkzeuge
OSMC - Open Source Monitoring Conference7

8. Agenda
• Application Performance Management
• inspectIT – The OpenSource APM solution
• Gathering and Visualizing Timeseries Data
• Application Performance Testing & Diagnosis
• WEB Performance Analysis

9. Application Performance Management

10. OSMC - Open Source Monitoring Conference10
Application Performance Management …
…
umfasst den gesamten
Software-Lebenszyklus
…
deckt alle
Systemebenen ab
…
erfordert Zusammenarbeit
unterschiedlicher Bereiche
… ist ein Querschnittsthema!
Anforderungen
Entwurf
EntwicklungTest
Betrieb
Netzwerk
Anwendung
System
Clients
Entwicklung
BetriebBusiness

11. Load Testing
Web Perf.
Testing
Performance
Modelling
Monitoring in
Production Synthetic
Monitoring
Real User
Monitoring
Root Cause
Diagnosis
Regression
Testing
APM
11
Requirements
Design
DevelopmentTesting
Operations
Network
App
System
Clients
Development
OperationsBusiness
… ist Vielseitig und Komplex!
OSMC - Open Source Monitoring Conference

12. OSMC - Open Source Monitoring Conference12
Umfangreiche
Funktionalität
Ausgereift
Bei großen Unternehmen
im Einsatz
Kommerzielle APM Werkzeuge
Gartner‘s Magic Quadrant for Application Performance Monitoring Suites (12/2015),
Cameron Haight, Will Cappelli, Federico De Silva
http://www.dynatrace.com/en/gartner-magic-quadrant-application-performance-monitoring-2015.html
Partner

13. Lizenzkosten
Vendor Lock-In
OSMC - Open Source Monitoring Conference13
Nachteile kommerzieller APM Werkzeuge
Typisches Modell: x $ pro Agent  Skalierung?
Microservices Internet of Things
[Abbildung: http://blog.wso2.com] [Abbildung: Christian Hinkelmann,http://nahverkehrhamburg.de]
Flexibilität Interoperabilität Nachhaltigkeit
Mobile Revolution
[Abbildung: https://uxmag.com]
 Anpassung an eigene
Bedürfnisse
 Fehlerbehebung
 Datenquellen
 Analysewerkzeuge
 Andere APM Werkzeuge
 Produkt eingestellt
 Strategiewechsel des
APM-Herstellers

14. 14
Open Source Werkzeuge im APM-Bereich
Load Testing
Performance
Modeling
Web Performance
Analysis
System &
Resources Monitoring
Real User Monitoring
JRat
JMemProf
Low-Level
Performance Profiling
Monitoring &
Application Deep Dive
OSMC - Open Source Monitoring Conference

15. inspectIT
The Open Source APM solution

16. OSMC - Open Source Monitoring Conference16
What is inspectIT?
pen Source
8 |2
Plattf rm
Pareto-Prinzip
• Fokus auf Hauptfunktionalität
• Erfahrung aus APM Projekten
Plattform-Prinzip
• Integration mit Werkzeugen
• Erweiterbarkeit
Die Open-Source APM Lösung
• Entwicklung seit 2005
• Open Source seit 2015

17. Gathering and Visualizing Timeseries
Data

18. What is APM About?
OSMC - Open Source Monitoring Conference18
Estimate Business ImpactMonitor Application Health
Identify anomalies and
performance issues
Prod. EnvironmentReal Users
Request
Dashboards
Monitoring Tool
Problem Diagnosis

19. Gathering and Visualizing Timeseries Data
Time Series Databases
Graphing ToolsData Collectors
Custom Code
…
Persist data Query & Visualize
OSMC - Open Source Monitoring Conference19

20. Dashboards
OSMC - Open Source Monitoring Conference20

21. Dashboards
OSMC - Open Source Monitoring Conference21

22. The Flaw of Averages
The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty
Sam L. Savage, with illustrations by Jeff Danziger – http://flawofaverages.com
Gil Tene – https://www.youtube.com/watch?v=lJ8ydIuPFeU
://latencytipoftheday.blogspot.de/
OSMC - Open Source Monitoring Conference22

23. Dashboards
OSMC - Open Source Monitoring Conference23

24. Anomaly Detection & Alerting
OSMC - Open Source Monitoring Conference24

25. Missing Business Context
CPU utilization over time
How critical is this peak in CPU utilization?
Asking the right questions:
 How are the users affected?
 Which business transactions are affected?
 Are any time-critical batch jobs affected?
11pm 0am 1am 2am 3am 4am 5am 6am
100 %
50 %
0%
OSMC - Open Source Monitoring Conference25

26. Defining Business Context
OSMC - Open Source Monitoring Conference26

27. Defining Business Context
Hawkular BTM
OSMC - Open Source Monitoring Conference27

28. Problem Diagnosis
 Deep Dive into request execution flow necessary
 Level of Detail vs. Measurement Overhead
 Do not instrument frequently called methods (e.g. Getters, Setters, String class, etc.)
 Instrument system and component boundaries (entry and exit points)
OSMC - Open Source Monitoring Conference28

29. Configuring Instrumentation
OSMC - Open Source Monitoring Conference29

30. Application Performance Testing &
Diagnosis

31. What is Performance Testing About?
Identify & Diagnose
Performance Problems
Evaluate Application Behaviour
under Load
Test Environment
Monitoring Tool
Test Scalability of Application
 load
 e-2-e response times
 throughput
 detailed invocations
 system resource data
 deep dive information
fLoad Generator
Agent
Agent
Agent
Request
Response
OSMC - Open Source Monitoring Conference31

32. 0
1
2
3
1 2 3 4 5 6 7
ResponseTime
[s]
Execution Nr.
e-2-e response times
load data
detailed
measurement
data
System Under Test
Load Driver
Regression Tests
OSMC - Open Source Monitoring Conference32

33. WEB Performance Analysis

34. End User Experience?
[Image | http://cdn2.hubspot.net/hub/2224117/file-4141070752-jpg/blog-files/three-e1409846588737.jpg]
Satisfied Tolerating Frustrated
Performance User Actions Users
 Performance issues
 Functional Errors
 Network (Local ISPs, Mobile network carriers)
 Third party content providers
 First user action
 Last user action
 Did the visit convert?
 Did the visit bounce?
 Device
 Resolution
 Browser Versions
 Geolocation
What is Web Performance Analysis about?
OSMC - Open Source Monitoring Conference34

35. Measuring End-User Experience
OSMC - Open Source Monitoring Conference35

36. Real User
Testing
Synthetic
Monitoring
 Continuous external testing
 Known testing nodes
 Different Locations (ISP, Networks)
 No baseline traffic required
 Competitor Benchmark
 Availability testing (incl. 3rd Party)
e.g., every 30 mins from different nodes
Real User
Monitoring
 Real End Users
 Real Interaction
 Real traffic
External testing with all major
 browsers,
 operating systems,
 mobile devices
and real world data
QoS Validation
QoS ExpectationQoS Optimization
Anomaly
Testing and Monitoring Web Performance
OSMC - Open Source Monitoring Conference36

37. [http://www.webpagetest.org/performance_optimization.php?test=160623_DF_MV7&run=1&cached=0]
webpagetest.org
OSMC - Open Source Monitoring Conference37

38. OSMC - Open Source Monitoring Conference38
webpagetest.org

39. 39
Visueller Vergleich
http://www.webpagetest.org – Tests ausgeführt aus Dulles, VA - Chrome
OSMC - Open Source Monitoring Conference

40. 40
https://dashboard.sitespeed.io/dashboard/db/compare-multiple-sites
OSMC - Open Source Monitoring Conference
Synthetic Testing

41. OSMC - Open Source Monitoring Conference41
Be careful...

42. Conclusion
Monitoring &
Application Deep Dive
Load Testing
Performance
Modeling
Web Performance
Analysis
System &
Resources Monitoring
Real User Monitoring
JRat
JMemProf
Low-Level
Performance Profiling
Open-Source tools sufficiently
address APM needs!
There are lots of them for different purposes!
Combine them!
OSMC - Open Source Monitoring Conference42

43. Ask me anything