Shinken ist eine Nagios- Reimplementierung welche basierend auf einer modernen Architektur und erstaunlicher Performance auf große Infrastrukturen abzielt. Da es mit Nagios und seinen Features kompatibel ist, wird es bereits in Produktionsumgebungen eingesetzt. Die Kernfunktionalitäten und verteilte Infrastrukturen sind bereits umgesetzt, aber die IT und somit das Monitoring entwickeln sich stetig weiter. In dem Vortrag wird auf diese Entwicklungen wie Business Rules und Business Impacts eingegangen. Darüber hinaus wird das neue Shinken-UI und CLI-Tool präsentiert.

1. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Shinken, Always go further
Gabès Jean & Gerhard Lausser
OSMC 2011, November 29
1 / 36
Shinken, Always go further

2. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
What is Shinken?
Shinken is an open source Nagios R like tool,
redesigned and rewritten from scratch. Its main goal
is to meet today's system monitoring requirements
while still following compatibility to Nagios R
2 / 36
Shinken, Always go further

3. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Shinken history
Start as a POC for Nagios R to enhance performances
... and really got huge performances
.. but also easy distributed architecture!
3 / 36
Shinken, Always go further

4. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Shinken history
Start as a POC for Nagios R to enhance performances
... and really got huge performances
.. but also easy distributed architecture!
No answers from Nagios devs about theses ideas
More ecient to continue this Python code than to backport
in C
No Nagios-devel branch, so we start our own project
3 / 36
Shinken, Always go further

5. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Not a one man show
4 / 36
Shinken, Always go further

6. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Technical enhancements
Python? Need bench
Let's look at the performances
Simple check (random result, 2% critical, in C)
Xeon 3Ghz 4 cores, 12Go ram
Checks with a 5min interval
No ndo/ido/broker modules. Just the core.
We search for the reaching the sky latency limit
5 / 36
Shinken, Always go further

7. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Technical enhancements
Performances
Maximum number of services checks in 5min :
Nagios core (3.2) : 30K
6 / 36
Shinken, Always go further

8. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Technical enhancements
Performances
Maximum number of services checks in 5min :
Nagios core (3.2) : 30K
Centreon-engine (last week git): 25K
6 / 36
Shinken, Always go further

9. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Technical enhancements
Performances
Maximum number of services checks in 5min :
Nagios core (3.2) : 30K
Centreon-engine (last week git): 25K
Icinga (last week git) : 31K
6 / 36
Shinken, Always go further

10. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Technical enhancements
Performances
Maximum number of services checks in 5min :
Nagios core (3.2) : 30K
Centreon-engine (last week git): 25K
Icinga (last week git) : 31K
Shinken : 120K
Conclusion : Python is not slow
6 / 36
Shinken, Always go further

11. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Technical enhancements
Shinken architecture...
A daemon for each role
7 / 36
Shinken, Always go further

12. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Technical enhancements
...an amazing architecture!
Loadbalanced architecture
Smart global conguration split
No central relational database : it just scales!
Even hot poller addition...
8 / 36
Shinken, Always go further

13. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Technical enhancements
...an amazing architecture!
Loadbalanced architecture
Smart global conguration split
No central relational database : it just scales!
Even hot poller addition...
High availability (RAID5 type)
No performance impacts
Retention data is also distributed (memcache, redis, ...)
8 / 36
Shinken, Always go further

14. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Technical enhancements
...an amazing architecture!
Loadbalanced architecture
Smart global conguration split
No central relational database : it just scales!
Even hot poller addition...
High availability (RAID5 type)
No performance impacts
Retention data is also distributed (memcache, redis, ...)
DMZ management
Lan to DMZ connexions only
Receiver daemon in the DMZ (traps, NSCA, ...)
8 / 36
Shinken, Always go further

15. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Technical enhancements
Datacenters and/or LAN distri-
bution
Two levels for load distribution (active/passive)
Realm : datacenter distribution, hierarchical tree
9 / 36
Shinken, Always go further

16. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Technical enhancements
Datacenters and/or LAN distri-
bution
Two levels for load distribution (active/passive)
Realm : datacenter distribution, hierarchical tree
Poller/reactionner tag : LAN(s) distribution
9 / 36
Shinken, Always go further

17. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Technical enhancements
Central data and notications
10 / 36
Shinken, Always go further

18. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Technical enhancements
On datacenter data and noti-
cations
11 / 36
Shinken, Always go further

19. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Technical enhancements
LAN level : a DMZ
12 / 36
Shinken, Always go further

20. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Technical enhancements
Discovery feature
... because it's always easier to start with real servers :)
Modular
Discovery scripts that output data
Used by creation rules
Samples discovery scripts : Nmap and VMware
Lot of sample creation rules
13 / 36
Shinken, Always go further

21. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
The today's IT need monitoring
core enhancements
14 / 36
Shinken, Always go further

22. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
The today's IT need monitoring
core enhancements
Sample of a today IT environnement
14 / 36
Shinken, Always go further

23. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
Business impact
Some end-user apps are more important than others
15 / 36
Shinken, Always go further

24. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
Business impact
Some end-user apps are more important than others
IT is not important by itself
15 / 36
Shinken, Always go further

25. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
Business impact
Some end-user apps are more important than others
IT is not important by itself
If an IT element is a root problem of an important impact, it
becomes important
15 / 36
Shinken, Always go further

26. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
Business impact
Some end-user apps are more important than others
IT is not important by itself
If an IT element is a root problem of an important impact, it
becomes important
Don't tag IT, only end-user apps
15 / 36
Shinken, Always go further

27. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
Business impact
Some end-user apps are more important than others
IT is not important by itself
If an IT element is a root problem of an important impact, it
becomes important
Don't tag IT, only end-user apps
Notication ltering (min_business_impact)
Modulation : business impact changes during the day/week
15 / 36
Shinken, Always go further

28. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
Qualication app impact
No SMS
16 / 36
Shinken, Always go further

29. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
Production app impact
The admin will need a coee :)
17 / 36
Shinken, Always go further

30. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
Business rules
Need to see one app as one element
Managers like summaries, not details
18 / 36
Shinken, Always go further

31. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
Business rules
Need to see one app as one element
Managers like summaries, not details
With powerful rules ( | Xof: X,Y,Zof:)
18 / 36
Shinken, Always go further

32. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
Business rules
Need to see one app as one element
Managers like summaries, not details
With powerful rules ( | Xof: X,Y,Zof:)
Need to be in the core to keep root problems/impact
BI linking
Dene on your host/services with just check command
18 / 36
Shinken, Always go further

33. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
Business rules
Need to see one app as one element
Managers like summaries, not details
With powerful rules ( | Xof: X,Y,Zof:)
Need to be in the core to keep root problems/impact
BI linking
Dene on your host/services with just check command
Can be useful for network installations too
18 / 36
Shinken, Always go further

34. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
Example of a business rules
19 / 36
Shinken, Always go further

35. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
VMotion support
20 / 36
Shinken, Always go further

36. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
VMware auto-linking VMo-
tion
21 / 36
Shinken, Always go further

37. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
VMware auto-linking VMo-
tion
Virtual environnements are everywhere
Cannot follow all VM=ESX links with just at les conf
21 / 36
Shinken, Always go further

38. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
VMware auto-linking VMo-
tion
Virtual environnements are everywhere
Cannot follow all VM=ESX links with just at les conf
Shinken can ask vSphere links, and auto-create dependencies
21 / 36
Shinken, Always go further

39. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
VMware auto-linking VMo-
tion
Virtual environnements are everywhere
Cannot follow all VM=ESX links with just at les conf
Shinken can ask vSphere links, and auto-create dependencies
And if a VMotion is done, it can detect it, and re-link without
restart!
21 / 36
Shinken, Always go further

40. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Monitoring core enhancement
VMware auto-linking VMo-
tion
Virtual environnements are everywhere
Cannot follow all VM=ESX links with just at les conf
Shinken can ask vSphere links, and auto-create dependencies
And if a VMotion is done, it can detect it, and re-link without
restart!
Here for VMware, but can work with whatever you want
21 / 36
Shinken, Always go further

41. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Low Shinken acceptance in UIs
An UI need :
Strong separation between problems and impacts
22 / 36
Shinken, Always go further

42. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Low Shinken acceptance in UIs
An UI need :
Strong separation between problems and impacts
Focus on business impacts
22 / 36
Shinken, Always go further

43. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Low Shinken acceptance in UIs
An UI need :
Strong separation between problems and impacts
Focus on business impacts
Dependencies are the keys, show them all!
22 / 36
Shinken, Always go further

44. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Low Shinken acceptance in UIs
An UI need :
Strong separation between problems and impacts
Focus on business impacts
Dependencies are the keys, show them all!
Aggregate all load balanced elements
22 / 36
Shinken, Always go further

45. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Low Shinken acceptance in UIs
An UI need :
Strong separation between problems and impacts
Focus on business impacts
Dependencies are the keys, show them all!
Aggregate all load balanced elements
HA by design
22 / 36
Shinken, Always go further

46. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Shinken WebUI!
Problems and impacts in dierent views
23 / 36
Shinken, Always go further

47. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Shinken WebUI!
Problems and impacts in dierent views
ALL is sorted by business impacts
23 / 36
Shinken, Always go further

48. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Shinken WebUI!
Problems and impacts in dierent views
ALL is sorted by business impacts
Dependencies are show as trees and graphs
23 / 36
Shinken, Always go further

49. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Shinken WebUI!
Problems and impacts in dierent views
ALL is sorted by business impacts
Dependencies are show as trees and graphs
Shinken architecture integrated (aggregation and HA)
23 / 36
Shinken, Always go further

50. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Shinken WebUI!
Problems and impacts in dierent views
ALL is sorted by business impacts
Dependencies are show as trees and graphs
Shinken architecture integrated (aggregation and HA)
No relational database backend of course!
23 / 36
Shinken, Always go further

51. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Fresh start
Very visual (dependencies, alerts, graphs)
24 / 36
Shinken, Always go further

52. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Fresh start
Very visual (dependencies, alerts, graphs)
HTML5 everywhere
24 / 36
Shinken, Always go further

53. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Fresh start
Very visual (dependencies, alerts, graphs)
HTML5 everywhere
Only useful infos are shown, others are hidden by default
24 / 36
Shinken, Always go further

54. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Fresh start
Very visual (dependencies, alerts, graphs)
HTML5 everywhere
Only useful infos are shown, others are hidden by default
Linkable with other UIs (like PNP)
24 / 36
Shinken, Always go further

55. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Fresh start
Very visual (dependencies, alerts, graphs)
HTML5 everywhere
Only useful infos are shown, others are hidden by default
Linkable with other UIs (like PNP)
Even your boss will understand it!
24 / 36
Shinken, Always go further

56. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Fresh start
Very visual (dependencies, alerts, graphs)
HTML5 everywhere
Only useful infos are shown, others are hidden by default
Linkable with other UIs (like PNP)
Even your boss will understand it!
and so will night shift operators
24 / 36
Shinken, Always go further

57. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Two main (incompatible) user
types
Boss : want to see end-user apps (and why it's down...)
Admins : want to see what IT elements are the problems
25 / 36
Shinken, Always go further

58. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Two main (incompatible) user
types
Boss : want to see end-user apps (and why it's down...)
Admins : want to see what IT elements are the problems
Root problems VS impact view
No one want to see both element types
All sorted by business impact of course!
25 / 36
Shinken, Always go further

59. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Sample case
26 / 36
Shinken, Always go further

60. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Boss : Twitter like for (huge)
impacts
27 / 36
Shinken, Always go further

61. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Admin : problems (and only
them)
28 / 36
Shinken, Always go further

62. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Admin : problems (and only
them)
28 / 36
Shinken, Always go further

63. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Admin : problems view
29 / 36
Shinken, Always go further

64. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Problem detail page
30 / 36
Shinken, Always go further

65. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Impact detail page
31 / 36
Shinken, Always go further

66. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Dependency graph
32 / 36
Shinken, Always go further

67. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
Get your UI
Just test it : http://demo-shinken.web4all.fr:7767
Like Shinken, the UI is modular (like for PNP inclusion)
Like the rest of Shinken, it's Python : rapid development!
33 / 36
Shinken, Always go further

68. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
I forgot about the Shinken
bench...
34 / 36
Shinken, Always go further

69. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
UI enhancement
I forgot about the Shinken
bench...
The 120k run WAS with the WebUI running
34 / 36
Shinken, Always go further

70. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Conclusion
Where is Shinken going?
The distributed architecture is done (and powerful!)
35 / 36
Shinken, Always go further

71. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Conclusion
Where is Shinken going?
The distributed architecture is done (and powerful!)
Lot of improvements in the core compare to Nagios R
35 / 36
Shinken, Always go further

72. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Conclusion
Where is Shinken going?
The distributed architecture is done (and powerful!)
Lot of improvements in the core compare to Nagios R
UI in a good way
35 / 36
Shinken, Always go further

73. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Conclusion
Where is Shinken going?
The distributed architecture is done (and powerful!)
Lot of improvements in the core compare to Nagios R
UI in a good way
Project from a tool point of view to a true solution
35 / 36
Shinken, Always go further

74. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Conclusion
Where is Shinken going?
The distributed architecture is done (and powerful!)
Lot of improvements in the core compare to Nagios R
UI in a good way
Project from a tool point of view to a true solution
Shinken evaluation on initiative of the management, not admin
35 / 36
Shinken, Always go further

75. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Conclusion
Where is Shinken going?
The distributed architecture is done (and powerful!)
Lot of improvements in the core compare to Nagios R
UI in a good way
Project from a tool point of view to a true solution
Shinken evaluation on initiative of the management, not admin
Commercial solutions based on Nagios are looking for a plan B
Professional services are coming :)
35 / 36
Shinken, Always go further

76. Technical enhancements Monitoring core enhancement UI enhancement Conclusion
Conclusion
Thanks a lot
Thanks for your attention.
Got some questions?
36 / 36
Shinken, Always go further