Building Your First Data Science Applicatino in MongoDBMongoDB
Ā
Speaker: Robyn Allen, Software Engineer, Central Inventions
Level: 100 (Beginner)
Track: Tutorials
To provide a hands-on opportunity to work with real data, this session will center around a web-hosted quiz application which helps students practice math and memorize vocabulary. After experimenting with a small demonstration dataset (generated by each individual during the workshop), attendees will be guided through working with an anonymized dataset in MongoDB. No prior MongoDB experience is required but attendees are expected to download and install MongoDB Community Edition (available for free from mongodb.com) and have a working Python 3 environment of their choice (e.g., IDLE, free from python.org) installed on a laptop they bring to the workshop.
Prerequisites:
Attendees are expected to bring a laptop with the following software installed:
MongoDB 3.4.x Community Edition
The text editor or IDE of their choice
A working Python 3 environment of their choice
No prior MongoDB experience is required.
What You Will Learn:
- How to load a CSV file into MongoDB using mongoimport and then write queries (using the Mongo shell) to ensure the data appears as expected. Attendees will use a demo version of an online quiz app to generate a small data file of raw session data (which can be accessed via http://strawnoodle.com/api/testdata after logging in to the demo app and answering one or more quiz questions about MongoDB). After studying how the demo app stores session data, attendees will practice using mongoimport to import anonymized session data (provided during the workshop) into MongoDB.
- How to use the aggregation pipeline (in PyMongo) to implement more complicated queries and gain insights from data. Because the sample dataset contains data from a variety of users of different skill levels, queries can be designed which reveal summary statistics for the anonymous user cohort or specific performance of individual users. Participants will receive instruction in using MongoDB aggregation pipelines in order to write powerful, efficient queries with very few lines of code.
- How to write queries to analyze sample data from an online quiz app. Once the sample data has been loaded into MongoDB, participants will be guided in writing basic queries to examine the sample data. Participants will have an opportunity to write queries in the Mongo shell and in Python in order to familiarize themselves with syntax variations and key ideas. Participants will learn how to implement CRUD operations in PyMongo.
It's 10pm: Do You Know Where Your Writes Are?MongoDB
Ā
Speaker: Samantha Ritter, Software Engineer, MongoDB
Level: 200 (Intermediate)
Track: How We Build MongoDB
MongoDB 3.6 delivers three new features to help you develop resilient applications: retriable writes, a cluster-wide killOp command, and zombie cursor cleanup. These features share a common base, an idea called a logical session. This new cluster-wide concept of user state is the quiet magic that allows you to know, with certainty, the status of your operations. MongoDB engineer Samantha Ritter will describe the above features in-depth, discuss when and how logical sessions can be used by applications and administrators, and show you how we implemented sessions for large, distributed systems.
What You Will Learn:
- What logical sessions are and how they are implemented in the server
- How to leverage logical sessions for retriable writes
- How to pull the new cluster-wide killOp emergency break
Building Your First Data Science Applicatino in MongoDBMongoDB
Ā
Speaker: Robyn Allen, Software Engineer, Central Inventions
Level: 100 (Beginner)
Track: Tutorials
To provide a hands-on opportunity to work with real data, this session will center around a web-hosted quiz application which helps students practice math and memorize vocabulary. After experimenting with a small demonstration dataset (generated by each individual during the workshop), attendees will be guided through working with an anonymized dataset in MongoDB. No prior MongoDB experience is required but attendees are expected to download and install MongoDB Community Edition (available for free from mongodb.com) and have a working Python 3 environment of their choice (e.g., IDLE, free from python.org) installed on a laptop they bring to the workshop.
Prerequisites:
Attendees are expected to bring a laptop with the following software installed:
MongoDB 3.4.x Community Edition
The text editor or IDE of their choice
A working Python 3 environment of their choice
No prior MongoDB experience is required.
What You Will Learn:
- How to load a CSV file into MongoDB using mongoimport and then write queries (using the Mongo shell) to ensure the data appears as expected. Attendees will use a demo version of an online quiz app to generate a small data file of raw session data (which can be accessed via http://strawnoodle.com/api/testdata after logging in to the demo app and answering one or more quiz questions about MongoDB). After studying how the demo app stores session data, attendees will practice using mongoimport to import anonymized session data (provided during the workshop) into MongoDB.
- How to use the aggregation pipeline (in PyMongo) to implement more complicated queries and gain insights from data. Because the sample dataset contains data from a variety of users of different skill levels, queries can be designed which reveal summary statistics for the anonymous user cohort or specific performance of individual users. Participants will receive instruction in using MongoDB aggregation pipelines in order to write powerful, efficient queries with very few lines of code.
- How to write queries to analyze sample data from an online quiz app. Once the sample data has been loaded into MongoDB, participants will be guided in writing basic queries to examine the sample data. Participants will have an opportunity to write queries in the Mongo shell and in Python in order to familiarize themselves with syntax variations and key ideas. Participants will learn how to implement CRUD operations in PyMongo.
It's 10pm: Do You Know Where Your Writes Are?MongoDB
Ā
Speaker: Samantha Ritter, Software Engineer, MongoDB
Level: 200 (Intermediate)
Track: How We Build MongoDB
MongoDB 3.6 delivers three new features to help you develop resilient applications: retriable writes, a cluster-wide killOp command, and zombie cursor cleanup. These features share a common base, an idea called a logical session. This new cluster-wide concept of user state is the quiet magic that allows you to know, with certainty, the status of your operations. MongoDB engineer Samantha Ritter will describe the above features in-depth, discuss when and how logical sessions can be used by applications and administrators, and show you how we implemented sessions for large, distributed systems.
What You Will Learn:
- What logical sessions are and how they are implemented in the server
- How to leverage logical sessions for retriable writes
- How to pull the new cluster-wide killOp emergency break
This presentation describes how to configure and leverage ProxySQL with
AWS Aurora,
Azure Database for MySQL
and CloudSQL for MySQL.
It details the various benefits, configuration, and monitoring.
Kernel Recipes 2019 - Formal modeling made easyAnne Nicolas
Ā
Modeling parts of Linux has become a recurring topic. For instance, the memory model, the model for PREEMPT_RT synchronization, and so on. But the term "formal model" causes panic for most of the developers. Mainly because of the complex notations and reasoning that involves formal languages. It seems to be a very theoretical thing, far from our day-by-day reality.
Believe me. Modeling can be more practical than you might guess!
This talk will discuss the challenges and benefits of modeling, based on the experience of developing the PREEMPT_RT model. It will present a methodology for modeling the Linux behavior as Finite-State Machines (automata), using terms that are very known by kernel developers: tracing events! With the particular focus on how to use models for the formal verification of Linux kernel, at runtime, with low overhead, and in many cases, without even modifying Linux kernel!
Daniel Bristot de Oliveira
Nagios Conference 2014 - Jim Prins - Passive Monitoring with NagiosNagios
Ā
Jim Prins's presentation on Passive Monitoring with Nagios.
The presentation was given during the Nagios World Conference North America held Oct 13th - Oct 16th, 2014 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/conference
OSMC 2010 | NSClient++ - what's new? And what's coming! by Michael MedinNETWAYS
Ā
A simple yet powerful and secure monitoring daemon for Windows operating systems, this is what a NSClient++ aims to be. It is built for Nagios, but nothing in the daemon is actually Nagios specific and with little or no change NSClient++ could be also integrated into any monitoring software that supports running user tools for polling. In this presentation we are going to look at the new features of NSClient++ 0.3.x. But perhaps more importantly looking at whatās coming in the 0.4.x version!
Nagios Conference 2014 - Rodrigo Faria - Developing your PluginNagios
Ā
Rodrigo Faria's presentation on Developing your Plugin.
The presentation was given during the Nagios World Conference North America held Oct 13th - Oct 16th, 2014 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/conference
Lee Myers - What To Do When Nagios Notification Don't Meet Your Needs.Nagios
Ā
Lee Myers - What To Do When Nagios Notification Don't Meet Your Needs. - Lee will present how he overcame timeperiod issues, through the use of MK_Livestatus, Pushbullet, and scripts to notify of him of alerts while he is at work. All the user needs to do is execute a command at the start of their shift, and they will receive all their notifications until their shift ends.
Ajuste (tuning) del rendimiento de SQL Server 2008Eduardo Castro
Ā
En el siguiente webcast http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032438450&Culture=es-AR analizamos las herramientas de desempeƱo de SQL Server 2008 y cĆ³mo utilizarlas.
Saludos,
Ing. Eduardo Castro MartĆnez, PhD ā Microsoft SQL Server MVP
http://mswindowscr.org
http://comunidadwindows.org
Costa Rica
Technorati Tags: SQL Server
LiveJournal Tags: SQL Server
del.icio.us Tags: SQL Server
http://ecastrom.blogspot.com
http://ecastrom.wordpress.com
http://ecastrom.spaces.live.com
http://universosql.blogspot.com
http://todosobresql.blogspot.com
http://todosobresqlserver.wordpress.com
http://mswindowscr.org/blogs/sql/default.aspx
http://citicr.org/blogs/noticias/default.aspx
http://sqlserverpedia.blogspot.com/
Les comparto la presentaciĆ³n utilizada en la charla sobre optimizaciĆ³n de desempeƱo de SQL Server.
Saludos,
Eduardo Castro Martinez
http://ecastrom.blogspot.com
http://comunidadwindows.org
This presentation describes how to configure and leverage ProxySQL with
AWS Aurora,
Azure Database for MySQL
and CloudSQL for MySQL.
It details the various benefits, configuration, and monitoring.
Kernel Recipes 2019 - Formal modeling made easyAnne Nicolas
Ā
Modeling parts of Linux has become a recurring topic. For instance, the memory model, the model for PREEMPT_RT synchronization, and so on. But the term "formal model" causes panic for most of the developers. Mainly because of the complex notations and reasoning that involves formal languages. It seems to be a very theoretical thing, far from our day-by-day reality.
Believe me. Modeling can be more practical than you might guess!
This talk will discuss the challenges and benefits of modeling, based on the experience of developing the PREEMPT_RT model. It will present a methodology for modeling the Linux behavior as Finite-State Machines (automata), using terms that are very known by kernel developers: tracing events! With the particular focus on how to use models for the formal verification of Linux kernel, at runtime, with low overhead, and in many cases, without even modifying Linux kernel!
Daniel Bristot de Oliveira
Nagios Conference 2014 - Jim Prins - Passive Monitoring with NagiosNagios
Ā
Jim Prins's presentation on Passive Monitoring with Nagios.
The presentation was given during the Nagios World Conference North America held Oct 13th - Oct 16th, 2014 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/conference
OSMC 2010 | NSClient++ - what's new? And what's coming! by Michael MedinNETWAYS
Ā
A simple yet powerful and secure monitoring daemon for Windows operating systems, this is what a NSClient++ aims to be. It is built for Nagios, but nothing in the daemon is actually Nagios specific and with little or no change NSClient++ could be also integrated into any monitoring software that supports running user tools for polling. In this presentation we are going to look at the new features of NSClient++ 0.3.x. But perhaps more importantly looking at whatās coming in the 0.4.x version!
Nagios Conference 2014 - Rodrigo Faria - Developing your PluginNagios
Ā
Rodrigo Faria's presentation on Developing your Plugin.
The presentation was given during the Nagios World Conference North America held Oct 13th - Oct 16th, 2014 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/conference
Lee Myers - What To Do When Nagios Notification Don't Meet Your Needs.Nagios
Ā
Lee Myers - What To Do When Nagios Notification Don't Meet Your Needs. - Lee will present how he overcame timeperiod issues, through the use of MK_Livestatus, Pushbullet, and scripts to notify of him of alerts while he is at work. All the user needs to do is execute a command at the start of their shift, and they will receive all their notifications until their shift ends.
Ajuste (tuning) del rendimiento de SQL Server 2008Eduardo Castro
Ā
En el siguiente webcast http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032438450&Culture=es-AR analizamos las herramientas de desempeƱo de SQL Server 2008 y cĆ³mo utilizarlas.
Saludos,
Ing. Eduardo Castro MartĆnez, PhD ā Microsoft SQL Server MVP
http://mswindowscr.org
http://comunidadwindows.org
Costa Rica
Technorati Tags: SQL Server
LiveJournal Tags: SQL Server
del.icio.us Tags: SQL Server
http://ecastrom.blogspot.com
http://ecastrom.wordpress.com
http://ecastrom.spaces.live.com
http://universosql.blogspot.com
http://todosobresql.blogspot.com
http://todosobresqlserver.wordpress.com
http://mswindowscr.org/blogs/sql/default.aspx
http://citicr.org/blogs/noticias/default.aspx
http://sqlserverpedia.blogspot.com/
Les comparto la presentaciĆ³n utilizada en la charla sobre optimizaciĆ³n de desempeƱo de SQL Server.
Saludos,
Eduardo Castro Martinez
http://ecastrom.blogspot.com
http://comunidadwindows.org
Similar to Computer monitoring with the Open Monitoring Distribution (20)
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Ā
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
Ā
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Ā
Clients donāt know what they donāt know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clientsā needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
Ā
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. Whatās changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
Ā
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Ā
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
Ā
Computer monitoring with the Open Monitoring Distribution
1. Monitoring with Open Monitoring Distro
TONIGHTāS OUTLINE:
1. Overview of Nagios
2. Check_MK
3. What is the āOpen Monitoring Distributionā?
4. Operating a monitoring system
Kelvin Vanderlip
Oracle Linux systems administrator, Sunrider International, Torrance
kelvin@vanderlip.org
Kel Vanderlip 3-1-2012 UUASC-LA 1
2. Monitoring with Open Monitoring Distro
A thought for the night:
Better to remain silent and be thought a
fool than to speak out and remove all
doubt.
-- Abraham Lincoln (also attr. Confucius)
Kel Vanderlip 3-1-2012 UUASC-LA 2
3. Monitoring with Open Monitoring Distro
In the beginningā¦
Kel Vanderlip 3-1-2012 UUASC-LA 3
4. Monitoring with Open Monitoring Distro
Why do you care about monitoring?
ā¢You choose a job in which success depend on hard disks,
NFS, DNS, DHCP, NIS, mgetty, Cron jobs, postfix, routing,
FTP, swap space, fans, UPS systems, switches, CPU
registersā¦
Kel Vanderlip 3-1-2012 UUASC-LA 4
5. Monitoring with Open Monitoring Distro
So you ask you sole staffer āIs it runningā
He says āI donāt know. Can I install NetSaint?ā
Kel Vanderlip 3-1-2012 UUASC-LA 5
6. Monitoring with Open Monitoring Distro
Time passes:
āNetSaint is not affiliated with World Wide Digital Security, Inc. (WWDSI);
Richard S. Carson and Associates, Inc;
and the marks WEB SAINT, SAINT, SAINTWRITER, SAINTEXPRESS,
and SAINTBASIC owned by Richard S. Carson and Associateā
Kel Vanderlip 3-1-2012 UUASC-LA 6
7. Monitoring with Open Monitoring Distro
Meet Ethan Galstadt:
āThis website stands as as testament to a long-running Open Source project
that began with a simple idea in my mind. I had no inkling of the future success
that NetSaint (and later Nagios) would come by. I almost never released it to the
OSS community, but thank goodness I did. For without the constant flow of ideas
from NetSaint and Nagios users, the project would have died off a long time ago.
Cheers to everyone in the community who has participated in this project at
some point in their life. My hat is off to you...
-Ethan Galstad: Creator, Developer, Founder of NetSaint, Nagios, and Nagios Enterprises
-and happy participant in a wider movementā
Kel Vanderlip 3-1-2012 UUASC-LA 7
8. Monitoring with Open Monitoring Distro
As I said, how do you get started in the monitoring business?
Kel Vanderlip 3-1-2012 UUASC-LA 8
9. Monitoring with Open Monitoring Distro
Your server room grows, and you are still asking yourself: āIs it still working?ā
Kel Vanderlip 3-1-2012 UUASC-LA 9
10. Monitoring with Open Monitoring Distro
All about Nagios:
Nagios is a scheduling engine. It is written in C. In runs on Linux. Its an RPM and a DEB.
Input:
Text configuration files (lots and lots!)
Output:
Schedule many forks to run external monitoring applications,
some locally, some on remote servers.
Input:
Each called monitoring application returns status and performance information
Output:
status.dat, a āsnapshotā text file kept up to date several times a minute
describing the last state for each thing Nagios is checking
Kel Vanderlip 3-1-2012 UUASC-LA 10
11. Monitoring with Open Monitoring Distro
Status.dat is updated 3-6 times a minute:
host {
######################################## host_name=Compellent
# NAGIOS STATE RETENTION FILE modified_attributes=0
# check_command=check-mk-ping
# THIS FILE IS AUTOMATICALLY GENERATED check_period=24X7
# BY NAGIOS. DO NOT MODIFY THIS FILE! notification_period=24X7
######################################## event_handler=
info { has_been_checked=1
created=1330182965 check_execution_time=0.013
version=3.2.3 check_latency=0.135
last_update_check=0 check_type=0
update_available=0 current_state=0
update_uid=1330021387 last_state=0
last_version= last_hard_state=0
new_version= last_event_id=0
} current_event_id=0
program { current_problem_id=0
modified_host_attributes=0 last_problem_id=0
modified_service_attributes=0 plugin_output=OK - 10.10.99.79: rta 0.785ms, lost 0%
enable_notifications=1 long_plugin_output=
active_service_checks_enabled=1 performance_data=rta=0.785ms;200.000;500.000;0; pl=0%;40;80;; rtmax=1.628ms;;;; rtmin=0.426ms;;;;
passive_service_checks_enabled=1 last_check=1330182913
active_host_checks_enabled=1 next_check=1330182974
passive_host_checks_enabled=1 check_options=0
enable_event_handlers=1 current_attempt=1
obsess_over_services=0 max_attempts=1
obsess_over_hosts=0 normal_check_interval=1.000000
check_service_freshness=1 retry_check_interval=1.000000
check_host_freshness=0 state_type=1
enable_flap_detection=1 last_state_change=1330021647
enable_failure_prediction=1 last_hard_state_change=1330021647
process_performance_data=1 last_time_up=1330182914
global_host_event_handler= last_time_down=0
global_service_event_handler= last_time_unreachable=0
next_comment_id=40 notified_on_down=0
next_downtime_id=1 notified_on_unreachable=0
next_event_id=572 last_notification=0
next_problem_id=290 current_notification_number=0
next_notification_id=457 current_notification_id=0
} notifications_enabled=1
state_history=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
}
Kel Vanderlip 3-1-2012 UUASC-LA 11
12. Monitoring with Open Monitoring Distro
More status.dat:
service {
check_options=0
host_name=ebs-soa1
notified_on_unknown=0
service_description=CPU load
notified_on_warning=0
modified_attributes=0
notified_on_critical=0
check_command=check_mk-cpu.loads
current_notification_number=0
check_period=24X7
current_notification_id=0
notification_period=24X7
last_notification=0
event_handler=
notifications_enabled=1
has_been_checked=1
active_checks_enabled=0
check_execution_time=0.000
passive_checks_enabled=1
check_latency=0.316
event_handler_enabled=0
check_type=1
problem_has_been_acknowledged=0
current_state=0
acknowledgement_type=0
last_state=0
flap_detection_enabled=1
last_hard_state=0
failure_prediction_enabled=1
last_event_id=0
process_performance_data=1
current_event_id=0
obsess_over_service=1
current_problem_id=0
is_flapping=0
last_problem_id=0
percent_state_change=0.00
current_attempt=1
check_flapping_recovery_notification=0
max_attempts=1
state_history=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
normal_check_interval=1.000000
}
retry_check_interval=1.000000
state_type=1
last_state_change=1330021648
last_hard_state_change=1330021648
last_time_ok=1330182934
last_time_warning=0
last_time_unknown=0
last_time_critical=0
plugin_output=OK - 15min Load 0.19 at 8 CPUs
long_plugin_output=
performance_data=load1=0.25;40;80;0; load5=0.25;40;80;0; load15=0.19;40;80;0;
last_check=1330182934
next_check=0
The file goes on for about 4 more megabytes. You will never read this.
Kel Vanderlip 3-1-2012 UUASC-LA 12
13. Monitoring with Open Monitoring Distro
When you think of Nagios, you think of its web output:
Nagiosā CGI is a visualization engine. It is written in C. In runs on Linux.
Input:
status.dat
Output:
web pages describing whatās in status.dat
Input:
Mouse clicks from the operator
Output:
changes to what is viewed, and changes to Nagiosā current state
Kel Vanderlip 3-1-2012 UUASC-LA 13
15. Monitoring with Open Monitoring Distro
There are lots of books about using Nagios. I have read most of them,
and they all helped me out.
A good Nagios implementation is a study in organizational behavior.
If you run Nagios, and you find that no one else in the organization ever fixes anything
Based on Nagios findings, stop looking at the screen and start talking to people.
Socially, using Nagios successfully forces you to involve your co-workers. They
Have to ābuy inā to the Nagios outputs, which means they have to understand
what it does and how it reports its findings.
Managerially, keeping the email notification flood from Nagios under control is a
pre-requisite if you want anyone to actually use an email as a basis for corrective action.
Festival (a loud speaker) and SMS work great. An report based on an SQL query works
great.
Creating a Navy-like āOfficer of the watchā worked in the U.K.
Kel Vanderlip 3-1-2012 UUASC-LA 15
16. Monitoring with Open Monitoring Distro
Doing Nagios means:
Visit a server, and poke around. List what is important to check;
For each important thing you want to check:
ļ¼Find or write some code to check it
ļ¼Set limits which your code can test to decide whether what you are
checking is OK, or not
ļ¼Schedule the code to run over and over
ļ¼If the test is not OK, send a message to the interested party (email seems
to be a favorite).
Kel Vanderlip 3-1-2012 UUASC-LA 16
17. Monitoring with Open Monitoring Distro
ļ¼Find or write some code to check it
Grab a check from Nagios libexec apps ā C, Perl, Python, bash ā
and put it where it can perform the check
ļ¼Set limits so your code can decide whether itās OK or not
Configure command line parameters for the check where it will
be called
ļ¼Schedule the code to run over and over
Reconfigure Nagiosās inputs to include the check and run it,
perhaps using a transport mechanism
ļ¼If the test is not OK, send a message to the interested party
Nagios checks return state to Nagios, which can fork to send
notifications
Kel Vanderlip 3-1-2012 UUASC-LA 17
18. Monitoring with Open Monitoring Distro
Nagios transport systems:
ACTIVE CHECKS
PASSIVE CHECKS
EXPORT STATE
SSH WORKS AS WELLā¦
Kel Vanderlip 3-1-2012 UUASC-LA 18
19. Monitoring with Open Monitoring Distro
PROBLEMS WITH THE TRADITIONAL NAGIOS APPROACH:
How many times do you have to visit each server?
How many times do you have to modify Nagiosās input files?
How many times to you discover something you are not monitoring?
Is all this worth it?
Kel Vanderlip 3-1-2012 UUASC-LA 19
20. Monitoring with Open Monitoring Distro
Home is no better. Can you count the servers?
Kel Vanderlip 3-1-2012 UUASC-LA 20
21. Monitoring with Open Monitoring Distro
Welcome to Check_MK!
Kel Vanderlip 3-1-2012 UUASC-LA 21
22. Monitoring with Open Monitoring Distro
So how about a new approach to managing Nagios?
1. Write a shell script which check everything you can think of checking on a Linux
box in one operation
2. Send this script to each server once. Configure each serverās xinetd so that the
script can be called using port 6556
3. Remotely run this script and feed its output to a process which writes a separate
Nagios configuration for each āserviceā found
4. Schedule Nagios to run a single check once a minute: call the remote shell script
over port 6556, and process the result in the ācheckā itself
5. The check returns each individual āserviceā measurement it finds to Nagios by
writing to the Nagios passive āexternal command fileā
Kel Vanderlip 3-1-2012 UUASC-LA 22
23. Monitoring with Open Monitoring Distro
ļ¼ Write a shell script which check everything you can think of checking on a Linux
box in one operation
It is already written for you by M.K., for HP-UX, Linux and Windows, probably others.
ļ¼ Send this script to each server once. Configure each serverās xinetd so that the
script can be called using port 6556
Installing the shell script, creating directories, and reconfiguring and restarting xinetd
are done for you by the check_mk_agent.rpm or .deb
Kel Vanderlip 3-1-2012 UUASC-LA 23
24. Monitoring with Open Monitoring Distro
Getting the check_mk agent installed on a Linux box:
OMD[torrance]:~$ scp /home/kelvinv/check_mk-agent-1.1.12p6-1.noarch.rpm
root@ebsprod-is1:
root@ebsprod-is1's password:
OMD[torrance]:~/etc/check_mk$ ssh ebsprod-is1
torrance@ebsprod-is1's password:
[root@ebsprod-is1 ~]# rpm -Uhv check_mk-agent-1.1.12p6-1.noarch.rpm
Preparing... ###########################################
[100%]
1:check_mk-agent ###########################################
[100%]
Activating startscript of xinetd
Reloading xinetd...
Reloading configuration: [ OK ]
Kel Vanderlip 3-1-2012 UUASC-LA 24
25. Monitoring with Open Monitoring Distro
#!/bin/bash
# +------------------------------------------------------------------+
# | ____ _ _ __ __ _ __ |
# | / ___| |__ ___ ___| | __ | / | |/ / |
# | | | | '_ / _ / __| |/ / | |/| | ' / |
# | | |___| | | | __/ (__| < | | | | . |
# | ____|_| |_|___|___|_|____|_| |_|_|_ |
# | |
# | Copyright Mathias Kettner 2010 mk@mathias-kettner.de |
# +------------------------------------------------------------------+
#
# This file is part of Check_MK.
# The official homepage is at http://mathias-kettner.de/check_mk.
#
# check_mk is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by
# the Free Software Foundation in version 2. check_mk is distributed
# in the hope that it will be useful, but WITHOUT ANY WARRANTY; with-
# out even the implied warranty of MERCHANTABILITY or FITNESS FOR A
# PARTICULAR PURPOSE. See the GNU General Public License for more de-
# ails.
# Remove locale settings to eliminate localized outputs where possible
export LC_ALL=C
unset LANG
export MK_LIBDIR="/usr/lib/check_mk_agent"
export MK_CONFDIR="/etc/check_mk"
# Make sure, locally installed binaries are found
PATH=$PATH:/usr/local/bin
Kel Vanderlip 3-1-2012 UUASC-LA 25
26. Monitoring with Open Monitoring Distro
More tests in check_mk_agent.linux:
echo '<<<check_mk>>>'
echo Version: 1.1.12p6
echo AgentOS: linux
echo PluginsDirectory: $PLUGINSDIR
echo LocalDirectory: $LOCALDIR
echo AgentDirectory: $MK_CONFDIR
# If we are called via xinetd, try to find only_from configuration
if [ -n "$REMOTE_HOST" ]
then
echo -n 'OnlyFrom: '
echo $(sed -n
'/^service[[:space:]]*check_mk/,/}/s/^[[:space:]]*only_from[[:space:]]*=[[:space:]]*(.*)/1/p'
/etc/xinetd.d/* | head -n1)
fi
# Partitionen (-P verhindert Zeilenumbruch bei langen Mountpunkten)
# Achtung: NFS-Mounts werden grundsaetzlich ausgeblendet, um
# Haenger zu vermeiden. Diese sollten ohnehin besser auf dem
# Server, als auf dem Client ueberwacht werden.
echo '<<<df>>>'
df -PTlk -x smbfs -x tmpfs -x cifs -x iso9660 -x udf -x nfsv4 | sed 1d
# VMWare shows its own filesystems with 'vdf'. Just one
# problem: it outputs not 7 but only 6 columns
if which vdf > /dev/null
then
vdf -P | grep ^/vmfs/volumes | sed 's/ / vmfs /'
fi
Kel Vanderlip 3-1-2012 UUASC-LA 26
27. Monitoring with Open Monitoring Distro
More tests in check_mk_agent.linux:
# Check mount options. Filesystems may switch to 'ro' in case
# of a read error.
echo '<<<mounts>>>'
grep ^/dev < /proc/mounts
# processes including username, without kernel processes
echo '<<<ps>>>'
ps ax -o user,vsz,rss,pcpu,command --columns 10000 | sed -e 1d -e 's/ *([^
]*) *([^ ]*) *([^ ]*) *([^ ]*) */(1,2,3,4) /'
Kel Vanderlip 3-1-2012 UUASC-LA 27
29. Monitoring with Open Monitoring Distro
ļ¼ Remotely run this script and feed its output to a process which writes a separate
Nagios configuration for each āserviceā found
Edit /opt/omd/<site>/etc/check_mk/main.mk to add a host, or use WATO, then:
> check_mk āI <hostname>
ļ¼ Schedule Nagios to run a single check once a minute: call the remote shell script
over port 6556, and process the result in the ācheckā itself
ļ¼ The check returns each individual āserviceā measurement it finds to Nagios by
writing to the Nagios passive āexternal command fileā
> check_mk āO
ļ¼ Update the whole Nagios configuration for a server which has a new
configuration
> check_mk āII <hostname>
> check-mk -O And loop foreverā¦
Kel Vanderlip 3-1-2012 UUASC-LA 29
30. Monitoring with Open Monitoring Distro
Again, to test and see what check_mk_agent will report on your server, install the RPM and
then, locally, run
> telnet localhost 6556
To see what configuration has been created for Nagios, look at these files on the
Nagios server:
> less /opt/omd/<site>/etc/nagios/conf.d/check_mk_objects.cfg
> less /opt/omd/<site>/etc/nagios/conf.d/check_mk_templates.cfg
Kel Vanderlip 3-1-2012 UUASC-LA 30
31. Monitoring with Open Monitoring Distro
Charts and graphs:
Long-term performance history is important. SLAs, correlating things over
time.
Nagios keeps almost no history.
MySQL can save history, but needs maintenance (it fills up and Nagios stalls).
Besides, you still need to āvisualizeā what is going on.
RRD is a great database service to keep temporal history. It never fills up.
RRD includes visualization tools (graphs)
Traditionally, it has been a job to incorporate RRD into Nagios, usually using
3rd party packages.
Kel Vanderlip 3-1-2012 UUASC-LA 31
32. Monitoring with Open Monitoring Distro
āPNP is an addon to Nagios which analyzes performance data provided by
plugins and stores them automatically into RRD-databases (Round Robin
Databases, see RRD Tool).
During development of PNP we set value on easy installation and little
maintenance while running it. An administrator should do other things than
configure graphing tools. ā
Kel Vanderlip 3-1-2012 UUASC-LA 32
33. Monitoring with Open Monitoring Distro
Besides configuring Nagios and RRD, other things an administrator should be doing include
Documentation.
Wouldnāt it be great if you could move between the Nagios CGI screens, the PNP4Nagios
Charts and a documentation Wiki?
DokuWiki has worked for me.
Also used in OMD for users, passwords, privileges across OMD applications (eg NagViz)
āDokuWiki is a standards compliant, simple to use Wiki, mainly aimed at
creating documentation of any kind. It is targeted at developer teams,
workgroups and small companies. It has a simple but powerful syntax which
makes sure the datafiles remain readable outside the Wiki and eases the
creation of structured texts. All data is stored in plain text files ā no
database is required. ā
Kel Vanderlip 3-1-2012 UUASC-LA 33
34. Monitoring with Open Monitoring Distro
Hereās the punch line:
Kel Vanderlip 3-1-2012 UUASC-LA 34
35. Monitoring with Open Monitoring Distro
OMD Quick introduction
First install the package matching your operating system:
# zypper install omd-0.50-sles11sp1-25.x86_64.rpm
Now create a monitoring instance (OMD calls this "site"):
# omd create UULAC
And let's start Nagios and all other processes:
# omd start UULAC
Other OMD features:
ā¢Run several monitoring sites in parallel
ā¢Install and use several different versions of OMD in parallel
ā¢Easily update, duplicate, rename and manage sites
Kel Vanderlip 3-1-2012 UUASC-LA 35
36. Monitoring with Open Monitoring Distro
What OMD contains, Page 1
nagios-3.2.3 The current version Nagios
nagios-plugins Standard external apps which take and report measurements
Nsca The listener for passive checks from remote servers
check_nrpe The check application which calls checks on remote hosts
Shinken-0.6.99 (drop-in Nagios replacement, a whole world to explore)
Nagvis The management-level view of state ā live maps, schematics
Pnp4nagios RRD and useful graphs. Compare services across hosts.
rrdtool/rrdcached
Check_MK Godās gift to the sysadmin
MK Livestatus replace status.dat with a callable data provider
Multisite Easily add additional monitoring sites.
Kel Vanderlip 3-1-2012 UUASC-LA 36
37. Monitoring with Open Monitoring Distro
What OMD contains, Page 2
Dokuwiki A nice no-SQL wiki linked from Check_MKās screens
Thruk A Perl CGI to view Nagios state (unexplored)
Mod-Gearman Process queue manager, reduces Nagios fork load
check_logfiles Locally read log files and report to Nagios
check_oracle_health Locally perform several Oracle DB checks
check_mysql_health Locally perform several Oracle DB checks
Jmx4perl (unexplored)
check_webinject wget-like web site checker, easy to use from Nagios
check_multi The all singing, all dancing, Python-writing Nagios check
Kel Vanderlip 3-1-2012 UUASC-LA 37
38. Monitoring with Open Monitoring Distro
The Check_MK dashboard (actually called āMultisiteā):
Kel Vanderlip 3-1-2012 UUASC-LA 38
39. Monitoring with Open Monitoring Distro
# KCV Dec 2011
snmp_default_community = 'public'
You configure check_mk by editing
~/etc/check_mk/main.mk: snmp_communities = [
( "SunriderR0!", ["UCS"] ),
( "SunriderR0!", ["Compellent"] ),
]
monitoring_host = "nagios",
ntp_default_levels = (10, 80.0, 110.0)
# hosts not added in WATO
all_hosts = [
"copy-server|linux|dev|tcp",
"ebsprod-ap1|linux|dev|tcp",
"ebsprod-ap2|linux|dev|tcp",
"ebsprod-db1|linux|dev|tcp",
"ebsprod-db2|linux|dev|tcp",
"fortunedelight|linux|dev|tcp",
"ip158|linux|dev|tcp",
"istore-1|linux|dev|tcp",
"istore-uat|linux|dev|tcp",
"landing-page|windows|ping",
"pci-kickstart|dev|linux|tcp",
"soa11g|linux|dev|tcp",
"xbiz1-ap1|linux|dev|tcp",
"xbiz3-db1|linux|dev|tcp",
"xuat1-is1|linux|dev|tcp",
]
Kel Vanderlip 3-1-2012 UUASC-LA 39
46. Monitoring with Open Monitoring Distro
Copies of the check_mk_agent scripts are installed here:
This demonstrates how OMD keeps its versions nicely separated.
OMD[torrance]:~/etc/check_mk$ locate check_mk_agent
/opt/omd/versions/0.50/share/check_mk/agents/check_mk_agent.aix
/opt/omd/versions/0.50/share/check_mk/agents/check_mk_agent.freebsd
/opt/omd/versions/0.50/share/check_mk/agents/check_mk_agent.hpux
/opt/omd/versions/0.50/share/check_mk/agents/check_mk_agent.linux
/opt/omd/versions/0.50/share/check_mk/agents/check_mk_agent.solaris
/opt/omd/versions/0.50/share/check_mk/agents/windows/check_mk_agent.cc
/opt/omd/versions/0.50/share/check_mk/agents/windows/check_mk_agent.exe
/opt/omd/versions/0.50/share/doc/check_mk/treasures/check_mk_agent.hp
/opt/omd/versions/0.52/share/check_mk/agents/check_mk_agent.aix
/opt/omd/versions/0.52/share/check_mk/agents/check_mk_agent.freebsd
/opt/omd/versions/0.52/share/check_mk/agents/check_mk_agent.hpux
/opt/omd/versions/0.52/share/check_mk/agents/check_mk_agent.linux
/opt/omd/versions/0.52/share/check_mk/agents/check_mk_agent.solaris
/opt/omd/versions/0.52/share/check_mk/agents/windows/check_mk_agent.cc
/opt/omd/versions/0.52/share/check_mk/agents/windows/check_mk_agent.exe
Kel Vanderlip 3-1-2012 UUASC-LA 46
47. Monitoring with Open Monitoring Distro
Example of adding a custom check run on a remote host using ssh:
In main.mk:
legacy_checks = [
( ( "check_by_ssh_kel!'/opt/nrpe/libexec/check_ypwhich.sh'","YP client",
True), [ "itauxap1", "itauxap2" ] ),
]
define command{
command_name check_by_ssh_kel
command_line $USER1$/check_by_ssh -H $HOSTADDRESS$ -t 20 -l kelvinv -C $ARG1$
}
On itauxap1 at /opt/nrpe/libexec/check_ypwhich.sh
#!/bin/sh
SERVER=`ypwhich`
if [ -z $SERVER ]
then
echo "CRITICAL: ypwhich NULL"
exit 2
fi
if [ $SERVER != "itauxap1.sunrider.com" ]
then
echo "CRITICAL: ypwhich INCORRECT: $SERVER"
exit 2
fi
echo ypwhich OK: $SERVER
exit 0
Kel Vanderlip 3-1-2012 UUASC-LA 47
48. Monitoring with Open Monitoring Distro
Another custom check, returns count of printer queue depth on
HP-UX.
KISS!
#more check_lpstat-o.sh
#!/bin/sh
COUNT=`lpstat -o | wc -l`
echo lpstat queue $COUNT | queue=$COUNT
exit 0
Kel Vanderlip 3-1-2012 UUASC-LA 48
49. Monitoring with Open Monitoring Distro
livestatus.o:
It is a replacement output method for Nagiosās status.dat
Like NDO, uses Nagios Event Broken API and loads as a module into Nagios.
Unlike NDO, does not write; just responds to queries
Used by Check_MK_Multisite, NagVis, Thruk to populate CGIs with data.
Of course, its automatically set up in OMDā¦
In nagios.cfg:
broker_module=/usr/local/lib/mk-livestatus/livestatus.o
/var/run/nagios/rw/live
Anyone using āLivestatus Query Languageā?
Kel Vanderlip 3-1-2012 UUASC-LA 49
50. Monitoring with Open Monitoring Distro
livescheck:
It is a replacement output method for Nagiosās heavy fork()
Lightweight (100k RAM) helper process, called by Nagios to execute external
applications.
New, only in latest distro, I have not used it yet.
In nagios.cfg:
broker_module=...../livestatus.o livecheck=/omd/sites/mysite/lib/
mk-livestatus/livecheck
OMD includes the gearmand helper process, works for me.
Kel Vanderlip 3-1-2012 UUASC-LA 50
51. Monitoring with Open Monitoring Distro
MultiSite:
Itās the CGI created by Mithias - āCheck_MKā ā it is what you see in your browser.
āMultisite allows each user to customize the builtin views or create completely new
views. This is done in the GUI by flexibly combining datasources, layouts, filters, sortings,
groupings, column-painters and inter-view-links. The idea behind is, that the
administrators of the monitoring system should be able to create custom views for their
users or customers, while those are presented a GUI as simple as possible.ā
Reads data from livestatus.o, so refresh is almost instant ā triggered by Nagios events, I
think.
Allows āmulti-siteā Nagios monitoring to be trivially easy:
ā¢Set up more than one site using OMD (local or distributed)
ā¢Edit āmultisite.mkā
ā¢Watch the world ā I have Hong Kong, they have me.
Includes a configurable sidebar, I have not been there yet.
Kel Vanderlip 3-1-2012 UUASC-LA 51
52. Monitoring with Open Monitoring Distro
Check_MK things we might not have touched on tonight:
Python I have not had to look at it yet!
WATO GUI management for Multisite
Application-level monitoring Aggregation, BI services
Logwatch grep your favorite logs and read from the GUI
Windows ācheck_mk_agent.exe installā
NagViz The management view ā pays your bills!
Mailing lists sign up, they are active.
Kel Vanderlip 3-1-2012 UUASC-LA 52
53. Monitoring with Open Monitoring Distro
Iām out of slides!
Questions?
Kelvin Vanderlip
kelvin@vanderlip.org
Kel Vanderlip 3-1-2012 UUASC-LA 53