Conferencia de Federico Cabiddu sobre HOMER presentada en VoIP2DAY 2016. Más información: www.voip2day.com

1. #HOMER: The VoIP & RTC Time Machine
100% Open Source VoIP Monitoring and RTC Troubleshooting Tools
Written by: Lorenzo Mangani, Alexandr Dubovikov, Federico Cabiddu
Project Supporters & Contributors: Joseph Jackson, Giacomo Vacca, Eugen Biegler,
Shlomi Gutman, Ilya Malchin, Doug Smith

2. #TROUBLESHOOTING NIGHTMARE
And how to break out of it with HOMER and SIPCAPTURE
Whether you operate a Hosted PBX, a large Class4/5 softswitch or the next RTC Service,
Monitoring and Managing operational issues across networks and systems can be a
tedious, time-consuming task involving multiple high-risk and uncoordinated steps:
● Identify and Access critical production systems and nodes involved
● Replicate existing issues over and over with stressed Customers
● Capture the right Packets and application Logs using too many tools
● Correlate and Filter case details required for Technical Escalations
● Export and Retain all critical process data for business compliance
… and things get worse with size, complexity and virtualization!
PROBLEM
SYSTEM ACCESS
ROOT CAUSE
CALL TRACING LOG COLLECTION
DATA EXPORT
REPEAT?
SUBSCRIBER LOCATE CORRELATION
WHERE’S THAT CALL?
…. AND WHERE DID IT
FAIL?!?
?

3. #TROUBLESHOOTING NIGHTMARE DREAM
And how to break out of it with HOMER and SIPCAPTURE READY TO SERVE!
WHERE’S THAT CALL?
LET ME ASK..
HOMER!
Whether you operate a Hosted PBX, a large Class4/5 softswitch or the next RTC Service,
Monitoring and Managing operational issues over vast networks and systems can be a
tedious, time-consuming task involving multiple high-risk and uncoordinated steps:
UNTIL YOU MEET HOMER!
ROOT CAUSEPROBLEM
● Identify and Access critical production systems and nodes involved
● Replicate existing issues over and over with stressed Customers
● Capture the right Packets and application Logs using too many tools
● Correlate and Filter case details required for Technical Escalations
● Export and Retain all critical process data for business compliance
… and things get worse with size, complexity and virtualization!

4. Meet #HOMER = the VoIP & RTC Time Machine
100% Open Source VoIP Monitoring and Troubleshooting Tools
HOMER is a carrier-grade, scalable, OpenSource RTC and VoIP Capture and Monitoring
application suite deployed and trusted by hundreds of Operators, Enterprises and Developers
worldwide. Designed to ease the process of mirroring, indexing and storing large amounts of
signaling, logs and statistics, HOMER provides instant advantages including:
● Secure, Centralized access to Search and Filter past and present data
● Distributed Capture
● Call-Flow visualizations of all RTC sessions with multi-protocol support
● Automatic Correlation of B2BUA Legs, Logs and Quality Reports
● Statistics and Analytics to detect traffic profiles, attacks, anomalies
● Protocol Drill-Down for Signaling and RTP / RTCP media packets
● Unlimited Capture Agents with customizable HEP data chunks
● PCAP Exporting and Sharing functionality for 3rd party apps
...and much more!
HERE’S YOUR
SESSION, STICKY!
WHERE’S THAT CALL?
LET ME ASK..
HOMER!
INVITE (SDP) ...
ROOT CAUSEPROBLEM

5. Meet #HOMER = Brief History of SIPCAPTURE project
100% Open Source VoIP Monitoring and Troubleshooting Tools
VOICE DEVELOPER
LOVING OSS VOICE?
JOIN US!
2011
2013
2014
2015
2016
NOW
HOMER ½.x
Founded by Alexandr Dubovikov & Lorenzo Mangani
Early concept, basic User-Interface, HEP v1
HOMER 3.x
User Interface, Data Charts & Statistics, Call-Flows
HEP v2 integration across multiple platforms
Heavy Rotation concepts and External API
HOMER 5.x
Modular & AngularJS, Visualization Libraries
Additional Protocols in HEP v3, Logs support
WebRTC support, RTP/RTCP Media Statistics
Distributed Capture & Statistics with Autocorrelation
HOMER CLOUD:
HSP v1: Homer Statistics Protocol
External Series support for Alarms and Notifications
Big-Data Analytics and Business Intelligence
More Protocols…. Old and new ;)

6. HOMER is all about providing equal capture opportunities for everyone!
Users deploying HOMER can pick and choose their preferred HEP Capture
Server, Database Server, Statistics Backend and Web Server to match their
existing knowledge and/or live environment to get productive in no time.
The same freedom applies to the capture AGENTS front thanks to the
widespread native support for the open HEP/EEP encapsulation protocol
across leading Open-Source VoIP and Real-Time Communication platforms
and used to seamlessly duplicate packets, logs and other data to HOMER
80/tcp
9060/udp
Apache2 or
NGINX+PHP
Kamailio / OpenSIPS
MySQL or
PostgreSQL
N
ET
#HOMER CaptureArchitecture Elements
Options, Options, Options!
HEP SOCKET
● Kamailio
● OpenSIPS
CORE DATABASE
● MySQL/InnoDB
● PostgreSQL
STATS DATABASE
● Elasticsearch
● InfluxDB
● Graylog
WEB SERVER
● Apache2 + mod_php
● Nginx + php_fpm
Graylog
InfluxDB
Elastic

7. Feeding #HOMER
WE NEED SOME
CAPTURE AGENTS!
Looks and Sounds Great! How do we feed my calls and data to HOMER ?
H E P E E P
S I P
R T P R T C P
L O G S C D R
Q O SJ S O N
S D P

8. #HOMER CaptureArchitecture Elements
Options, Options, Options!
GIVE ME SOME HEP!
Capture Servers
sipgrepCaptagent
Capture Agents
Janus web:RTC
Meetecho
RTC:Engine
SIP:WISE
SPIDR
GENBAND

9. Asterisk + HEP/EEP Configuration
Example Usage of the Integrated Capture Agent for Monitoring
HEP/EEP
Agent
S I P
R T C P
S D P
HEP/EEP
Agent
MODULE
SIPTRACE
MODULE
SIPTRACE
Enabling HEP/EEP on supported platforms is as simple!
Check our WIKI for instruction on how to configure any:
Asterisk 12+ ships with the res_hep capture
module, supporting the HEP/EEP protocol and
used to mirror SIP packets packets from the pjsip
stack and effortlessly mirrors them to HOMER.
Asterisk 12+ also ships with res_hep_rtcp providing
RTCP information from the message bus in HEP/EEP to
HOMER using the res_hep module for transmission.
Using this module, Homer users can receive live call
quality monitoring for all channels in PJSIP Asterisk 12+
systems. To enable the functionality, simply load the
res_hep_rtcp module alongside the res_hep module:
; res_hep Module configuration for Asterisk
[general]
enabled = yes
capture_address = 10.0.0.1:9060
capture_password = foo
capture_id = 1234
HEP
HEP
HEP
HEP

10. Captagent
CAPTAGENT 6.2 Universal Capture Agent
Bleeding Edge HEP/EEP support features cross-platform
No Integrated Agent? No Problem!
CaptAgent is a flexible, asynchronous, and
completely modular packet capture framework,
ready for virtually any network protocol - past,
present and future - all 100% Open Source and
built to deeply integrate with the HOMER core.
CaptAgent natively supports voice protocols,
uses the latest HEP/EEP Encapsulation protocol
feature set. To install the latest version of
Captagent clone a fresh copy from our git repo:
cd /usr/src
git clone https://github.com/sipcapture/captagent.git
cd captagent
./build.sh
./configure
make && make install
Captagent
S I P
R T C P
S D P
NEXT:
sipgrep
sipgrep is SIP console capture and troubleshooting tool
able to act as a quick on-demand HEP/EEP capture
agent with powerful filtering capabilities, natively able to
mirror captured packets to a collector to enrich and
empower console driven voice troubleshooting:
sngrep 1.x from Irontec/Kaian introduced a new HEP/EEP
command line option (-H) and dedicated settings
(eep.send) to send capture data to Homer and to run
headless as a capture agent:
Just want to capture some packets, quick?

11. HEPIPE HEP/EEP Log Harvester
Logs, CDRs, Custom Protocols and much more!
Did someone say Logs?
HEPIPE (as in hep-pipe) is a simple yet powerful tool
to harvest system logs and other event files
(rows, events, cdrs, etc) to extract data using
regex filters and ship correlated HEP packets to
a central capture server such as HOMER
HEPIPE is ideal as a HEP/EEP prototyping
agent and features a few specialized versions:
● HEPIPE ESL for FreeSwitch
● HEPIPE Janus for Janus WebRTC Gateway
HEP is also supported by NODE-LOGSTASH
with many input plugins and GROK parsing:
https://github.com/bpaquet/node-logstash/
L O G S
C D R
J S O N
NEXT: sipgrep

12. Now that we have some data, let’s see how we can use HOMER ourselves!
Discover #HOMER
Captagent

13. Using HOMER #5
Dashboardand Widgetmanagement
YOUR WIDGETS HERE
HOMER 5.x ships with a powerful AngularJS
User Interface providing a multi user dashboard
system, easy to extend with widgets and
applications producing responsive data
visualizers, form processors and more.
HOMER users can easily and instantly:
● Search for Messages and Sessions
● Display and Filter results in real time
● Visualize data using D3 and Flot Charts
● Export, Share and Integrate results

14. Troubleshooting VoIP and
Real-Time Communications
protocol sessions in HOMER
is quick, fun and easy!
Signaling forks are
automatically correlated and
B2BUA legs aggregated and
displayed using the powerful
visual Call-Flow display
supporting unlimited number
of hops.
Using HOMER #5
ProtocolSearchandFlowCorrelation

15. Application Logs can be easily
added to the mix to provide
synchronized extraction from any
source format from a variety of
generic and custom built HEP
Agents.
You don’t even have to find them
back - HOMER will automatically
find and pair any received data
when a corresponding signaling
session is user selected - easy!
Using HOMER #5
ProtocolSearchandFlowCorrelation

16. Despite what you might have heard...
HOMER does support media!
We just prefer a “Privacy Friendly” approach!
RTP and RTCP media streams statistics
can be produced by supported Capture
Agents and sent over HEP Protocol to
HOMER without invading user privacy
and without the requirement to store or
rotate PCAP files.
Media Statistics are automatically
indexed by HOMER and paired with their
originating signaling sessions
automatically, providing unprecedented
granularity for investigating call quality
on per-call basis or globally.
Using HOMER #5
ProtocolSearchandFlowCorrelation

17. Using HOMER #5
ProtocolSearchandFlowCorrelation
Wait, there’s even more under the hood!
User-Agent generated X-RTP-Stats and
RTCP-XR push reports supported by a large
number of Enterprise VoIP Phones are
natively digested by HOMER and can
provide a fully correlated view of client-side
experience in metrics, which alongside all the
other network determined reports allows
engineers and support ops to determine the
origin of any call quality related issue faster
than ever, side by side.
Awesome RTCP Stats enhanced in cooperation with our friends @Voicenter

18. bash <( curl -s https://cdn.rawgit.com/sipcapture/homer-installer/master/homer_installer.sh
Ready to install? HOMER comes with several automated methods to get started:
Puppet Master? A dedicated module is available and ready to use on our repository at https://github.com/sipcapture
None of the above? Get in touch with our team to get professional assistance and support to get started!
Install HOMER 5 in 5 minutes
SIPCAPTURE basic stack using Homer-Installer on supported OSs
docker run -it -v /tmp/homer_mysql/:/var/lib/mysql --name homer5 -p 80:80 -p 9060:9060/udp sipcapture/homer-docker
Docker Fan? No Problem! We ship both single-container and multi-container setups, ready to pull and run wild:
80/tcp
9060/udp
Apache2 or
NGINX+PHP
Kamailio / OpenSIPS
MySQL or
PostgreSQL
N
ET
Graylog
InfluxDB
Elastic
Bare System? Get a vanilla Debian 8 or CentOS 7 net-install image up and running and fire our “all-in-one” installer:

19. AWESOMESTUFF!
… but…
WE WANT MORE?
ALRIGHT..!
I already have HOMER up and running in production - is there anything NEW in store?
WHAT’s
UPwith
HOMER?

20. HOMER provides an extensible set of onboard Statistics
and generates a number of metrics used to render visual
charts and perform arbitrary detections over time ranges.
HOMER statistics in stock setups are saved to the same
database backend used to store the captured packets.
PROS: No additional moving-parts
CONS: Shared database IO resources
NEW:Big-Data & HOMER
OnBoard vs. ExternalStatistics

21. This “All-in-One” database approach works great for small
and medium setups focusing on troubleshooting mostly,
but presents major drawbacks and limitations for intensive
usage of statistics while the packet capture engine is busy
inserting and indexing data from multiple agents at high
rates. So how do we solve this bottleneck?
Sounds like an ideal Big Data challenge!
NEW:Big-Data & HOMER
OnBoard vs. ExternalStatistics
request timeout
request timeout
request timeout
request timeout

22. To provide flexibility for large and complex deployments
our team implemented support in capture servers for
optionally exporting any custom metrics and aggregated
statistics to just about any external API. HOMER with both
OpenSIPS and Kamailio already supports:
✓ Elasticsearch
✓ InfluxDB
✓ Graylog
NEW:Big-Data & HOMER
OnBoard vs. ExternalStatistics

23. NEW:Big-Data & HOMER
OnBoard vs. ExternalStatistics
Extensive Queries and Complex Aggregations can easily
be performed without impacting the core capture servers,
enhancing performance and guaranteeing vast scalability.
HOMER users and integrators can now reach far beyond
simple protocol troubleshooting and leverage collected
metrics to perform Analytics and Business Intelligence
interpolating voice data in conjunction the rest of their
business to answer new questions and reach new goals.

24. NEW:Big-Data & HOMER
OnBoard vs. ExternalStatistics
Alerts and visual Reports can automatically be
programmed, scheduled and generated based on changes
in any of your data according to granular logic and with
notifications sent via email, API calls and opening the way
for brand new use cases and scenarios such as fail
predictions and early warning systems.
https://github.com/elasticfence/kaae
There’s more coming!
Interested? Come by our Stand and talk with our Team!
MAJOR THANKS TO
Razvan Crainea, Eugen Biegler
for their amazing support prototyping this section!

25. Time’s UP! Thanks for attending our Talk!
Come visit our booth and ask us (almost) anything!
SIPCAPTURE @GITHUB http://sipcapture.org + http://sipcapture.io
HOMER @GITHUB http://github.com/sipcapture/homer
CAPTAGENT @GITHUB http://github.com/sipcapture/captagent
MAILING-LIST @USERS https://groups.google.com/forum/#!forum/homer-discuss