Hawkular is an open source monitoring project that is the successor to JBoss ON (RHQ). It provides REST services for collecting and storing metrics and for alerting. Hawkular started in 2014 and provides solutions for monitoring containers, applications, middleware, and IoT devices. It includes projects for services and alerts, metrics storage, and formerly application performance monitoring (which is now handled by Jaeger). Hawkular integrates with ManageIQ and is used to provide middleware management within CloudForms.

1. Hawkular
Successor of the JBoss ON (RHQ)
JBoss User Group Korea
tedwon
2017-11-29

2. Hawkular
● Open source monitoring project
● Next generation monitoring project of JBoss ON (RHQ since 2006)
○ Hawkular started end of 2014
● Apache License v2
● Provide REST services
● Providing a generic solutions to common problems.
○ From collecting rain sensors data and send an SMS on rain,
○ Monitor docker containers
○ Application Performance Monitoring,
● Adopted to the Middleware management solution in the ManageIQ project

3. For who ?
● IoT entusiast who needs to collect metrics and possibly need to trigger alerts
● Operators who are looking for a solution to store metrics from statsD, collectd,
syslog…​
● Developers of solutions who need long term timeseries database storage
● Users of ManageIQ who are looking for Middleware management
● Users of Kubernetes/Heapster who wants to store docker container metrics in a
long term timeseries database storage

4. Hawkular Projects
● Hawkular Services & Alerts
○ flagship base project
○ based on WildFly
○ provides services
■ store metrics,
■ alert on metrics,
■ keep a graph view of an inventory
● Hawkular Metrics
○ provide TimeSeries Database (TSDB) services
○ Time Series Metrics Engine based on Cassandra for scalability
○ offers a REST API to store and retrieve metrics.
● Hawkular APM
○ Application Performance Management
○ visualize which layer of an application time is spent
○ deprecated in favor of CNCF's Jaeger

5. Red Hat Middleware Management with CloudForms
● ManageIQ project
● Middleware Manager for Red Hat Cloudforms Technology Preview
○ https://access.redhat.com/announcements/2820611
● Provide centralized management of all the JBoss middleware products from
within the CloudForms user interface.
● Red Hat Middleware Management 7.0.TechPreview
○ https://access.redhat.com/documentation/en-us/red_hat_middleware_management/7.0.techpr
eview/html-single/installing_red_hat_middleware_management_with_cloudforms/

6. Hawkular Services Quick Start

7. Hawkular Services Quick Start
● Let’s run Hawkular
○ http://www.hawkular.org/hawkular-services/docs/quickstart-guide/
● Run Cassandra
○ docker run --name hawkular-cassandra -e CASSANDRA_START_RPC=true -d
cassandra:3.0.9
● Run Hawkular Services
○ docker run --name hawkular-services --link=hawkular-cassandra -e
CASSANDRA_NODES=hawkular-cassandra -p 8080:8080 hawkular/hawkular-services
○ docker exec -it hawkular-services bash
○ http://localhost:8080/

8. Hawkular Services Quick Start
● STEP 1: ADD METRICS
● STEP 2: READ THE METRICS
● STEP 3: ADD ALERTING
● STEP 4: TEST THE ALERT TRIGGER
● Metrics REST API
○ http://www.hawkular.org/hawkular-metrics/docs/user-guide/
○ http://www.hawkular.org//docs/rest/rest-metrics.html

9. Hawkular Services Quick Start
● STEP 1: ADD METRICS
○ metrics_day_1.json => weather data collected every 30 min for the whole day of July 15th
2016.
○ curl -v -u jdoe:password -X POST http://localhost:8080/hawkular/metrics/gauges/temperature/raw -d
@metrics_day_1.json -H "Content-Type: application/json" -H "Hawkular-Tenant: myTenant"

10. Hawkular Services Quick Start - Read Metrics
● STEP 2: READ THE METRICS
● from when to when we want the values, and ordered by time
○ curl -v -u jdoe:password -X GET
"http://localhost:8080/hawkular/metrics/gauges/temperature/raw?start=1468578600000&end=
1468594800001&order=ASC" -H "Content-Type: application/json" -H "Hawkular-Tenant:
myTenant"

11. Hawkular Services Quick Start - Read Metrics

12. Hawkular Services Quick Start - Downsampled
● STEP 2: READ THE METRICS
● Want to know the minimum, maximum and average values for buckets that
would last 2 hours
○ curl -v -u jdoe:password -X GET
"http://localhost:8080/hawkular/metrics/gauges/temperature/stats?bucketDuration=2h&start=1
468533600000&end=1468618200001" -H "Content-Type: application/json" -H
"Hawkular-Tenant: myTenant"

13. Hawkular Services Quick Start - Downsampled

14. Hawkular Services Quick Start
● STEP 3: ADD ALERTING
○ curl -v -u jdoe:password -X POST http://localhost:8080/hawkular/alerts/import/all -d
@trigger_definition.json -H "Content-Type: application/json" -H "Hawkular-Tenant: myTenant"

16. Hawkular Services Quick Start
● STEP 4: TEST THE ALERT TRIGGER
○ clear console
○ curl -v -u jdoe:password -X POST
http://localhost:8080/hawkular/metrics/gauges/temperature/raw -d @metrics_day_2.json -H
"Content-Type: application/json" -H "Hawkular-Tenant: myTenant"
● In the logs of Hawkular Services, you should see 3 INFO messages showing
that the conditions were met

17. Hawkular Alerts Quick Start with UI

18. Hawkular Alerts Quick Start with UI
● You can
○ Add New Trigger
○ Ack Alert
○ Resolve Alert
● Run Hawkular Alerts UI
○ docker stop hawkular-services
○ docker stop hawkular-cassandra
○ docker ps
○ cd hawkular-alerts/examples/tutorial/lessons
○ docker rm lessons_hawkularAlerts_1
○ docker-compose up
○ http://localhost:8080/hawkular/alerts/ui
○ ./tutorial-data.sh => generate random sample data every 5secs
○ https://github.com/hawkular/hawkular-alerts/blob/master/examples/tutorial/lessons/lesson-04-ui.adoc

19. Hawkular Alert Lifecycle
https://www.slideshare.net/ponceballesteros/hawkular-alerting

20. Hawkular APM with Jaeger

21. Hawkular APM with Jaeger
● Want to trace performance issue on distributed polyglot systems e.g. msa
○ Java
○ Go
○ Python
○ Node.js
○ C++
● Hawkular developed own contemporary distributed tracing systems
○ http://www.hawkular.org/blog/2016/09/19/hawkular-apm-on-msa.html
○ for application performance management
○ for distributed & micro-service applications
● e.g., Zipkin, Google Dapper

22. Hawkular APM with Jaeger
● Hawkular deprecated Hawkular APM project
○ http://www.hawkular.org/blog/2016/09/19/hawkular-apm-on-msa.html
● Instead, Hawkular Team collaborate on the Jaeger project
○ http://www.hawkular.org/blog/2017/04/19/hawkular-apm-jaeger.html
● Uber’s Yuri Shkuro and Red Hat’s Gary Brown, both core contributors to the
Jaeger Distributed Tracing project
○ Hawkular Team contributing to OpenTracing standard
● Distributed tracing based on OpenTracing standard
● OpenTracing is the vendor neutral open standard for distributed tracing
○ Adopted by CNCF (Cloud Native Computing Foundation)

23. ● https://github.com/uber/jaeger
● http://opentracing.io/documentation/pages/quick-start
● https://eng.uber.com/distributed-tracing/
● Run Jaeger
○ docker run -e COLLECTOR_ZIPKIN_HTTP_PORT=9411 -p5775:5775/udp -p6831:6831/udp
-p6832:6832/udp -p5778:5778 -p16686:16686 -p14268:14268 -p9411:9411
jaegertracing/all-in-one:latest
● Jaeger UI
○ http://localhost:16686/
Let’s run Jaeger

24. Jaeger Example
● Spring Boot App /chaining call => Spring Boot /hello => WildFly Swarm /hello
○ https://github.com/pavolloffay/opentracing-java-examples
○ http://www.hawkular.org/blog/2017/06/9/opentracing-spring-boot.html
○ http://www.hawkular.org/blog/2017/07/opentracing-jaxrs.html

25. Jaeger Example

26. Jaeger Example

27. Jaeger - a Distributed Tracing System

28. References
● http://www.hawkular.org/
● http://www.hawkular.org/overview/
● https://access.redhat.com/articles/2221611
● http://www.hawkular.org/hawkular-services/docs/installation-guide/
● http://www.hawkular.org/hawkular-services/docs/quickstart-guide/
● http://www.hawkular.org/hawkular-services/docs/user-guide/inventory/
● http://www.hawkular.org/hawkular-services/docs/user-guide/inventory/inventory-v3-details.html
● http://www.hawkular.org/hawkular-apm/
● http://jaeger.readthedocs.io/en/latest/
● http://www.hawkular.org/hawkular-metrics/docs/user-guide/
● http://www.hawkular.org/hawkular-metrics/docs/user-guide/installation.html
● http://www.hawkular.org/hawkular-metrics/docs/user-guide/configuration.html
● https://www.slideshare.net/ponceballesteros/hawkular-alerting
● http://www.hawkular.org/community/docs/developer-guide/alerts-v1.html
● http://www.hawkular.org/community/docs/developer-guide/alerts-v2.html
● https://github.com/ManageIQ/manageiq-providers-hawkular/blob/master/GETTING_STARTED.md
● https://github.com/hawkular/hawkular-alerts/tree/master/examples/hello-world
● http://highscalability.com/blog/2010/4/27/paper-dapper-googles-large-scale-distributed-systems-tracing.html
● http://www.hawkular.org/blog/2016/09/19/hawkular-apm-on-msa.html
● https://eng.uber.com/distributed-tracing/
● https://www.slideshare.net/YuriShkuro/distributed-tracing-at-uber-scale-creating-a-treasure-map-for-your-monitoring-data
● http://www.hawkular.org/blog/2017/03/24/distributed-tracing-with-camel.html
● https://www.cncf.io/blog/2017/09/13/cncf-hosts-jaeger/
● https://blog.openshift.com/openshift-commons-briefing-82-distributed-tracing-with-jaeger-prometheus-on-kubernetes/

29. Thank you