Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Metrics, Logs, Transaction Traces, Anomaly Detection at Scale

4,336 views

Published on

Published in: Data & Analytics
  • DOWNLOAD FULL BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Metrics, Logs, Transaction Traces, Anomaly Detection at Scale

  1. 1. Processing Metrics, Logs & Traces … at Scale Otis Gospodnetić
  2. 2. WHO WHY WHAT HOW
  3. 3. WHO HQ: Brooklyn People: Everywhere
  4. 4. WHO Otis Gospodnetić Sematext founder Apache member Book author ex-Lucene/Solr dev
  5. 5. WHO Services Solr Elasticsearch* Kafka Spark HBase Cassandra... * We’ve got serious Solr & Elasticsearch ninjas on the team!
  6. 6. WHY
  7. 7. WHO Clients want Performance Bottlenecks Tuning Scaling WHY
  8. 8. WHO Before you can fix things need to know what to fix WHY
  9. 9. WHY We need….INSIGHT Performance Metrics! Anomalies! Logs!
  10. 10. WHY i.e. Need Tools! Metrics monitoring Log searching Anomaly alerting
  11. 11. OSS Use the (Open) Source, Luke
  12. 12. OSS OpenTSDB InfluxDB Ganglia Graphite Nagios ELK ...
  13. 13. OSS http://blog.sematext.com/2015/04/22/monitoring-stream-processing-tools-cassandra-kafka-and-spark/
  14. 14. OSS “I have an ELK stack that has been suffering as of late. The logstash service will continually crash, the elasticsearch cluster is hardly in the green, and it is taking a constant amount of maintenance.”
  15. 15. WHAT
  16. 16. WHAT SPM → monitoring Logsene → logging On PremisesCloud http://sematext.com/spm http://sematext.com/logsene
  17. 17. WHAT http://blog.sematext.com/2015/04/22/monitoring-stream-processing-tools-cassandra-kafka-and-spark/
  18. 18. WHAT SPM Logsene
  19. 19. HOW
  20. 20. WHAT Agent Java Node.js Want Traces? Embed it! Collectd ⇒ SIGAR for OS Flume SpilloverChannel ES API
  21. 21. WHAT Interesting finds Variable Collectd support Collectd ⇒ SIGAR Apache Flume Elasticsearch Stats API Metrics 2nd class citizen
  22. 22. WHAT Transaction Tracing Java Bytecode Instrumentation Bottleneck finder AppMap maker
  23. 23. WHAT Custom Pointcuts <method signature="java.lang.String com.company. example.Service#getUserName(com.company.model. Company company)"/>
  24. 24. Write-agg vs. Read-agg
  25. 25. Anomalies > Thresholds
  26. 26. WHAT Alerts Heartbeats Thresholds Anomalies
  27. 27. WHAT Anomaly Detection ExponentialSTDFromMA KNN ... boolean result = anomalyCount / (notAnomalyCount + anomalyCount) >= 3d / 4d;
  28. 28. WHAT Anomaly issues Warn early / create noise Normal abnormalities Slow change
  29. 29. Scalable Data Stores
  30. 30. http://blog.sematext.com/2015/06/09/docker-monitoring-support/
  31. 31. Logging
  32. 32. Hot vs. Cold HOT COLD
  33. 33. Drop, don’t Delete HOT COLD drop
  34. 34. Pull, don’t Push GET QUEUE pull ES
  35. 35. Beware of Aggregations Circuit Breakers
  36. 36. http://blog.sematext.com/2014/10/06/top-5-most-popular-log-shippers/
  37. 37. Thank you! @otisg otis@sematext.com @sematext http://sematext.com

×