Advertisement
Advertisement

More Related Content

Similar to Metrics 2.0 @ Monitorama PDX 2014(20)

Advertisement

Metrics 2.0 @ Monitorama PDX 2014

  1.    
  2.     by niteroi @ panoramio.com
  3.     vimeo.com/43800150
  4.    
  5.    
  6.    
  7.     problems Metrics 2.0 concepts implementations & examples
  8.     Mostly graphite
  9.     terminology sync
  10.     (1234567890, 82) (1234567900, 123) (1234567910, 109) (1234567920, 77) db15.mysql.queries_running host=db15 mysql.queries_running
  11.     Problems
  12.     Vimeo.com pagerequests/s? server X write perf?
  13.     Finding metrics Browse hierarchies Dashboard search .. which keywords? Search in source code/documentation? Ask around ...
  14.     stats.hits.vimeo_com stats_counts.hits.vimeo_com stats.*.requesthostport. vimeo_com_80
  15.     Meaning, difference Unit? Where and how.. hard Prefixes Understanding metrics
  16.     collectd.db.disk.sda1.disk_time .write
  17.     Terminology? Which field is where? Total so far? From zero per datapoint? Aggregate? Which? Point at t=x describes which timeframe? Understanding metrics
  18.     Change agent?
  19.     Unclear, inconsistent terminology, format tightly coupled lack information
  20.     O(S*P*A)    S = # Sources      P = # People      A = # Aggregators    
  21.    
  22.    
  23.     times N
  24.     graph definitions are redundant and a time sink.
  25.    
  26.     http://litlquest.com/forest-trees/see-forest-trees-2
  27.     metrics 2.0 concepts
  28.     Self-describing Standardized Orthogonal dimensions
  29.     stats.timers.dfs5. proxy-server.object.GET.200. timing.upper_90
  30.     { server: dfvimeodfsproxy5, http_method: GET, http_code: 200, unit: ms, metric_type: gauge, stat: upper_90, swift_type: object }
  31.     allow more characters unit: Req/s, site: vimeo.com, ...
  32.     Metadata meta: { src: proxy.py:458, from: diamond }
  33.     Conceptual model vs wire protocol vs storage
  34.     metrics20.org
  35.     SI + IEC B Err Warn Conn Job File Req ... MB/s Err/d Req/h ...
  36.     Immediate understanding of metrics Minimize time to graphs, alerting rules, debugging compatibility & flexibility in tooling
  37.     Implementations & examples
  38.    
  39.     Carbon-tagger … stats.gauges.host.foo 125 1234567890 service=foo instance=host target_type=gauge unit=B 123 1234567890 …
  40.    
  41.     Statsdaemon unit=B unit=B ... unit=ms unit=ms ... unit=B/s unit=ms stat=mean unit=ms stat=upper_90 ...
  42.     Keep metric tags in sync with data
  43.     Graph Explorer
  44.    
  45.     Graph­Explorer queries 101 proxy-server swift server:regex unit=ms (AND)
  46.    
  47.    
  48.    
  49.    
  50.    
  51.    
  52.    
  53.     upper_90 (or stat=upper_90) from <datetime> to <datetime> avg over <timespec> (5M, 1h, 3d, ...)
  54.     Compare object put/get stack … http_method:(PUT|GET) swift_type=object avg by http_code,server
  55.    
  56.     Comparing servers http_method:(PUT|GET) group by unit,target_type avg by http_code, swift_type,http_method
  57.    
  58.     transcode unit=Job/s avg over <time> from <datetime> to <datetime>
  59.     Note: data is obfuscated
  60.     Bucketing sum by zone:eu-west|us- east|ap-southeast|us-west| sa-east|vimeo-df|vimeo-lv group by state
  61.     Note: data is obfuscated
  62.     Compare job states per region (zones  bucket) group by zone
  63.     Note: data is obfuscated
  64.     Unit conversion unit=Mb/s network server:regex sum by server
  65.    
  66.    
  67.     Integration Metric unit=B/s Query unit=TB
  68.    
  69.     Deriving Metric unit=B Query unit=GB/d
  70.    
  71.     Bonus round
  72.    
  73.    
  74.    
  75.    
  76.    
  77.    
  78.    
  79.    
  80.     Dashboard definition  queries = [ 'cpu usage sum by core', 'mem unit=B !total group by type:swap', 'stack network unit=Mb/s', 'unit=B (free|used) group by =mountpoint' ]
  81.    
  82.    
  83.    
  84.     Future Work
  85.     ● Storage aggregation rules ● graphite API functions such as  cumulative, summarize and  smartSummarize ● consolidateBy & Graph  renderers
  86.    
  87.     Self-describing & standardized stat=upper/lower/mean/... target_type=counter..
  88.     Select your view
  89.     From: dygraphs.com
  90.     Facet based suggestions
  91.     unit=Err/s
  92.     Conclusion structured self­describing  standardized metrics = enabler
  93.     Conclusion Manual composing  should be last  resort, not default
  94.     Conclusion This sucks – Tell me why – What should we do instead? This is neat! – Help me make it better – Adopt native metrics 2.0, structured_metrics
  95.     Seen in this presentation: metrics20.org vimeo.github.io/graph-explorer github.com/vimeo/timeserieswidget github.com/vimeo/carbon-tagger github.com/vimeo/statsdaemon github.com/Dieterbe/anthracite github.com/graphite-ng github.com/vimeo/graphite-influxdb github.com/vimeo/smoketcp github.com/vimeo/tailgate twitter.com/Dieter_be dieter.plaetinck.be You might also like:
Advertisement