Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Time Series Analysis by JavaScript LL matsuri 2013

2,689 views

Published on

Published in: Technology, Business

Time Series Analysis by JavaScript LL matsuri 2013

  1. 1. Time Series analysis by JavaScript LL matsuri 2013/08/24 1
  2. 2. Time Series analysis by JavaScript 前処理 LL matsuri 2013/08/24 2
  3. 3. {twitter: “muddydixon”} ✓Data Mining ✓Data Visualization ✓Love D3.js ✓working @ NIFTY 3
  4. 4. NIFTY Cloud 4
  5. 5. Visualization ✓What is Visualization? communicating information clearly and effectively 5
  6. 6. Visualization on Web ✓Profit ✓Watch KGI / KPI continuously ✓Share visualization via URL ✓empowered representation by HTML5 / css3 -> D3.js 6
  7. 7. Visualization on Web : D3.js 7
  8. 8. Time Series Visualization ✓Visualize “time series” of ✓KGI / KP of services ✓Metrics of system resources 8
  9. 9. Sample {"time":"2013-01-02T08:54:31.000Z","cpu":0.6283,"memory":0.4427} {"time":"2013-01-02T08:54:22.000Z","cpu":0.6103,"memory":0.6494} {"time":"2013-01-02T08:54:24.000Z","cpu":0.6096,"memory":0.4888} {"time":"2013-01-02T08:54:25.000Z","cpu":0.6811,"memory":0.641} {"time":"2013-01-02T08:54:14.000Z","cpu":0.6028,"memory":0.6452} {"time":"2013-01-02T08:54:06.000Z","cpu":0.5565,"memory":0.4346} {"time":"2013-01-02T08:54:10.000Z","cpu":0.4161,"memory":0.358} {"time":"2013-01-02T08:55:01.000Z","cpu":0.5256,"memory":0.5969} {"time":"2013-01-02T08:55:47.000Z","cpu":0.6145,"memory":0.3213} {"time":"2013-01-02T08:56:24.000Z","cpu":0.6896,"memory":0.5109} {"time":"2013-01-02T08:56:35.000Z","cpu":0.443,"memory":0.633} {"time":"2013-01-02T08:56:11.000Z","cpu":0.4746,"memory":0.4777} {"time":"2013-01-02T08:56:42.000Z","cpu":0.503,"memory":0.5732} {"time":"2013-01-02T08:56:38.000Z","cpu":0.4731,"memory":0.4024} {"time":"2013-01-02T08:57:01.000Z","cpu":0.405,"memory":0.5203} {"time":"2013-01-02T08:57:54.000Z","cpu":0.4496,"memory":0.5324} {"time":"2013-01-02T08:57:04.000Z","cpu":0.3528,"memory":0.3145} {"time":"2013-01-02T08:57:49.000Z","cpu":0.6659,"memory":0.4285} {"time":"2013-01-02T08:57:19.000Z","cpu":0.3237,"memory":0.6979} {"time":"2013-01-02T08:58:31.000Z","cpu":0.4045,"memory":0.6581} {"time":"2013-01-02T08:58:45.000Z","cpu":0.3134,"memory":0.3261} {"time":"2013-01-02T08:58:44.000Z","cpu":0.4382,"memory":0.419} {"time":"2013-01-02T08:58:38.000Z","cpu":0.6983,"memory":0.3493} {"time":"2013-01-02T08:58:34.000Z","cpu":0.6645,"memory":0.6272} {"time":"2013-01-02T08:59:54.000Z","cpu":0.604,"memory":0.3287} {"time":"2013-01-02T08:59:25.000Z","cpu":0.574,"memory":0.4856} {"time":"2013-01-02T08:59:19.000Z","cpu":0.4775,"memory":0.3998} {"time":"2013-01-02T08:59:34.000Z","cpu":0.5047,"memory":0.6702} {"time":"2013-01-02T08:59:46.000Z","cpu":0.672,"memory":0.5021} {"time":"2013-01-02T08:59:53.000Z","cpu":0.3278,"memory":0.6679} {"time":"2013-01-02T08:59:29.000Z","cpu":0.4494,"memory":0.3319} 9
  10. 10. 可視化エンジニア マネージャー デイリーで 売上データ見せて 了解しました 10
  11. 11. 可視化エンジニア マネージャー やっぱり、月次 と週次で …了解しました 11
  12. 12. 可視化エンジニア マネージャー あと売上の平均 と分散も うっ、つらい 12
  13. 13. Summarize keyvalues = {} for d in data time = new Date(d.time.getFullYear(), d.time.getMonth(), d.time.getDate(), d.time.getHours(), d.time.getMinutes()) keyvalues[time] = [] unless keyvalues[time] keyvalues[time].push d aggregated = [] for time, values of keyvalues obj = {time: time, cpu: 0, memory: 0} for d in values obj.cpu += d.cpu obj.memory += d.memory aggregated.push obj console.log aggregated 13
  14. 14. Problems Large Time perspectives / Statistics Avoid to get data each time Hourly / Daily / Weekly / Monthly / Yearly Summary / Average / Stdev 14
  15. 15. Problems メンドクサイ 15
  16. 16. Series.js 16
  17. 17. Series.js ✓Providing utility methods for time series ✓sum, sum square, mean, median, quantile ✓variance, standard devience, covariance, correlation ✓auto covariance, auto correlation ✓Providing aggregation ✓minutes, hour, day, week, month, year ✓Coming soon ✓auto regression, spectrum ✓moving average, holt-winters, exponential moving average 17
  18. 18. Series.js APIs example # class methods Series.sum([1,2,3,4,5]) #=> 15 Series.mean([1,2,3,4,5]) #=> 3 # instance methods s = new Series() s.put d for d in [1,2,3,4,5] s.mean() #=> 3 18
  19. 19. Series.js APIs example # accessor Series.y((d)-> d.v).sum([{v:1},{v:2},{v:3},{v:4},{v: 5}]) #=> 15 s = new Series().y((d)-> d.v) s.put d for d in [{v:1}, {v:2}, {v:3}, {v:4}, {v:5}] s.mean() #=> 3 19
  20. 20. Series.js APIs example # aggregation data = [{t: 1, v:1}, {t: 1, v:2}, {t:2, v:3}, {t:2, v:4}, {t:1, v:5}] Series.y((d)-> d.v).aggregation(Series.sum).key((d)- > d.t)(data) #=> [{t:1, v: 8}, {t: 2, v: 7}] 20
  21. 21. Sample {"time":"2013-01-02T08:54:31.000Z","cpu":0.6283,"memory":0.4427} {"time":"2013-01-02T08:54:22.000Z","cpu":0.6103,"memory":0.6494} {"time":"2013-01-02T08:54:24.000Z","cpu":0.6096,"memory":0.4888} {"time":"2013-01-02T08:54:25.000Z","cpu":0.6811,"memory":0.641} {"time":"2013-01-02T08:54:14.000Z","cpu":0.6028,"memory":0.6452} {"time":"2013-01-02T08:54:06.000Z","cpu":0.5565,"memory":0.4346} {"time":"2013-01-02T08:54:10.000Z","cpu":0.4161,"memory":0.358} {"time":"2013-01-02T08:55:01.000Z","cpu":0.5256,"memory":0.5969} {"time":"2013-01-02T08:55:47.000Z","cpu":0.6145,"memory":0.3213} {"time":"2013-01-02T08:56:24.000Z","cpu":0.6896,"memory":0.5109} {"time":"2013-01-02T08:56:35.000Z","cpu":0.443,"memory":0.633} {"time":"2013-01-02T08:56:11.000Z","cpu":0.4746,"memory":0.4777} {"time":"2013-01-02T08:56:42.000Z","cpu":0.503,"memory":0.5732} {"time":"2013-01-02T08:56:38.000Z","cpu":0.4731,"memory":0.4024} {"time":"2013-01-02T08:57:01.000Z","cpu":0.405,"memory":0.5203} {"time":"2013-01-02T08:57:54.000Z","cpu":0.4496,"memory":0.5324} {"time":"2013-01-02T08:57:04.000Z","cpu":0.3528,"memory":0.3145} {"time":"2013-01-02T08:57:49.000Z","cpu":0.6659,"memory":0.4285} {"time":"2013-01-02T08:57:19.000Z","cpu":0.3237,"memory":0.6979} {"time":"2013-01-02T08:58:31.000Z","cpu":0.4045,"memory":0.6581} {"time":"2013-01-02T08:58:45.000Z","cpu":0.3134,"memory":0.3261} {"time":"2013-01-02T08:58:44.000Z","cpu":0.4382,"memory":0.419} {"time":"2013-01-02T08:58:38.000Z","cpu":0.6983,"memory":0.3493} {"time":"2013-01-02T08:58:34.000Z","cpu":0.6645,"memory":0.6272} {"time":"2013-01-02T08:59:54.000Z","cpu":0.604,"memory":0.3287} {"time":"2013-01-02T08:59:25.000Z","cpu":0.574,"memory":0.4856} {"time":"2013-01-02T08:59:19.000Z","cpu":0.4775,"memory":0.3998} {"time":"2013-01-02T08:59:34.000Z","cpu":0.5047,"memory":0.6702} {"time":"2013-01-02T08:59:46.000Z","cpu":0.672,"memory":0.5021} {"time":"2013-01-02T08:59:53.000Z","cpu":0.3278,"memory":0.6679} {"time":"2013-01-02T08:59:29.000Z","cpu":0.4494,"memory":0.3319} 21
  22. 22. Summarize with Series.js Series.y((d)-> d.cpu).aggregation(Series.sum) .minute((d)-> d.time)(data) #=> [{ t: 'Sun Jan 13 2013 23:29:00 GMT+0900 (JST)', y: 2.0095 }, { t: 'Sun Jan 13 2013 23:30:00 GMT+0900 (JST)', y: 3.4370 }, { t: 'Sun Jan 13 2013 23:31:00 GMT+0900 (JST)', y: 1.58180 }, { t: 'Sun Jan 13 2013 23:32:00 GMT+0900 (JST)', y: 2.2742 }, { t: 'Sun Jan 13 2013 23:33:00 GMT+0900 (JST)', y: 3.4541 }, { t: 'Sun Jan 13 2013 23:34:00 GMT+0900 (JST)', y: 4.6035 }, { t: 'Sun Jan 13 2013 23:35:00 GMT+0900 (JST)', y: 2.1165 }, { t: 'Sun Jan 13 2013 23:36:00 GMT+0900 (JST)', y: 1.6177 }, 22
  23. 23. TODO Performance Refactoring Document 23
  24. 24. TODO Client side 24
  25. 25. TODO Client side さっきpushしました 25
  26. 26. CROSS 2014 26
  27. 27. 以下、ビールを飲んで 笑顔な技術者の顔が 続いていると 思ってください 27

×