Your SlideShare is downloading. ×
Statim, time series interface for Perl.
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Statim, time series interface for Perl.

562
views

Published on

Published in: Technology, Economy & Finance

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
562
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Statimtime series interface for Perl Thiago Rondon YAPC::Brasil 2011
  • 2. “Science progressesthrought observation”-- Isaac Newton
  • 3. “The greatest value of a pictureis that is forces us to notice whatwe never expected to see”-- John Tukey
  • 4. Where ?• Astronomy • Apps• Medicine • Operating System• Stock Market • Networks• Meteorology • Robotics• Biometrics • Events• Geology • ....
  • 5. Wait! let’s seeChart::Cliker examples.
  • 6. Line
  • 7. StackedLine
  • 8. Bar
  • 9. StackedBar
  • 10. Area
  • 11. StackedArea
  • 12. Bubble
  • 13. CandleStick
  • 14. Pie
  • 15. PolarArea
  • 16. What ?• time series is a sequence of data points, measured typically at successive times spaced at uniform time intervals.• interface define a set of methods and conventions that provides a consistent time series interface.
  • 17. Why ?• real time statistics.• simple way to deploy.• support openstandards• distributed input• provides default methods, functions, tools and etc.
  • 18. Motivation• independent• many writers• abstract layer on top of other databases to map and reduce.• Portability• easy integration
  • 19. But, we need more.• 3D,Visualizing data.• Euclidean distance • Find similar patterns • Anomaly detection • Rule discovery • Frequent pattern discovery
  • 20. Architecture TCP TCP Server Client Perlstorage schema engine Client REST REST Server Client
  • 21. Conventions• data collection• period (time units)• step and timestamp.• type of aggregation• fields
  • 22. Engine• AnyEvent • TCP • REST
  • 23. Storage• Key Interface • Redis • Memcached• Table Interface • DBI • File• RRD Interface • RRD
  • 24. Schema{ "collection" : { "period" : "84600", "aggregate" : "sum", "fields" : { "foo" : "count", "bar" : "enum", "jaz" : "enum" } }}
  • 25. Command• add, get and set• period, step• version, quit
  • 26. Commands• add collection foo:jaz bar:1• OK 1• add collection foo:jaz bar:1• OK 2• add collection foo:wow bar:10• OK 10• get collection foo:jaz bar• OK 2• get collection bar• OK 12• set collection foo:jaz bar:2• OK 2
  • 27. Commands with “ts”• You can pass timestamp with “ts”.• add collection foo:bar ts:123467890 bar:1• You can get a new series with “ts”• get collection foo:bar ts:1234567880-1234567890 bar
  • 28. Commands with step• You can pass step with “step”.• add collection foo:bar step:-10 bar:1• You can get a new series with “step”• get collection foo:bar step:-10..-20 bar
  • 29. step vs timestamp• You can get the step in server by:• step 1234567890• Or make the calc in the client.
  • 30. timestamp vs step• Timestamp make easy life to insert data• Step make easy life to get data
  • 31. New time series by periods.• get collection bar:jaz ts:$ts1-$ts2 foo:max• get collection bar:* ts:$ts1-$ts2 foo:max• get collection bar:jaz ts:$ts1-$ts3 foo:min• get collection ts:$ts1-$ts3 foo:sum• get collection ts:$ts1-$ts3 foo:avg• get collection ts:$ts1-$ts3 foo:distinct
  • 32. New time series by periods.• get collection ts:$ts1-$ts2 foo:trend(5)• get collection ts:$ts1-$ts2 foo:anomaly(100)• get collection ts:$ts1-$ts3 foo:frequency(200)
  • 33. Chart !• You can use any lib to do that.• Server-side, eg. Chart::Cliker.• Client-site, eg. html5 and javascript libs.
  • 34. REST• You can extend to any protocol.• statim-rest : /method/[args]
  • 35. Thanks.• http://github.com/maluco/Statim• http://search.cpan.org/dist/Statim• @thiagorondon