• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
What's new in v11 - Fluentd Casual Talks #3 #fluentdcasual
 

What's new in v11 - Fluentd Casual Talks #3 #fluentdcasual

on

  • 4,619 views

 

Statistics

Views

Total Views
4,619
Views on SlideShare
1,475
Embed Views
3,144

Actions

Likes
5
Downloads
2
Comments
0

14 Embeds 3,144

http://frsyuki.hatenablog.com 2078
http://dev.classmethod.jp 576
http://blog.livedoor.jp 310
http://feedly.com 79
https://twitter.com 62
http://www.feedspot.com 10
http://wiki.dev.qops.net 8
http://digg.com 8
http://confluence.nubee.com 4
http://rss.ameba.jp 4
http://reader.aol.com 2
http://frsyuki.hatenablog.comrg 1
https://www.commafeed.com 1
http://webcache.googleusercontent.com 1
More...

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    What's new in v11 - Fluentd Casual Talks #3 #fluentdcasual What's new in v11 - Fluentd Casual Talks #3 #fluentdcasual Presentation Transcript

    • What s new in v11 Sadayuki Furuhashi fluentd.org
    • Non-stop restart v10 (old): supervisor process listen TCP socket worker process heartbeat
    • Non-stop restart v10 (old): supervisor process Downtime! heartbeat worker process restart initializing...
    • Non-stop restart v11: supervisor process listen TCP socket worker process heartbeat
    • Non-stop restart Zero downtime! v11: supervisor process listen TCP socket heartbeat worker process restart initializing...
    • Multiprocess supervisor process heartbeat worker process <worker> ... </worker> <worker> ... </worker> <worker> ... </worker>
    • Multiprocess worker process <worker> <worker> ... </worker> <worker> ・ in_tail ・ in_forward ・ out_groupcounter ・ out_VERY_UNSTABLE ・ out_webhdfs </worker> </worker> Separate important logging from experimental plugins
    • Error stream v10 (old): <source> <match **> {“event”: 1, ...} chunk event OK {“event”: 2, ...} ERROR! {“event”: 3, ...} ? {“event”: 4, ...} chunk ERROR! {“event”: 5, ...} ? {“event”: 6, ...} ? Error handling depends on plugin (throw away! etc.)
    • Error stream v10 (old): <source> 特殊ラベル <match **> {“event”: 1, ...} event chunk OK {“event”: 2, ...} ERROR! {“event”: 3, ...} OK {“event”: 4, ...} chunk ERROR! {“event”: 5, ...} OK {“event”: 6, ...} OK <label @ERROR> <match **> ... OK OK Fluentd core handles error events
    • Plugin version management v11: $ cat /etc/td-agent/Gemfile gem “fluent-plugin-s3”, “~> 0.11.3” gem “fluent-plugin-webhdfs”, “~> 0.2.2” gem “fluentd”, “~> 0.11.2” $
    • Log level per plugin v11: <source> <match **> type forward type mongo log_level error log_level debug </source> </match>
    • Variables in a config file v11: <source> type forward port “#{ENV[‘FLUENTD_LISTEN_PORT’]}” </source>
    • Variables in a config file v11?: <server> @set port 9800 @include “/etc/fluentd/server.conf” </server> <server> @set port 9801 @include “/etc/fluentd/server.conf” </server>
    • Streaming processing without tag hacking v10 (old): <match raw.**> type flowcounter add_tag_prefix counted </match> <match counted.**> type growthforecast </match> v11: <match **> type flowcounter <match **> type growthforecast </match> </match>
    • v11 Reliability & Non-stop restart Performance Multiprocess Error stream Management Plugin version management Log level per plugins Configuration Variables in a config file Streaming processing without tag hacking