• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
An Introduction to Fluent & MongoDB Plugins
 

An Introduction to Fluent & MongoDB Plugins

on

  • 6,328 views

 

Statistics

Views

Total Views
6,328
Views on SlideShare
6,140
Embed Views
188

Actions

Likes
16
Downloads
99
Comments
1

16 Embeds 188

http://wiki.home.wols.org 68
http://foreign-business.blogspot.com 29
http://us-w1.rockmelt.com 26
http://a0.twimg.com 23
http://factory.joshfire.com 13
https://twitter.com 10
http://www.twylah.com 5
http://www.onlydoo.com 3
http://mobile.home.wols.org 3
http://foreign-business.blogspot.mx 2
http://factory-joshfire.herokuapp.com 1
http://teknikjawabbmstpm.blogspot.com 1
http://www.blogger.com 1
http://paper.li 1
http://tweetedtimes.com 1
https://cybozulive.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    An Introduction to Fluent & MongoDB Plugins An Introduction to Fluent & MongoDB Plugins Presentation Transcript

    • Fluent Structured logging Pluggable architecture Reliable forwarding e Event Collector ServiceSadayuki FuruhashiTreasure Data, Inc.@frsyuki
    • Fluent Structured logging Pluggable architecture Reliable forwarding e Event Collector ServiceSadayuki FuruhashiTreasure Data, Inc.@frsyuki
    • log
    • log log log log
    • log log log log
    • log log log log
    • Fluent Structured logging Pluggable architecture Reliable forwardinge Event Collector Service
    • Fluent Structured logging Pluggable architecture Reliable forwarding e Event Collector ServiceSadayuki FuruhashiTreasure Data, Inc.@frsyuki
    • Fluent Structured logging Pluggable architecture Reliable forwarding e Event Collector ServiceSadayuki FuruhashiTreasure Data, Inc.@frsyuki
    • log log log log
    • log log log log
    • log log
    • log log
    • <match ...> type mongo_backup capped_size 100m log <store> tcp type tcp host 192.168.0.13 ... </store></match>
    • log log log log
    • <match mongo.**> type mongo database fluent collection test log # Following attibutes are optional host fluenter port 10000 # Other buffer configurations here</match>
    • Mon Nov 14 23:36:22 [conn13] run command admin.$cmd { replSetGetStatus: 1 }Mon Nov 14 23:36:22 [conn13] command admin.$cmd command: { replSetGetStatus: 1 } ntoreturn:1 reslen:571 0msMon Nov 14 23:36:22 [conn13] run command admin.$cmd { ismaster: 1 }Mon Nov 14 23:36:22 [conn13] command admin.$cmd command: { ismaster: 1 } ntoreturn:1 reslen:234 0msMon Nov 14 23:36:22 [conn13] run command admin.$cmd { replSetGetStatus: 1 } Input { _id : ..., time: Mon Nov 14 23:36:22, log key1 : “[conn13]”, key2 : “command”, key3 : ”admin.$cmd”, key4 : { “ismaster”: 1 }, value : “0ms”, } Output
    • aggregate aggregate aggregate aggregate log log log log key1 key2 key3 shuffle aggregate aggregate aggregate aggregate
    • <source> type tail format /^(?<time>[^ ]* [^ ]* [^ ]* [^ ]*) (?<key1>[^ ]*) (?<key2>[^ ]*) (?<key3>[^ ]*)(?<value1>[^ ]*)$/ time_format %a %b %e %H:%M:%S path /var/log/something.log tag aggr_hostneme</source> <metrics> <server> name one_key name host1 partition_by m host host1 each_key key1 port 24224 </metrics> </server> <metrics> <server> name two_keys name host2 partition_by m host host2 each_key key2,key3 port 24224 value_key value1 </server> type float ... </metrics>
    • Mon Nov 14 23:36:22 [conn13] run command admin.$cmd { replSetGetStatus: 1 }Mon Nov 14 23:36:22 [conn13] command admin.$cmd command: { replSetGetStatus: 1 } ntoreturn:1 reslen:571 0msMon Nov 14 23:36:22 [conn13] run command admin.$cmd { ismaster: 1 }Mon Nov 14 23:36:22 [conn13] command admin.$cmd command: { ismaster: 1 } ntoreturn:1 reslen:234 0msMon Nov 14 23:36:22 [conn13] run command admin.$cmd { replSetGetStatus: 1 } Input{ _id : "399e94941cacf13eeb3f808e8ac00981", delta1 delta2 delta3 name : one_key, partition : "2011-11-14 19:17" key : { key1 : "PeriodicTask::Runner" }, delta4 delta5 count : 30, value : { response : 1024 } delta6} Output
    • key1 key3 shuffle key2aggregate aggregate aggregate ... shard key1 shard key3 shard key2 aggregate aggregate ...
    • Fluent Structured logging Pluggable architecture Reliable forwarding e Event Collector ServiceSadayuki FuruhashiTreasure Data, Inc.@frsyuki