Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

San Antonio DevOps: Fluentd


Published on

An overview of fluentd for the San Antonio DevOps Meetup

Published in: Technology
  • Be the first to comment

  • Be the first to like this

San Antonio DevOps: Fluentd

  1. 1. Hart Hoover | @hhoover Josh O’Brien | @OBrienCommaJosh @SADevOps
  2. 2. fluentd
  3. 3. Log all the things to all the things!
  4. 4. …in JSON!
  5. 5. input&{& &&file&{& &&&&path&=>&"/var/log/http.log"& &&}& }& filter&{& &&grok&{& &&&&match&=>&[&"message",&"%{IP:client}&% {WORD:method}&%{URIPATHPARAM:request}&% {NUMBER:bytes}&%{NUMBER:duration}"&]& &&}& }
  6. 6. Fluentd Events Time Tag Record passed from source or adding a parsed time message routing in fluentd JSON
  7. 7. #&receive&events&via&HTTP&& <source>&& type&http&& port&8888&& </source>
  8. 8. #&read&logs&from&a&file&& <source>&& type&tail
 path&/var/log/httpd.log& format&apache
 tag&apache.access&& </source>
  9. 9. #&DOCKER&OMG& <source>& &&type&tail& &&format&json& &&path&/var/lib/docker/containers/ID/IDYjson.log& &&pos_file&/var/lib/docker/containers/ID/IDY json.log.pos& &&tag&docker.container& &&rotate_wait&5& &&read_from_head&true& </source>
  10. 10. #&save&alerts&to&a&file&& <match&alert.**>&& type&file&& path&/var/log/fluent/alerts&& </match>
  11. 11. #&save&access&logs&to&MongoDB&& <match&apache.access>&& type&mongo& database&apache& collection&log&& </match>&
  12. 12. #&Post&to&IRC& <match&**>& &&type&irc& &&host&localhost& &&port&6667& &&channel&fluentd& &&nick&fluentd& &&user&fluentd& &&real&fluentd& &&message&notice:&%s&[%s]&%s& &&out_keys&tag,time,message& &&time_key&time& &&time_format&%Y/%m/%d&%H:%M:%S& &&tag_key&tag& </match>
  13. 13. #&forward&other&logs&to&servers&& <match&**>&& type&forward& &&<server>&& &&&&host& &&&&weight&20&& &&</server>& &&<server>&& &&&&host& &&&&weight&60&& &&</server>
  14. 14. #&Send&logs&to&ElasticSearch& <match&**>& &&type&elasticsearch& &&logstash_format&true& &&host&localhost& &&port&9200& &&index_name&fluentd& </match>
  15. 15. Client Libraries Ruby Python Java PHP Node.JS Scala
  16. 16. Install with Chef Vagrantfile & kitchen.yml included!
  17. 17. Possibly Better
  18. 18. @SADevOps