Your SlideShare is downloading. ×

Fluentd and MySQL

12,562

Published on

about fluent-plugin-mysql (super lightening talk presentation)

about fluent-plugin-mysql (super lightening talk presentation)

Published in: Technology
2 Comments
9 Likes
Statistics
Notes
No Downloads
Views
Total Views
12,562
On Slideshare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
26
Comments
2
Likes
9
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Fluentd and MySQL MySQL Casual Talks Vol.3 (2012/04/19) LT @tagomoris NHN Japan Corp.12年4月20日金曜日
  • 2. Whats Fluentd ? • Log collector daemon we all loves. • Message (key-value pairs) without schema12年4月20日金曜日
  • 3. Storage systems with Fluentd • HDFS of Hadoop • Amazon S3 • MongoDB • Online aggregation and graphs12年4月20日金曜日
  • 4. RDBMS with Fluentd? • Schema is fine! • Schema-less fluentd message is not suitable for RDBMS.... $ gem search -r fluent-plugin-mysql *** REMOTE GEMS *** $12年4月20日金曜日
  • 5. If it doesnt exists, Ill write it. 無いのなら 作ってしまえ プラグイン12年4月20日金曜日
  • 6. fluent-plugin-mysql • Fluentd buffered output plugin • https://github.com/tagomoris/fluent-plugin-mysql • Released just before this LT! • Now, we can do gem install fluent-plugin-mysql12年4月20日金曜日
  • 7. Schema ?12年4月20日金曜日
  • 8. We have mysql_json ! • "mysql_json - a MySQL UDF for parsing JSON" • http://blog.kazuhooku.com/2011/09/mysqljson-mysql-udf-for-parsing-json.html • https://github.com/kazuho/mysql_json12年4月20日金曜日
  • 9. Simple configuration <source>   type forward </source> <match test.*>   type mysql   host localhost   database applogs   username root   sql INSERT INTO tbl SET jsondata=?   format json   flush_interval 1s </match>12年4月20日金曜日
  • 10. Coooooooool!!!! mysql> select * from tbl where json_get(jsondata, pospos) like moge%; +----+----------------------------------------------+ | id | jsondata                                     | +----+----------------------------------------------+ |  2 | {"hoge":1,"pospos":"mogemogemogemogemogege"} | +----+----------------------------------------------+ 1 row in set (0.00 sec)12年4月20日金曜日
  • 11. Thanks ! (See you next in Fluentd Casual Talks!)12年4月20日金曜日
  • 12. Schema.... • fluent-plugin-mysql also supports schema-full insert. • like below: <match test.*>   type mysql   host localhost   database applogs   username root   key_names status,path,rhost,agent,referer table tbl columns status,path,rhost,agent,referer   flush_interval 1s </match>12年4月20日金曜日

×