Fluentd meetup intro

  • 96 views
Uploaded on

 

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
96
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Fluentd
  • 2.   Meetup
  • 3.   Group Introduction @fluentd www.fluentd.org Tuesday, June 25, 13
  • 4. WHAT’S
  • 5.   FLUENTD? •JSON-BASED
  • 6.   LOG
  • 7.   COLLECTOR •OPEN
  • 8.   SOURCE
  • 9.   (APACHE
  • 10.   2.0) •PRODUCTION
  • 11.   READY Tuesday, June 25, 13
  • 12. JSON-BASED •“EVENT”
  • 13.   IS
  • 14.   THE
  • 15.   SMALLEST
  • 16.   UNIT
  • 17.   OF
  • 18.   DATA
  • 19.    FOR
  • 20.   FLUENTD •THE
  • 21.   FORMAT
  • 22.   IS
  • 23.   JSON •INTERNALLY,
  • 24.   WE
  • 25.   USE
  • 26.   MESSAGEPACK
  • 27.    (JSON-LIKE
  • 28.   BINARY
  • 29.   SERIALIZER) Tuesday, June 25, 13
  • 30. WHY
  • 31.   JSON!? MAINTAINABLE
  • 32.   LOGGING! Tuesday, June 25, 13
  • 33. HOW
  • 34.   LOGGING
  • 35.   IS
  • 36.   DONE
  • 37.   AT MANY
  • 38.   PLACES Tuesday, June 25, 13
  • 39. HOW
  • 40.   LOGGING
  • 41.   IS
  • 42.   DONE
  • 43.   AT MANY
  • 44.   PLACES LOG
  • 45.    FILES DATA
  • 46.    USER
  • 47.   1 DATA
  • 48.    USER
  • 49.   2 INTERMED IATE
  • 50.   DB DATA
  • 51.    USER
  • 52.   3 DATA
  • 53.    USER
  • 54.   2’S
  • 55.    BOSS DATA
  • 56.    USER
  • 57.   1’S
  • 58.    BOSS Tuesday, June 25, 13
  • 59. HOW
  • 60.   LOGGING
  • 61.   IS
  • 62.   DONE
  • 63.   AT MANY
  • 64.   PLACES LOG
  • 65.    FILES DATA
  • 66.    USER
  • 67.   1 DATA
  • 68.    USER
  • 69.   2 INTERMED IATE
  • 70.   DB DATA
  • 71.    USER
  • 72.   3 DATA
  • 73.    USER
  • 74.   2’S
  • 75.    BOSS DATA
  • 76.    USER
  • 77.   1’S
  • 78.    BOSSuser 1’s script user 2’s script DBA’s script user 1’s script 2 user 2’s script 2 user 3’s script... Tuesday, June 25, 13
  • 79. HOW
  • 80.   LOGGING
  • 81.   IS
  • 82.   DONE
  • 83.   AT MANY
  • 84.   PLACES LOG
  • 85.    FILES DATA
  • 86.    USER
  • 87.   1 DATA
  • 88.    USER
  • 89.   2 INTERMED IATE
  • 90.   DB DATA
  • 91.    USER
  • 92.   3 DATA
  • 93.    USER
  • 94.   2’S
  • 95.    BOSS DATA
  • 96.    USER
  • 97.   1’S
  • 98.    BOSSuser 1’s script user 2’s script DBA’s script user 1’s script 2 user 2’s script 2 user 3’s script... FORMAT CHANGE! Tuesday, June 25, 13
  • 99. HOW
  • 100.   LOGGING
  • 101.   IS
  • 102.   DONE
  • 103.   AT MANY
  • 104.   PLACES LOG
  • 105.    FILES DATA
  • 106.    USER
  • 107.   1 DATA
  • 108.    USER
  • 109.   2 INTERMED IATE
  • 110.   DB DATA
  • 111.    USER
  • 112.   3 DATA
  • 113.    USER
  • 114.   2’S
  • 115.    BOSS DATA
  • 116.    USER
  • 117.   1’S
  • 118.    BOSSuser 1’s script user 2’s script DBA’s script user 1’s script 2 user 2’s script 2 user 3’s script... FORMAT CHANGE! Tuesday, June 25, 13
  • 119. HOW
  • 120.   LOGGING
  • 121.   IS
  • 122.   DONE
  • 123.   AT MANY
  • 124.   PLACES LOG
  • 125.    FILES DATA
  • 126.    USER
  • 127.   1 DATA
  • 128.    USER
  • 129.   2 INTERMED IATE
  • 130.   DB DATA
  • 131.    USER
  • 132.   3 DATA
  • 133.    USER
  • 134.   2’S
  • 135.    BOSS DATA
  • 136.    USER
  • 137.   1’S
  • 138.    BOSSuser 1’s script user 2’s script DBA’s script user 1’s script 2 user 2’s script 2 user 3’s script... FORMAT CHANGE! :-( Tuesday, June 25, 13
  • 139. HOW
  • 140.   LOGGING
  • 141.   IS
  • 142.   DONE
  • 143.    WITH
  • 144.   FLUENTD Tuesday, June 25, 13
  • 145. HOW
  • 146.   LOGGING
  • 147.   IS
  • 148.   DONE
  • 149.    WITH
  • 150.   FLUENTD FLUEND DATA
  • 151.    USER
  • 152.   1 DATA
  • 153.    USER
  • 154.   2 INTERMED IATE
  • 155.   DB DATA
  • 156.    USER
  • 157.   3 DATA
  • 158.    USER
  • 159.   2’S
  • 160.    BOSS DATA
  • 161.    USER
  • 162.   1’S
  • 163.    BOSS APACHE/ NGINX SYSLOG LOGS
  • 164.   W/ IN-HOUSE
  • 165.    FORMAT Tuesday, June 25, 13
  • 166. HOW
  • 167.   LOGGING
  • 168.   IS
  • 169.   DONE
  • 170.    WITH
  • 171.   FLUENTD FLUEND DATA
  • 172.    USER
  • 173.   1 DATA
  • 174.    USER
  • 175.   2 INTERMED IATE
  • 176.   DB DATA
  • 177.    USER
  • 178.   3 DATA
  • 179.    USER
  • 180.   2’S
  • 181.    BOSS DATA
  • 182.    USER
  • 183.   1’S
  • 184.    BOSS APACHE/ NGINX SYSLOG LOGS
  • 185.   W/ IN-HOUSE
  • 186.    FORMAT JSON JSON JSON JSON JSON JSON in_syslog in_tail customized in_tail Tuesday, June 25, 13
  • 187. HOW
  • 188.   LOGGING
  • 189.   IS
  • 190.   DONE
  • 191.    WITH
  • 192.   FLUENTD FLUEND DATA
  • 193.    USER
  • 194.   1 DATA
  • 195.    USER
  • 196.   2 INTERMED IATE
  • 197.   DB DATA
  • 198.    USER
  • 199.   3 DATA
  • 200.    USER
  • 201.   2’S
  • 202.    BOSS DATA
  • 203.    USER
  • 204.   1’S
  • 205.    BOSS APACHE/ NGINX SYSLOG LOGS
  • 206.   W/ IN-HOUSE
  • 207.    FORMAT JSON JSON JSON JSON JSON JSON in_syslog in_tail customized in_tail FORMAT CHANGE! Tuesday, June 25, 13
  • 208. HOW
  • 209.   LOGGING
  • 210.   IS
  • 211.   DONE
  • 212.    WITH
  • 213.   FLUENTD FLUEND DATA
  • 214.    USER
  • 215.   1 DATA
  • 216.    USER
  • 217.   2 INTERMED IATE
  • 218.   DB DATA
  • 219.    USER
  • 220.   3 DATA
  • 221.    USER
  • 222.   2’S
  • 223.    BOSS DATA
  • 224.    USER
  • 225.   1’S
  • 226.    BOSS APACHE/ NGINX SYSLOG LOGS
  • 227.   W/ IN-HOUSE
  • 228.    FORMAT JSON JSON JSON JSON JSON JSON in_syslog in_tail customized in_tail FORMAT CHANGE! MODIFY JUST
  • 229.   HERE! Tuesday, June 25, 13
  • 230. HOW
  • 231.   LOGGING
  • 232.   IS
  • 233.   DONE
  • 234.    WITH
  • 235.   FLUENTD FLUEND DATA
  • 236.    USER
  • 237.   1 DATA
  • 238.    USER
  • 239.   2 INTERMED IATE
  • 240.   DB DATA
  • 241.    USER
  • 242.   3 DATA
  • 243.    USER
  • 244.   2’S
  • 245.    BOSS DATA
  • 246.    USER
  • 247.   1’S
  • 248.    BOSS APACHE/ NGINX SYSLOG LOGS
  • 249.   W/ IN-HOUSE
  • 250.    FORMAT JSON JSON JSON JSON JSON JSON in_syslog in_tail customized in_tail FORMAT CHANGE! MODIFY JUST
  • 251.   HERE! Tuesday, June 25, 13
  • 252. WHY
  • 253.   JSON!? MAINTAINABLE
  • 254.   LOGGING! Tuesday, June 25, 13
  • 255. OPEN
  • 256.   SOURCE! •APACHE
  • 257.   2.0 •COMMUNITY!
  • 258.   (1,100+
  • 259.   STARS
  • 260.   ON
  • 261.   GITHUB) •160+
  • 262.   PLUGINS
  • 263.   (MY
  • 264.   DEMO
  • 265.   WILL
  • 266.   USE
  • 267.   ONE
  • 268.   OF
  • 269.    THEM) Tuesday, June 25, 13
  • 270. PRODUCTION
  • 271.   READY •USED
  • 272.   IN
  • 273.   PRODUCTION
  • 274.   WITH
  • 275.   4,000+
  • 276.   SERVERS •CAN
  • 277.   PROCESS
  • 278.   ~15,000
  • 279.   MSGS/SEC
  • 280.   PER
  • 281.   CORE •GOOD
  • 282.   COMMUNITY
  • 283.   SUPPORT
  • 284.   (ACTIVE
  • 285.    MAILING
  • 286.   LIST) Tuesday, June 25, 13
  • 287. WHAT’S
  • 288.   FLUENTD? •JSON-BASED
  • 289.   LOG
  • 290.   COLLECTOR •OPEN
  • 291.   SOURCE
  • 292.   (APACHE
  • 293.   2.0) •PRODUCTION
  • 294.   READY Tuesday, June 25, 13
  • 295. DEMO! Tuesday, June 25, 13
  • 296. DEMO! Chrome browser
  • 297.    log fluentd your
  • 298.    phone! customized in_tail out_twilio Tuesday, June 25, 13
  • 299. DEMO! Chrome browser
  • 300.    log fluentd your
  • 301.    phone! customized in_tail out_twilio “CALL
  • 302.   ME
  • 303.   DEFINITELY IF
  • 304.   MY
  • 305.   EMPLOYEE
  • 306.   IS
  • 307.   SURFIN’
  • 308.   FACEBOOK.COM” Tuesday, June 25, 13