Safely Drinking From The Fire Hose   @jschaumaJan SchaumannSeñor Network Security Engineerjschauma@etsy.comB60D A9F7 0D89 ...
I <3 logs!                              @jschauma                                        web logs                         ...
Log Bongzilla, aka Splunk                           @jschauma            Logs go in…Is this how Octocat    came to be?    ...
Splunk Alerts FTW!           @jschauma  YO DAWG, I HERD YOU LIKE LOGS  SO I PUT SOME LOGS IN YOUR LOGSSO YOU CAN SPLUNK WH...
sudo make me a sandwich        @jschauma                          2   08/28/12
Know your patterns.                                             @jschauma                   VPN ConnectionsJuly 4th was a ...
That was unexpected…   @jschauma
XSS detection                          @jschauma             Announcement of            Bug Bounty program:            htt...
Geolocate all the things!        @jschauma                            3   08/28/12
XSS detection                                              @jschaumaIP          : 79.182.16.1 - bzq-79-182-16-1.red.bezeqi...
SQLi detection                                    @jschaumaIP          : 216.185.114.219 – unknownGeolocation : Jurong Eas...
Know when people can’t log in…        @jschauma                                 2   08/28/12
High number of failed logins                               @jschaumaAdmin             : <username> (<internal login>, <sit...
Geolocate all the things!        @jschauma                            4   08/28/12
“Unexpected” login detection                         @jschaumaAdmin      : <username> (<internal login>, <site login>)IP  ...
I said: “Please insert girder!”   @jschauma
Identify scrapers.                                     @jschaumaAdmin      : <username> (<internal login>, <site login>)IP...
Re-re-re-re-re-CAPTCHA                                      @jschaumasource=”info.log" reCAPTCHA status="incorrect" | tran...
Of Liars and Outliers (good book, btw)                   @jschaumawtf happened   here?                                    ...
This talk was too long!                      @jschauma   Log it now, log it all.   Geolocate all the things.   Build profil...
Upcoming SlideShare
Loading in...5
×

Safely Drinking from the Data Waterhose

4,028

Published on

An ingite talk given at DataGotham 2012 about how we extract security related events and alerts from our logs. I repeated the same talk at DevOpsDays NYC 2013.

Published in: Technology, Design
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,028
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
21
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Safely Drinking from the Data Waterhose

  1. 1. Safely Drinking From The Fire Hose @jschaumaJan SchaumannSeñor Network Security Engineerjschauma@etsy.comB60D A9F7 0D89 544A 79957D25 5A5B 4375 275F 0BB5
  2. 2. I <3 logs! @jschauma web logs mail logs system logsvpn logs 2 08/28/12
  3. 3. Log Bongzilla, aka Splunk @jschauma Logs go in…Is this how Octocat came to be? ts come ler out ri ty a secu 2 08/28/12
  4. 4. Splunk Alerts FTW! @jschauma YO DAWG, I HERD YOU LIKE LOGS SO I PUT SOME LOGS IN YOUR LOGSSO YOU CAN SPLUNK WHILE YOU SPLUNK 2 08/28/12
  5. 5. sudo make me a sandwich @jschauma 2 08/28/12
  6. 6. Know your patterns. @jschauma VPN ConnectionsJuly 4th was a Wednesday People making up for People slacking off early last week? on a Friday, eh? 5 08/28/12
  7. 7. That was unexpected… @jschauma
  8. 8. XSS detection @jschauma Announcement of Bug Bounty program: http://is.gd/UTZ5wD code push to address reported vulnerabilities 6 08/28/12
  9. 9. Geolocate all the things! @jschauma 3 08/28/12
  10. 10. XSS detection @jschaumaIP : 79.182.16.1 - bzq-79-182-16-1.red.bezeqint.netGeolocation : Even Yehuda, 02, IL Whois : *SE4-DRP*, RIPE, BEZEQINT-BROADBANDRequests : 146 Method : GET URL : /suggest_username.php?first-name=test&last-name=onerror%3Dalert(0)%3E&email=shai%40exploit.co.il 13 minutes after we announced our security bug bounty program http://is.gd/UTZ5wD Method : POST URL : /your/profile Data : ufb_avatar_url=&gender=female&city3=&new_city="><img src=x onerror=prompt(1);>&new_region=&new_countrycode=&new_latlon=,&city3_dup="><img src=x’[…] 6 08/28/12
  11. 11. SQLi detection @jschaumaIP : 216.185.114.219 – unknownGeolocation : Jurong East, 00, SGWhois : ThePlanet.com Internet Services, Inc., ARIN, NET216Requests : 20 Method : GET URL : /listing/102946830/womens-shirt-beige-tunic-womens-blouse?ref=999999.9%27+union+all+select+0x31303235343830303536%2C0x31303235343830303536%2C0x31303235343830303536%2C0x31303235343830303536%2C0x31303235343830303536%2C0x31303235343830303536%2C0x31303235343830303536%2C0x31303235343830303536%2C0x31303235343830303536%2C0x31830303536%2C0x31303235343830303536%2C0x31303235343830303536%2C0x31303235343830303536%2C0x31303235343830303536+and+%27x%27%3D%27x Method : GET URL : /category/furniture?page=499999%27%20union%20 select%20unhex(hex(version()))%20 […] 6 08/28/12
  12. 12. Know when people can’t log in… @jschauma 2 08/28/12
  13. 13. High number of failed logins @jschaumaAdmin : <username> (<internal login>, <site login>)IP : 64.124.192.210 - 64.124.192.210.t01419-07.above.netGeolocation : Brooklyn, NY, USWhois : ETSY Inc, ARIN, NET64# of failed logins : 13 doesn’t know what he’s doing; do not trust!Admin : jschauma (jschauma, jschauma)IP : 207.38.139.33 - 207-38-139-33.c3-0.avec-ubr2.nyr- avec.ny.cable.rcn.comGeolocation : New York, United StatesWhois : RCN Corporation, ARIN, NET207# of failed logins : 16 6 08/28/12
  14. 14. Geolocate all the things! @jschauma 4 08/28/12
  15. 15. “Unexpected” login detection @jschaumaAdmin : <username> (<internal login>, <site login>)IP : 83.160.48.31 - a83-160-48-31.adsl.xs4all.nlGeolocation : Rotterdam, 11, NLWhois : XS4ALL Internet BV, RIPE, DEMON-NL-DSLAdmin : <username> (<internal login>, <site login>)IP : 217.192.56.102 – unknownGeolocation : Zurich, 25, CHWhois : The Hub Zuerich Assoc., RIPE, THE-HUB-ZUERICH-NET Admin : <username> (<internal login>, <site login>)IP : 24.231.49.240 - unknownGeolocation : Nassau, 23, BS Whois : Cable Bahamas, ARIN, CABLEBAHAMAS-NETAdmin : <username> (<internal login>, <site login>)IP : 200.49.191.120 - map120.network49.191.tigo.net.gtGeolocation : Guatemala City, 07, GTWhois : COMCEL GUATEMALA S.A., LACNIC 6 08/28/12
  16. 16. I said: “Please insert girder!” @jschauma
  17. 17. Identify scrapers. @jschaumaAdmin : <username> (<internal login>, <site login>)IP : 50.17.73.70 - ec2-50-17-73-70.compute-1.amazonaws.comGeolocation : Ashburn, VA, US Whois : Amazon.com, Inc., ARIN, NET50Provider : Amazon AWS Count :7Admin : <username> (<internal login>, <site login>)IP : 207.228.237.110 – unknownGeolocation : New York, NY, USWhois : HopOne Internet Corporation, ARIN, NET207Provider : HopOne Count :1 6 08/28/12
  18. 18. Re-re-re-re-re-CAPTCHA @jschaumasource=”info.log" reCAPTCHA status="incorrect" | transaction ip | where eventcount > 50 | table ip,eventcount | sort -eventcount 6 08/28/12
  19. 19. Of Liars and Outliers (good book, btw) @jschaumawtf happened here? Ooh, right… this: http://is.gd/fognju http://is.gd/0hRDLY http://is.gd/WxcA0r 6 08/28/12
  20. 20. This talk was too long! @jschauma Log it now, log it all. Geolocate all the things. Build profiles. (Creepy, I know.) Reduce false positives. (Whitelists!) Have defined reactions to all alerts. Notice the outliers. Explain them. That’s all, folks! Thanks! 2 08/28/12
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×