Safely Drinking from the Data Waterhose

4,365
-1

Published on

An ingite talk given at DataGotham 2012 about how we extract security related events and alerts from our logs. I repeated the same talk at DevOpsDays NYC 2013.

Published in: Technology, Design
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,365
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
22
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Safely Drinking from the Data Waterhose

  1. Safely Drinking From The Fire Hose @jschaumaJan SchaumannSeñor Network Security Engineerjschauma@etsy.comB60D A9F7 0D89 544A 79957D25 5A5B 4375 275F 0BB5
  2. I <3 logs! @jschauma web logs mail logs system logsvpn logs 2 08/28/12
  3. Log Bongzilla, aka Splunk @jschauma Logs go in…Is this how Octocat came to be? ts come ler out ri ty a secu 2 08/28/12
  4. Splunk Alerts FTW! @jschauma YO DAWG, I HERD YOU LIKE LOGS SO I PUT SOME LOGS IN YOUR LOGSSO YOU CAN SPLUNK WHILE YOU SPLUNK 2 08/28/12
  5. sudo make me a sandwich @jschauma 2 08/28/12
  6. Know your patterns. @jschauma VPN ConnectionsJuly 4th was a Wednesday People making up for People slacking off early last week? on a Friday, eh? 5 08/28/12
  7. That was unexpected… @jschauma
  8. XSS detection @jschauma Announcement of Bug Bounty program: http://is.gd/UTZ5wD code push to address reported vulnerabilities 6 08/28/12
  9. Geolocate all the things! @jschauma 3 08/28/12
  10. XSS detection @jschaumaIP : 79.182.16.1 - bzq-79-182-16-1.red.bezeqint.netGeolocation : Even Yehuda, 02, IL Whois : *SE4-DRP*, RIPE, BEZEQINT-BROADBANDRequests : 146 Method : GET URL : /suggest_username.php?first-name=test&last-name=onerror%3Dalert(0)%3E&email=shai%40exploit.co.il 13 minutes after we announced our security bug bounty program http://is.gd/UTZ5wD Method : POST URL : /your/profile Data : ufb_avatar_url=&gender=female&city3=&new_city="><img src=x onerror=prompt(1);>&new_region=&new_countrycode=&new_latlon=,&city3_dup="><img src=x’[…] 6 08/28/12
  11. SQLi detection @jschaumaIP : 216.185.114.219 – unknownGeolocation : Jurong East, 00, SGWhois : ThePlanet.com Internet Services, Inc., ARIN, NET216Requests : 20 Method : GET URL : /listing/102946830/womens-shirt-beige-tunic-womens-blouse?ref=999999.9%27+union+all+select+0x31303235343830303536%2C0x31303235343830303536%2C0x31303235343830303536%2C0x31303235343830303536%2C0x31303235343830303536%2C0x31303235343830303536%2C0x31303235343830303536%2C0x31303235343830303536%2C0x31303235343830303536%2C0x31830303536%2C0x31303235343830303536%2C0x31303235343830303536%2C0x31303235343830303536%2C0x31303235343830303536+and+%27x%27%3D%27x Method : GET URL : /category/furniture?page=499999%27%20union%20 select%20unhex(hex(version()))%20 […] 6 08/28/12
  12. Know when people can’t log in… @jschauma 2 08/28/12
  13. High number of failed logins @jschaumaAdmin : <username> (<internal login>, <site login>)IP : 64.124.192.210 - 64.124.192.210.t01419-07.above.netGeolocation : Brooklyn, NY, USWhois : ETSY Inc, ARIN, NET64# of failed logins : 13 doesn’t know what he’s doing; do not trust!Admin : jschauma (jschauma, jschauma)IP : 207.38.139.33 - 207-38-139-33.c3-0.avec-ubr2.nyr- avec.ny.cable.rcn.comGeolocation : New York, United StatesWhois : RCN Corporation, ARIN, NET207# of failed logins : 16 6 08/28/12
  14. Geolocate all the things! @jschauma 4 08/28/12
  15. “Unexpected” login detection @jschaumaAdmin : <username> (<internal login>, <site login>)IP : 83.160.48.31 - a83-160-48-31.adsl.xs4all.nlGeolocation : Rotterdam, 11, NLWhois : XS4ALL Internet BV, RIPE, DEMON-NL-DSLAdmin : <username> (<internal login>, <site login>)IP : 217.192.56.102 – unknownGeolocation : Zurich, 25, CHWhois : The Hub Zuerich Assoc., RIPE, THE-HUB-ZUERICH-NET Admin : <username> (<internal login>, <site login>)IP : 24.231.49.240 - unknownGeolocation : Nassau, 23, BS Whois : Cable Bahamas, ARIN, CABLEBAHAMAS-NETAdmin : <username> (<internal login>, <site login>)IP : 200.49.191.120 - map120.network49.191.tigo.net.gtGeolocation : Guatemala City, 07, GTWhois : COMCEL GUATEMALA S.A., LACNIC 6 08/28/12
  16. I said: “Please insert girder!” @jschauma
  17. Identify scrapers. @jschaumaAdmin : <username> (<internal login>, <site login>)IP : 50.17.73.70 - ec2-50-17-73-70.compute-1.amazonaws.comGeolocation : Ashburn, VA, US Whois : Amazon.com, Inc., ARIN, NET50Provider : Amazon AWS Count :7Admin : <username> (<internal login>, <site login>)IP : 207.228.237.110 – unknownGeolocation : New York, NY, USWhois : HopOne Internet Corporation, ARIN, NET207Provider : HopOne Count :1 6 08/28/12
  18. Re-re-re-re-re-CAPTCHA @jschaumasource=”info.log" reCAPTCHA status="incorrect" | transaction ip | where eventcount > 50 | table ip,eventcount | sort -eventcount 6 08/28/12
  19. Of Liars and Outliers (good book, btw) @jschaumawtf happened here? Ooh, right… this: http://is.gd/fognju http://is.gd/0hRDLY http://is.gd/WxcA0r 6 08/28/12
  20. This talk was too long! @jschauma Log it now, log it all. Geolocate all the things. Build profiles. (Creepy, I know.) Reduce false positives. (Whitelists!) Have defined reactions to all alerts. Notice the outliers. Explain them. That’s all, folks! Thanks! 2 08/28/12
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×