All Anton's Top11 Log Lists


Published on

All Anton's Top11 Log Lists: top reasons to collect, review, analyze and secure logs.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

All Anton's Top11 Log Lists

  1. 1. Originally posted on my blog “Security Warrior” – reposted here all in one place. DISCLAIMER: Security is a rapidly changing field of human endeavor. Threats we face literally change every day; moreover, many security professionals consider the rate of change to be accelerating. On top of that, to be able to stay in touch with such ever-changing reality, one has to evolve with the space as well. Thus, even though I hope that this document will be useful for to my readers, please keep in mind that is was possibly written years ago. Also, keep in mind that some of the URL might have gone 404, please Google around. Anton's "Top 11 Reasons to Collect and Preserve Computer Logs", presented in no particular order: 1. Before anything else, do you deal with credit cards? Patient info? Are you a government org under FISMA? A financial org? You have to keep'em - stop reading further. 2. What if there is a law or a regulation that requires you to retain logs - and you don't know about it yet? Does the world "compliance" ring a bell? 3. An auditor comes and asks for logs. Do you want to respond "Eh, what do you mean?"? 4. A system starts crashing and keeps doing so. Where is the answer? Oops, it was in the logs - you just didn't retain them ... 5. Somebody posts a piece of your future quarterly report online. Did John Smith did it? How? If not him, who did? Let's see who touched this document, got logs? 6. A malware is rampant on your network. Where it came from? Who spreads it? Just check the logs - but only if you have them saved. 7. Your boss comes and says 'I emailed you this and you ignored it!!' - 'No, you didn't!!!' Who is right? Only email logs can tell! 8. Network is slow; somebody is hogging the bandwidth. Let's catch the bastard! Is your firewall logging? Keep the info at least until you can investigate. 9. Somebody added a table to your database. Maybe he did something else too - no change control forms were filed. Got database log management? How else would you know? 10. Disk space is cheap; tape is cheaper still. Save a log! Got SAN or NAS? Save a few of them! 11. If you plan to throw away a log record, think - are you 100% sure you won't need it, ever? Exactly! :-) Keep it. Anton's "Top 11 Reasons to Look at Your Logs", presented in no particular order:
  2. 2. 1. The first reason is again disarmingly simple (is it :-)). Read PCI DSS lately? Glanced at HIPAA? Suffer under FISMA? Yup, all of the above say that you must not only have, but also review logs periodically. 2. Are you 0wned? How do you know if all your logs are stashed on a tape in a closet? Look at them! Now!! 3. An incident happens. Really, who needs extra motivation to look at logs in this case? Duh! Logs for incident response is a "no-brainer" use case for log review. 4. Users - from CEO to a janitor. You might have to know what they do on your IT systems! How? Read the logs! Everybody leaves tracks. 5. Logged system errors. Sometimes they are stupid, sometimes - benign. However, often they mean that "stuff" is about to hit the fan. Periodic review of logs reveals them and saves the day. 6. Network slowed to a crawl? Applications are slooow? Server is not ... well, serving? :-) Where is the answer? In the logs, but you need to read them and understand them. 7. That policy you wrote a few months ago. Anybody following that? Anybody remembers that? Halloooo! Check the logs and you'd know. 8. You know your auditor might check your logs. But did you know they might also check whether you looked at them? Did'ya? Review the logs and leave the record of this activity! 9. Change can be good. But then again, it may be the sign that your controls are lacking. Who changes what and when? From what and to what? Just review the logs. 10. Now, you hate looking at logs. You have too many! In this case, look at a specific subset of logs that you never saw before- NBS. Or just deploy log management that can do it for you. 11. Logs can help you predict the future (if you review, know and love them :-)). Don't believe it? If you read them for long enough, you develop an ability to - gasp!- predict the future, albeit mostly future problems :-) Anton’s "Top 11 Reasons to Secure and Protect Your Logs", presented in no particular order: 1. Let's review why you are reviewing logs. Will logs that might have been changed by somebody, somewhere, somehow still be useful for items 1-11 from here? No? Secure them! 2. Oooh, logs in court? Challenges abound! To respond to them, one needs to protect the logs so you can claim that they are both authentic and reliable. 3. A human error still beats an evil hacker as the main cause of IT problems. Are your logs safe from it? Available when needed? Protect them from crashes and other faults!
  3. 3. 4. PCI DSS just says so: "Secure audit trails so they cannot be altered." Wonna do it- or pay the fines? 5. Do you protect financial records? Identity info? Passwords? Some of it ends up in logs - thus making them more sensitive. Secure the C-I-A of logs! 6. Do you look at logs during incident investigation? Do you want them to be "true" or full of random (if creative...) cr*p, inserted by the guilty party? Secure the logs! 7. Think that "attacks vs logging" are theoretical? Think again. Are your logs safe or vulnerable? Is your logging tool 0wned? 8. Syslog + UDP = log injection. Are you protected (reliable TCP, confirmed delivery, encryption - SSH, SSL, VPN)? 9. Why change logs? No, really, why change logs? If you never change logs - and you never should - hash them right away after collection to make them immutable. 10. Logs are backed up on tape - who will see them? Well, whoever restores the tape, that's who! Encrypt them to protect them from accidental and malicious disclosure if tape is lost. 11. Why log access to logs? Same reason why you had the logs in the first place - to review who did what. Who broke through and stole the logs? Who browsed them without permission? Only logs will tell - if you have them! Anton’s "Top 11 Reasons to Analyze Your Logs" Don't just read your logs; analyze them. Why? Here are the reasons: 1. Seen an obscure log message lately? Me too - in fact, everybody have. How do you know what it means (and logs usually do mean something) without analysis? At the very least, you need to bring additional context to know what some logs mean. 2. Logs often measure in gigabytes and soon will in terabytes; log volume grows all the time - it passed a limit of what a human can read a long time ago, it then made simple filtering 'what logs to read' impossible as well: automated log analysis is the only choice. 3. Do you peruse your logs in real time? This is simply absurd! However, automated real-time analysis is entirely possible (and some logs do crave for your attention ASAP!) 4. Can you read multiple logs at the same time? Yes, kind of, if you print them out on multiple pages to correlate (yes, I've seen this done :-)). Is this efficient? God, no! Correlation across logs of different types is one of the most useful approaches to log analysis. 5. A lot of insight hides in "sparse" logs, logs where one record rarely matters, but a large aggregate does (e.g. from one "connection allowed" firewall log to a scan pattern). Thus, the only way to extract that insight from a pool of data is through algorithms (or, as some say, visualization) 6. Ever did a manual log baselining? This is where you read the logs and learn which ones are normal for your environment. Wonna do it again? :-) Log baseline
  4. 4. learning is a useful and simple log analysis technique, but humans can only do it for so much. 7. OK, let's pick the important logs to review. Which one are those? The right answer is "we don't know, until we see them." Thus, to even figure out which logs to read, you need automated analysis. 8. Log analysis for compliance? Why, yes! Compliance is NOT only about log storage (e.g. see PCI DSS). How to highlight compliance-relevant messages? How to see which messages will lead to a violation? How do you satisfy those "daily log review" requirements? Thru automated analysis, of course! 9. Logs allow you to profile your users, your data and your resources/assets. Really? Yes, really: such profiling can then tell you if those users behave in an unusual manner (in fact, the oldest log analysis systems worked like that). Such techniques may help reach the holy grail of log analysis: automatically tell you what matters for you! 10. Ever tried to hire a log analysis expert? Those are few and far between. What if your junior analysts can suddenly analyze logs just as well? One log analysis system creator told me that his log data mining system enabled exactly that. Thus, saving a lot of money to his organization. 11. Finally, can you predict future with your logs? I hope so! Research on predictive analytics is ongoing, but you can only do it with automated analysis tools, not with just your head alone (no matter how big :-)) ... Finally, HUMOROUS (posted on April 1st) Anton’s Top 11 Reasons to Hate Logs You thought I am done with my Top 11 lists? Nah... here is one more, which actually is designed to bite you in the ass on a certain date. So, "Top 11 Reasons to HATE Logs ... With a Passion." 1. Read any logs lately? Got bored in 5 minutes - or survived for the whopping 10? Congrats, you score a point! But logs are still boooooooooooooooooooooooooooooring. 2. One log, two logs, 10 logs.... 1,000,000,000 logs: rabbits and hamsters cannot match the speed with whichlogs multiply. Don't you just hate that? 3. You keep hearing people refer to "log data." Then you run 'tail /var/log/messages' and see text in pidgin English. Where is my data? Hate it! 4. "Real hackers don't get logged": thus logs are seen as useless - and hated by some "hard core" security pros!
  5. 5. 5. If people lie to you, you hate it. Logs do lie too (see 'false positives') - and they are hated too. 6. 'Transport error 202 message repeated 3456 times.' Niiiiice. Now go fix that! Fix what? Ah, hate the log obscurity! 7. Why are there 47 different ways to log that "connection from A to B was established OK?" Or 21 way to say "user logged in OK?" No, really? Why? Who can I kill to stop this insanity? 8. You MUST do XYZ with logs for compliance. Or you are going to jail, buddy! No, sorry, we can't tell you what XYZ is. Maybe in 7 years; for now, just store everything. 9. 'Critical error: process completed successfully' and 'Operation successfully failed' engender deep and lasting hatred of logs in most people. They just do ... 10. The book called "Ugliest Logs Ever!" is a fat tome, covering every log source from a Linux system all the way to databases and CRM. Bad logs are popular! Bad logs are all the rage among the programmers! Bad logs are here to stay. Bad logs that mean nothing power the log hatred. 11. "Logs: can't live with them, can't live without them" :-) Hate them we might for different reasons, but we still must collect, protect, review, and analyze them ... ABOUT AUTHOR: This is an updated author bio, added to the paper at the time of reposting in 2009. Dr. Anton Chuvakin ( is a recognized security expert in the field of log management and PCI DSS compliance. He is an author of books "Security Warrior" and "PCI Compliance" and a contributor to "Know Your Enemy II", "Information Security Management Handbook" and others. Anton has published dozens of papers on log management, correlation, data analysis, PCI DSS, security management (see list . His blog is one of the most popular in the industry. In addition, Anton teaches classes and presents at many security conferences across the world; he recently addressed audiences in United States, UK, Singapore, Spain, Russia and other countries. He works on emerging security standards and serves on the advisory boards of several security start-ups. Currently, Anton is developing his security consulting practice, focusing on logging and PCI DSS compliance for security vendors and Fortune 500
  6. 6. organizations. Dr. Anton Chuvakin was formerly a Director of PCI Compliance Solutions at Qualys. Previously, Anton worked at LogLogic as a Chief Logging Evangelist, tasked with educating the world about the importance of logging for security, compliance and operations. Before LogLogic, Anton was employed by a security vendor in a strategic product management role. Anton earned his Ph.D. degree from Stony Brook University.