This document discusses log standards and future trends in logging. It outlines the current state of log chaos, with no standard formats, schemas, taxonomies, or transport mechanisms. This results in logs being difficult to analyze and make sense of. The document then discusses past attempts at log standards that failed and proposes that a common event expression standard could help bring order to the chaos by establishing common log syntax, taxonomy, transport, and recommendations. This would allow for improved log management, correlation, and security analysis capabilities. It also notes that growing log volumes will continue to pose challenges and will need to be addressed.

1. Log Standards and Future TrendsDr. Anton ChuvakinSecurity Warrior Consultingwww.securitywarriorconsulting.comAugust 2010

2. OutlineWorld of logs todayLog chaos? Order is sorely needed!Past attempts to bring order to chaos!Why all failed?What does the future hold?Logging trends of the next few years

3. Log Data OverviewFrom Where?What Logs?Firewalls/intrusion prevention

4. Routers/switches

5. Intrusion detection

6. Servers, desktops, mainframes

7. Business applications

8. Databases

9. Anti-virus

10. VPNs

11. Audit logs

12. Transaction logs

13. Intrusion logs

14. Connection logs

15. System performance records

16. User activity logs

17. Various alerts and other messagesFrom Log Analysis to Log ManagementThreatprotection and discoveryIncidentresponse and forensicsRegulatory compliance and auditInternal policies and procedure complianceIT system and network troubleshootingSystem performancemanagement

18. Log Chaos: Login<18> Dec 17 15:45:57 10.14.93.7 ns5xp: NetScreendevice_id=ns5xp system-warning-00515: Admin User chuvakinhas logged on via Telnet from 10.14.98.55:39073 (2002-12-17 15:50:53) <57> Dec 25 00:04:32:%SEC_LOGIN-5-LOGIN_SUCCESS:LoginSuccess [user:antonc] [Source:10.4.2.11] [localport:23] at 20:55:40 UTC Fri Feb 28 2006<122> Mar 4 09:23:15 localhostsshd[27577]: Accepted password for anton from ::ffff:192.168.138.35 port 2895 ssh2<13> Fri Mar 17 14:29:38 2006 680 Security SYSTEM User Failure Audit ENTERPRISE Account Logon Logon attempt by: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0 Logon account: ACHUVAKIN

19. Log Chaos Everywhere!No standard formatNo standard schema No standard meaningNo taxonomyNo standard transportNo shared knowledge on what to log and howNo logging guidance for developersNo standard API / libraries for log production

20. Result?%PIX|ASA-3-713185 Error: Username too long - connection abortedAug 11 09:11:19 xx null pif ? exit! 0 ERROR: transport error 202: send failed: Successuserenv[error] 1030 RCI-CORP\wsupx No description available

21. So, what can we expect?

22. If We Don’t Stop It …MORE logs (learn what’s a “petabyte”!)Distributed logging -> WIDELY distributed logging across applications, systems, etcMore REALLY bad logs from custom applicationsWe work harder – and still MISS important things in logs (see VzBIR 2010!)BIG log DATA comes and kills us! 

23. Cloud to the Rescue?Question: do you think “cloud” will make logging better due to APIs, XML, structured data, etc?Answer: "If your security and trust models suck now, you'll be pleasantly surprised by the lack of change when you move to cloud“Chris Hoff @ Cisco

24. Any solutions, Anton?

25. Standards: The ONLY Way Out!FIRST: make it easier to know what logs tell us!Easier to report on logs and explain the reportsDeeper insight into future problems Easier system interoperabilityCommon logging practicesEasier to explain what is in the logs to management and non-IT people

26. What Becomes Possible?All those super-smart people at SIEM vendors can stop parsing and start analyzingWhat the events mean? Consequences? Actions? Maybe even prediction?Different systems can mitigate consequences of each others’ failuresWe can finally tell the developers “what to log?” and have them “get it!”

27. Various Logging Standards by TypeLog formatExample: Syslog, a non-standard standard Example: IDMEF, a failed standard Log contentsNo standard to speak of: logs = trash can because application developers dump what they want there (and how they want!)Log transportExample: Syslog (TCP/UDP port 514)Logging practices / recommendationsExample: NIST 800-92 (for security only)