Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to Use OWASP Security Logging

2,577 views

Published on

OWASP Security Logging API easily extends your current log4j and logback logging with impressive features helpful for security, diagnostics/forensics, and compliance. Slide deck presentation from OWASP AppSecEU 2016 in Rome.

Published in: Software

How to Use OWASP Security Logging

  1. 1. How To Use OWASP Security Logging by August Detlefsen, Sytze van Koningsveld, and Milton Smith
  2. 2. About the Presenters August Detlefsen (California) Senior Application Security Consultant with more than eighteen years experience in software development and information security. August authored several Burp Suite extensions, CodeMagi’s Clickjacking Defense, and book author for Iron-Clad Java: Building Secure Web Applications. Twitter @codemagi or augustd@codemagi.com Sytze van Koningsveld (Netherlands) Senior Software Developer at KLM specialised in security and quality assurance with over eighteen years of Java development experience. sytze.vankoningsveld@owasp.org Milton Smith (California) Security principal developing cloud security tools at Oracle. Previously Milton was the leader for Java Platform Security and the Java Security Track at Oracle’s JavaOne conference. Past security principal at Yahoo. Twitter @spoofzu or milton.smith@owasp.com
  3. 3. Exercises Exercises for this demo are available on GitHub: https://github.com/augustd/securitylogging https://github.com/augustd/securitylogging-webapp
  4. 4. Security Logging Background Why do we need a security logging platform?
  5. 5. OWASP Security Logging Project Started in 2014, project born out of the need for a logger with better support for security Implemented in Java, popular platform and language Built with open standards(SLF4J), open source project written to open logging standards, SLF4J
  6. 6. Security & Compliance Distinct from Diagnostics • Logger priorities, debug, info, warn, fatal - meaningless for security & logging • Retention, losing diagnostics log messages is a pain, losing security logs raises eyebrows, losing compliance logs - ouch! • Context, knowing the action or activity is not good enough. Need to know who, what, where, when.
  7. 7. What Would a Better Logging Platform Look Like? • 3-broader use cases, diagnostics, security, and compliance • Framework encouraging robust logging, current frameworks leave what to log and when to log up to developers. Improved automation for common use-cases • Legacy support, must add some value to older applications or 3rd party applications where we don’t have source code
  8. 8. Why Use Security Logging? Powerful features w/automation, associate current logged on user w/activities, log system state on start for later forensics, import into SIEM tools, log trends like heap space, open file counts, users logged on, etc. Let us help you log, most logging systems put bits on disk. What, when, and where to log is important. Let us help. Get going fast, know how to use log4j? Leverage your existing skills. You're ready to go!
  9. 9. Building A Better Logging Framework Java Logging Log4j/Log4j 2 logbackLog Platforms SLF4J OWASP Security Logging Interface Security & Compliance
  10. 10. SLF4J and JSR-47 Confused? Specifications dogfight Subtle but important differences, logger inheritance, log level names For details see, http://www.jajakarta.org/log4j/jakarta-log4j- 1.1.3/docs/critique.html
  11. 11. Benefits of OWASP Security Logging Security logging encourages positive design
  12. 12. Benefit, Designed for 3-Use Cases: Diag,Sec,Comp Diagnostics/Forensics - What just happened? History of memory usage? History of security events ? What command line args executed app? Disk use over time. Security - Door open/closed, user logged in/out, resource created/read/update/deleted, information classification Compliance - Log messages remotely, sign logs, discourage tampering
  13. 13. Security Log Events Successful logins Failed logins Log outs Changed password or security questions Profile changes, such as change of email address Password reset attempts Authorization failures Changes to privilege levels Input validation failures Any other sensitive operation…
  14. 14. Benefit, Encourage Improved Logging Via Automation Standalone Application - log command line arguments, system environment variables, Java system properties J2EE/Servlet - All standalone logging + HTTP Request Parameters like current user logged on
  15. 15. Benefit, Popular Logging Support & Ease of Use Popular logging platforms, support for popular platforms like Java logging, log4j, log4j 2, logback Large base of developer knowledge, years of experience w/these logging platforms Open source & commercial support, many development organizations offer creative products services in this space
  16. 16. Introduction to OWASP Security Logging Security logging encourages positive design
  17. 17. Planning Your Project Logging Formalize your objectives, diagnostics/forensics, security, compliance Map features to your objectives, understand/implement the features that support your projects needs New project or legacy, for new projects you can use a battery of features. However, even old projects that console log (e.g. System.out) receive some benefits.
  18. 18. https://www.owasp.org/index.php/OWASP_Security_Logging_Project Quick Start, info to get started Source Code, GitHub project Java code Issue Tracker, report bugs, feature requests Messaging leaders, work in progress. Temporary OWASP leaders email list or issue tracker link
  19. 19. Including OWASP Security Logging Binaries GitHub releases, download release binaries from project, https://github.com/javabeanz/owasp-security-logging/releases
  20. 20. Building and Dependency Resolution Maven Central, include Maven dependency declaration in your project POM <dependency> <groupId>org.owasp</groupId> <artifactId>security-logging-log4j</artifactId> <version>LATEST</version> </dependency> <dependency> <groupId>org.owasp</groupId> <artifactId>security-logging-logback</artifactId> <version>LATEST</version> </dependency> log4j logback
  21. 21. Running Test Cases & Project Badges OWASP Security Logging hosted on Github https://github.com/javabeanz/owasp-security-logging : • Continuous Integration with Travis • Quality assurance with Codecov, Codacy and Versioneye • Security analysis with Coverity • License and Maven version badge • Core infrastructure badge in progress • Many more : collaboration, deployment, project management, …. : https://github.com/integrations
  22. 22. Community Support, Suggestions, Contributing OWASP Security logging github page offers : • issue management • #owaspsecurity-logging channel on OWASP Slack for chat • wiki pages for documentation – contributing : clone the git repo, create a pull request for your change. If code change passes the tests, builds OK, and badges green then the pull request is accepted
  23. 23. Help Us Think of a Better Name for This Project OWASP Security Logging Project name is too long! We invite ideas for a distinctive name and logo. A single word and simple project icon would be best, clean, simple for everyone remember
  24. 24. Introduction to Security Logging Features Features to encourage positive design & save time
  25. 25. Feature, Security Markers Federal and State government agencies as well as companies supporting those agencies are often required to classify information. Log routing, log messages with privileged classifications to secure logs Exclude sensitive, exclude log messages with privileged classifications from being logged
  26. 26. Feature, Log HTTP Session Parameters Sometimes is helpful to have information associated with the session associated with log messages SessionPlugin, adds the current user logged on to the web application to Mapped Diagnostic Context(MDC). Information easily used to include/correlate user id with activity in log messages
  27. 27. Feature, Log HTTP Session Parameters (cont) ForwardedIPAddressPlugin, add remote IP address to the MDC by using value of X-Forwarded-For in header appended by load balancer IPAddressPlugin, add remote IP address to the MDC by using value of HttpServletRequest.getRemoteAddr() in header UserNamePlugin, grab HttpServletRequest.getAttribute(“username”) and place value in MDC
  28. 28. Feature, Log Command Line Args on Startup Log the command line arguments that initialized your program. Useful if your application has problems. SecurityUtil.logCommandLineArguments(args); Use WebApplicationInitializer in Spring web applications
  29. 29. Feature, Log System Environment on Startup Shell variables can be useful to diagnose problems your application may be experiencing. Do this to log your environment properties. SecurityUtil.logShellEnvironmentVariables();
  30. 30. Feature, Log System Properties on Startup Knowing the Java System properties at startup (or other times) can be helpful. Log them easily by doing this. SecurityUtil.logJavaSystemProperties();
  31. 31. Feature, Interval Logging Beneficial for diagnostics/forensics to keep record of system state for later follow-up You want this every in your logs every 15-sec, 20:10:10.204 [Thread-0] INFO Watchdog: MemoryTotal=64.5MB, FreeMemory=58.2MB, MaxMemory=947.7MB, Threads Total=5, Threads New=0, Threads Runnable=3, Threads Blocked=0, Threads Waiting=2, Threads Terminated=0 Add this code, IntervalLoggerController wd = SecurityLoggingFactory.getControllerInstance(); wd.start();
  32. 32. Feature, Redirection Streams, System.out/err Redirect the system streams of your legacy console logging code to your SLF4J logger. Set this on start-up. SecurityUtil.bindSystemStreamsToSLF4J(); If you need to disable for some reason do this, SecurityUtil.unbindSystemStreams();
  33. 33. Feature, Filtering Sensitive Log Messages There are also times where it’s desirable to filter unstructured data within log messages. Fields like SSN, password, are examples. An example of what to code, LOGGER.info("userid={}", userid); LOGGER.info(SecurityMarkers.CONFIDENTIAL, "password={}", password); 2014-12-16 13:54:48,860 [main] INFO - userid=joebob 2014-12-16 13:54:48,860 [main] [CONFIDENTIAL] INFO - password=***********
  34. 34. Attendee Lab No substitute for hand-on experience
  35. 35. If you have a laptop AND if you don’t If you brought a laptop you can participate in the coding exercise. If not, don’t worry. You can shoulder surf or watch us on the big screen.
  36. 36. Exercise: HelloWorld w/Security Logging Step 1, download securitylogging from GitHub https://github.com/augustd/securitylogging
  37. 37. Exercise: HelloWorld w/Security Logging Step 2, integrate securitylogging with your favorite IDE
  38. 38. Exercise: HelloWorld w/Security Logging Step 3, update POM.xml to include SLF4J logger (log4J 2 or logback) ... </plugins> </build> <dependencies> <dependency> <groupId>org.owasp</groupId> <artifactId>security-logging-log4j</artifactId> <version>1.1.2</version> </dependency> </dependencies> </project>
  39. 39. Exercise: HelloWorld w/Security Logging Step 4, download Maven dependencies
  40. 40. Exercise: HelloWorld w/Security Logging Step 4, compile and run the project You should see something like this, 17:00:30.984 [main] INFO com.owasp.securitylogging.bin.HelloWorld - It's alive! Log message outside log4j 2
  41. 41. Exercise: HelloWorld w/Security Logging INFORMATION, at this point you have a functioning program that implements log4j 2 and OWASP Security Logging. Now for some fun!
  42. 42. Exercise: HelloWorld with Security Markers Step 5, add the following code to HelloWorld.java to tag log events as security-specific. Add after logger.info() call. logger.info(SecurityMarkers.SECURITY_SUCCESS, "User '{}' logged in", "augustd"); logger.error(SecurityMarkers.SECURITY_FAILURE, "User '{}' attempted to access invalid account '{}'", "snidely", 5555785);
  43. 43. Exercise: HelloWorld with Security Markers Step 5, in log4j.xml, modify the PatternLayout definition to include markers: <Console name="Console" target="SYSTEM_OUT"> <PatternLayout pattern="%d{HH:mm:ss.SSS} %marker [%t] %-5level %logger{36} - %msg%n"/> </Console>
  44. 44. Exercise: HelloWorld with Startup Properties Step 5, compile and run the project. Your output should look like this: 13:30:55.370 SECURITY SUCCESS [main] INFO com.owasp.securitylogging.bin.HelloWorld - User 'augustd' logged in 13:30:55.370 SECURITY FAILURE [main] ERROR com.owasp.securitylogging.bin.HelloWorld - User 'snidely' attempted to access invalid account '5555785' ...
  45. 45. Exercise: HelloWorld with Startup Properties Step 6, add the following code to HelloWorld.java to print system properties on startup. Add after logger.info() call. // log command line arguments SecurityUtil.logCommandLineArguments(args); // log shell environment variables SecurityUtil.logShellEnvironmentVariables(); // log java system properties SecurityUtil.logJavaSystemProperties();
  46. 46. Exercise: HelloWorld with Startup Properties Step 6, compile and run the project. Your output should look like this: 17:37:30.678 [main] INFO com.owasp.securitylogging.bin.HelloWorld - It's alive! 17:37:30.690 [main] INFO org.owasp.security.logging.util.SecurityUtil - Env, PATH=/usr/bin:/bin:/usr/sbin:/sbin 17:37:30.690 [main] INFO org.owasp.security.logging.util.SecurityUtil - Env, SHELL=/bin/bash 17:37:30.690 [main] INFO org.owasp.security.logging.util.SecurityUtil - Env, JAVA_STARTED_ON_FIRST_THREAD_4018=1 17:37:30.690 [main] INFO org.owasp.security.logging.util.SecurityUtil - Env, APP_ICON_4018=../Resources/Eclipse.icns 17:37:30.690 [main] INFO org.owasp.security.logging.util.SecurityUtil - Env, JAVA_MAIN_CLASS_7073=com.owasp.securitylogging.bin.HelloWorld 17:37:30.690 [main] INFO org.owasp.security.logging.util.SecurityUtil - Env, USER=milton ...
  47. 47. Exercise: HelloWorld with Stream Redirection Step 7, add the following code to HelloWorld.java to caputure logging to system streams by legacy or commericial programs. Add after the interval logging code, SecurityUtil.bindSystemStreamsToSLF4J(); // Intercept system streams. SecurityUtil.bindSystemStreamsToSLF4J(); // Test stream interception System.out.println("This is a system.out"); System.err.println("This is a system.err");
  48. 48. Exercise: HelloWorld with Stream Redirection Step 7, check to see logging to system streams is redirected. Notice how the log message routed to log4j 2 includes time, msg priority, etc: 12:34:09.084 [main] INFO org.owasp.security.logging.util.SecurityUtil - SysProp, sun.cpu.isalist= 12:34:09.084 [main] INFO org.owasp.security.logging.util.SecurityUtil - This is a system.out 12:34:09.084 [main] ERROR org.owasp.security.logging.util.SecurityUtil - This is a system.err
  49. 49. Exercise: HelloWorld with Interval Logging Step 8, now add the following code to HelloWorld.java to add interval logging. Add after SecurityUtil.logJavaSystemProperties(); type: // start the interval logger IntervalLoggerController wd = SecurityLoggingFactory.getControllerInstance(); wd.start();
  50. 50. Exercise: HelloWorld with Interval Logging Step 8, with interval logging on you should see a few messages print at the end of your log each 15-sec like this: Log message outside log4j 2 17:47:53.714 [Thread-1] INFO org.owasp.security.logging.util.DefaultIntervalLoggerView - Watchdog: MemoryTotal=64.5MB, MemoryFree=56.2MB, MemoryMax=954.7MB, ThreadNew=0, ThreadRunnable=3, ThreadBlocked=0, ThreadWaiting=2, ThreadTerminated=0, INFORMATION: to exit you need to press stop in debugger or call wd.stop() in your code
  51. 51. Exercise: HelloWorld with Interval Logging CONGRATULATIONS, your program now has: • command line arg logging • logging shell environment properties • logging Java system properties • Intercepting System streams • and interval logging every 15-secs ...but wait, there’s more!
  52. 52. Exercise: HelloWorld Web App w/Security Logging Step 1, download securitylogging-webapp from GitHub https://github.com/augustd/securitylogging-webapp
  53. 53. Exercise: HelloWorld Web App Step 2, integrate securitylogging-webapp with your favorite IDE The logging API is already included in pom.xml
  54. 54. Exercise: HelloWorld Web App Step 3, compile and run the project You should see something like this, Info: Loading application [securitylogging-webapp] at [/securitylogging-webapp] Info: securitylogging-webapp was successfully deployed in 237 milliseconds.
  55. 55. Exercise: HelloWorld Web App Step 4, Hit the web app URL: http://localhost:8080/securitylogging- webapp/HelloWorld?name=august You should see something like this: Info: 18:03:08.729 SECURITY SUCCESS [http-listener-1(1)] INFO org.owasp.securitylogging.webapp.HelloWorld - User august logged in
  56. 56. Exercise: HelloWorld Web App At this point you have a functioning web app that implements log4j 2 and OWASP Security Logging. Now for some fun!
  57. 57. Exercise: HelloWorld Web App Step 5, update web.xml to include MDC Filter: ... <filter> <filter-name>LoggingFilter</filter-name> <filter-class>org.owasp.security.logging.mdc.MDCFilter </filter-class> <init-param> <!-- component name is a free-from text value --> <param-name>ProductName</param-name> <param-value>securitylogging-webapp</param-value> </init-param> </filter>
  58. 58. Exercise: HelloWorld Web App Step 5, map the MDC Filter to all URLs in your app: <filter-mapping> <filter-name>LoggingFilter</filter-name> <url-pattern>/*</url-pattern> </filter-mapping>
  59. 59. Exercise: HelloWorld Web App Step 5, Add some plugins to MDC Filter: <filter> … <init-param> <param-name>ipAddress</param-name> <param-value>org.owasp.security.logging.mdc.plugins.ForwardedIPAddressPlugin </param-value> </init-param> <init-param> <param-name>username</param-name> <param-value>org.owasp.security.logging.mdc.plugins.UsernamePlugin </param-value> </init-param> </filter>
  60. 60. Exercise: HelloWorld Web App Step 5, update log4j2.xml to include MDC info in layouts: ... <Appenders> <Appender type="console" name="Console"> <PatternLayout pattern="%d{HH:mm:ss.SSS} %marker [%t] %mdc{username}:%mdc{session} %mdc{ipAddress} %mdc{productName} %-5level %logger{36} - %msg%n"/> </Appender> </Appenders>
  61. 61. Exercise: HelloWorld Web App Step 6, Hit the web app URL: http://localhost:8080/securitylogging- webapp/HelloWorld?name=august Your logs should show something like this: Info: 18:24:21.077 SECURITY SUCCESS [http-listener-1(3)] august:019397a165a49dd2a03e5e3002dd0505219796e00296b33ec82107e5d0da6 3da 127.0.0.1 securitylogging-webapp INFO org.owasp.securitylogging.webapp.HelloWorld - User august logged in Automatically, every request, no code required!
  62. 62. Exercise: HelloWorld Web App CONGRATULATIONS, your program now has: • Automatic gathering of diagnostic data • Diagnostic data added to every log statement • Done all in configuration, no coding required ...but wait, there’s (going to be) more!
  63. 63. Ideas for the Future Possible future directions with security logging
  64. 64. Forward Looking Information The ideas presented are forward look future ideas for platform secure features. No guarantees are provided that any ideas described in this section will be implemented in future releases. We present these ideas for the purpose of gauging public interest and support
  65. 65. Idea, High Frequency Ring Logging When diagnostic or forensic incident occurs it’s often desirable to understand the state of the system prior to event of interest. To achieve this, two logs are necessary. The normal low frequency application log and a short duration high frequency log High Freq Ring Logger, 10-15 mins of highly detailed diagnostic information. Overwritten as necessary by the system
  66. 66. Idea, Improved Message Correlation Explore different ways to correlate messages with each other. For example, time/date establish a timeline, user id is useful, etc. Maybe other types of information like application instance ID. Scoping rules different for each application, single user, multi-user, service instance, etc.
  67. 67. Idea, Async Message Logging Beneficial for some situations to log a message to a queue and return execution to the caller rather than block while message is being sent. Also useful to have offline logging and forward messages later when Internet connectivity is restored
  68. 68. Idea, Guaranteed Delivery In some situations like non-repudiation reliable logging is essential. A hypothetical example of operation would be, client logs message, the client is blocked until message is sent to log server and successfully logged after which the client is unblocked. On errors when a message cannot be logged a runtime exception can be thrown. This allows callers to rollback activities if logging is not possible
  69. 69. Idea, Improved J2EE Logging Plugin code used to correlate user ID with log messages can be improved to allow callers to specify arbitrary HTTPRequest parameters. For example, different web applications provide a number of custom request attributes that may provide additional meaningful context to log messages
  70. 70. Idea, Transport Encryption/Compression Safe and efficient data transport, HTTPS support w/gzip and deflate compression
  71. 71. Idea, Authenticated Client Logging X.509 client certs, for cases where authenticated client end- point is desirable. Industrial strength secure point to point encryption. Password or OAuth2, encrypted password solution using PBKDF2. Lighter weight, easier to manage than client certs. Also may be a good option for IoT devices that have limited software/hardware resources
  72. 72. Idea, Signed Log Messages In some scenarios it may be advantageous to sign messages on the client to ensure they are free from tampering when received on the server
  73. 73. About the Presenters August Detlefsen (California) Senior Application Security Consultant with more than eighteen years experience in software development and information security. August authored several Burp Suite extensions, CodeMagi’s Clickjacking Defense, and book author for Iron-Clad Java: Building Secure Web Applications. Twitter @codemagi or augustd@codemagi.com Sytze van Koningsveld (Netherlands) Senior Software Developer at KLM specialised in security and quality assurance with over eighteen years of Java development experience. sytze.vankoningsveld@owasp.org Milton Smith (California) Security principal developing cloud security tools at Oracle. Previously Milton was the leader for Java Platform Security and the Java Security Track at Oracle’s JavaOne conference. Past security principal at Yahoo. Twitter @spoofzu or milton.smith@owasp.com

×