Your SlideShare is downloading. ×
0
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Developing Highly Instrumented Applications with Minimal Effort
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Developing Highly Instrumented Applications with Minimal Effort

2,148

Published on

Presentation from Silicon Valley Code Camp 2013. Related code on github: …

Presentation from Silicon Valley Code Camp 2013. Related code on github:

* https://github.com/hoserdude/mvcmusicstore-instrumented
* https://github.com/hoserdude/spring-petclinic-instrumented
* https://github.com/hoserdude/nodecellar-instrumented

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,148
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Many products promise to provide insights and value from whatever data you throw at them. There’s no doubt they can provide something, but you can’t escape the law of GIGO.
  • As a result, it’s on you as a developer, product manager and business to care about the data you are creating. This presentation will demonstrate some best practices on how to go about this, regardless of your platform.
  • Now for some definitions. There is a lot of stuff generated by the system – let’s classify the stuff broadly and narrow on what I am going to be talking about.
  • Quick review of patterns we’ll use. Interception is the secret sauce here. We want to be able to get in front of every call to every object or component in our system. That lets us see what data is being passed around, who is doing it, and how long it took.
  • Indent the bullets
  • Transcript

    • 1. Developing highly instrumented applications with minimal effort Tim Hobson Principal Engineer, Chief Caffeine Officer - Intuit
    • 2. About Me 2
    • 3. Agenda Concepts • Quality Data • Application Instrumentation • Application Logging Theory • Patterns • Supporting Frameworks • Supporting Components Practice • .Net Example • Java Example • Node.js Example GOAL: You should be able to apply this to your application today 3
    • 4. Concepts
    • 5. If You Do Nothing… OR 5
    • 6. There’s No Free Lunch Garbage In, Garbage Out Insight! 6
    • 7. Quality In, Insight Out Most apps start with only the framework or app server logging (or nothing!) None of the above is interesting to the business or the developer. You can’t get if you don’t give – there are many ways to give, and many classes of data to provide. 7
    • 8. Classes of System Output System Instrumentation • JMX/WMI/SNMP monitoring • Apache/IIS/nginx access logs App Logging App Instrumentation • • • • • • • • Intentional Business Transactions Overtly triggers alerts Source of business metrics • Aids in troubleshooting failures, bugs Cross-cutting (free) App Activity Passively triggers alerts Source of performance data • Source of usage data 8
    • 9. Theory and Best Practices
    • 10. Best Practices (© splunk>) Create human readable events Clearly timestamp events Use key-value pairs Be aware of multi-value fields Log unique identifiers 10
    • 11. Best Practices (© Tim) Global timestamps (UTC – 2013-08-21 22:43:31,990) Context setting (who/what/where/when/how) Categories/taxonomy (what tier, what component) Timing (time everything!) Security (never log sensitive data: password=***) Consistency in naming – (action=purchase; sale=oct13; productId=123123) – (action=buy; promo=oct13; sku=123123) 11
    • 12. Example Output Context 2013-08-21 22:55:36,504; LogLevel=INFO; sid=q3prv41kt511vzojytnx1d42; rid=6500583; userLogin=(null); ipAddress=0.0.0.0; thread=249; category=Web.Controllers.BaseWebController; msg=RequestInfo; server=ws001prod; url=https://myapp.com/account/logon; method=GET; languages=en-US,en;q=0.8; referrer=https://myapp.com/members/dashboard; userAgent=Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36; userId=123456; controller=Account; action=Logon; Metrics 2013-08-21 22:55:36,519; LogLevel=INFO; sid=q3prv41kt511vzojytnx1d42; rid=6500583; userLogin=(null); ipAddress=0.0.0.0; thread=249; category=TraceInterceptor; timeTracing=Service.Users.GetUser, time=3; 12
    • 13. Patterns Pipeline – Filter – Interceptor Dependency injection – Proxy Pointcuts/aspects 13
    • 14. HTTP Pipelines var app = express(); app.use(func1); app.use(func2); app.use(func3); app.use(func4); app.use(func5); app.use(func6); app.use(func7); app.use(func8); app.listen(8080); Java (from Oracle Docs) ASP.Net (from MSFT Docs) 14 express.js
    • 15. Dependency Injection • Enforces interface-based programming • Forbids circular references • Lends itself to testability • Flexibility in changing implementations • For our purposes: cross-cutting capabilities 15
    • 16. Intercepting Calls With Dynamic Proxies Interceptor Dependency A Implementation DI Container 16 Interceptor Interceptor Proxy Interceptor Interceptor Interceptor Caller Interceptor Proxy Interceptor Interceptor Proxy Dependency B
    • 17. Supporting Frameworks The Front Door The Inner Sanctum Annotations & Attributes • ASP.Net Modules/ActionFi lters • Java Servlet Filters/Spring Interceptors • Node.js interceptors • Unity Dynamic Proxies • Spring @AspectJ Pointcuts • JavaScript Mixins • @Instrumentable • [Instrumentable] 17
    • 18. Supporting Components .Net • Log4Net • NLog Java node.js • LogBack • SLF4J • Log4J 18 • SenchaLabs Connect • Winston
    • 19. Intervention!
    • 20. Sample App Intervention 3 Platforms 3 Sample Apps 20 3 Interventions
    • 21. Goals Leverage an HTTP pipeline for context Leverage aspects for interception Apply best practices Minimize impact on existing code 21
    • 22. MVC Music Store Intervention (Demo) Missing dependency injection/interception framework: Unity Missing logging library: Log4Net, Buche 1. 2. 3. 4. 5. 6. 7. 8. Add DI framework (Unity) Configure aspects (Interception) Configure controller factory, container locator Set up logger (Log4Net) Configure log pattern and targets (Log4Net.xml) Wire up logging interceptor Wire up LogActionFilter and BaseLoggingController Deploy! 22
    • 23. Spring Pet Clinic Intervention (Demo) 1. Set up logger a. Configure log pattern (to support context data) b. Configure targets (rotating file) 2. Create LogAspect.java and @Instrumentable attribute a. Capture calling context b. Start/stop timing 3. Create LogInterceptor.java a. Capture request metadata b. Set MDC c. Start/stop timing 4. Configure aspect bean 1. Configure interceptor bean 1. Annotate the methods we care about 23
    • 24. NodeCellar Intervention (Demo) Missing Interception Framework: Scarlet Missing Logging Library: Winston 1. 2. 3. 4. 5. 6. Set up Scarlet Configure log pattern Configure method interception Create logging interceptor Create LoggingFilter Deploy! 24
    • 25. Closing Thoughts
    • 26. Got Log? Now Get Intimate With Your App Ops Dashboards Business Dashboards Quality Assurance Pro-Active Service Degradation Alerting SLA Tracking Security Alerting 2 6 Customer Support Performance Metrics Pre-Release Sanity Testing
    • 27. Key Takeaways • It is YOUR responsibility as a developer to provide useful operational and business data. • It is not hard, and most of it is for free once you have the patterns in place. • The same patterns and practices can be applied to practically any platform, and any type of application or service. • When you provide consistent and predictable data others can build on your greatness 27
    • 28. Get the Code @hoserdude Spring Pet Clinic Intervention: https://github.com/hoserdude/spring-petclinic-instrumented MVC Music Store Intervention: https://github.com/hoserdude/mvcmusicstore-instrumented NodeCellar Intervention: https://github.com/hoserdude/nodecellar-instrumented 28
    • 29. Intuit Speakers @ Silicon Code Camp 2013: SATURDAY 9:45 a.m. - Ramakrishna Kollipara – “Complete Automation of Performance Testing” 1:45 p.m. - Joe Wells - “QBO: Journey From legacy Java app to a Client-side HTML5 app” 3:30 p.m. - Naga Addagadde & Sangeeta Narang – “Intuit APIs for Financial Transaction Aggregation” 5:00 p.m. Ted Drake –“Hitting the Accessibility High Notes with ARIA” SUNDAY 9:15 a.m. - Eugene Krivopaltsev –“Building Native Mobile Apps with Custom Views” 1:15 p.m. - Tim Hobson – “Developing Highly Instrumental Applications with Minimal Effort” For more information about joining our organization visit our booth or connect with our onsite recruiter: Chriscox_recruiter@intuit.com You don't want to miss out on a chance to win this cool headset. Stop by our booth to enter!
    • 30. THANK YOU Want to talk more? I’ll be at the Intuit booth today 3-5 PM. 30

    ×