Building Observable Applications w/ Node.js -- BayNode Meetup, March 2014
•
8 likes•4,667 views
Learn how to build observable applications with Node.js so you can quickly debug and fix bugs in production.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Building Observable Applications w/ Node.js -- BayNode Meetup, March 2014
Similar to Building Observable Applications w/ Node.js -- BayNode Meetup, March 2014 (20)
Recently uploaded
Recently uploaded (20)
Building Observable Applications w/ Node.js -- BayNode Meetup, March 2014
- 1. Observable Applications w/ Node.js Yunong Xiao, Senior Node.js Engineer, UI Platform, Netflix @yunongx, yunong@netflix.com March 2015 BayNode The Bay Area Node.js Meetup
- 2. Node.js @ Netflix • Website www.netflix.com • Discovery • Playback • Acquisition • Account Management • Internal Services • TV/Mobile/Devices
- 4. Increased Errors Increased Latency Memory Leak
- 5. What Now? “It is a capital mistake to theorize before one has DATA. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.” Sherlock Holmes -A Scandal in Bohemia
- 12. vasync • https://github.com/davepacheco/node-vasync • Similar to caolan/async. • Added observability.
- 13. async f() state failed f() successful f() pending f() finished f() # of errors
- 14. vasync • Make API requests. • Persist state to file system. • Query database. • Update caches. Async workflow management is error prone
- 17. How Do I See this in Prod? • Logs • Core Dumps • REPL • HTTP API
- 18. node-bunyan • https://github.com/trentm/node-bunyan • Streaming JSON logging library for JS. (node and browser)
- 19. Streaming JSON • One JSON object per line. e.g. Twitter’s streaming API. • Perfect for machine processing. i.e. works with Unix tools such as grep(1), cut(1), … • Illegible for Humans
- 20. Streaming JSON
- 21. bunyan(1) CLI
- 22. Features • Lightweight API • Log levels: trace, debug, info, warn, error, fatal • Extensible Streams interface. • Custom object rendering with serializers. • DTrace support.
- 23. Demo
- 24. DTrace • Dynamic tracing framework. • Available on Mac OS X (dev env). `man dtrace` • Node.js DTrace USDT provider: https:// github.com/chrisa/node-dtrace-provider • Pure JS probes.
- 25. bunyan -p Demo
- 26. Production Tracing • Production environment is Ubuntu Linux on EC2. • Future work: get probe access. Possibilities: • SystemTap • LTTng (Linux Tracing Toolkit Next Gen) • perf_events markers (currently a proposed patch)
- 27. restify • http://restifyjs.com • Production tested REST framework. • Observability. • Metrics. • First class Bunyan integration. • DTrace.
- 28. Vanilla App
- 29. Vanilla App
- 30. Logging
- 31. Request Capture Stream • Tradeoffs, Perf vs Verbosity. • Captures all log statements including trace in memory. • Dump all logs only on error.
- 32. Audit Logs
- 33. Audit Logsstatus code request ID (UUID) URL request headers response headers individual handler timers req latency
- 36. Logging • Native Bunyan integration. • Request capture stream. • Audit logs with each req. • Scoped child loggers with each req.
- 37. restify + Bunyan • Streaming JSON. • Processing using Unix tools is easy. • Helpful tools: • Unix: cut(1), wc(1), grep(1), awk(1), perl(1), … • JSON: https://github.com/trentm/json (npm install -g json) • daggr: https://github.com/joyent/daggr (npm install -g daggr)
- 38. Examples • Find all logs for a specific request. • Count the # of non-200 responses. • Show all requests that took longer than 200ms.
- 39. Advanced Example Request latency distribution by URL
- 40. DTrace
- 41. Demo
- 42. distributed processing • elasticsearch • spark • hive bunyanrestify audit logs req id req headers req latency handler latencies URL res codeErrors res headers Processing w/ Unix tools • grep(1) • awk(1) • cut(1) • wc(1) • sort(1) • json(1) • bunyan(1) • … req scoped logs application specific context req capture stream dumps all logs on error DTrace* req latency handler latencies * Where available restify metrics
- 43. Increased Errors Increased Latency
- 44. restify Interested? Call for contributors. restifyjs.com
- 45. The Killer Combo • vasync: https://github.com/davepacheco/node-vasync • Observable async operations. • bunyan: https://github.com/trentm/node-bunyan • Streaming JSON logs. • restify: restifyjs.com • Observable REST applications. • Unix Tools • Stream of text model. Many tools to manipulate JSON logs.
- 46. Data Data Data
- 47. Thanks • Questions? • yunong@netflix.com • @yunongx • http://restifyjs.com March 2015 BayNode The Bay Area Node.js Meetup