Tommi Reiman discusses optimizing Clojure performance and abstractions. He shares lessons learned from optimizing middleware performance and JSON serialization. Data-driven approaches can enable high performance while maintaining abstraction. Reitit is a new routing library that aims to have the fastest performance through techniques like compiled routing data. Middleware can also benefit from data-driven approaches without runtime penalties. Overall performance should be considered but not obsessively, as many apps do not require extreme optimization.

1. Performance &
Abstractions
Clojure Finland 28.9.2017
Tommi Reiman
@ikitommi

2. Or …
The Story of learning how to
write fast low-level Clojure code
for greater good fun & profit.

4. The Story begins
• 1+ year ago, chat with Alan Malloy about
middleware performance
• Had no clue (compojure-api)
• Measured some

5. The Story begins
• 1+ year ago, chat with Alan Malloy about
middleware performance
• Had no clue (compojure-api)
• Measured some
• Oh my.

6. Performance

7. Performance
• Experienced by the End Users
• Your System performance
• Your application
• Core & libraries

8. Performance
• Experienced by the End Users
• Your System performance
• Your application
• Core & libraries
è Does the low-level perf matter?

9. Performance
• Measuring
– clojure.core/time
– Criterium (https://github.com/hugoduncan/criterium)
• Analysing
– Source code
– Decompiler
– no.dissamble
– Profilers
– JVM options

10. Performance
• Always measure, but don’t trust the results
• Are we measuring the right thing?
• Performance And Lies byTom Crayford
– http://yellerapp.com/posts/2015-06-29-performance-
and-lies.html

11. The JVM
• Inlining
– http://normanmaurer.me/blog/2014/05/15/Inline-all-the-
Things/
• Unrolled SmallVectors
– https://dev.clojure.org/jira/browse/CLJ-1517
– https://github.com/google/guava/issues/1268
– Polymorphism
• Monomorphic
• Bimorphic
• Megamorphic

12. HikariCP
https://brettwooldridge.github.io/HikariCP/

13. The usual suspects
• Protocols over Multimethods
• Records over Maps
• Transducers
• Regexps
• Destructuring
• Too high abstraction (walkers, zippers, get-in,…)
• Too long methods
• Too much code

14. DEMO

15. … back to the story
• Trying to understand and find quick wins for the
compojure-api middleware pipeline
• Had already ported Kekkonen from Middleware
into Pedestal Interceptors
– … and the performance degraded
• I know nothing, need to learn.

16. Three samples
(And the order or magnitude)

17. JSON

18. Cheshire
• https://github.com/dakrone/cheshire
• clojure-json had really nice features (custom encoders),
but was slow; clj-json had no features, but was fast.
Cheshire encodes JSON fast, with added support for
more types and the ability to use custom encoders.
• Global extensions via add-encoder!
• It’s an awesome lib

19. JSONista
• https://github.com/metosin/jsonista
• Clojure library for fast JSON encoding and
decoding.
• Embrace Java (Jackson)
• Explicit configuration
• Need some testing
• Merge with Cheshire?

20. https://github.com/metosin/jsonista/blob/master/test/jsonista/json_perf_test.clj

21. Takeaways
• Performance
– Embrace Java to go fast
– Avoid: dynamic binding, large methods, (Clojure) inlining
– Always be measuring
• Api
– Explicit configuration over global mutable state

22. Content-negotiation

23. Ring-middleware-format
• https://github.com/ngrunwald/ring-middleware-format
• Ring middleware for parsing parameters and
emitting responses in JSON or other formats
• It’s a great lib

24. Muuntaja
• https://github.com/metosin/muuntaja
• Clojure library for fast http format negotiation,
encoding and decoding. Standalone library, but
ships with adapters for ring (async) middleware &
Pedestal-style interceptors.
• Complete rewrite of r-m-f
• “how fast can we go with Clojure?”

25. https://github.com/metosin/muuntaja/blob/master/test/muuntaja/core_perf_test.clj

26. https://github.com/metosin/muuntaja/blob/master/test/muuntaja/core_perf_test.clj

27. Takeaways
• Performance
– Single middleware/interceptor
– Java-backed memoization
– Avoid run-time regexps
– Micro-opimize.*: no destructurings, unroll generic
functions, records & field access instead of hash-lookups,
no dynamic binding, small methods to support JVM
inlining (profile it!), streaming etc.

28. Explicit configuration

29. compojure-api perf
For real??!?!

30. Routing

31. Lot’s of existing libs
• Ataraxy, Bidi, Compojure, Pedestal, …
• All great libs, but none is perfect
• Most are not optimized for perf
• Looking out of the box
– routers in GO, JavaScript and C++

33. reitit
• https://github.com/metosin/reitit
• A friendly data-driven router for Clojure(Script).
• Tiny core, optional layers (ring, coercion)
• A hobby project, aiming to be
– fastest ever
– best parts of existing libs
– new features & abstractions

36. The perf
• Best parts of Pedestal routing
• The low-level perf-knobs from previous libs
• Route flattening & re-ordering
• Compiled route data
• Compiled middleware (and interceptors)

37. Different routers
• Linear-router
• Lookup-router
• Mixed-router
• (Compiled-router)
• (Prefix-tree-router)

39. https://github.com/metosin/reitit/blob/master/perf-test/clj/reitit/opensensors_routing_test.clj

41. http://delved.org/What-if-I-told-you-that-HTTP-can-be-fast/#/29

43. Middleware & Interceptors
• Middleware
– Opaque functions
– Great Performance
– CPS, higher stacks
• Interceptors
– Need an Interpreter, shallow stacks
– Data-driven, easy to extend
– Slower to run

45. Data-driven middleware?
• Like Pedestal interceptors, with :wrap key
• Extra data for docs & behavior
• Middleware registries? (Macchiato & Duct)
• Manual & Automatic composition
• Optional :requires & :provides
• Advanced visualization & debugging
• Zero runtime penalty
• Better perf when ”compiled”

49. Final words

50. Performance
• For many apps, don’t need to think of it
• Think in web-scale & big-data
• Understanding perf makes you a better
programmer
• Quick wins with the usual suspects
• Read the source, time & criterium
• Don’t trust the benchmarks
• Always be measuring

51. Abstractions
• Data-driven enables big wins for perf,
abstraction and configuration
• Data can be transformed &/ compiled for
effective runtime
• Both ahead-of-time & delayed computation

52. Coming up
• Finalizing the libs, go try them out!
• Follow on twitter for updates on libs & life

53. Coming up
• Finalizing the libs, go try them out!
• Follow on twitter for updates on libs & life
• Cool projects on pipeline, hiring both
programmers & designers, Helsinki &Tampere
– Poke me after the talk or tommi@metosin.fi

54. Thanks!
Clojure Finland 28.9.2017
Tommi Reiman
@ikitommi