New Features in   Clojure    Rich Hickey
Overview• Unified primitive semantics• Map key equality• Improved interaction between dynamic  binding and thread pools• Ex...
Objective - Unify          Semantics• Unify primitive and boxed integer semantics • Currently, boxed integer arithmetic   ...
Objective - Make Fast Numerics Simple• Would like to have literal numbers be used  as primitives when possible • Ditto pri...
Unified Integer Semantics• The semantics of primitive longs, plus throw on  overflow  • This is the only semantic that works...
Numeric Features• Literal numbers are now treated implicitly  as primitives  • Locals known to be numbers are    automatic...
Numeric Features• longs also match ints, and doubles match floats • long->int is checked• Can’t force boxed representation ...
Auto-promotion• No longer the default• Still available explicitly• ’ is now a constituent character (like #)• +, -, *, inc...
Objective - Align Map Keys and Numeric =• Users frustrated when insert with one key  representation, lookup fails with ano...
Map Key Equality• Hash maps and sets now use = for keys • this will make maps and sets unify 42 and    42N, since using =•...
Objective - Make Binding  Work with Threads • When work is sent to thread-pool threads,   e.g. via agent sends or future c...
Binding Issues• Intended to be used within one thread• Don’t support a notion of task or unit of  work crossing threads• U...
Thread Issues• Threads don’t align with work tree• And when threads used for subtasks, not  usually explicitly launched as...
Shared Bindings• Allow subtasks in thread pool threads to  share bindings with parent task• Make binding access safe from ...
Binding Conveyance• Bindings are now automatically shared with  threads used for future calls and agent  sends  • and anyt...
Binding and Scopes• Much of this work driven by needs of  resource scopes • Natural fit for binding • But need same notion ...
Objective - Pay for      What You Use• All defs currently create fully dynamically  rebindable vars• Incur cost for every ...
Why Top Level Vars?• Traditional context-bound dynamic  variables, like *out* et al• Dynamically bound fns can be used for...
Be Explicit about      Dynamic Intent• Change default semantic from dynamically  rebindable to not• Must declare vars dyna...
Metadata Features• New ^:keyword metadata • turns into ^{:keyword true}• Metadata stacks - ^:dynamic ^:private foo • Merge...
Remove Overhead• Remove binding check from access to non-  dynamics• Remove bound check from normal access • instead retur...
Presume Stability• Replace via defn to fix bugs case still remains • infrequent, but would like to avoid pain of    static ...
Objective - SupportPrimitive Args/Returns• Clojure supports primitive arithmetic  locally, and primitives in deftype/defre...
Issues• fn is specified by an interface (IFn), taking  and returning Objects  • fns must still satisfy that interface• How ...
IFn$LOD...• New set of single-method nested IFn  interfaces  • Corresponding to all combinations of    Object/long/double,...
More Features• ^long and ^double hints supported on args• hint for return goes on arg vector • e.g. (defn foo ^long [x] .....
Future Fns• Primitive-taking fn interfaces open door to  non-boxing HOFs• One more step to dynamically typed, yet  interna...
Summary• New features enable much faster code• Unified semantics reduce complexity • all features are still available   • a...
Upcoming SlideShare
Loading in …5
×

Conjfeatures

318 views

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
318
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
10
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Conjfeatures

    1. 1. New Features in Clojure Rich Hickey
    2. 2. Overview• Unified primitive semantics• Map key equality• Improved interaction between dynamic binding and thread pools• Explicitly dynamic vars• Primitive support in fns• This is all work-in-progress!
    3. 3. Objective - Unify Semantics• Unify primitive and boxed integer semantics • Currently, boxed integer arithmetic promotes to bigint as needed • Primitive arithmetic throws on overflow • Means that you always have to be explicit about primitives • difficulty with representation choices • incidental boxing changes meaning
    4. 4. Objective - Make Fast Numerics Simple• Would like to have literal numbers be used as primitives when possible • Ditto primitive method returns• Coercion and primitive ops currently for experts only • Easy to fail to optimize
    5. 5. Unified Integer Semantics• The semantics of primitive longs, plus throw on overflow • This is the only semantic that works for both boxed and primitive• For bigint arithmetic, specify bigint operands • new literal bigint format: 42N• Polymorphic boxed numeric tower still in place• bigints now contagious, no more auto- reductions
    6. 6. Numeric Features• Literal numbers are now treated implicitly as primitives • Locals known to be numbers are automatically coerced to long or double• More automatic conversions • int matches long, float matches double• Consistent canonic integer boxing • If fits in long, is Long
    7. 7. Numeric Features• longs also match ints, and doubles match floats • long->int is checked• Can’t force boxed representation from primitive coercion: • was always a bug to presume otherwise • (class (int 42)) => Long • ditto boxed values coming from arrays of primitives, and primitive returns from interop
    8. 8. Auto-promotion• No longer the default• Still available explicitly• ’ is now a constituent character (like #)• +, -, *, inc, dec do auto-promotion • never return primitives • Unless you need to work with arbitrary- precision integers where BigInt contagion is insufficient, there is no reason to use them.
    9. 9. Objective - Align Map Keys and Numeric =• Users frustrated when insert with one key representation, lookup fails with another• Use (type flexible) equivalence for map = • but narrow equiv, only same category numbers are = • so floats not = to integers • but == still available for that• new clojure.lang.BigInt class
    10. 10. Map Key Equality• Hash maps and sets now use = for keys • this will make maps and sets unify 42 and 42N, since using =• Will still use stricter .equals when calling through java.util interfaces• Not there yet • this will require renaming Associative.containsKey, since semantics will now differ
    11. 11. Objective - Make Binding Work with Threads • When work is sent to thread-pool threads, e.g. via agent sends or future calls, the work isnt done with the same binding set as the invocation (without manual effort) • Yet that is often the expectation/desire • Makes transparent parallelization difficult • Current tools for propagating bindings are expensive
    12. 12. Binding Issues• Intended to be used within one thread• Don’t support a notion of task or unit of work crossing threads• Unsafe if used between threads • thus, must be copied • i.e., new bindings created for subtask thread
    13. 13. Thread Issues• Threads don’t align with work tree• And when threads used for subtasks, not usually explicitly launched as children • thread pool threads (re-)used instead• Need better notion of subtasks not aligned with thread stack
    14. 14. Shared Bindings• Allow subtasks in thread pool threads to share bindings with parent task• Make binding access safe from subtask threads • Add volatile semantics for binding values • Ensure vars only set! from binding thread• Sharing bindings is extremely fast • Just use same (immutable) binding map in subtask thread
    15. 15. Binding Conveyance• Bindings are now automatically shared with threads used for future calls and agent sends • and anything built on them, e.g. pmap• This reroots the binding frame for thread pool thread prior to your work fn running • so bindings in work fn ‘win’• Just works
    16. 16. Binding and Scopes• Much of this work driven by needs of resource scopes • Natural fit for binding • But need same notion of task tree • and without conveyance, would have same thread problems/surprises as bindings• Scopes work can now proceed
    17. 17. Objective - Pay for What You Use• All defs currently create fully dynamically rebindable vars• Incur cost for every access • Is it dynamically bound? • Is it unbound? • What’s the current value• Yet most vars (defns) have rarely-changing values, never dynamically bound
    18. 18. Why Top Level Vars?• Traditional context-bound dynamic variables, like *out* et al• Dynamically bound fns can be used for error handling• Replace fns with fixed versions • Doesn’t require rebinding• Mutable globals
    19. 19. Be Explicit about Dynamic Intent• Change default semantic from dynamically rebindable to not• Must declare vars dynamic using • ^:dynamic metadata • or .setDynamic on manually created Vars• Better for consumers• Enables significant optimizations
    20. 20. Metadata Features• New ^:keyword metadata • turns into ^{:keyword true}• Metadata stacks - ^:dynamic ^:private foo • Merges rather than replaces • same as ^{:dynamic true :private true} foo
    21. 21. Remove Overhead• Remove binding check from access to non- dynamics• Remove bound check from normal access • instead return Unbound object • has the var as field, prints in toString • implements IFn and throws on any invoke, with good message
    22. 22. Presume Stability• Replace via defn to fix bugs case still remains • infrequent, but would like to avoid pain of static (i.e., re-eval of caller code)• Minimize overhead • cache var values in fn • global var rev, inc on any var alteration • check rev on fn entry and reload vars if changed • else use cached values
    23. 23. Objective - SupportPrimitive Args/Returns• Clojure supports primitive arithmetic locally, and primitives in deftype/defrecord • But not in fn arguments or returns • Means you can’t break up your logic, or write helper functions, without introducing boxing
    24. 24. Issues• fn is specified by an interface (IFn), taking and returning Objects • fns must still satisfy that interface• How will callers know about primitive params/return, and how to invoke? • Support ^long and ^double type hints on args/return
    25. 25. IFn$LOD...• New set of single-method nested IFn interfaces • Corresponding to all combinations of Object/long/double, to arity 4• Compiler will compile invokePrim methods in addition to IFn virtual invoke methods• When compiling direct invocation of prim fn, compiler will look at arglists metadata and make call to invokePrim method• var still contains IFn object for HOFs etc
    26. 26. More Features• ^long and ^double hints supported on args• hint for return goes on arg vector • e.g. (defn foo ^long [x] ...) • so supports multi-arity with varied returns• Other reference type hints allowed as well • Just for hints, not enforced
    27. 27. Future Fns• Primitive-taking fn interfaces open door to non-boxing HOFs• One more step to dynamically typed, yet internally primitive, map and reduce • When combined with collections of primitives • and chunked seqs
    28. 28. Summary• New features enable much faster code• Unified semantics reduce complexity • all features are still available • auto-promotion is explicit • dynamic binding is explicit• Optimization more automatic, less work, less subtle• Pave the way for scopes and primitive HOFs

    ×