The document discusses new features in Clojure including unifying primitive and boxed integer semantics, making fast numerics simpler, aligning map key equality with numeric equality, improving interaction between dynamic binding and thread pools, and adding support for primitive arguments and returns in functions. These changes aim to reduce complexity, enable optimizations, and pave the way for additional future improvements.
2. Overview
• Unified primitive semantics
• Map key equality
• Improved interaction between dynamic
binding and thread pools
• Explicitly dynamic vars
• Primitive support in fns
• This is all work-in-progress!
3. Objective - Unify
Semantics
• Unify primitive and boxed integer semantics
• Currently, boxed integer arithmetic
promotes to bigint as needed
• Primitive arithmetic throws on overflow
• Means that you always have to be explicit
about primitives
• difficulty with representation choices
• incidental boxing changes meaning
4. Objective - Make Fast
Numerics Simple
• Would like to have literal numbers be used
as primitives when possible
• Ditto primitive method returns
• Coercion and primitive ops currently for
experts only
• Easy to fail to optimize
5. Unified Integer Semantics
• The semantics of primitive longs, plus throw on
overflow
• This is the only semantic that works for both
boxed and primitive
• For bigint arithmetic, specify bigint operands
• new literal bigint format: 42N
• Polymorphic boxed numeric tower still in place
• bigints now contagious, no more auto-
reductions
6. Numeric Features
• Literal numbers are now treated implicitly
as primitives
• Locals known to be numbers are
automatically coerced to long or double
• More automatic conversions
• int matches long, float matches double
• Consistent canonic integer boxing
• If fits in long, is Long
7. Numeric Features
• longs also match ints, and doubles match floats
• long->int is checked
• Can’t force boxed representation from
primitive coercion:
• was always a bug to presume otherwise
• (class (int 42)) => Long
• ditto boxed values coming from arrays of
primitives, and primitive returns from
interop
8. Auto-promotion
• No longer the default
• Still available explicitly
• ’ is now a constituent character (like #)
• +', -', *', inc', dec' do auto-promotion
• never return primitives
• Unless you need to work with arbitrary-
precision integers where BigInt contagion
is insufficient, there is no reason to use
them.
9. Objective - Align Map
Keys and Numeric =
• Users frustrated when insert with one key
representation, lookup fails with another
• Use (type flexible) equivalence for map =
• but narrow equiv, only same category
numbers are =
• so floats not = to integers
• but == still available for that
• new clojure.lang.BigInt class
10. Map Key Equality
• Hash maps and sets now use = for keys
• this will make maps and sets unify 42 and
42N, since using =
• Will still use stricter .equals when calling
through java.util interfaces
• Not there yet
• this will require renaming
Associative.containsKey, since semantics
will now differ
11. Objective - Make Binding
Work with Threads
• When work is sent to thread-pool threads,
e.g. via agent sends or future calls, the work
isn't done with the same binding set as the
invocation (without manual effort)
• Yet that is often the expectation/desire
• Makes transparent parallelization difficult
• Current tools for propagating bindings are
expensive
12. Binding Issues
• Intended to be used within one thread
• Don’t support a notion of task or unit of
work crossing threads
• Unsafe if used between threads
• thus, must be copied
• i.e., new bindings created for subtask
thread
13. Thread Issues
• Threads don’t align with work tree
• And when threads used for subtasks, not
usually explicitly launched as children
• thread pool threads (re-)used instead
• Need better notion of subtasks not aligned
with thread stack
14. Shared Bindings
• Allow subtasks in thread pool threads to
share bindings with parent task
• Make binding access safe from subtask threads
• Add volatile semantics for binding values
• Ensure vars only set! from binding thread
• Sharing bindings is extremely fast
• Just use same (immutable) binding map in
subtask thread
15. Binding Conveyance
• Bindings are now automatically shared with
threads used for future calls and agent
sends
• and anything built on them, e.g. pmap
• This reroots the binding frame for thread
pool thread prior to your work fn running
• so bindings in work fn ‘win’
• Just works
16. Binding and Scopes
• Much of this work driven by needs of
resource scopes
• Natural fit for binding
• But need same notion of task tree
• and without conveyance, would have
same thread problems/surprises as
bindings
• Scopes work can now proceed
17. Objective - Pay for
What You Use
• All defs currently create fully dynamically
rebindable vars
• Incur cost for every access
• Is it dynamically bound?
• Is it unbound?
• What’s the current value
• Yet most vars (defns) have rarely-changing
values, never dynamically bound
18. Why Top Level Vars?
• Traditional context-bound dynamic
variables, like *out* et al
• Dynamically bound fns can be used for
error handling
• Replace fns with fixed versions
• Doesn’t require rebinding
• Mutable globals
19. Be Explicit about
Dynamic Intent
• Change default semantic from dynamically
rebindable to not
• Must declare vars dynamic using
• ^:dynamic metadata
• or .setDynamic on manually created Vars
• Better for consumers
• Enables significant optimizations
20. Metadata Features
• New ^:keyword metadata
• turns into ^{:keyword true}
• Metadata stacks - ^:dynamic ^:private foo
• Merges rather than replaces
• same as ^{:dynamic true :private true} foo
21. Remove Overhead
• Remove binding check from access to non-
dynamics
• Remove bound check from normal access
• instead return Unbound object
• has the var as field, prints in toString
• implements IFn and throws on any
invoke, with good message
22. Presume Stability
• Replace via defn to fix bugs case still remains
• infrequent, but would like to avoid pain of
static (i.e., re-eval of caller code)
• Minimize overhead
• cache var values in fn
• global var rev, inc on any var alteration
• check rev on fn entry and reload vars if
changed
• else use cached values
23. Objective - Support
Primitive Args/Returns
• Clojure supports primitive arithmetic
locally, and primitives in deftype/defrecord
• But not in fn arguments or returns
• Means you can’t break up your logic, or
write helper functions, without
introducing boxing
24. Issues
• fn is specified by an interface (IFn), taking
and returning Objects
• fns must still satisfy that interface
• How will callers know about primitive
params/return, and how to invoke?
• Support ^long and ^double type hints on
args/return
25. IFn$LOD...
• New set of single-method nested IFn
interfaces
• Corresponding to all combinations of
Object/long/double, to arity 4
• Compiler will compile invokePrim methods
in addition to IFn virtual invoke methods
• When compiling direct invocation of prim fn,
compiler will look at arglists metadata and
make call to invokePrim method
• var still contains IFn object for HOFs etc
26. More Features
• ^long and ^double hints supported on args
• hint for return goes on arg vector
• e.g. (defn foo ^long [x] ...)
• so supports multi-arity with varied returns
• Other reference type hints allowed as well
• Just for hints, not enforced
27. Future Fns
• Primitive-taking fn interfaces open door to
non-boxing HOFs
• One more step to dynamically typed, yet
internally primitive, map and reduce
• When combined with collections of
primitives
• and chunked seqs
28. Summary
• New features enable much faster code
• Unified semantics reduce complexity
• all features are still available
• auto-promotion is explicit
• dynamic binding is explicit
• Optimization more automatic, less work, less
subtle
• Pave the way for scopes and primitive HOFs