The presentation I gave at Riviera Scala Clojure Group on Scala profiling techniques (http://www.meetup.com/riviera-scala-clojure/events/54559732/)

1. SCALA PROFILING
(for Java developers)
Filippo Pacifici
mercoledì 23 maggio 12

2. Who am I
• Filippo Pacifici
• Twitter: OddId_
• mail: filippo.pacifici@gmail.com
• Blog: http://outofmemoryblog.blogspot.com
• One of the 9M devs thinking Java is not so bad
• recently started looking at Scala
mercoledì 23 maggio 12

3. What’s this all about?
• Apply Java profiling methods to Scala
programs
• How do we deal with performance in
Java?
• Is it the same in Scala?
• How do we optimize a JVM for Scala
applications?
mercoledì 23 maggio 12

4. Java profiling 101
mercoledì 23 maggio 12

5. Java profiling 101
• Goals:
• troubleshoot performance problems
• estimate application performance
• estimate application scalability
mercoledì 23 maggio 12

6. Java profiling 101
• Ehi, I use Scala, why should I care about Java
profiling?
• Scala compiled in byte code and runs in a
JVM
• We can profile a Scala application as it
was Java
• We can use the same tools
• I am not aware of alternatives
mercoledì 23 maggio 12

7. Java methods profiling
• Tracks methods invocations
• Runtime instrumentation
• Time analysis
mercoledì 23 maggio 12

8. Java memory profiling
• Dumps heap content
• Browse objects in the heap
• Memory usage analysis
mercoledì 23 maggio 12

9. Profiling tools
• Method profiling
• Java Visual VM (embedded in JDK)
• Yourkit, Dynatrace, etc.
• Memory profiling
• Eclipse MAT (www.eclipse.org/
mat)
• Yourkit, Dynatrace, etc.
mercoledì 23 maggio 12

10. Back to Scala...
• We won’t find Scala specific constructs
• Need to know how Scala is translated
into bytecode
• Goals:
• Identify methods generated by Scala
compilers
• Characterize Scala data structures in
memory
mercoledì 23 maggio 12

11. Scala Functions
mercoledì 23 maggio 12

12. Short discouraging
comparative demo
mercoledì 23 maggio 12

13. Scala functions vs byte code
• Classic functions converted in methods
• First class function do not exist in Java
• Anonymous classes extending
scala.runtime.AbstractFunction
• apply method to execute.
mercoledì 23 maggio 12

14. AbstractFunction
• AbstractFunction2
• takes 2 input parameters
• One apply method per
combination of input and
output types
• example apply.mcFID
• F= returns float
• I takes one Int
• D takes one Double
mercoledì 23 maggio 12

15. AbstractFunction
• Find the call in the profile:
• anonfun => instance of the anonymous class
• main$1 => first anonymous class defined in
main method
mercoledì 23 maggio 12

16. Functions in the heap
• Each instance of AbstractFunction present
in the heap
• Very small impact
• Stateless
mercoledì 23 maggio 12

17. Where do we use them?
• Our program did not contain any anonymous
function, right?
• AbstractFunction used:
• For first class functions
• For closures
• For partially applied functions
• To manage for loops blocks
• To manage filter logic in for loops
mercoledì 23 maggio 12

18. Performance impact
• Is this a performance impact?
• Scala compiler performs optimizations:
• Same anonymous functions reused
(avoid multiple instantiations)
• Anonymous functions doing the same
thing are shared
• Attention to partially applied:
• New function created.
mercoledì 23 maggio 12

19. Some examples
mercoledì 23 maggio 12

20. Scala data structures
(Collections in heap dump)
mercoledì 23 maggio 12

21. Scala Lists
• A Scala view:
• Linked lists (single link)
• abstract class List + two case classes: ::
and Nil
mercoledì 23 maggio 12

22. Scala Lists
• A byte code view:
• Case classes become inner classes:
• :: becomes $colon$colon
• Nil becomes Nil
mercoledì 23 maggio 12

23. Scala Lists
• Heads and elements have the same type
mercoledì 23 maggio 12

24. Scala Lists
• What about mutable lists?
• ListBuffer
• Wrapper on a Linked List
• Keeps an additional reference to the
last element: last0
mercoledì 23 maggio 12

25. Scala Sets
• Immutable sets
• scala.collection.immutable.Set
• Case classes for different sizes
• HashSet over 5 elements
mercoledì 23 maggio 12

26. Scala Maps
• Immutable:
• small number of elements:
scala.collections.immutable.Map$MapN
• N = number of elements
• over 5 elements:
scala.collection.immutable.HashMap
• Mutable: scala.collection.mutable.HashMap
mercoledì 23 maggio 12

27. Map and Sets examples
mercoledì 23 maggio 12

28. Primitive types boxing
mercoledì 23 maggio 12

29. Primitive types and
generics
• Type parameters cannot be primitive in
generic types.
• Scala systematically boxes and unboxes
them to Object
• scala.runtime.BoxesRunTime methods
mercoledì 23 maggio 12

30. Basic tuning tips
mercoledì 23 maggio 12

31. Exploit tail recursion
• Long recursion
• Long stack
• Performance impact on stack size
• Scala compiler recognizes tail recursion
• Recursive call must be the last operation
of the method
• Scala transforms it into iterative form
mercoledì 23 maggio 12

32. Can’t exploit tail
recursion?
• If (and only if) you run out of stack space
(frequent java.lang.StackOverflowError):
• -Xss JVM option sets stack size
• example: -Xss2048k
• Normally limited at OS level
• Each thread statically allocates stack size:
• pay attention
mercoledì 23 maggio 12

33. Memory structure
• Optimize for small, short lived objects
• Anonymous functions:
• small
• frequently instantiated
• Use a big young space
• GC is fast and frequent
• Objects do not get promoted
mercoledì 23 maggio 12

34. Memory structure
• What about the perm gen?
• Anonymous classes reused
• No insane usage of proxies
• No specific issues
mercoledì 23 maggio 12

35. Which GC should I use?
• Depends on your application requirements
• The same consideration done for Java
still hold
• Need throughput : parallel GC
• Need response time : CMS
• You are brave : G1
mercoledì 23 maggio 12

36. Questions?
mercoledì 23 maggio 12