The document discusses various techniques for profiling Java applications, including sampling, tracing, and busy loops. It notes that profiling is needed to optimize performance, but that some profiling techniques can introduce overhead that shifts the balance. The key areas to focus on optimizing first are the application's own building blocks like database and web service access.

1. Is your profiler speaking to you?
Anton Arhipov

2. Anton Arhipov

5. Batch / ETL
Input: ~10k rows
Output: ~20M rows
j.u.Date / j.u.Calendar
Java 1.4 HP-UX
Initial execution time: 25 hours :-(
After optimizations: 10 hours
After redesign: 30 minutes :-)
2005
My Story

6. That moment
when you look smart,
but feel stupid

7. https://twitter.com/shipilev/status/578193813946134529

8. A
B
C
D E
Performance
Code complexity

9. Improving
Optimizing
Last 5%
Last 1%
A
B
C
D E

10. Profiling
CPU
Tracing Sampling
Memory
Usage Allocation
We are only talking about this part today

11. JVM has ability to produce thread dump:
Press Ctrl+Break on Windows
kill -3 PID on *nix
"http-nio-8080-exec-1"#40 daemon prio=5 os_prio=31 tid=0x00007fd7057ed800 nid=0x7313 runnable
java.lang.Thread.State: RUNNABLE
at o.s.s.p.w.OwnerController.processFindForm(OwnerController.java:89)
at s.r.NativeMethodAccessorImpl.invoke0(Native Method)
at s.r.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at s.r.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at j.l.r.Method.invoke(Method.java:497)
Sampling

12. Sampling interval : 20 ms
Sample count: 4

13. Sampling interval : 1 ms
Sample count: 100+

14. Sampling
main()
foo()
bar()
baz()
never captured :(

15. Safepoint bias
main()
foo()
bar()
baz()
Safepoints

16. Safepoint bias
main()
foo()
bar()
baz()
Safepoints

17. Taming safepoint bias
Java Mission Control
Proprietary protocol
Java 7u40
Honest Profiler
https://github.com/RichardWarburton/honest-profiler
AsyncGetCallTrace
JVMTI
NB! not documented

18. public0void0businessMethod()0{
00long0start0=0System.currentTimeMillis();
00work();
00Profiler.log(System.currentTimeMillis()Cstart);0
}
Tracing
(Instrumentation)

19. Tracing
public0void0businessMethod()0{
00long0start0=0System.nanoTime();
00work();
00Profiler.log(System.nanoTime()Cstart);
}

20. Tracing
public0void0businessMethod()0{
00Profiler.start(“businessMethod");0
00try0{
0000work();
00}0finally0{
0000Profiler.log("businessMethod");
00}
}

21. System.nanoTime
System.currentTimeMillis
Parallel thread updating easily accessible
memory location
sleep-wakup-update
yield-wakup-update
busy-loop-update

22. class0Profiler0{0
00Loop0loop;
00public0static0void0start(String0method)0{
0000long0now0=0loop.getTime();0
0000…
00}0

23. public0class0Loop0implements0Runnable0{
0000private0volatile0long0time;
0000public0void0run()0{
00000000while0(running)0{
000000000000time0=0System.nanoTime();
000000000000sleep();
00000000}
0000}0
0000public0final0long0getTime()0{
00000000return0time;
0000}0
Busy Loop

24. Nano seconds put into perspective
Reading memory is not free.
It takes cycles = nanoseconds
Each (software) layer is not free.
JVM,JNI,OS,HW
http://shipilev.net/blog/2014/nanotrusting-nanotime/
Nanotrusting the NanoTime

25. Nano seconds put into perspective
http://shipilev.net/blog/2014/nanotrusting-nanotime/
Nanotrusting the NanoTime
granularity_nanotime: 26.300 +-0.205 ns
latency_nanotime: 25.542 +-0.024 ns
granularity_nanotime: 29.322 +-1.293 ns
latency_nanotime: 29.910 +-1.626 ns
granularity_nanotime: 371,419 +-1,541 ns
latency_nanotime: 14,415 +-0,389 ns
Linux
Solaris
Windows

26. Sleep timer: time-slicing & scheduling
Minimum sleep times:
Win8: 1.7 ms (1.0 ms using JNI + socket poll)
Linux: 0.1 ms
OS X: 0.05 ms
VirtualBox + Linux: don’t ask :)
Still uses 25-50% of CPU core
Yield?
Windows scheduler skips yielded threads in case of
CPU starvation

27. public0class0Loop0implements0Runnable0{
0000private0volatile0long0time;
0000public0void0run()0{
00000000while0(running)0{
000000000000time0=0System.nanoTime();
000000000000sleep();
00000000}
0000}0
0000private0void0sleep()0{
000000if0(!MiscUtil.isWindows())0{
000000000Thread.yield();
000000}
0000}
Busy Loop

28. public0class0Loop0implements0Runnable0{
0000private0volatile0long0time;
0000public0void0run()0{
00000000while0(running)0{
000000000000time0=0System.nanoTime();
000000000000sleep();
00000000}
0000}0
0000private0void0sleep()0{
000000if0(!MiscUtil.isWindows())0{
000000000Thread.yield();
000000}
0000}
Busy Loop

29. Busy Loop
public0class0Loop0implements0Runnable0{
0000private0volatile0long0time;
0000public0void0run()0{
00000000while0(running)0{
000000000000time0=0System.nanoTime();
000000000000sleep();
00000000}
0000}0
0000private0void0sleep()0{
000000if0(!MiscUtil.isWindows())0{
000000000Thread.yield();
000000}
0000}

30. public0class0Loop0implements0Runnable0{
0000private0volatile0long0time;
0000public0void0run()0{
00000000while0(running)0{
000000000000time0=0System.nanoTime();
000000000000sleep();
00000000}
0000}0
0000private0void0sleep()0{
000000if0(!MiscUtil.isWindows())0{
000000000Thread.yield();
000000}
0000}
Busy Loop

31. main()
foo()
bar()
baz()
Tracing
relatively higher overhead for fast methods :(

32. Tracing
main()
foo()
bar()
baz()
2
3
4

33. Tracing
main()
foo()
bar()
baz()
2
3
5
boo()
Balance shift due to tracing overhead

37. Database access
Web Services
RMI
Various application layers
3rd-party components
HTTP session
Exceptions
Caches
File system access
View rendering
Serialization
GC
What else?
What are your building blocks?

38. Most improvement gained here!
This is where we should focus first!
A
B
C
D E