Thread with care: concurrency pitfalls in Java [Iași CodeCamp 25th October 2014]

Thread with care: concurrency pitfalls in Java
Luigi Lauro (Enteprise Architect - Product Line Group Risk Management)
UniCredit Business Integrated Solutions
Iași, 25 October 2014

Thread with care
Concurrency pitfalls in Java
We live in a world where parallel computing is becoming more and more
ubiquitous: from large-scale GPU-based computing grids for bitcoin mining and
montecarlo simulations, to 16+ cores desktop processors, 5760 shader cores
GPUs, and octa-core smartphones.
Leveraging this computing power in an effective way while also
guaranteeing correctness requires a deep understanding of the threading
capabilities and of the memory model behaviors hidden behind the language
primitives that we use every day while coding.
Yet, concurrency is one of the least taught subject in programming courses
around the world, and one of the most obscure topic for the developers, often
also very senior ones.
We'll explore the topic together by walking through cases in the Java world. We'll
also delve into novel approaches that can simplify it, such as Actors and
Software Transactional Memory.
DISCLAIMER: this presentation was made for a speech at CodeCamp. It was a very interactive presentation, so most of the explanations / teaching
were given by my voice, and are not present in text in the presentation, because I wanted to force the people to think, talk, interact.
I hope you can understand anyways the topic and what I wanted to give as an overall message.
2 Iași, 25 October 2014 Luigi Lauro - UniCredit Business Integrated Solutions

Agenda
1. Concurrent and Parallel Computing
2. Why parallel computing matters?
3. Threading in Java
4. Common Pitfalls
3
5. Solutions
6. Novel Approaches
Iași, 25 October 2014 Luigi Lauro - UniCredit Business Integrated Solutions

Concurrent and Parallel Computing
What’s the difference?
Concurrent computing
Computing in which computations are executing during overlapping time
periods – concurrently – instead of sequentially (one completing before
the next starts)
Parallel computing
Computing in which computations are carried out simultaneously, in
parallel
They have the same hardware/software requirements?
What’s the relationship between them?
One is a super-set of the other? They are mutually exclusive?
They share the same risks, the same dangers?

Why parallel computing matters?
From Moore to Amdahl
Moore’s Law
over the history of
computing hardware, the
number of transistors in a
dense integrated circuit
doubles approximately
every two years
Amdahl’s Law
Maximum speedup of parallel
computing is limited by the
fraction of the problem that
must be performed
sequentially
What does it mean for performance?
What does it mean for the programmer?
What does it mean for the user?

Threading in Java
So easy, yet so complex
 EASY
 DANGEROUS
 COMPLEX
 NOT KNOWN
 NOT TAUGHT
 NOT DOCUMENTED
 BITES YOU IN THE (_!_)
… really?

Threading in Java
Quotes on parallel computing
“The way the processor industry is going, is to add more and more cores, but nobody knows
how to program those things. I mean, two, yeah; four, not really; eight, forget it.”
Steve Jobs, Apple
“Everybody who learns concurrency thinks they understand it, ends up finding mysterious
races they thought weren’t possible, and discovers that they didn’t actually understand it yet
after all.”
Herb Sutter, chair of the ISO C++ standards committee, Microsoft.
“I decided long ago to stick to what I know best. Other people understand parallel machines
much better than I do; programmers should listen to them, not me, for guidance on how to
deal with simultaneity.”
Donald Knuth, Professor Emeritus at Stanford University
“Debugging is twice as hard as writing the code in the first place. Therefore, if you write the
code as cleverly as possible, you are, by definition, not smart enough to debug it.”
Brian Kernighan, professor at Princeton University

Threading in Java
 EASY
Can it get easier than this?
Can you do without threads?
Where are threads?

Threading in Java
 DANGEROUS
What is Thread Safety?
A class is thread-safe if it behaves correctly when accessed from multiple threads,
regardless of the scheduling or interleaving of the execution of those threads by the runtime
environment, and with no additional synchronization or other coordination on the part of the
calling code.
How it becomes dangerous?
Without proper synchronization the JVM is designed to exploit any possible optimization,
relaxing the guarantees and semantics of the java code in multi-threading environments, in
order to ensure maximum performance for the code when single-threaded.
How can I fix it?
Basically the assumption is that you know when your classes will be accessed by multiple
threads, you know what the dangers are, and you will take proper actions to notify the
JVM of this, and ensure proper behavior using the tools java offers (synchronized, volatile,
etc…)
And of course you know that perfectly right?

Threading in Java
 COMPLEX
Understanding concurrent/parallel programming and threading in Java, means
understanding in details the Java Language Specification, and how Java Memory
Model works (JSR 133), and the partial-ordering rules (“happens-before”) that are
defined there
“The Java Memory Model describes what behaviors are legal in multithreaded code, and
how threads may interact through memory. It describes the relationship between variables
in a program and the low-level details of storing and retrieving them to and from memory or
registers in a real computer system. It does this in a way that can be implemented correctly
using a wide variety of hardware and a wide variety of compiler optimizations.”
Brian Goetz

Threading in Java
 COMPLEX
What you need to know
Safely share mutable state across threads
Ensure proper visibility and avoid reading stale/partial values
Manage correctly JVM low-level optimization to not affect the software semantics
Avoid deadlocks and livelocks situations
Know what is thread-safe and what not and use them accordingly
Properly use the tools java offers (immutable objects, volatile keyword, implicit/explicit locking, atomic variables,
synchronized/concurrent collections, blocking queues and deques, latches, semaphores, barriers, phasers, executors,
FutureTask, fork-join framework, parallel bulk operations, non-blocking synchronization, thread and stack confinement,
client-side locking, safe publication, fail-fast iterators, lock striping, piggybacking, copy-on-write)
Minimize critical sections, avoid unneeded bottlenecks and maximize code performance in multi-core scenarios
You tick all the checkboxes right?

Threading in Java
 NOT KNOWN
The seniors and gurus who know it, they THINK they know it, but actually, they don’t
The ones who do know it for real, are the ones that will tell you that it’s a very
complex topic, and that they really don’t know it all
Where do you think you stand here?
Most junior programmers don’t know it
Most senior programmers don’t know it
Most “guru“ programmers don’t know it

Threading in Java
 NOT TAUGHT
In Education: while threading is so ubiquitous, it’s considered an ‘advanced topic’ and it’s
mostly not taught at all in schools and universities. Not to mention the fact that most
teachers do not actually know anything about it (there are exceptions of course! )
In Books: most people who write java books, do not really understand the topic, and
therefore do not really teach it (example: most sold and known Java book by Bruce Eckel
took 10+ years and 4 editions to include a proper and correct discussion of the topic)
In Online Articles: … they are too busy speaking about “The Cloud” (whatever that means)
or showing you how to build in 5 minutes and 10 easy steps a “HelloWorld” web application
in the last fancy, trendy framework
In Companies: you have to deliver for yesterday, something based on the requirements
that will come in tomorrow…. you think you have time to think about thread safety??
Did someone or something teach you about this topic?

Threading in Java
 NOT DOCUMENTED
Documentation is one of the most powerful tools for managing thread safety. Users look to the
documentation to find out if a class is thread-safe, and maintainers look to the documentation to
understand the implementation strategy so they can maintain it without inadvertently compromising
safety
but….
Most of Java frameworks, API, libraries, technologies very poorly document their thread-safety and
threading behaviors, if they do at all.
Even official java libraries are no exception: more often than not, you have to make assumptions
based on common sense, because java technologies specification are silent, or at least
unforthcoming, about thread safety guarantees and requirements for interfaces to implement
Try to look for words “thread” or “concurrent” in JDBC specifications
Is a ServletContext thread-safe? A HttpSession? A DataSource?

Threading in Java
 BITES YOU IN THE ***
Multi-threading bugs are the kind of bug that:
Can wreck havoc: visibility/reordering issues can make your application read old or even completely wrong values in a
variable, dead and live locks situation can make your application hang totally or run and use 100% of the hardware
resources without doing any progress. Think about what this can do to the real-world application you manage.
You will be fired faster than you can read the Exception message.
Are impossible to replicate: for 1.000.000.000 times it works correctly, then it doesn’t work correctly for one time, then it
works correctly again for another 1.000.000.000 times. Good luck debugging that.
Choose their target well: they work correctly on your development machine (one processor, less cores, client JVM), they
fail on the production machine (multi-processors, more cores, server JVM). That’s unfair I know.
You are naked against them: you have almost no tools to defend yourself. How can you detect or debug something that
happens once every 3 months and manifest in completely unknown behavior? The only way is pain-stakingly going through
*ALL* your code to find where the threading issue is. Good luck with that too.
Do you still want to write multi-threaded software?

Threading in Java
… really? Yes. Really.
Seems simple enough, isn’t it?

Threading in Java
Challenge
You think you know the answer?
#ReplyIfYouCan
Post on twitter using hashtag #ReplyIfYouCan the solution to this challenge
Anyone can participate, you have time until Sunday 26th October 2014, 23.59 to
submit your answer. At that time, I will post the ‘solution’ to the challenge.
The best answer will be selected for a special prize offered by UniCredit Business
Integrated Solutions, and will be given a job opportunity for an IT career in Iasi
Anyone up to the challenge?

Threading in Java
So, where are the problems?
 ATOMICITY
Whenever a program has instructions that should be logically connected one to the other (i.e. “Check Then Act” or “If Then
Else” logic), I have to ensure that the instructions are executed in an atomic way (all, or none) in a multi-threading
environment, otherwise race conditions could arise that violate the program intended behavior
 VISIBILITY
In a multi-threading environment, due to optimizations techniques used by the VM and JIT Compiler, there is no guarantee of
visibility of mutable state across threads. Whenever I share state across threads, I have to make sure to take the proper
steps to notify this to the VM, so to enforce necessary actions to achieve visibility
 ORDERING
The JVM specifications allows the JVM to freely re-order instructions in any program, and not execute them strictly in
order, but instead re-ordering them in a more efficient way for performance purpose, as long as this re-ordering do not
affect the intended behavior in single-thread environment. This effectively means that in multi-threading environment I
have to understand what the re-ordering could mean, and take proper action to tell the JVM not to do it, in case it would
break the correct program execution
Scary stuff isn’t it?

Common Pitfalls
If you have done this, don’t worry, you are not alone
How would you fix this?
 ATOMICITY

Common Pitfalls
 ATOMICITY

Common Pitfalls
 VISIBILITY

Common Pitfalls
 ORDERING

Solutions
…. Ahhh finally!
Now I KNOW how to fix this!
 STATELESS
 CONFINEMENT
 VOLATILE
 LOCKING
 IMMUTABLE
THREAD-SAFE CLASSES

Solutions
…. Ahhh finally!
 STATELESS
if you have no state, you have no risk

Solutions
…. Ahhh finally!
if your state can only be accessed by one thread, you’re safe
 CONFINEMENT

Solutions
…. Ahhh finally!
when you have shared state, and you need to enforce visibility ONLY, use volatile
 VOLATILE

Solutions
…. Ahhh finally!
when you have shared state, and you need to enforce visibility AND atomicity, synchronize
 LOCKING

Solutions
…. Ahhh finally!
 IMMUTABLE
Immutable objects are always thread-safe

Solutions
…. Ahhh finally!
when you can’t do something, ask someone else to do it
 THREAD-SAFE CLASSES

Solutions
Let’s see what Java offers us…
Fail-fast and fail-safe iterators: almost all non-thread safe java collections offers fail-fast iterators, that
in most cases gives ConcurrentModificationException when used incorrectly, to help detecting threading
issues, and most concurrent collections offers fail-safe iterators, which are thread-safe by default
Easy ways to ensure immutability and to force confinement: final keyword, ThreadLocal* classes
Direct un-optimized memory access: with keyword “volatile” you can force the read and write directly
to central memory and ensure no caching/optimization interfere (and no instruction re-ordering since
Java5, when it became also a full memory barrier), giving you a fast and easy solution to simple visibility
issues
Implicit and Explicit Locking: you have at your disposal exclusive implicit locking through the
“synchronized” keyword, but you can also explicit lock classes such as ReentrantLock,
ReentrantReadWriteLock, that gives more flexibility over dealing with lock unavailability and greater control
over queueing behavior (fair vs unfair lock, read-write, etc…)
Atomic classes: many atomic classes that are thread-safe, are very fast (due to native support for
Compare-and-Swap CAS hardware CPU instructions) and also allows thread-safe compound operations
like addAndGet, compareAndSet, etc.. (AtomicInteger, AtomicLongArray, AtomicReference, etc…)
Learn these tools and use them!

Solutions
Let’s see what Java offers us…
Thread-safe classes and wrappers: many thread-safe classes for a lot of different operations
(StringBuffer, etc…), thread-safe collections (CopyOnWriteArrayList, etc…), and library wrappers to make
collections thread-safe through exclusive locking (Collections.synchronizedList/Map/Set, etc…)
Concurrent collections: concurrent collections that are thread-safe, very fast, and with support for
compound operations, through implementation approaches like copy-on-write, lock-striping, piggybacking
and other non-blocking lock-free advanced synchronization techniques (CopyOnWriteArrayList/Set,
ConcurrentHashMap, ConcurrentSkipListSet/Map, ConcurrentLinkedQueue/Deque,
PriorityBlockingQueue, LinkedTransferQueue, SynchronousQueue, etc…)
Synchronizers: when you need to coordinate the work and lifecycle of multiple threads, you have a lot
of different classes to help you do so (CountDownLatch, CyclicBarrier, Semaphore, Phaser, Exchanger,
etc…)
Threading Tools: when you need to manage multiple threads for tasks and pool/re-use them, you have
a lot of threading pool possibilities, executor services and tools to facilitate the job (FutureTask, Executor,
ExecutorService, ThreadFactory, ScheduledExecutorService, CompletionService, Fork-join framework,
Thread.isInterrupted()/interrupted(), daemon threads, etc..)
Learn these tools and use them!

Novel Approaches
Going forward

Novel Approaches
Going forward
 LOCKING

Novel Approaches
Going forward
 LOCKS
PRO
Available: synchronization primitives are
available by default in any modern
programming language, Java included
Flexible: by mixing all the tools that you
have at your disposal, you can achieve and
implement exactly the concurrency model and
architecture you need for your purpose
Fast: nothing beats being able to write your
very own thread-synchronization code at the
lowest level possible, using volatile, atomics,
locks, synchronizers
CONS
Knowledge: require a very deep knowledge
of the Java Language Specification and Java
Memory Model to fully grasp the concepts
behind and master them (“happens-before”)
Complicated: managing threading
primitives at the lowest level possible is often a
big task, and cumbersome, it’s a big effort on
its own
Error-prone: it’s very easy to forget to
properly handle one corner case, and suddenly
turn one perfectly thread-safe class into a
broken one

Novel Approaches
Going forward
 ACTORS

Novel Approaches
Going forward
 ACTORS
Actors are very lightweight concurrent entities. They hold the state, and encapsulate the programming
logic, and interact with each other through asynchronous messages and message queues. They raise the
abstraction level and make it much easier to write, test, understand and maintain concurrent and/or
distributed systems. You focus on workflow—how the messages flow in the system—instead of low level
primitives like threads, locks and socket IO
PRO
Safe: no shared state, and no blocking, so
your software is safe “by default”
High Level: you don’t have to deal with
threading primitives, and you can focus on the
real task at hand
CONS
Restructuring & Rethinking: if you want to
use actors, ALL your code will need to use
actors, and all your code will need to be
written, designed and thought using actors and
message handling
Are you ready to *THINK* in actors?

Novel Approaches
Going forward
 SOFTWARE TRANSACTIONAL MEMORY

Novel Approaches
Going forward
 SOFTWARE TRANSACTIONAL MEMORY
Software transactional memory (STM) is a concurrency control mechanism analogous to database
transactions for controlling access to shared memory in concurrent computing. It is an alternative to lock-based
synchronization. A transaction in this context occurs when a piece of code executes a series of
reads and writes to shared memory. These reads and writes logically occur at a single instant in time;
intermediate states are not visible to other (successful) transactions.
PRO
Increased concurrency: thanks to
optimistic approach no thread needs to wait for
a resource, and different threads can safely
access same resource
Straightforward: you just need to mark the
transaction sections that you need to be
atomic, but you don’t have to reason about the
state, variables, conditions
CONS
Repeatable operations: every critical
sections needs to be re-executed if it fails to
‘commit’, so it means that side-effects could
trigger twice, and that do-once operations,
such as I/O, cannot be handled this way
Overhead: maintaining the log, and doing
transactions verification and commits cost time
and performance, and so does repeating
operations again and again potentially

Thread with care: concurrency pitfalls in Java
Luigi Lauro - Luigi.Lauro@unicredit.eu
Online Articles:
http://docs.oracle.com/javase/tutorial/essential/concurrency/
http://www.vogella.com/tutorials/JavaConcurrency/article.html
http://baptiste-wicht.com/posts/2010/05/java-concurrency-part-1-threads.html
http://www.javaworld.com/article/2078809/java-concurrency/
http://adit.io/posts/2013-05-15-Locks,-Actors,-And-STM-In-Pictures.html
Get in touch with me at Luigi.Lauro@unicredit.eu for asking to have this presentation, for
questions, doubts, solutions, discussions, insults, suggestions, and pretty much everything!
That’s all folks!
Books:
Java Concurrency in Practice - Brian Goetz
Concurrent Programming in Java - Doug Lea
The Art of Multiprocessor Programming - Maurice Herlihy
Programming Concurrency on the JVM - Venkat Subramaniam
Seven Concurrency Models in Seven Weeks - Paul Butcher

Thread with care: concurrency pitfalls in Java [Iași CodeCamp 25th October 2014]

Recommended

Recommended

More Related Content

Similar to Thread with care: concurrency pitfalls in Java [Iași CodeCamp 25th October 2014]

Similar to Thread with care: concurrency pitfalls in Java [Iași CodeCamp 25th October 2014] (20)

Recently uploaded

Recently uploaded (20)

Thread with care: concurrency pitfalls in Java [Iași CodeCamp 25th October 2014]