Loom and concurrency latest

Structured Concurrency And
Project Loom
Srinivasan Raghavan
Principal Software Engineer
Veracode

Safe Harbor Statement
The following is for information purposes only and not an endorsement from the company I work for

❖ Little's law
❖ Constrains and beneﬁts of request pre thread model
❖ Asynchronous models beneﬁts and pit falls
❖ Need for Structured Concurrency
❖ Project Loom
❖ Q and A
Program Agenda

Little's law
L = λW
L - Concurrency should be supported
W - Mean latency
- Incoming of requestλ

Little's law
❖ There are various factors affecting the capacity of the servers L
❖ TCP sessions the os can make and handling of IO
❖ The number of threads T which the VM, OS can handle
❖ How much of each thread T is blocking which impacts W
❖ If L >>> T and W is high the we have un-responsiveness
❖ Then we should up extra resources to scale more

Request Per Thread
❖ The usual programming model is Request per thread model.
❖ Where incoming request are sent to thread and there serial step by step execution of the code
with in the thread
❖ Very easy to reason this code as its synchronous code and imperative
❖ But this uses the hardware badly
❖ There is no parallel use of cpu for computation related problem
❖ And lets the code depends on some external IO then its blocked util the entire operation is
ﬁnished
❖ And the this leads to bad performance

Asynchronous non blocking model
❖ Asynchronous frameworks came where one defines the list of tasks in a form
steps and the steps can execute at different point of time and threads
❖ The steps are given in domain specific way and so framework specific
❖ Good for the hardware
❖ Hard to maintain code and hard to reason code and profile it
❖ Throws out all effective abstraction of imperative programming built over
decades

Moving away from Request per Thread model

Imperative and Blocking
double placeDeposit(ﬁnal double value) {
var result = compute("SGD->USD", value);
result = compute("addInterest", result);
result = compute("subtractCharges", result);
return result;
}
double compute(String op, double x) {
switch (op) {
case "SGD->USD":
return getSgdUsd() * x;
case "addInterest":
return getFederalFundsRate() * x + x;
case "subtractCharges":
return x - getChargeCalculator(x);
default:
}
throw new UnsupportedOperationException();
}

Non Blocking -Wild West
void placeDeposit(double value) {
compute("SGD->USD", value, (r, e) -> {
LOGGER.log(Level.INFO, "First");
if (Objects.nonNull(e)) {
LOGGER.log(ERROR, e);
} else {
compute("addInterest", r, (r1, e1) -> {
LOGGER.log(Level.INFO, "second");
if (Objects.nonNull(e1)) {
LOGGER.log(ERROR, e1);
} else {
compute("subtractCharges", r1, (r2, e2) -> {
LOGGER.log(Level.INFO, "third");
if (Objects.nonNull(e2)) {
LOGGER.log(ERROR, e2);
} else {
publishToSomeRandomQueue(r2);
}
});
}
});
}
});
}
void compute(String op, double x, BiConsumer<Double, Exception> blk) {

Monads
❖ A pure function is a subroutine which takes an input (argument )and returns
a value or throws an exception
❖ Monads gives pure function super power. Then have like three parts
❖ Monadic value : wraps a value with a context that talks to composer
❖ Monadic function : returns a monadic value
❖ Composer: Composes a monadic value with the monadic function

Monads in JDK
public <R> Stream<R> ﬂatMap(Function<? super T, ? extends Stream<? extends R>> mapper)
public Optional ﬂatMap(Function<? super T, ? extends Optional<? extends U>> mapper)
public CompletableFuture thenCompose( Function<? super T, ? extends CompletionStage> fn)

Non Blocking -Tamed
/**
* @implNote Hope this would scale things up.
*/
public void placeDeposit(ﬁnal double value) {
compute("SGD->USD", value)
.thenCompose(r -> compute("addInterest", r))
.thenCompose(r -> compute("subtractCharges", r))
.whenComplete((r, e) -> {
if (Objects.nonNull(e)) Non
LOGGER.log(Level.ERROR, e);
} else {
publishToRandomPlace(r);
}
});
}
CompletableFuture<Double> compute(String op, double x) {
switch (op) {
case "SGD->USD":
return supplyAsync(() -> getSgdUsd() * x);
case "addInterest":
return supplyAsync(() -> getFederalFundsRate() * x +
return supplyAsync(() -> x - getChargeCalculator(x));
}
}

Non Blocking -Simplified
/**
* @implNote Hope this would scale things up.
*/
public void placeDeposit(ﬁnal double value) {
CompletableFuture.supplyAsync(() -> compute("SGD->USD", value))
.thenApplyAsync((r) -> compute("addInterest", r))
.thenApplyAsync((r) -> compute("subtractCharges", r))
} else {
publishToRandomPlace(r);
}
});
}
double compute(String op, double x) {
switch (op) {
case "SGD->USD":
return getSgdUsd() * x;
case "addInterest":
return getFederalFundsRate() * x + x;
return x - getChargeCalculator(x);
}
}

Monads Becoming harmful
CompletableFuture<Double> placeDepositMonadicReturn(ﬁnal double value) {
return CompletableFuture.supplyAsync(() -> compute("SGD->USD", value))
.thenApplyAsync((r) -> compute("addInterest", r))
.thenApplyAsync((r) -> compute("subtractCharges", r))
} else {
LOGGER.log(Level.INFO, "Computation Success");
}
});
}

Notes on Structured Programming
❖ Dijkstra major concern was abstraction. He wanted to write programs that are
too big to hold in your head all at once.
❖ One needs to parts of the program as black box. So that one doesn’t need to
know implementation details of a sub-routine
❖ He was also very concerned on the induction for scaling things
❖ goto: the destroyer of abstraction
❖ So when one deviates from the sequential ﬂow it becomes hard to reason
about it

Notes on Structured Concurrency
❖ Structured Concurrency tries to remove “go” statements from languages by
mandating join() and cancellation() for a block executing concurrent code.
❖ This ensures certainty in sub-routine output.
❖ This can open automatic resource cleanups
❖ Makes testing easy
❖ Get back the goodness of imperative abstractions which we are used and has
worked quite well for so many years

J.l.Thread
❖ 25 year old idea
❖ Threads are always mapped to os threads
❖ OS thread are more generic and designed for the variety different tasks
❖ Os thread memory can grow but not shrink
❖ In general the Thread can take KB to MB of memory
❖ Bounded executers help manage these more efﬁciently

Virtual Threads
❖ Virtual Threads which handled completely in VM and are lightweight
❖ They are scheduled and multiplexed on a carrier thread
❖ Virtual Threads can be suspended and resumed
❖ So we can create millions of Virtual Threads and be safe that memory doesn’t
run out

Continuation
public static void main(String[] args) {
final var scope = new ContinuationScope("Test");
final var continuation = new java.lang.Continuation(scope, () -> {
System.out.println("Before");
java.lang.Continuation.yield(scope);
System.out.println("After");
});
continuation.run();
System.out.println(continuation.isDone());
continuation.run();
System.out.println(continuation.isDone());
}
Before
false
After
true
Process finished with exit code 0

VirtualThread = Continuation + Schedular
class VirtualThread extends Thread {
private static final ContinuationScope VTHREAD_SCOPE = new ContinuationScope("VirtualThreads
“);
private static final Executor DEFAULT_SCHEDULER = defaultScheduler();
private static final ScheduledExecutorService UNPARKER = delayedTaskScheduler();
private final Executor scheduler;
private final Continuation cont;
private final Runnable runContinuation;
——-

VirtualThread = Continuation + Scheduler
public static void park(Object blocker) {
Thread t = Thread.currentThread();
setBlocker(t, blocker);
if (t.isVirtual()) {
VirtualThreads.park();
} else {
U.park(false, 0L);
}
setBlocker(t, (Object)null);
}
public static void unpark(Thread thread) {
if (thread != null) {
if (thread.isVirtual()) {
VirtualThreads.unpark(thread);
} else {
U.unpark(thread);
}
}
}

Structured concurrency with Java
public static void structuredConcurrency() {
var deadline = Instant.now().plusSeconds(10);
ﬁnal var exec = Executors.newVirtualThreadExecutor()
.withDeadline(deadline);
System.out.println("Start Structure");
try (exec) {
exec.submit(() -> System.out.println("Start Task1"));
exec.submit(() -> System.out.println("Start Task2"));
}
System.out.println("End Structure");
}
Start Structure
Start Task2
Start Task1
End Structure
Process ﬁnished with exit code 0

placeDeposit() with Structured Concurrency
public static Double placeDeposit(ﬁnal double value) {
ﬁnal var exec = Executors.newVirtualThreadExecutor().withDeadline(deadline);
try (exec) {
return compute("SGD->USD", value, exec)
.thenCompose(r -> compute("addInterest", r, exec))
.thenCompose(r -> compute("subtractCharges", r, exec)).join();
} catch (CompletionException ex) {
throw new RuntimeException(ex.getCause());
}
}

Exploiting parallelism
public int sumAccumulatorPattern() {
var sum = 0;
for (int i = 0; i < 100; i++) {
sum = sum + i;
}
return sum;
}
public int sumInParallel() {
return IntStream.range(0, 100)
.parallel().sum();
}

public static void downloadParallel() {
final var urls = List.of("https://www.google.com", "https://edition.cnn.com");
final var factory = Thread.builder().virtual().factory();
final var exec = Executors.newUnboundedExecutor(factory).withDeadline(deadline);
final var httpClient = HttpClient.newBuilder()
.connectTimeout(Duration.ofSeconds(5))
.executor(exec)
.build();
try (exec) {
final var tasks = new ArrayList<CompletableFuture<String>>();
for (final var url : urls) {
final var request = HttpRequest.newBuilder()
.GET()
.uri(URI.create(url))
.build();
LOGGER.log(DEBUG, "Downloading the content for the url " + url);
tasks.add(httpClient.sendAsync(request, BodyHandlers.ofString())
.thenApply(r -> handleResponse(r, url)));
}
tasks.stream().map(CompletableFuture::join).forEach(System.out::println);
} catch (CompletionException ex) {
throw new RuntimeException(ex.getCause());
}
}

Limitations of Virtual Thread
❖ Currently doesn’t work well with synchronized block. Can’t yield while
holding a lock
❖ Object.wait() inside the synchronized block the carrier thread
❖ Continuation doesn’t work with native frames
❖ Java.util.concurrent retuning is needed

References
❖ https://wiki.openjdk.java.net/display/loom/Structured+Concurrency
❖ https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-
considered-harmful/
❖ https://www.cs.utexas.edu/~EWD/ewd02xx/EWD249.PDF
❖ https://scholar.google.com/scholar?
cluster=15335993203437612903&hl=en&as_sdt=0,5
❖ https://wiki.openjdk.java.net/display/loom/Main
❖ http://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1.html

Loom and concurrency latest

More Related Content

What's hot

Similar to Loom and concurrency latest

Recently uploaded

Loom and concurrency latest