Implementing efficient spinlocks in userspace is not possible yet in Linux, even after years of different approaches and proposed solutions.The main gap to achieve it is the lack of ABI providing an easy and low-overhead way to check if the current lock holder is running or not. In this session, we are going to present the problem, and to propose a solution for it using the restartable sequences infrastructure as means to expose the thread state to userspace in a cheap way, without requiring system calls. RFC: https://lore.kernel.org/lkml/20230529191416.53955-1-mathieu.desnoyers@efficios.com/ (c) Linux Plumbers Conference 2023 15-15 Nov Omni Richmond Hotel, Richmond, VA (US) https://lpc.events/event/17/page/198-lpc-2023-overview

1. Userspace adaptive spinlock with rseq
André Almeida, Igalia
Mathieu Desnoyers, EfficiOS

2. Use case
● Calling a mutex_lock() -> futex() requires a context switch
● Context switch can be more expensive than the critical section ("microcontention")
● Some apps have a complete userspace lock implementation without syscalls to avoid this cost
● Let's allow userspace to correctly spin!
● rt_mutex uses adaptive spinlock

3. The challenge
● A lock contender should spin if the lock owner is running
● Or sleep otherwise
● How can we tell an userspace thread is running or not?
● In a very fast way, without syscalls

4. rseq
● rseq code is already integrated with task scheduler and has a fast uAPI
● No syscalls to read information set by the kernel
● In a similar fashion, rseq was reused for getcpu as well

5. sched_state
● A new field for struct rseq: sched_state
● Updated by the kernel when process is scheduled out and in
● Now, the only thing left for userspace is to check if a thread is on a given CPU or not.
● If it is, it can safely spin, else, it goes the normal path (sleep using futex())

6. rseq struct cache line
struct rseq_sched_state {
__u32 version;
__u32 state;
__u32 tid;
};
struct rseq {
[...]
__u32 mm_cid;
__u32 padding1;
__u64 sched_state_ptr;
char end[];
}

7. robustness?
● This interface requires shared thread information
● Before a thread dies, it free the lock in the exit path
● Barriers around the sched_state ensure that every reader is reading something meaningful
● However, this mechanism isn’t robust for shared memory/multi process
● But we don’t even have access to other processes' thread area in the first place
● Is this a concern?

8. Correctness
● Scheduler cannot take a page fault, so there’s no guarantee that the sched_state is always correct
● Statistically it should work anyways, it’s an edge case
● Worse case: it gets faulted in on next return to userspace
● Workaround: hook into the page fault handler, and populate it with new data when it gets faulted
in for read