Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Cause of Infinite Loop in Wait-free Bounded Queue
1. int dequeue() {
uint64_t cur_slot = front++;
int order = cur_slot / size;
cur_slot = cur_slot % size;
while (true) {
uint64_t flag = slot[cur_slot].flag;
if (flag % 2 == 0 || flag / 2 != order) {
//if (slot[cur_slot].flag % 2 == 0 || slot[cur_slot].flag / 2 != order) {
this_thread::yield();
} else {
int ret = slot[cur_slot].key;
slot[cur_slot].flag++;
return ret;
}
}
}
Commented out if statement has potential hazard(unexpected behavior).
Queue will suffer infinite loop if slot[cur_slot].flag's value has been
modified by other dequeuing thread during the execution of if statement.
The full code can be found here
Cause of Possible Infinite Loop in Wait-free Queue
void enqueue(int key) {
uint64_t cur_slot = rear++;
int order = cur_slot / size;
cur_slot = cur_slot % size;
while (true) {
uint64_t flag = slot[cur_slot].flag;
if (flag % 2 == 1 || flag / 2 != order) {
this_thread::yield();
} else {
slot[cur_slot].key = key;
slot[cur_slot].flag++;
break;
}
}
}
2. data
■
flag 1
Q(0) Q(1) Q(2) Q(3) Q(4)
...
4
I will briefly demonstrate possible hazard scenario with 4 threads.
Assume all threads already called enqueue/dequeue function.
Their cur_slot value is showed in gray box.
Let's ignore all other function calls between them.
Current running thread & code section will be highlighted by red box.
int dequeue() {
uint64_t cur_slot = front++;
int order = cur_slot / size;
cur_slot = cur_slot % size;
while (true) {
uint64_t flag = slot[cur_slot].flag;
if (flag % 2 == 0 || flag / 2 != order) {
//if (slot[cur_slot].flag % 2 == 0 || slot[cur_slot].flag / 2 != order) {
this_thread::yield();
} else {
int ret = slot[cur_slot].key;
slot[cur_slot].flag++;
return ret;
}
}
}
void enqueue(int key);
... ...
4 9 9Assume size = 5
enq(■) deq() enq(■) deq()
3. ...
4
First thread successfully enqueued ■, now second & fourth thread enters
the if statement simultaneously.
int dequeue() {
uint64_t cur_slot = front++;
int order = cur_slot / size;
cur_slot = cur_slot % size;
while (true) {
uint64_t flag = slot[cur_slot].flag;
if (flag % 2 == 0 || flag / 2 != order) {
//if (slot[cur_slot].flag % 2 == 0 || slot[cur_slot].flag / 2 != order) {
this_thread::yield();
} else {
int ret = slot[cur_slot].key;
slot[cur_slot].flag++;
return ret;
}
}
}
void enqueue(int key);
... ...
4 9 9
slot[cur_slot].flag
!= 0
must continue if
statement...
enq(■) deq() enq(■) deq()
data
■
flag 1
Q(0) Q(1) Q(2) Q(3) Q(4)
Assume size = 5
4. enq(■) deq() enq(■) deq()
...
4
int dequeue() {
uint64_t cur_slot = front++;
int order = cur_slot / size;
cur_slot = cur_slot % size;
while (true) {
uint64_t flag = slot[cur_slot].flag;
if (flag % 2 == 0 || flag / 2 != order) {
//if (slot[cur_slot].flag % 2 == 0 || slot[cur_slot].flag / 2 != order) {
this_thread::yield();
} else {
int ret = slot[cur_slot].key;
slot[cur_slot].flag++;
return ret;
}
}
}
void enqueue(int key);
... ...
4 9 9
slot[cur_slot].flag
!= 0
must continue if
statement...
slot[cur_slot].flag
!= 0
must continue if
statement...
They both realized that `slot[cur_slot].flag == 0` is false,
so they need to check second part(slot[cur_slot].flag / 2 != order)
data
■
flag 1
Q(0) Q(1) Q(2) Q(3) Q(4)
Assume size = 5
5. ...
4
second thread realized that `slot[cur_slot].flag / 2 != order` is false.
Because(obviously) both side has the value of 0.
Now it proceeds dequeuing(■).
int dequeue() {
uint64_t cur_slot = front++;
int order = cur_slot / size;
cur_slot = cur_slot % size;
while (true) {
uint64_t flag = slot[cur_slot].flag;
if (flag % 2 == 0 || flag / 2 != order) {
//if (slot[cur_slot].flag % 2 == 0 || slot[cur_slot].flag / 2 != order) {
this_thread::yield();
} else {
int ret = slot[cur_slot].key;
slot[cur_slot].flag++;
return ret;
}
}
}
void enqueue(int key);
... ...
4
slot[cur_slot].flag /
2 != order is false!
enq(■) deq() enq(■) deq()
data
■
flag 1
Q(0) Q(1) Q(2) Q(3) Q(4)
Assume size = 5 9 9
6. ...
After second thread's dequeue. Now the Q(4) is empty and waiting for the
enqueue. But instead of enqueue thread, fourth thread (which was yielded
after executing half of if statement) is scheduled in.
int dequeue() {
uint64_t cur_slot = front++;
int order = cur_slot / size;
cur_slot = cur_slot % size;
while (true) {
uint64_t flag = slot[cur_slot].flag;
if (flag % 2 == 0 || flag / 2 != order) {
//if (slot[cur_slot].flag % 2 == 0 || slot[cur_slot].flag / 2 != order) {
this_thread::yield();
} else {
int ret = slot[cur_slot].key;
slot[cur_slot].flag++;
return ret;
}
}
}
void enqueue(int key);
... ...
enq(■) deq() enq(■) deq()
data
flag 2
Q(0) Q(1) Q(2) Q(3) Q(4)
Assume size = 5 4 4 9 9
7. ...
Because second thread has increased flag value, unexpectedly
`slot[cur_slot].flag / 2 != order` returns false(value == 1).
Dequeue has been finished even before execution of proper matching
enqueue function call(■). Now fourth thread receives garbage value(■).
int dequeue() {
uint64_t cur_slot = front++;
int order = cur_slot / size;
cur_slot = cur_slot % size;
while (true) {
uint64_t flag = slot[cur_slot].flag;
if (flag % 2 == 0 || flag / 2 != order) {
//if (slot[cur_slot].flag % 2 == 0 || slot[cur_slot].flag / 2 != order) {
this_thread::yield();
} else {
int ret = slot[cur_slot].key;
slot[cur_slot].flag++;
return ret;
}
}
}
void enqueue(int key);
... ...
Assume size = 5
enq(■) deq() enq(■) deq()
slot[cur_slot].flag /
2 != order is false!
data
flag 2
Q(0) Q(1) Q(2) Q(3) Q(4)
4 4 9 9
8. ...
After two unwilling dequeue functions are done, third thread will never
proceed if statement. Because `flag % 2` will always return true.
Next dequeuing thread will also yield forever because of `flag / 2 != order`
will remain true forever.
... ...
enq(■) deq() enq(■) deq()
data
flag 3
Q(0) Q(1) Q(2) Q(3) Q(4)
int dequeue() {
uint64_t cur_slot = front++;
int order = cur_slot / size;
cur_slot = cur_slot % size;
while (true) {
uint64_t flag = slot[cur_slot].flag;
if (flag % 2 == 0 || flag / 2 != order) {
//if (slot[cur_slot].flag % 2 == 0 || slot[cur_slot].flag / 2 != order) {
this_thread::yield();
} else {
int ret = slot[cur_slot].key;
slot[cur_slot].flag++;
return ret;
}
}
}
void enqueue(int key) {
uint64_t cur_slot = rear++;
int order = cur_slot / size;
cur_slot = cur_slot % size;
while (true) {
uint64_t flag = slot[cur_slot].flag;
if (flag % 2 == 1 || flag / 2 != order) {
this_thread::yield();
} else {
slot[cur_slot].key = key;
slot[cur_slot].flag++;
break;
}
}
}
deq()
14
flag % 2
== 1 is true!
yield!
flag / 2 != order
is true!
yield!
Assume size = 5 4 4 9 9