2. 2 |
Cautionary Statement
This presentation contains forward-looking statements concerning Advanced Micro Devices, Inc. (AMD) such AMD’s vision, mission and focus; AMD’s market
opportunity and total addressable markets; AMD’s technology and architecture roadmaps; the features, functionality, performance, availability, timing and expected
benefits of future AMD products and product roadmaps; AMD’s path forward in data center, PCs and gaming; AMD’s market and financial momentum; and the
expected benefits from the acquisition of Xilinx, Inc., which are made pursuant to the Safe Harbor provisions of the Private Securities Litigation Reform Act of 1995.
Forward-looking statements are commonly identified by words such as "would," "may," "expects," "believes," "plans," "intends," "projects" and other terms with
similar meaning. Investors are cautioned that the forward-looking statements in this presentation are based on current beliefs, assumptions and expectations, speak
only as of the date of this presentation and involve risks and uncertainties that could cause actual results to differ materially from current expectations. Such
statements are subject to certain known and unknown risks and uncertainties, many of which are difficult to predict and generally beyond AMD's control, that could
cause actual results and other future events to differ materially from those expressed in, or implied or projected by, the forward-looking information and statements.
Investors are urged to review in detail the risks and uncertainties in AMD’s Securities and Exchange Commission filings, including but not limited to AMD’s most
recent reports on Forms 10-K and 10-Q. AMD does not assume, and hereby disclaims, any obligation to update forward-looking statements made in this
presentation, except as may be required by law.
3. 3 |
Problem statement & Outline
On GNU/Linux GDB, it is not possible to use Ctrl-C to interrupt programs that block SIGINT:
• Type Ctrl-C => nothing happens, the debug session gets stuck
• In some cases, it is not possible to get back the prompt
Talk outline:
• Explain why that does not work
• Why this turned out important for the AMD GPU target
• The plan to fix it
4. 4 |
First, a quick Unix process model primer
• A session is UNIX's way of grouping processes that share a controlling tty
• A session can have a controlling tty ("teletype", aka, terminal)
• A session contains process groups
• A process group contains processes
• At most one process group in a session can be a foreground process group
• Typing the interrupt character on a tty => kernel sends SIGINT to:
• all processes of the foreground process group
• in the session (if any)
• which has that tty as controlling tty
• The default interrupt character is ^C (Ctrl-C)
• Can change with e.g., "$ stty intr ^G"
5. 5 |
SIGINT blocked example
int main () {
/* Block SIGINT. */
sigset_t new_mask;
sigaddset (&new_mask, SIGINT);
sigprocmask (SIG_BLOCK, &new_mask, NULL);
while (true) {
sleep (1);
}
}
(gdb) r
Starting program: sigint-blocked
^C^C^C
*nothing*
6. 6 |
sigwait example
int main () {
/* Block SIGINT. */
sigset_t new_mask;
sigaddset (&new_mask, SIGINT);
sigprocmask (SIG_BLOCK, &new_mask, NULL);
/* Make sigwait wait for all signals. */
sigset_t signal_set;
int sig;
sigfillset (&signal_set);
sigwait (&signal_set, &sig);
}
(gdb) r
Starting program: sigwait
^C
[Inferior 1 (process 2585683)
exited normally]
(gdb)
Same thing with signalfd
Bug 9425 - When using "sigwait" GDB doesn'ttrap SIGINT.
Ctrl+C terminates program when should break gdb
First reported on 2007-09-20:
https://sourceware.org/bugzilla/show_bug.cgi?id=9425
Rejected as kernel problem:
https://bugzilla.kernel.org/show_bug.cgi?id=9039#c1
7. 7 |
What is Ctrl-C supposed to be under GDB?
Ctrl-C has "supervisor" role in GDB
• GDB action vs action on inferior
Ctrl-C when running under GDB is:
• "please ask GDB to stop the program so I can inspect it"
• not: "kill the program" (as in, send-signal-kill)
Note how default is to not pass SIGINT => consistent with above view:
• 'c' after Ctrl-C resumes inferior rather than killing it
• must explicitly use 'kill' or 'signal' after a Ctrl-C to interrupt the child process (vs just stop it)
8. 8 |
Problem is consequence of how GDB implements Ctrl-C
• When GDB spawns a new process, it:
• starts the process in the same terminal as GDB
• puts it in a new process group
• makes that the foreground process group in the session
• When the process is running, e.g., after "continue"
• GDB put the inferior's process group in the foreground
• the user types Ctrl-C - kernel sends a SIGINT to the foreground process group (the inferior)
• GDB intercepts the SIGINT via ptrace before inferior sees it
• GDB stops all the other threads of the inferior (if all-stop)
• GDB presents the "Program received SIGINT" stop to the user
• If SIGINT is masked out, signal is left pending, not delivered
• If program is using sigwait, a pending SIGINT is consumed, but not delivered
9. 9 |
GDB / Inferior share terminal (GDB juggles terminal settings)
GDB
Inferior
process
GDB's
terminal
fork()/exec()
stdout
stderr
stdin
User
keyboard input
terminal output • GDB and inferior share the session / terminal
• GDB puts inferior in a new process group
• GDB saves/restores gdb/inferior terminal settings appropriately
stdout
stderr
stdin
10. 10 |
Inferior
process
GDB / Inferior share terminal (GDB in foreground)
GDB
GDB's
terminal
2. SIGINT
aborts/quits
current
command
User
1. Ctrl-C • GDB is in the foreground (user can type GDB commands)
• Readline terminal settings apply
• Ctrl-C => SIGINT to GDB
• SIGINT handler sets global quit_flag flag
• Mainline code calls QUIT macro, which aborts current command
• QUIT macro calls sprinkled throughout GDB, in loops
11. 11 |
GDB / Inferior share terminal (Inferior in foreground)
GDB
Inferior
process
GDB's
terminal
User
1. Ctrl-C
0. After "(gdb) c":
• GDB restores inferior terminal settings
• Inferior is put in the foreground
• Ctrl-C results in SIGINT sent to inferior
2. SIGINT
Kernel sends SIGINT to
terminal's foreground progress
group
3. SIGCHLD
• ptrace intercepts SIGINT, stops thread, and
wakes ptracer via SIGCHLD
• gdb:
1. calls waitpid and learns about SIGINT
stop
2. stops all threads (if all-stop mode)
3. presents "Program received SIGINT"
4. saves inferior terminal settings
5. restores its own terminal settings
6. puts itself in foreground (back to
previous slide)
12. 12 |
GDB / Inferior share terminal (Inferior in foreground, SIGINT blocked)
GDB
Inferior
process
GDB's
terminal
User
1. Ctrl-C
0. After "(gdb) c", and SIGINT is blocked in inferior:
• GDB restores inferior terminal settings
• Inferior is put in the foreground
• Ctrl-C results in SIGINT pending in inferior, but not delivered
2. SIGINT
Kernel sends SIGINT to terminal's foreground progress group
But, signal remains pending, never delivered...
Thus, ptrace never intercepts it
13. 13 |
GDB / Inferior share terminal (Inferior in foreground, sigwait)
GDB
Inferior
process
GDB's
terminal
User
1. Ctrl-C
0. After "(gdb) c", and inferior uses sigwait:
• GDB restores inferior terminal settings
• Inferior is put in the foreground
• Ctrl-C results in SIGINT queued in inferior, but not delivered
2. SIGINT
Kernel sends SIGINT to terminal's foreground progress group
sigwait consumes pending signal, technically signal is never delivered...
Thus, ptrace does not intercept it
14. 14 |
Ctrl-C and AMDGPU – why important?
(gdb) info threads
Id Target Id Frame
1 Thread ... (LWP 476966) main () from libhsa-runtime64.so.1
2 Thread ... (LWP 476969) in ioctl () at syscall-template.S:78
4 Thread ... (LWP 477504) in ioctl () at syscall-template.S:78
* 5 AMDGPU Wave 1:1:1:1 (0,0,0)/0 my_kernel () at kernel.cc:41
…
• Kernel sends SIGINT to process (host side)
• But all host threads are stopped
• ptrace will not intercept the SIGINT, until a
host thread is resumed, and the
signal delivered
• But user does not have prompt, so, hang...
(gdb) set scheduler-locking on
(gdb) thread 5 # a GPU thread
(gdb) c
^C^C^C^C^C^C^C^C^C^C^C
* nothing, no way to get back the prompt *
15. 15 |
The Fix – give inferiors their own tty
Make SIGINT always reach GDB first, not the inferior
Put inferiors in their own terminal/session created by GDB
• no need to juggle inferior terminal's settings
That is:
• GDB creates a pseudo-terminal (pty) master/slave pair
• GDB makes the inferior run with the pty slave as its terminal
• GDB pumps output/input on the pty master end.
Since:
• inferior has own session/terminal => GDB's remains the foreground process in its terminal
• Ctrl-C SIGINT always reaches GDB first, since GDB is in the terminal's foreground
GDB is then free to handle SIGINT and interrupting the program any way it sees fit
16. 16 |
Before: GDB / Inferior share terminal
GDB
Inferior
process
GDB's
terminal
fork()/exec()
stdout
stderr
stdin
User
keyboard input
terminal output • GDB and inferior share the session / terminal
• GDB puts inferior in a new process group
• GDB saves/restores gdb/inferior terminal settings appropriately
stdout
stderr
stdin
17. 17 |
After: Inferior is given its own terminal (GDB forwards I/O)
GDB
Inferior
process
Pseudo terminal
slave
(/dev/pts/...)
Pseudo terminal
master
GDB's
terminal
fork()/exec()
stdout
stderr
stdin
stdout
stderr
stdin
write()
read()
User
keyboard input
terminal output • Inferior PTY has its own terminal settings, GDB doesn't need to
save/restore them
• GDB's tty must be in raw mode when forwarding input/output, so
that escape codes, rn, etc. are fully forwarded uninterpreted
18. 18 |
Ctrl-C in GDB's terminal always reaches GDB
GDB
Inferior
process
Pseudo terminal
slave
(/dev/pts/...)
GDB's
terminal
3. GDB
stops process
2. SIGINT
User
1. Ctrl-C
• Inferior is in foreground of its own terminal
• GDB is in foreground of its own terminal
(More about stopping inferior later)
19. 19 |
I/O forwarding (output)
1. Open an unused pty master/slave pair using standard posix_openpt / grantpt / unlockpt
2. Start inferior with pty slave as terminal (as if "(gdb) tty $PTY_SLAVE_NAME")
3. Register master end of pty in the event loop
• inferior writes to its terminal
=> event loop wakes up and handler flushes inferior output to GDB's stdout
That handles output. But what about input?
20. 20 |
I/O forwarding (input)
Input when GDB has the prompt != Input when the inferior is running
GDB already has logic to switch terminal modes at the right spots depending on
whether GDB or the inferior "have the terminal":
a) target_terminal::inferior() -> child_terminal_inferior()
1. Set stdin to raw mode
2. Register stdin-forwarder handler in the event loop
b) target_terminal::ours() -> child_terminal_ours()
1. Delete stdin-forwarder handler from the event loop and restore readline terminal settings
2. async_enable_stdin later re-installs GDB's normal stdin->readline event-loop handler, when it's time to show the
prompt
21. 21 |
How to tell GDB to spawn inferior in GDB's terminal?
Still useful to be able to tell GDB to spawn inferior in same terminal & session as GDB (like today)
• Say, you want to run + detach, and let the detached inferior continue printing to the terminal
POSIX says "/dev/tty" => terminal for the current process
=> special-casing "set inferior-tty /dev/tty" seemed like an obvious choice
Leaves "(gdb) set inferior-tty" with no arguments free for a different behavior
=> whatever is the default in the platform
22. 22 |
GDB controls when inferior output is flushed to the screen
(gdb) c &
Continuing.
(gdb) help
List of classes of commands:
aliases -- User-defined aliases of other commands.
Hello world!
breakpoints -- Making program stop at certain points.
data -- Examining data.
…
(gdb)
(gdb) c &
Continuing.
(gdb) help
List of classes of commands:
aliases -- User-defined aliases of other commands.
breakpoints -- Making program stop at certain points.
data -- Examining data.
…
(gdb)
Hello world!
Avoids inferior output being interspersed with GDB's output (background/non-stop mode)
• Inferior output is never flushed while GDB is printing something itself
• E.g., "help" while inferior is running
Before: After:
23. 23 |
TUI & inferior output flushing
With the TUI:
• could hook in tui_puts, instead of inferior outputting to screen directly
• we could even redirect inferior I/O to its own TUI window if we wanted
• no more need for Ctrl-L to refresh the screen
Debugging curses programs with the TUI would no longer work as today, though
=> ASCII escape sequence wouldn't be interpreted by the TUI's command window...
Unless... we made TUI's command window a real terminal emulator. :-)
=> TUI within the TUI within the TUI?
Still, that limitation wouldn't seem too important for non-gdb hackers?
=> "(gdb) tty /dev/tty"
=> or, run the curses inferior in its own terminal window
24. 24 |
One last problem: session leaders and SIGHUP
Session leaders have special properties:
• session leader exits => SIGHUP to all processes in the foreground process group
Recall primer:
• In Unix every process belongs to a process group which in turn belongs to a session
• Session (SID) → Process Group (PGID) → Process (PID)
More:
• The first process in the process group becomes the process group leader
• The first process in the session becomes the session leader
• Every session can have one TTY associated with it
• Only a session leader can take control of a TTY
25. 25 |
The follow-fork SIGHUP problem
pts/1 pts/2 pts/2
| | |
gdb---fork_parent---fork_child
|
session leader
If the spawned inferior is the session leader itself, and,
- it forks, and,
- if the parent (the session leader) exits,
=> the child gets a SIGHUP (and likely dies)
26. 26 |
After:
1. gdb
|
pts/1
2. gdb(1)---gdb(2)
3. gdb(1)---gdb(2)---gdb(3)
4. gdb(1)---gdb(2)---shell(3)
session leader
|
5. gdb(1)---gdb(2)---inf(3)
| | |
pts/1 pts/2 pts/2
SIGHUP: The fix – apply double-fork technique
Before:
1. gdb(1)
|
pts/1
2. gdb(1)---gdb(2)
3. gdb(1)---shell(2)
4. gdb(1)---inferior(2)
| |
pts/1 pts/1
Make GDB insert a process between
itself and the inferior it first runs, whose
only job is to be the session leader. Akin
to double-fork technique for daemonizing.
Before:
• GDB forks
• child GDB execs the shell
• shell execs the final inferior program
After:
• GDB forks
• child GDB forks again, and parent
becomes session leader process
• grandchild GDB execs the shell
• shell execs the final inferior program
27. 27 |
SIGHUP: The fix, follow-fork scenario
pts/1 pts/2 pts/2 pts/2
| | | |
gdb---gdb---fork_parent---fork_child
|
session leader
Make GDB insert an extra process between itself and
the inferior it first runs, whose only job is to be the
session leader.
Session leader process is not ptraced
Session leader process stays around until both:
1. its child exits
2. gdb closes the inferior's terminal
(It doesn't exit after #1 as there may still be grandchildren
attached to its session)
28. 28 |
The run+detach SIGHUP problem
pts/1 pts/2
| |
gdb- X -inferior
|
session leader
1. (gdb) run
2. (gdb) detach
3. GDB destroys the inferior's terminal (pts/2)
4. detached inferior is session leader and loses terminal
and so gets SIGHUP (and usually dies)
Surprising number of GDB testcases do "run" + "detach"
29. 29 |
SIGHUP: The fix, detach scenario
pts/1 pts/2 pts/2
| | |
gdb- X -gdb---inferior
|
session leader
1. (gdb) run
2. (gdb) detach
3. GDB destroys the inferior's terminal (pts/2)
4. detached inferior remains alive but loses terminal so
output goes nowhere
Detached process no longer gets SIGHUP. Output is lost,
but we could make GDB warn in this situation ("inferior
will lose terminal. Are you sure you want to detach?").
If you really want to be able to run + detach + continue
seeing inferior output, then you really want to run the
inferior in gdb's terminal, as before. You can do that with:
(gdb) tty /dev/tty
This disables the spawning of the extra session-leader
process and reactivates the gdb/inferior terminal settings
juggling.
30. 30 |
Still need to stop inferiors that block SIGINT
Now GDB always sees the Ctrl-C request
We still have the original problem, though – cannot stop inferior with SIGINT if inferior blocks SIGINT
31. 31 |
Fix: Stop inferiors with SIGSTOP
Make GDB stop the inferior with SIGSTOP instead => SIGSTOP is special and cannot be blocked
User-visible behavior change in all-stop mode -- with Ctrl-C, instead of:
Thread 1 "threads" received signal SIGINT, Interrupt.
You'll get:
Thread 1 "threads" stopped.
Which is what you already got with the "interrupt" command in non-stop mode
You can still pass a SIGINT to the program, using:(gdb) signal SIGINT
32. 32 |
Stop inferior with SIGSTOP, continued
The stopping is done from common code, via the existing target_stop
1. Ctrl-C to GDB -> sets quit flag
2. Default quit handler marks infrun event handler for Ctrl-C [new]
3. Infrun's new ctrl-c event handler:
a) if all-stop mode:
1. Synchronously stops all threads
2. If possible, keeps focus on the previously-selected thread [new, not possible before]
3. Presents stop via normal_stop
b) if non-stop mode:
1. Asynchronously stops one thread [preserving behavior], preferentially previously-selected thread [new]
Could change the non-stop behavior: Drop to prompt without stopping anything? Stop the world? Other?
33. 33 |
Upstreaming status
Last version sent to gdb-patches here (2021-06-14):
[PATCH v2 00/16] Interrupting programs that block/ignore SIGINT
https://inbox.sourceware.org/gdb-patches/20210614212410.1612666-1-pedro@palves.net/
Also in the users/palves/ctrl-c branch on sourceware:
https://sourceware.org/git/?p=binutils-gdb.git;a=shortlog;h=refs/heads/users/palves/ctrl-c
Updated series should appear on gdb-patches soon™
