Plan for Today
Between libc and the kernel
PS3 Benchmarking Results
Project Time

7 November 2013

University of Virginia cs4414

1
Rust Runtime

Recap
run::Process::new(program, argv, options)
spawn_process_os(prog, args, env, dir, in_fd, …)
fork()
Today
libc: fork()
linux kernel: fork syscall

7 November 2013

University of Virginia cs4414

2
libstd/rt/io/native/process.rs

#[cfg(unix)]
fn spawn_process_os(prog: &str, args: &[~str],
env: Option<~[(~str, ~str)]>, dir: Option<&Path>,
in_fd: c_int, out_fd: c_int, err_fd: c_int) -> SpawnProcessResult {
…
#[cfg(not(target_os = "macos"), not(windows))]
unsafe fn set_environ(envp: *c_void) {
extern {
static mut environ: *c_void;
}
environ = envp;
}
unsafe {

let pid = fork();
if pid < 0 {
fail!("failure in fork: {}", os::last_os_error());
} else if pid > 0 {
return SpawnProcessResult {pid: pid, handle: ptr::null()};
}
… // 25 lines of failure-handing code
}

7 November 2013

University of Virginia cs4414

3
Test Program
use std::libc::funcs::posix88::unistd::fork;
#[fixed_stack_segment]
fn main() {
let pid = unsafe { fork() } ;
println(fmt!("pid = %?", pid));
}

7 November 2013

University of Virginia cs4414

> rustc fork.rs
> ./fork
pid = 0i32
pid = 15039i32
$ ./fork
pid = 15043i32
pid = 0i32
4
use std::libc::funcs::posix88::unistd::fork;

> rustc -O -S fork.rs
> wc -l fork.S
72 fork.S

#[fixed_stack_segment]
fn main() { unsafe { fork() } ; }
.section
__TEXT,__text,regular,pure_instructio
ns
.align 4, 0x90
__ZN4main18h8b6694fe33a5855ag4
v0.0E:
.cfi_startproc
leaq -2097152(%rsp), %r11
cmpq %gs:816, %r11
ja LBB0_2
movabsq $2097152, %r10
movabsq $0, %r11
callq ___morestack
ret
LBB0_2:
pushq %rbp
Ltmp2:
.cfi_def_cfa_offset 16
Ltmp3:
.cfi_offset %rbp, -16
movq %rsp, %rbp

7 November 2013

Ltmp4:
.cfi_def_cfa_register %rbp
popq %rbp

jmp

_fork

.cfi_endproc
.globl _main
.align 4, 0x90
_main:
.cfi_startproc
cmpq %gs:816, %rsp
ja LBB1_2
movabsq $8, %r10
movabsq $0, %r11
callq ___morestack
ret
LBB1_2:
pushq %rbp
Ltmp7:
.cfi_def_cfa_offset 16
Ltmp8:
.cfi_offset %rbp, -16
movq %rsp, %rbp

University of Virginia cs4414

Ltmp9:
.cfi_def_cfa_register %rbp
movq %rsi, %rax
movq %rdi, %rcx
movq %rsi, %rax
movq %rdi, %rcx
leaq __ZN4main18h8b6694fe33a5855ag4v0.0E(%rip), %rsi
xorl %edi, %edi
movq %rcx, %rdx
movq %rax, %rcx
popq %rbp
jmp __ZN8unstable4lang5start17hf72eb8b3c3a0a9ac4v0.8E
.cfi_endproc

.section
__DATA,__data
.globl __rust_crate_map_toplevel
.align 4
__rust_crate_map_toplevel:
.long 1
.space 4
.quad __rust_mod_map
.quad __rust_crate_map_std_0.8_6c65cf4b443341b1
.quad 0
.zerofill __DATA,__bss,__rust_mod_map,16,3
.section
__TEXT,__const
.globl _rust_abi_version
.align 3
_rust_abi_version:
.quad 1

.subsections_via_symbols

5
Could actual call to kernel
fork be a regular call?

7 November 2013

University of Virginia cs4414

6
Rust Runtime

Entering the Kernel
run::Process::new(program, argv, options)
spawn_process_os(prog, args, env, dir, in_fd, …)
fork()
libc: fork()
linux kernel: fork syscall

7 November 2013

University of Virginia cs4414

7
Supervisor Mode
Kernel code needs (or at least uses)
special privileges!
What would happen if user-level code could
just jump into kernel code?

7 November 2013

University of Virginia cs4414

8
Entering the Kernel
User-Level Code
…
movl $SYS_fork,%eax
int
$0x80
int instruction generates an interrupt

7 November 2013

University of Virginia cs4414

9
Traditional PC Design
Programmable
Interrupt
Controller

CPU

(PIC)

Keyboard
7 November 2013

Interval Timer
University of Virginia cs4414

10
Page 2213 of Intel x86 Manual:
http://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-manual-325462.pdf

Modern x86 Design:
“APIC” = “Advanced PIC”

7 November 2013

University of Virginia cs4414

11
Page 2213 of Intel x86 Manual:
http://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-manual-325462.pdf

What should generate a
“Local Interrupt”?

7 November 2013

What should generate an
“External Interrupt”?

University of Virginia cs4414

12
7 November 2013

University of Virginia cs4414

13
7 November 2013

University of Virginia cs4414

14
7 November 2013

University of Virginia cs4414

15
…
movl $SYS_fork,%eax
int
$0x80

Programmable
Interrupt
Controller

(PIC)

7 November 2013

University of Virginia cs4414

Handling
Syscall
Interrupts

CPU
16
7 November 2013

University of Virginia cs4414

17
7 November 2013

University of Virginia cs4414

18
Intel manual, p. 146:

7 November 2013

University of Virginia cs4414

19
7 November 2013

University of Virginia cs4414

20
Rust Runtime

Running in Supervisor Mode
run::Process::new(program, argv, options)
spawn_process_os(prog, args, env, dir, in_fd, …)
fork()
int 0x80

libc: fork()
jumps into kernel code
sets supervisor mode

linux kernel: fork syscall
7 November 2013

University of Virginia cs4414

21
PS3 Bakeoff Winners
13.2, 5701.3

Average Response Time (milliseconds)

6,000

5,000
9.7, 3908.1

4,000

217.1, 3902.7

reference zhtta

3,000

2,000
44.0, 989.7
39.8, 960.8

1,000

225.2, 531.3
5.5, 0.6

0

0

50

100

150

200

Total Duration (seconds)
5 November 2013

University of Virginia cs4414

23
8pm Friday
Rouss/Robertson Hall Room 120
7 November 2013

University of Virginia cs4414

24
Decoy Project!

7 November 2013

University of Virginia cs4414

25
7 November 2013

University of Virginia cs4414

26
13.2, 5701.3

6,000

Average Response Time (milliseconds)

Kiet, Mark, Tanmoy
5,000
9.7, 3908.1

4,000

217.1, 3902.7

reference zhtta

3,000

2,000
44.0, 989.7
39.8, 960.8

Hong, Jireh, Marshall

Chris, Tong, Yicheng

1,000

225.2, 531.3
5.5, 0.6

0

0

Harriet, Kevin, Zeming
50

100

150

200

Total Duration (seconds)
5 November 2013

University of Virginia cs4414

27
Charge
Find a team and project!

Decoy projects are only allowed in security classes.
Sneaking around my house is no longer permitted.

7 November 2013

University of Virginia cs4414

28

Crossing into Kernel Space

  • 2.
    Plan for Today Betweenlibc and the kernel PS3 Benchmarking Results Project Time 7 November 2013 University of Virginia cs4414 1
  • 3.
    Rust Runtime Recap run::Process::new(program, argv,options) spawn_process_os(prog, args, env, dir, in_fd, …) fork() Today libc: fork() linux kernel: fork syscall 7 November 2013 University of Virginia cs4414 2
  • 4.
    libstd/rt/io/native/process.rs #[cfg(unix)] fn spawn_process_os(prog: &str,args: &[~str], env: Option<~[(~str, ~str)]>, dir: Option<&Path>, in_fd: c_int, out_fd: c_int, err_fd: c_int) -> SpawnProcessResult { … #[cfg(not(target_os = "macos"), not(windows))] unsafe fn set_environ(envp: *c_void) { extern { static mut environ: *c_void; } environ = envp; } unsafe { let pid = fork(); if pid < 0 { fail!("failure in fork: {}", os::last_os_error()); } else if pid > 0 { return SpawnProcessResult {pid: pid, handle: ptr::null()}; } … // 25 lines of failure-handing code } 7 November 2013 University of Virginia cs4414 3
  • 5.
    Test Program use std::libc::funcs::posix88::unistd::fork; #[fixed_stack_segment] fnmain() { let pid = unsafe { fork() } ; println(fmt!("pid = %?", pid)); } 7 November 2013 University of Virginia cs4414 > rustc fork.rs > ./fork pid = 0i32 pid = 15039i32 $ ./fork pid = 15043i32 pid = 0i32 4
  • 6.
    use std::libc::funcs::posix88::unistd::fork; > rustc-O -S fork.rs > wc -l fork.S 72 fork.S #[fixed_stack_segment] fn main() { unsafe { fork() } ; } .section __TEXT,__text,regular,pure_instructio ns .align 4, 0x90 __ZN4main18h8b6694fe33a5855ag4 v0.0E: .cfi_startproc leaq -2097152(%rsp), %r11 cmpq %gs:816, %r11 ja LBB0_2 movabsq $2097152, %r10 movabsq $0, %r11 callq ___morestack ret LBB0_2: pushq %rbp Ltmp2: .cfi_def_cfa_offset 16 Ltmp3: .cfi_offset %rbp, -16 movq %rsp, %rbp 7 November 2013 Ltmp4: .cfi_def_cfa_register %rbp popq %rbp jmp _fork .cfi_endproc .globl _main .align 4, 0x90 _main: .cfi_startproc cmpq %gs:816, %rsp ja LBB1_2 movabsq $8, %r10 movabsq $0, %r11 callq ___morestack ret LBB1_2: pushq %rbp Ltmp7: .cfi_def_cfa_offset 16 Ltmp8: .cfi_offset %rbp, -16 movq %rsp, %rbp University of Virginia cs4414 Ltmp9: .cfi_def_cfa_register %rbp movq %rsi, %rax movq %rdi, %rcx movq %rsi, %rax movq %rdi, %rcx leaq __ZN4main18h8b6694fe33a5855ag4v0.0E(%rip), %rsi xorl %edi, %edi movq %rcx, %rdx movq %rax, %rcx popq %rbp jmp __ZN8unstable4lang5start17hf72eb8b3c3a0a9ac4v0.8E .cfi_endproc .section __DATA,__data .globl __rust_crate_map_toplevel .align 4 __rust_crate_map_toplevel: .long 1 .space 4 .quad __rust_mod_map .quad __rust_crate_map_std_0.8_6c65cf4b443341b1 .quad 0 .zerofill __DATA,__bss,__rust_mod_map,16,3 .section __TEXT,__const .globl _rust_abi_version .align 3 _rust_abi_version: .quad 1 .subsections_via_symbols 5
  • 7.
    Could actual callto kernel fork be a regular call? 7 November 2013 University of Virginia cs4414 6
  • 8.
    Rust Runtime Entering theKernel run::Process::new(program, argv, options) spawn_process_os(prog, args, env, dir, in_fd, …) fork() libc: fork() linux kernel: fork syscall 7 November 2013 University of Virginia cs4414 7
  • 9.
    Supervisor Mode Kernel codeneeds (or at least uses) special privileges! What would happen if user-level code could just jump into kernel code? 7 November 2013 University of Virginia cs4414 8
  • 10.
    Entering the Kernel User-LevelCode … movl $SYS_fork,%eax int $0x80 int instruction generates an interrupt 7 November 2013 University of Virginia cs4414 9
  • 11.
    Traditional PC Design Programmable Interrupt Controller CPU (PIC) Keyboard 7November 2013 Interval Timer University of Virginia cs4414 10
  • 12.
    Page 2213 ofIntel x86 Manual: http://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-manual-325462.pdf Modern x86 Design: “APIC” = “Advanced PIC” 7 November 2013 University of Virginia cs4414 11
  • 13.
    Page 2213 ofIntel x86 Manual: http://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-manual-325462.pdf What should generate a “Local Interrupt”? 7 November 2013 What should generate an “External Interrupt”? University of Virginia cs4414 12
  • 14.
    7 November 2013 Universityof Virginia cs4414 13
  • 15.
    7 November 2013 Universityof Virginia cs4414 14
  • 16.
    7 November 2013 Universityof Virginia cs4414 15
  • 17.
    … movl $SYS_fork,%eax int $0x80 Programmable Interrupt Controller (PIC) 7 November2013 University of Virginia cs4414 Handling Syscall Interrupts CPU 16
  • 18.
    7 November 2013 Universityof Virginia cs4414 17
  • 19.
    7 November 2013 Universityof Virginia cs4414 18
  • 20.
    Intel manual, p.146: 7 November 2013 University of Virginia cs4414 19
  • 21.
    7 November 2013 Universityof Virginia cs4414 20
  • 22.
    Rust Runtime Running inSupervisor Mode run::Process::new(program, argv, options) spawn_process_os(prog, args, env, dir, in_fd, …) fork() int 0x80 libc: fork() jumps into kernel code sets supervisor mode linux kernel: fork syscall 7 November 2013 University of Virginia cs4414 21
  • 23.
  • 24.
    13.2, 5701.3 Average ResponseTime (milliseconds) 6,000 5,000 9.7, 3908.1 4,000 217.1, 3902.7 reference zhtta 3,000 2,000 44.0, 989.7 39.8, 960.8 1,000 225.2, 531.3 5.5, 0.6 0 0 50 100 150 200 Total Duration (seconds) 5 November 2013 University of Virginia cs4414 23
  • 25.
    8pm Friday Rouss/Robertson HallRoom 120 7 November 2013 University of Virginia cs4414 24
  • 26.
    Decoy Project! 7 November2013 University of Virginia cs4414 25
  • 27.
    7 November 2013 Universityof Virginia cs4414 26
  • 28.
    13.2, 5701.3 6,000 Average ResponseTime (milliseconds) Kiet, Mark, Tanmoy 5,000 9.7, 3908.1 4,000 217.1, 3902.7 reference zhtta 3,000 2,000 44.0, 989.7 39.8, 960.8 Hong, Jireh, Marshall Chris, Tong, Yicheng 1,000 225.2, 531.3 5.5, 0.6 0 0 Harriet, Kevin, Zeming 50 100 150 200 Total Duration (seconds) 5 November 2013 University of Virginia cs4414 27
  • 29.
    Charge Find a teamand project! Decoy projects are only allowed in security classes. Sneaking around my house is no longer permitted. 7 November 2013 University of Virginia cs4414 28