Running a container app in the container is easy, attaching a custom app to a running container is a bit trickier. But, what if I wanted to run any arbitrary binary in any arbitrary running container? Common wisdom says it's impossible. Is it ? This talk dives into containers internals, just above the kernel surface and demonstrates that this is, indeed possible. With a bit of C magic and ptrace.

1. Container Torture
Run any binary, in any container
Jean-Tiare Le Bigot
System Developer

2. Sailabove
2
R&D project
Goal: understand the Challenges
what are the at-scale issues?
what are the multi-tenancy challenges?
how does it work?

3. Inspection / Introspection
3
How do you inspect /proc?
How do you inspect files?
How do you inject files?

4. Inspection / Introspection
4
You don’t trust the container
You can’t make asumptions
You can’t alter the container

5. Let’s fix it

6. Goal / Scope
Inspect any container FS
it includes you, /proc
with standard, unpatched tools
sftp access for all

7. The road ahead
What is a container?
How (not) to enter a container?
Run a static host binary in a container
Run a dyn host binary in a container
Run a TUI host binary in a container
Stretch goal: sftp (and we'll stretch)

8. What is a container

9. Anatomy of a container
A container is a group of processes
Sharing the same namespaces.

10. Introducing namespaces
Introduced with Linux 2.4.19 (2002)
Allow processes to have dedicated
Mountpoints
Network stack
PIDs
…

11. Anatomy of a container « / »
DONE
umount()
pivot_root()
mount()
clone()  Host
/bin
/container
/dev
/etc
/usr
 Container
/bin
/container
/dev
/etc
/usr

12. Anatomy of a container « / »
DONE
umount()
pivot_root()
mount()
clone()  Host
/bin
/container
/dev
/etc
/usr
 Container
/bin
/container
/bin
/dev
/etc
/host
/usr
/dev
/etc
/usr

13. Anatomy of a container « / »
DONE
umount()
pivot_root()
mount()
clone()  Host
/bin
/container
/dev
/etc
/usr
 Container
/bin
/dev
/etc
/host
/bin
/container
/dev
/etc
/usr
/usr

14. Anatomy of a container « / »
DONE
umount()
pivot_root()
mount()
clone()  Host
/bin
/container
/dev
/etc
/usr
 Container
/bin
/dev
/etc
/host
/usr

15. Anatomy of a container « / »
DONE
umount()
pivot_root()
mount()
clone()
Independent FS

16. How to enter a container?

17. Introducing setns

18. Introducing setns
Introduced in Linux 3.0
Join an existing Namespace
Building block for: nsenter, docker exec
int fd = open(“/proc/<target pid>/ns/<ns name>”)
setns(fd, <ns type>)

19. setns pattern #1 – custom binary
Host Container
setns()

20. setns pattern #2 – container binary
Host Container
setns()

21. What about host binaries
Only the 2 previous patterns
Independant FS
This is impossible
This is were the fun begins

22. What about host binaries
Easy  patch
Hard  automatic code rewrite
Ambitious  at runtime

23. We’ll need some magic
ptrace, the grey

24. Introducing ptrace
Introduced in Linux 1.0 / SVr4 (1983)
Building block for: strace/gdb/…
Features:
Trace processes
Mess with processes (easy)
Interract with processes (trickier)

25. What we’ll do
Inject setns() syscall
At the right time
Assume Linux >= 2.14
Assume X86_64

26. Run static binaries

27. Anatomy of a syscall
process kernel tracer

28. Anatomy of a syscall
process kernel tracer
handle_call()
handle_ret()

29. Anatomy of a syscall
process kernel tracer
handle_call()
handle_ret()
What if we…
 Alter orig_rax?
 Alter rip?
 Alter rax?
inject_syscall()

30. Injecting setns
1. Open each Namespace fd
2. Leak them in the target process
3. Start ptrace-ing
4. Wait until exec
5. Inject each setns calls
6. Clean your room

31. Show me the code!

32. // Wait until exec syscall is done
do {
wait_for_syscall(child, NULL);
wait_for_syscall(child, &regs);
} while(regs.orig_rax != SYS_execve);
Show me the code!
OVH group/Sailabove: booth 17 sailabove.com

33. // Enter namespaces + cleanup
for(ns=namespaces; ns->proc_name; ns++)
{
inject_setns(child, ns->fd, 0);
inject_close(child, ns->fd);
}
Show me the code!
OVH group/Sailabove: booth 17 sailabove.com

34. Thank you!
Jean-Tiare Le Bigot
@oyadutaf
Jean-tiare.le-bigot@corp.ovh.com

35. Agenda
Section Three
Title
Title
Title
Section Four
Title
Title
Title
Section One
Title
Title
Title
Section Two
Title
Title
Title

36. Section Name
Subtitle

39. RGB
0–149–203
RGB
0–167–224
RGB
20–175–229
RGB
41–183–234
RGB
51–193–241
RGB
66–198–243
RGB
117–209–243
RGB
209–173–43
RGB
238–201–49
RGB
244–214–98
RGB
203–86-86
RGB
240-100-101
RGB
244–142–142
RGB
57–77–85
RGB
216–222–223
39

40. 40
Photo Caption
Insert photo, then send to back in order to show the
rounded border. DELETE this text box

41. —Author Name
“Here’s the space
for an awesome quote.”
41