BSidesDelhi 2018: Finding Memory Bugs with the Address Sanitizer
•
0 likes•558 views
Presenter: Siddharth Muralee Abstract: This talk explains why Sanitizers are important and how they can be used to help increase the efficiency of mainstream fuzzing. I will be delving into detail about how they work with relevant information about instrumentation and mapping. We will be looking at how the kernel implementation of the Address Sanitizer with a focus on the implementation in NetBSD. In the end of the talk I will be discussing some small examples of userland and kernel land bugs we can detect with the Address Sanitizer and I will use the opportunity to compare the Address Sanitizer with other tools out there.
More Related Content
What's hot
What's hot (20)
Similar to BSidesDelhi 2018: Finding Memory Bugs with the Address Sanitizer
Similar to BSidesDelhi 2018: Finding Memory Bugs with the Address Sanitizer (20)
Recently uploaded
Recently uploaded (20)
BSidesDelhi 2018: Finding Memory Bugs with the Address Sanitizer
- 1. FINDING MEMORY BUGS WITH THE BSIDES DELHI 2018
- 2. FINDING MEMORY BUGS WITH THE ADDRESS SANITIZER BSIDES DELHI $ WHOAMI ▸Siddharth Muralee (R3x) ▸Third year CSE @ Amrita Vishwa Vidyapeetham ▸CTF player - Team bi0s ▸Core organising team @ InCTF and InCTFj ▸Reverse Engineering/Exploit Development ▸GSoC ’18 with NetBSD
- 3. FINDING MEMORY BUGS WITH THE ADDRESS SANITIZER BSIDES DELHI AGENDA ▸What are Sanitizers ? ▸What is an Address Sanitizer(ASan)? ▸How does the Address Sanitizer work? ▸The Kernel Address Sanitizer(KASan) ▸Bug report examples ▸Address Sanitizer vs other tools
- 4. FINDING MEMORY BUGS WITH THE ADDRESS SANITIZER BSIDES DELHI WHAT IS A SANITIZER? ▸ Programming tool to detect computer program errors. ▸ Compiler Instrumented ▸ The various Sanitizers are : ▸ Address Sanitizer ▸ Memory Sanitizer ▸ Thread Sanitizer ▸ Undefined Behaviour Sanitizer
- 5. FINDING MEMORY BUGS WITH THE ADDRESS SANITIZER BSIDES DELHI ADDRESS SANITIZER (ASAN) ▸Open source tool developed by Google. ▸Detects memory corruption bugs - Overflows and UAFs ▸Implemented in Clang, GCC and Xcode. ▸Easy to use just compile with -fsanitize=address. ▸Run-time library ▸A pretty Amazing Fuzzer aide
- 6. FINDING MEMORY BUGS WITH THE ADDRESS SANITIZER BSIDES DELHI ADDRESS SANITIZER - PREREQUISITE KNOWLEDGE ▸Shadow Memory ▸Storing metadata corresponding to each piece of application data. ▸Each Address Mapped to a Shadow memory offset ▸Compiler Instrumentation ▸Compiler adds instructions during compilation which allows some information
- 7. FINDING MEMORY BUGS WITH THE ADDRESS SANITIZER BSIDES DELHI ADDRESS SANITIZER WORKING ▸Each Memory access is modified using compiler instrumentation. Prevents unintended memory read/write.
- 8. FINDING MEMORY BUGS WITH THE ADDRESS SANITIZER BSIDES DELHI ADDRESS SANITIZER WORKING ▸Run time library replaces malloc and free. ▸Memory around chunks (memory blocks allocated using malloc) are poisoned. Prevents Heap overflow bugs. ▸Freed memory placed in quarantine list . Prevents Use- after-free bugs from being missed. ▸Parts of the stack are also poisoned. This is to avoid Stack overflow bugs
- 9. FINDING MEMORY BUGS WITH THE ADDRESS SANITIZER BSIDES DELHI SHADOW MEMORY ▸ ASan maps 1 byte of Application data to 1 bit of shadow memory. ▸ Shadow bit ▸ 1 - Unaddressable ▸ 0 - Addressable ▸ Total Shadow region size = Application size / 8 ▸ Each byte needs to be converted to the corresponding shadow memory address.
- 10. FINDING MEMORY BUGS WITH THE ADDRESS SANITIZER BSIDES DELHI SHADOW MEMORY ▸ Shadow Address = (Addr >> 3) + Shadow offset. ▸ Shadow offset = 0x20000000 Address Addr >> 3 Shadow Address 0xffffffff 0x1fffffff 0x3fffffff 0x00000000 0x00000000 0x20000000
- 11. DEMO OFF BY ONE BUG
- 12. FINDING MEMORY BUGS WITH THE ADDRESS SANITIZER BSIDES DELHI KERNEL ADDRESS SANITIZER (KASAN) ▸Gcc and Clang come with a build in option -fsanitize=kernel- address ▸You can build the entire kernel with the Address Sanitizer using the kernel config file. ▸Provides an API which the kernel has to implement. ▸Is implemented as a feature in the Linux Kernel, OS X and now NetBSD !!!
- 13. FINDING MEMORY BUGS WITH THE ADDRESS SANITIZER BSIDES DELHI IMPLEMENTING KASAN ▸Allocation and population of the Shadow buffer during boot ▸Need to write interceptors for kernel allocator functions to update shadow memory. (Multiple allocators) ▸Kernel VA space needs to be properly managed. ▸Implement Quarantine lists in page allocation ▸Bug Reporting infrastructure needs to be written
- 14. KASAN WORKING FINDING MEMORY BUGS WITH THE ADDRESS SANITIZER BSIDES ptr - points to 0xdeadbeef shadow_ptr - shadow address kmem_to_shadow - converts kernel address to shadow offset.
- 15. NETBSD KASAN -VA SPACE MAPPING FINDING MEMORY BUGS WITH THE ADDRESS SANITIZER BSIDES
- 16. FINDING MEMORY BUGS WITH THE ADDRESS SANITIZER BSIDES DELHI Fix buffer overflow, detected by kASan. ifconfig gif0 create ifconfig gif0 up [ 50.682919] kASan: Unauthorized Access In 0xffffffff80f22655: Addr 0xffffffff81b997a0 [8 bytes, read] [ 50.682919] #0 0xffffffff8021ce6a in kasan_memcpy <netbsd> [ 50.692999] #1 0xffffffff80f22655 in m_copyback_internal <netbsd> [ 50.692999] #2 0xffffffff80f22e81 in m_copyback <netbsd> [ 50.692999] #3 0xffffffff8103109a in rt_msg1 <netbsd> [ 50.692999] #4 0xffffffff8159109a in compat_70_rt_newaddrmsg1 <n [ 50.692999] #5 0xffffffff81031b0f in rt_newaddrmsg <netbsd> [ 50.692999] #6 0xffffffff8102c35e in rt_ifa_addlocal <netbsd> [ 50.692999] #7 0xffffffff80a5287c in in6_update_ifa1 <netbsd> [ 50.692999] #8 0xffffffff80a54149 in in6_update_ifa <netbsd> [ 50.692999] #9 0xffffffff80a59176 in in6_ifattach <netbsd> [ 50.692999] #10 0xffffffff80a56dd4 in in6_if_up <netbsd> [ 50.692999] #11 0xffffffff80fc5cb8 in if_up_locked <netbsd> [ 50.703622] #12 0xffffffff80fcc4c1 in ifioctl_common <netbsd> [ 50.703622] #13 0xffffffff80fde694 in gif_ioctl <netbsd> [ 50.703622] #14 0xffffffff80fcdb1f in doifioctl <netbsd>
- 17. FINDING MEMORY BUGS WITH THE ADDRESS SANITIZER BSIDES DELHI ASAN VS OTHER TOOLS ASan Memcheck(Valgrind) Mudflap Technique Compiler Instrumentation Dynamic Binary Instrumentation Compiler Instrumentation Slowdown 2x 20x 2x - 20x Types of Bugs Heap and Stack overflows, UAF, UAR UMR, Heap overflows and UAFs Heap overflows and UAFs UMR - Uninitialised Memory Reads UAF - Use After Free UAR - Use After Return
- 18. QUESTION S?
Editor's Notes
- Third year CSE , Amrita , Kollam. CTF player with team bi0s mainly focusing on Reverse engineering and exploitation. I am also a core member of the organising team for InCTF and InCTF junior , CTFs bi0s conducts for college and school students in India. I did my Google Summer of Code this year with the Amazing NetBSD Foundation.
- This talk aims at introducing you to the sanitizer family and in specific the Address Sanitizer. We will be looking at how the address Sanitizer works. We will also be looking into the kernel address sanitizer and how its implemented. In the end we will compare the address sanitizer with some similar tools.
- The Sanitizer is a programming tool used to detect program errors. The technology behind sanitizers being compiler instrumentation. There are several sanitizers created for various purposes like the Address, Memory , Thread and the Undefined Behavior Sanitizer.
- Now let’s look at the Address Sanitizer - It is a open source tool developed by google which is mainly used to detect memory corruption bugs - such as Stack and Heap overflows, Use after Frees etc The sanitizer comes as a compiler option in Clang, Gcc and Xcode. In the user space its linked with the executable as a run time library. The features that the address sanitizer has makes it a pretty cool fuzzing aide that is widely used even in the OSS fuzz tool which google has.
- Okay before look deeper into the address Sanitizer there two concepts that are very vital. One of them is the concept of Shadow memory. Shadow memory is a additional memory that is used by the address sanitizer to keep track of what data is legal and what data is illegal. Here we store metadata corresponding to each piece of application data. This metadata is updated with each instruction and is used to determine whether a instruction is legal or not. Compiler instrumentation - This is the underlying technology behind the address sanitizer - The Compiler adds certain instruction during the compilation process which allows the program to update and check the shadow memory(metadata) at every necessary point.
- Now let’s look at how the address sanitizer works. During compilation each memory access (that is a memory read or write) is modified to have a check whether the address is poisoned or not. An address is said to be poisoned when the program is not supposed to be access it. If a poisoned memory is accessed then we report an error.
- When the Run time library for ASan is linked with the executable it replaces the malloc and free function provided by the libc with its own. This allows it to allocate chunks with the memory around it poisoned like a redzone. This is done to prevent heap overflow bugs since there is now no possibility to overflow and write to another chunk. Also the chunks that are freed are now put into a quarantine list. This means that a chunk that has been recently used would not be used for a long time unless there is a shortage of memory. This is done to make sure that Use after free bugs are seen. If the freed chunk is again malloc’d then we miss it.
- That’s the main features that the address sanitizer provides. Now let’s take a better look at the Shadow memory to find out how the mapping works. The address sanitizer maps 1 byte of the application memory to 1 bit of shadow memory. So eight bytes of application memory is taken care by 1 byte of metadata. This is indeed a significant amount of memory - that is we require 1/8th of the total application memory to manage the shadow region. Also during each check/updation the bytes need to converted to the corresponding shadow memory address. This means that the shadow memory conversion process should be pretty simple otherwise we waste a lot of time there.
- So the equation to get the shadow address is to take the addr and divide it by 8 (>> 3) and then add it to a predetermined address called the shadow offset. If you look at the diagram you can see the address 0xffff is mapped to - 0x3ffff 0x00 - 0x2000000 This determines the bounds of the shadow memory. The shadow region is marked in light blue. The region marked in red is the shadow address range of the shadow memory region - this region is not supposed to be used. Hence every address in the entire range has been mapped to a shadow region and this has been done in a very simple process.
- The kernel address sanitizer is the result of the build in option fsanitize = kernel address. The entire kernel can be built with the address sanitizer using the kernel config file. Sadly, implementing kernel address sanitizer is not very easy - the compiler can only provide a set of API’s which the kernel has to implement. The kernel address Sanitizer has been implemented in the linux kernel, Mac OS X and Now its also there in NetBSD.
- There are a lot of things that the OS must implement to get the API working perfectly. Since running the address sanitizer as a shared library isn’t viable in such a situation we need to modify the OS kernel itself. Several things that need to taken care are. Making sure that the Shadow buffer is allocated and populated during the boot process. This is vital since the allocator functions are also used during the boot process. The Memory allocators like slab, slob in linux need to be modified so that they update the shadow memory during allocation and freeing of pages The kernel Virtual Address Space also needs to be managed so that no other allocations happen in the memory region corresponding to the shadow memory. We will take a look at the kernel VA space with resepect to NetBSD in a short while. Also we need to implement the quarantine list for the same reason as earlier. In the case of the kernel the allocator allocates chunks that are the in essence pages. These pages have to undergo the same procedure. We also have to write the bug reporting infrastructure. Since the bugs are supposed to be printed out by the kernel - either in the kernel log or as a kernel crash.
- Okay let’s consider a example in the kernel space and see what happens. Assume the kernel code has a pointer in the address 0xdeadbeef. The address is dereferenced at some point of time and we are writing something into it. Since the kernel is compiled with KASAN the compiler now has added certain functions __asan_store(). asan_store checks if the address is poisoned - for this it first passes the address to the kmem to shadow function which converts the address to the shadow address as we saw earlier and then the check_memory_poisoned function checks whether the memory has been poisoned or not. If its poisoned then we call kasan_report error - report the bug and exit else we continue execution. Similar functions exist for kmalloc and other implementations.
- This is the virtual address space of the NetBSD kernel for the amd64 architecture. Here you can see that there are several sections in the kernel and there are pointers containing the address of that section. The shadow buffer takes up around 128 Tb of space when it comes to the 64 bit kernel. Hence we placed it in a hole that’s not been used for any other purposes. Also we have to deal with some regions in a certain way and the others slightly different. The kernel memory, the user land, page tables and the Module Map are areas we need to take care of.
- This is a example bug report of a bug we found in the NetBSD kernel with Kasan. As you can see the output is slightly different and we are printing it in the kernel log. Kasan_memcpy instead of memcpy.
- Compared to Mudflap