The document discusses tips for malloc and free in C, including making your own malloc library for troubleshooting. It covers system calls like brk/sbrk and mmap/munmap that are used to allocate memory in user space. It also provides tips for the glibc malloc implementation, such as functions like mallopt, malloc_stats, and malloc_usable_size. Finally, it discusses two methods for hooking and replacing malloc - using LD_PRELOAD and dlsym, or the __malloc_hook mechanism.
Virtual File System in Linux Kernel
Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
Reverse Mapping (rmap) in Linux KernelAdrian Huang
Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
On July 6, 2021, MariaDB 10.6 became generally available (production ready). This presentation focuses on the most important aspects of it as well as the influence it has. Improvements to InnoDB, SYS Schema Adoption, and deprecated variables and engines are all part of this presentation.
Kernel Recipes 2017 - Understanding the Linux kernel via ftrace - Steven RostedtAnne Nicolas
Ftrace is the official tracer of the Linux kernel. It has been apart of Linux since 2.6.31, and has grown tremendously ever since. Ftrace’s name comes from its most powerful feature: function tracing. But the ftrace infrastructure is much more than that. It also encompasses the trace events that are used by perf, as well as kprobes that can dynamically add trace events that the user defines.
This talk will focus on learning how the kernel works by using the ftrace infrastructure. It will show how to see what happens within the kernel during a system call; learn how interrupts work; see how ones processes are being scheduled, and more. A quick introduction to some tools like trace-cmd and KernelShark will also be demonstrated.
Steven Rostedt, VMware
Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
Virtual File System in Linux Kernel
Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
Reverse Mapping (rmap) in Linux KernelAdrian Huang
Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
On July 6, 2021, MariaDB 10.6 became generally available (production ready). This presentation focuses on the most important aspects of it as well as the influence it has. Improvements to InnoDB, SYS Schema Adoption, and deprecated variables and engines are all part of this presentation.
Kernel Recipes 2017 - Understanding the Linux kernel via ftrace - Steven RostedtAnne Nicolas
Ftrace is the official tracer of the Linux kernel. It has been apart of Linux since 2.6.31, and has grown tremendously ever since. Ftrace’s name comes from its most powerful feature: function tracing. But the ftrace infrastructure is much more than that. It also encompasses the trace events that are used by perf, as well as kprobes that can dynamically add trace events that the user defines.
This talk will focus on learning how the kernel works by using the ftrace infrastructure. It will show how to see what happens within the kernel during a system call; learn how interrupts work; see how ones processes are being scheduled, and more. A quick introduction to some tools like trace-cmd and KernelShark will also be demonstrated.
Steven Rostedt, VMware
Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
Talk by Brendan Gregg for USENIX LISA 2019: Linux Systems Performance. Abstract: "
Systems performance is an effective discipline for performance analysis and tuning, and can help you find performance wins for your applications and the kernel. However, most of us are not performance or kernel engineers, and have limited time to study this topic. This talk summarizes the topic for everyone, touring six important areas of Linux systems performance: observability tools, methodologies, benchmarking, profiling, tracing, and tuning. Included are recipes for Linux performance analysis and tuning (using vmstat, mpstat, iostat, etc), overviews of complex areas including profiling (perf_events) and tracing (Ftrace, bcc/BPF, and bpftrace/BPF), and much advice about what is and isn't important to learn. This talk is aimed at everyone: developers, operations, sysadmins, etc, and in any environment running Linux, bare metal or the cloud."
Windows Kernel Exploitation : This Time Font hunt you down in 4 bytesPeter Hlavaty
In our recent work we targeted also win32k, what seems to be fruit giving target. @promised_lu made our own TTF-fuzzer which comes with bunch of results in form of gigabytes of crashes and various bugs. Fortunately windows make great work and in February most of our bugs was dead - patched, but not all of them…
Whats left were looking as seemingly unexploitable kernel bugs with ridiculous conditions. We decided to check it out, and finally combine it with our user mode bug & emet bypass. Through IE & flash we break down system and pointed out at weak points in defensive mechanism.
In this talk we will present our research dedicated for pwn2own event this year. We will describe kernel part of exploit in detail*, including bug description, resulting memory corruption conditions & caveats up to final pwn via one of our TTF bugs.
Throughout the talk we will describe how to break various exploit mitigations in windows kernel and why it is possible. We will introduce novel kernel exploitation techniques breaking all what stands { KASLR, SMEP, even imaginary SMAP or CFG } and bring you SYSTEM exec (from kernel driver to system calc).
* unfortunately bug was not fixed at the time of talk, so we do not exposed details about TTF vulnerability, and we skipped directly to some challenges during exploitation, and demonstrate how OS design can overpower introduced exploit mitigations.
Linux Kernel Booting Process (2) - For NLKBshimosawa
Describes the bootstrapping part in Linux, and related architectural mechanisms and technologies.
This is the part two of the slides, and the succeeding slides may contain the errata for this slide.
This tutorial covers all parallel replication implementation in MariaDB 10.0 and 10.1 and MySQL 5.6, 5.7 and 8.0 (including how it works in Group Replication).
MySQL and MariaDB have different types of parallel replication. In this tutorial, we present the different implementations that allow us to understand their limitations and tuning parameters. We cover how to make parallel replication faster and what to avoid for maximizing its benefits. We also present tests from Booking.com workloads.
Some of the subjects that are covered are group commit and optimistic parallel replication in MariaDB, the parallelism interval of MySQL and its Write Set optimization, and the ?slowing down the master to speed up the slave? optimization.
After this tutorial, you will know everything you need to implement and tune parallel replication in your environment. But more importantly, we will show how you can test parallel replication benefit in a non-disruptive way before deployment.
Kernel Recipes 2019 - No NMI? No Problem! – Implementing Arm64 Pseudo-NMIAnne Nicolas
As the name would suggest, a Non-Maskable Interrupt (NMI) is an interrupt-like feature that is unaffected by the disabling of classic interrupts. In Linux, NMIs are involved in some features such as performance event monitoring, hard-lockup detector, on demand state dumping, etc… Their potential to fire when least expected can fill the most seasoned kernel hackers with dread.
AArch64 (aka arm64 in the Linux tree) does not provide architected NMIs, a consequence being that features benefiting from NMIs see their use limited on AArch64. However, the Arm Generic Interrupt Controller (GIC) supports interrupt prioritization and masking, which, among other things, provides a way to control whether or not a set of interrupts can be signaled to a CPU.
This talk will cover how, using the GIC interrupt priorities, we provide a way to configure some interrupts to behave in an NMI-like manner on AArch64. We’ll discuss the implementation, some of the complications that ensued and also some of the benefits obtained from it.
Julien Thierry
Process Address Space: The way to create virtual address (page table) of user...Adrian Huang
Process Address Space: The way to create virtual address (page table) of userspace application.
Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
Доклад рассказывает об устройстве и опыте применения инструментов динамического тестирования C/C++ программ — AddressSanitizer, ThreadSanitizer и MemorySanitizer. Инструменты находят такие ошибки, как использование памяти после освобождения, обращения за границы массивов и объектов, гонки в многопоточных программах и использования неинициализированной памяти.
While CMake has become the de-facto standard buildsystem for C++, it's siblings CTest and CPack are less well known. This talk gives a lightspeed introduction into these three tools and then focuses on best practices on building, testing, and packaging.
Linux Performance Analysis: New Tools and Old SecretsBrendan Gregg
Talk for USENIX/LISA2014 by Brendan Gregg, Netflix. At Netflix performance is crucial, and we use many high to low level tools to analyze our stack in different ways. In this talk, I will introduce new system observability tools we are using at Netflix, which I've ported from my DTraceToolkit, and are intended for our Linux 3.2 cloud instances. These show that Linux can do more than you may think, by using creative hacks and workarounds with existing kernel features (ftrace, perf_events). While these are solving issues on current versions of Linux, I'll also briefly summarize the future in this space: eBPF, ktap, SystemTap, sysdig, etc.
Introduce F9 microkernel, new open source implementation built from scratch, which deploys modern kernel techniques, derived from L4 microkernel designs, to deep embedded devices.
:: https://github.com/f9micro
Characteristics of F9 microkernel
– Efficiency: performance + power consumption
– Security: memory protection + isolated execution
– Flexible development environment
XPDDS17: Reworking the ARM GIC Emulation & Xen Challenges in the ARM ITS Emu...The Linux Foundation
Part 1: Reworking the ARM GIC Emulation
The ARM Generic Interrupt Controller (GIC) provides some level of virtualization support in hardware. This still requires emulation of the distributor part, which has to integrate with the virtualization feature. Doing this in a performing and readable way is not trivial, especially the locking strategy tends to be complicated.
While extending the existing virtual GIC support in Xen to cover support for MSIs, some issues have been discovered which ask for some significant changes in the existing code.
The presentation will briefly describe the existing VGIC design and the issues we faced when trying to extend it. Based on this the changes will be presented and how they improve and ideally simplify the code.
Part 2: Xen Challenges in the ARM ITS Emulation
For being able to use MSIs on ARM systems in Xen domains we need to emulate the ARM GICv3 ITS controller. Its design is centered around a command queue located in normal system memory.
Emulating this in the Xen hypervisor brings some interesting challenges, ranging from safely accessing the guest memory and dealing with possible propagation of commands, to possible DOS attacks by domains keeping the emulation code busy.
The presentation outlines the main problems and how we hit Xen limits in emulating this correctly and efficiently. Also it presents our temporary workarounds and their drawbacks.
Memory Mapping Implementation (mmap) in Linux KernelAdrian Huang
Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
Valgrind is a GPL'd system for debugging and profiling Linux programs. With Valgrind's tool suite you can automatically detect many memory management and threading bugs, avoiding hours of frustrating bug-hunting, making your programs more stable. You can also perform detailed profiling to help speed up your programs.
Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
Talk by Brendan Gregg for USENIX LISA 2019: Linux Systems Performance. Abstract: "
Systems performance is an effective discipline for performance analysis and tuning, and can help you find performance wins for your applications and the kernel. However, most of us are not performance or kernel engineers, and have limited time to study this topic. This talk summarizes the topic for everyone, touring six important areas of Linux systems performance: observability tools, methodologies, benchmarking, profiling, tracing, and tuning. Included are recipes for Linux performance analysis and tuning (using vmstat, mpstat, iostat, etc), overviews of complex areas including profiling (perf_events) and tracing (Ftrace, bcc/BPF, and bpftrace/BPF), and much advice about what is and isn't important to learn. This talk is aimed at everyone: developers, operations, sysadmins, etc, and in any environment running Linux, bare metal or the cloud."
Windows Kernel Exploitation : This Time Font hunt you down in 4 bytesPeter Hlavaty
In our recent work we targeted also win32k, what seems to be fruit giving target. @promised_lu made our own TTF-fuzzer which comes with bunch of results in form of gigabytes of crashes and various bugs. Fortunately windows make great work and in February most of our bugs was dead - patched, but not all of them…
Whats left were looking as seemingly unexploitable kernel bugs with ridiculous conditions. We decided to check it out, and finally combine it with our user mode bug & emet bypass. Through IE & flash we break down system and pointed out at weak points in defensive mechanism.
In this talk we will present our research dedicated for pwn2own event this year. We will describe kernel part of exploit in detail*, including bug description, resulting memory corruption conditions & caveats up to final pwn via one of our TTF bugs.
Throughout the talk we will describe how to break various exploit mitigations in windows kernel and why it is possible. We will introduce novel kernel exploitation techniques breaking all what stands { KASLR, SMEP, even imaginary SMAP or CFG } and bring you SYSTEM exec (from kernel driver to system calc).
* unfortunately bug was not fixed at the time of talk, so we do not exposed details about TTF vulnerability, and we skipped directly to some challenges during exploitation, and demonstrate how OS design can overpower introduced exploit mitigations.
Linux Kernel Booting Process (2) - For NLKBshimosawa
Describes the bootstrapping part in Linux, and related architectural mechanisms and technologies.
This is the part two of the slides, and the succeeding slides may contain the errata for this slide.
This tutorial covers all parallel replication implementation in MariaDB 10.0 and 10.1 and MySQL 5.6, 5.7 and 8.0 (including how it works in Group Replication).
MySQL and MariaDB have different types of parallel replication. In this tutorial, we present the different implementations that allow us to understand their limitations and tuning parameters. We cover how to make parallel replication faster and what to avoid for maximizing its benefits. We also present tests from Booking.com workloads.
Some of the subjects that are covered are group commit and optimistic parallel replication in MariaDB, the parallelism interval of MySQL and its Write Set optimization, and the ?slowing down the master to speed up the slave? optimization.
After this tutorial, you will know everything you need to implement and tune parallel replication in your environment. But more importantly, we will show how you can test parallel replication benefit in a non-disruptive way before deployment.
Kernel Recipes 2019 - No NMI? No Problem! – Implementing Arm64 Pseudo-NMIAnne Nicolas
As the name would suggest, a Non-Maskable Interrupt (NMI) is an interrupt-like feature that is unaffected by the disabling of classic interrupts. In Linux, NMIs are involved in some features such as performance event monitoring, hard-lockup detector, on demand state dumping, etc… Their potential to fire when least expected can fill the most seasoned kernel hackers with dread.
AArch64 (aka arm64 in the Linux tree) does not provide architected NMIs, a consequence being that features benefiting from NMIs see their use limited on AArch64. However, the Arm Generic Interrupt Controller (GIC) supports interrupt prioritization and masking, which, among other things, provides a way to control whether or not a set of interrupts can be signaled to a CPU.
This talk will cover how, using the GIC interrupt priorities, we provide a way to configure some interrupts to behave in an NMI-like manner on AArch64. We’ll discuss the implementation, some of the complications that ensued and also some of the benefits obtained from it.
Julien Thierry
Process Address Space: The way to create virtual address (page table) of user...Adrian Huang
Process Address Space: The way to create virtual address (page table) of userspace application.
Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
Доклад рассказывает об устройстве и опыте применения инструментов динамического тестирования C/C++ программ — AddressSanitizer, ThreadSanitizer и MemorySanitizer. Инструменты находят такие ошибки, как использование памяти после освобождения, обращения за границы массивов и объектов, гонки в многопоточных программах и использования неинициализированной памяти.
While CMake has become the de-facto standard buildsystem for C++, it's siblings CTest and CPack are less well known. This talk gives a lightspeed introduction into these three tools and then focuses on best practices on building, testing, and packaging.
Linux Performance Analysis: New Tools and Old SecretsBrendan Gregg
Talk for USENIX/LISA2014 by Brendan Gregg, Netflix. At Netflix performance is crucial, and we use many high to low level tools to analyze our stack in different ways. In this talk, I will introduce new system observability tools we are using at Netflix, which I've ported from my DTraceToolkit, and are intended for our Linux 3.2 cloud instances. These show that Linux can do more than you may think, by using creative hacks and workarounds with existing kernel features (ftrace, perf_events). While these are solving issues on current versions of Linux, I'll also briefly summarize the future in this space: eBPF, ktap, SystemTap, sysdig, etc.
Introduce F9 microkernel, new open source implementation built from scratch, which deploys modern kernel techniques, derived from L4 microkernel designs, to deep embedded devices.
:: https://github.com/f9micro
Characteristics of F9 microkernel
– Efficiency: performance + power consumption
– Security: memory protection + isolated execution
– Flexible development environment
XPDDS17: Reworking the ARM GIC Emulation & Xen Challenges in the ARM ITS Emu...The Linux Foundation
Part 1: Reworking the ARM GIC Emulation
The ARM Generic Interrupt Controller (GIC) provides some level of virtualization support in hardware. This still requires emulation of the distributor part, which has to integrate with the virtualization feature. Doing this in a performing and readable way is not trivial, especially the locking strategy tends to be complicated.
While extending the existing virtual GIC support in Xen to cover support for MSIs, some issues have been discovered which ask for some significant changes in the existing code.
The presentation will briefly describe the existing VGIC design and the issues we faced when trying to extend it. Based on this the changes will be presented and how they improve and ideally simplify the code.
Part 2: Xen Challenges in the ARM ITS Emulation
For being able to use MSIs on ARM systems in Xen domains we need to emulate the ARM GICv3 ITS controller. Its design is centered around a command queue located in normal system memory.
Emulating this in the Xen hypervisor brings some interesting challenges, ranging from safely accessing the guest memory and dealing with possible propagation of commands, to possible DOS attacks by domains keeping the emulation code busy.
The presentation outlines the main problems and how we hit Xen limits in emulating this correctly and efficiently. Also it presents our temporary workarounds and their drawbacks.
Memory Mapping Implementation (mmap) in Linux KernelAdrian Huang
Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
Valgrind is a GPL'd system for debugging and profiling Linux programs. With Valgrind's tool suite you can automatically detect many memory management and threading bugs, avoiding hours of frustrating bug-hunting, making your programs more stable. You can also perform detailed profiling to help speed up your programs.
Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
Slab allocation is a memory management mechanism intended for the efficient memory allocation of kernel objects which displays the desirable property of eliminating fragmentation caused by allocations and deallocations.
This is an old presentation salvaged from archive.
http://tree.celinuxforum.org/CelfPubWiki/JapanTechnicalJamboree13
This is English translated version. Japanese version is here.
http://www.slideshare.net/tetsu.koba/basic-of-virtual-memory-of-linux
Android is NOT just 'Java on Linux'.
Android uses Linux kernel. But only kernel. I show you how different Android is from normal Linux systems.
Visit this page.
http://kobablog.wordpress.com/2011/05/22/android-is-not-just-java-on-linux/
1. Tips of
malloc & free
Making your own malloc
library for troubleshooting
2013.2.22 Embedded Linux Conference
Tetsuyuki Kobayashi
1
2. The latest version of this slide will
be available from here
http://www.slideshare.net/tetsu.koba/presentations
2
3. Who am I?
20+ years involved in embedded systems
10 years in real time OS, such as iTRON
10 years in embedded Java Virtual Machine
Now GCC, Linux, QEMU, Android, …
Blogs
http://d.hatena.ne.jp/embedded/ (Personal)
http://blog.kmckk.com/ (Corporate)
http://kobablog.wordpress.com/(English)
Twitter
@tetsu_koba 3
4. Today's topics
Prologue: Making your own malloc
library for troubleshooting
System calls to allocate memory in
user space
Tips of glibc's malloc
How to hook and replace malloc (and
pitfalls I fell)
dlmalloc
4
6. Typical troubles of heap memory
Corruption
crashed by SEGV at malloc or free.
looks malloc bug, but NOT
Who actually destroy heap?
Leaking
malloc'ed but not free'ed
damages silently
You want additional checking and
logging in malloc/free 6
7. Wrapping macro/fuction
#define malloc(x) debug_malloc(x)
Useful. But you can't cover all
malloc calling because ...
7
8. Explicit call for malloc
many standard library functions
use malloc internally
example) sprintf
C++ new operator uses malloc
internelly
8
9. Modify glibc(libc.so) directly?
libc source package is quite large
If you replace libc.so, it affects
whole system
not only for the debugee process
9
10. So I did was
making my own malloc library
easy to modify
use this only for the debugee
process
10
12. System calls to allocate
memory in user space
You need system call to allocate in
user space when you make your own
malloc library
There are 2 types of them
brk/sbrk
mmap/munmap/mremap
12
13. brk/sbrk
exists from ancient Unix
before virtual memory system
extends data segment
standard malloc library use these
system calls
You should not use these system calls
if your own malloc library co-exist with
standard malloc library 13
14. brk/sbrk extends data segment
Memory map of user process on old simple UNIX
Text Read Only
etext
Data
edata
Zero cleared Bss Read Write
end
Heap
At modern OS
start address of heap Extends by brk(2)/sbrk(2)
is randomized
Grows down automatically
Stack
14
16. mmap/munmap/mremap
newer system calls than brk/sbrk
integrate memory and file mapping
Glibc's malloc also use these when
large chunk (>= 128KB: default)
required
Use these when you implement your
own malloc library
16
17. Usage of mmap(2)
addr = mmap(NULL, size, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS, 0, 0);
if (MAP_FAILED == addr) {
perror("mmap");
abort();
}
You don't have to specify address. (set NULL)
Then kernel allocate memory from free space.
17
18. alloca(3)
By the way,
allocates memory in caller's stack
frame
frees automatically when the function
that called alloca() returns
same as local variables
machine and compiler dependent
be careful when stack size is small
especially multi-thread
18
20. mallopt
int mallopt(int param, int value)
configures glibc malloc such as
M_CHECK_ACTION
M_MMAP_THRESHOLD
M_TOP_PAD
M_TRIM_THRESHOLD
see man 3 mallopt
20
21. malloc_stats
void malloc_stats(void)
prints (on standard error) statistics
about heap like this
Arena 0:
system bytes = 135168
in use bytes = 128
Total (incl. mmap):
system bytes = 139264
in use bytes = 4224
max mmap regions = 1
max mmap bytes = 569344
21
22. malloc_usable_size
size_t malloc_usable_size(void *__ptr)
reports the number of usable allocated bytes
associated with allocated chunk __ptr
This size may be a bit bigger than the size
specified at malloc()
because of alignment of next data
This is useful when counting allocated total
size
increment size in hooked malloc
decrement size in hooked free 22
23. MALLOC_CHECK_
easy way to enable additional
checking in glibc malloc
with some overhead
environment variable
MALLOC_CHECK_
0: no check at all (no overhead)
1: check and print message if error
2: check and abort if error
23
24. __malloc_hook
glibc's malloc has its own hook
mechanism
global variables of function pointers
__malloc_hook
__realloc_hook
__memalign_hook
__free_hook
__malloc_initialize_hook
man malloc_hook for detail 24
25. mtrace
easy way to enable logging in glibc
malloc
see man 3 mtrace
There is tool to check log and find
leaking memory
see man 1 mtrace
implemented using __malloc_hook
This seems not thread safe
25
27. Hook and replace malloc
2 methods to hook malloc
LD_PRELOAD & dlsym
__malloc_hook
These do not require to recompile
other program and libraries
27
28. Using LD_PRELOAD & dlsym to
hook malloc
Use dynamic link mechanism
can not use when static linking
Make your own malloc dynamic link library
and set it to environment variable
LD_PRELOAD
Then your malloc is used prior to glibc's
malloc
You can get glibc's malloc address by
dlsym(3) 28
30. Hooking malloc by LD_PRELOAD
preload by LD_PRELOAD
your own library
get address by
dlsym(RTLD_NEXT, “malloc”)
malloc
malloc
glibc
executable
executable malloc
malloc
/libraries
/libraries output log or
record size ...
30
34. Pitfall #2
When compile with -pthread, it
crashes at the beginning. Why?
In multi-thread mode, dlsym() uses
calloc() at the first time.
calloc() requires dlsym()
dlsym() requires calloc() … !!
prepare special calloc() for the first
call of calloc().
34
35. st
Call special calloc at the 1 time
void *calloc (size_t n, size_t len)
{
void *ret;
void *caller; Just returns some static
allocated memory
if (no_hook) {
if (callocp == NULL) {
ret = my_calloc(n, len);
return ret;
}
return (*callocp)(n, len);
}
...
35
36. Using __malloc_hook variable to
hook malloc
function pointer variables for hooking
void *(*__malloc_hook)(size_t, const void*)
void*(*__realloc_hook)(void*, size_t, const void*)
void*(*__memalign_hook)(size_t, size_t, const void*)
void (*__free_hook)(void)
void (*__malloc_initialize_hook)(void)
36
38. Hooking malloc by
__malloc_hook
glibc
executable
executable malloc
malloc
/libraries
/libraries __malloc_hook
void_t *malloc(size_t bytes)
{
__malloc_ptr_t (*hook) (size_t, __const __malloc_ptr_t)
=__malloc_hook; my_malloc
my_malloc
if (hook != NULL)
return (*hook)(bytes, RETURN_ADDRESS (0));
Your own library
38
39. Thread unsafe example
static void *
my_malloc_hook(size_t size, const void *caller) __malloc_hook is not
{
void *result;
locked at all
/* Restore all old hooks */
__malloc_hook = old_malloc_hook;
/* Call recursively */
result = malloc(size);
/* Save underlying hooks */ In this moment
old_malloc_hook = __malloc_hook; malloc from other
thread does not
/* printf() might call malloc(), so protect it too. */ hook.
printf("malloc(%u) called from %p returns %pn",
(unsigned int) size, caller, result);
/* Restore our own hooks */
__malloc_hook = my_malloc_hook;
return result; 39
}
40. Workaround
Changing __malloc_hook variable is
not thread safe. (Actually these
variables are marked as 'deprecated')
Set once these hook variables at
initial time and don't touch after that.
You can not call back glibc's malloc.
link and replace to other malloc.
dlmalloc is good for this.
40
41. Which ?
If you replace malloc
You can use __malloc_hook with
care
Otherwise
use LD_PRELOAD & dlsym
41
42. Another pitfall
Almost program works fine with my
own malloc library. But some game
app. causes SEGV accessing null
pointer.
At first I doubt that malloc returns
NULL because heap runs out …
42
43. Behavior of malloc(size=0)
I thought malloc(0) returns NULL.
man malloc says:
“If size is 0, then malloc() returns
either NULL, or a unique pointer
value that can later be successfully
passed to free().”
glibc's malloc does the latter.
44. The game app. calls malloc(0) and
use the pointer without check!
so it causes null pointer access
I modified my malloc returns a unique
pointer even if size == 0
Then the game app. works fine with
my malloc library.
46. dlmalloc
by Doug Lea
http://g.oswego.edu/dl/html/malloc.html
easy to compile and use
can add prefix to all function names to
avoid conflict to standard malloc
functions (-DUSE_DL_PREFIX)
add -DUSE_LOCKS=1 for thread safe
Actually glibc's malloc is based on this 46
47. mspace of dlmalloc
can have multiple separate memory
spaces for heap
per thread, per functional module, ...
Good for troubleshooting
isolate heap of module in question
48. Usual single heap
(UI)
(UI) malloc() The single heap
(graphics)
(graphics)
(database)
(database)
In the same process
48
49. Using mspaces
Their own mspaces
(UI)
(UI)
mspace_malloc()
(graphics)
(graphics)
(database)
(database)
In the same process
49
50. Summary
Make your own malloc library rather
than modify glibc (libc.so).
Use mmap(2) to get memory.
__malloc_hook is not thread safe and
deprecated.
Use LD_PRELOAD & dlsym(3) to
hook glibc's malloc.
50