Hugepage

1. HugePage

2. Basics: Memory
● In Linux, memory is managed in blocks known as "pages".
● A page is the basic unit of virtual memory.
● The size of a page is "4096" bytes.
$ getconf PAGE_SIZE
4096
● In Linux the process address space is virtual.
● The CPU and the operating system have to remember which page belong to which process,
and where it is stored.
● The more pages you have, the more time it takes to find where the memory is mapped.
● When a process uses 1 GiB of memory, that's 262144 entries to look up (1GiB/4KiB).
● If one Page Table Entry consume 8bytes, that's 2 MiB (262144 x 8) to look­up.
● CPUs have a built­in memory management unit (MMU) that contains a list of these pages,
with each page referenced through a page table entry (PTE).

3. HugePage: Hardware Support
● Depending on the processor, there are two different HugePage sizes on
the x86_64 architecture: 2 MiB and 1 GiB
● If the CPU supports 2 MiB pages, it has the "PSE" cpuinfo flag.
● If the CPU supports 1 GiB pages, it has the "PDPE1GB" cpuinfo flag.
● /proc/cpuinfo shows whether the two flags are set.
$ grep pse /proc/cpuinfo | uniq
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr
● The page tables used by the 2 MiB pages are suitable for managing
multiple gigabytes of memory.
● The page tables used by the 1 GiB pages are best for scaling to
terabytes of memory.

4. HugePage: Kernel Support
● The support for HugePage is enabled in standard RHEL
kernel.
$ grep HUGETLB /boot/config­$(uname ­r)
CONFIG_HUGETLBFS=y
CONFIG_HUGETLB_PAGE=y
● The HugePage size can be determined by looking at
/proc/meminfo:
# cat /proc/meminfo |grep Hugepagesize
Hugepagesize: 2048 kB

5. HugePage: Size Based on Arch
● Huge pages sizes based on Architecture:
Architecture HugePage Size
i386 4MiB (2 MiB in PAE mode)
ia64 1MiB, 4MiB, 16MiB, 256MiB
ppc64 16MiB

6. HugePage: Setting
Temporary:
● Using /proc/sys/vm/nr_hugepages file.
Permanent:
● Using sysctl parameter vm.nr_hugepages in /etc/sysctl.conf file.
For Example;
# echo 1 > /proc/sys/vm/nr_hugepages
OR
# vim /etc/sysctl.conf
vm.nr_hugepages = 1
Note: Huge pages must be assigned at boot time. (Reboot the system)

7. HugePage: How much HugePages ?
● Ideally it is recommended to set
HugePages to half of the available RAM.
● But it is advisable to set HugePages
based on the requirement of the
application.

8. HugePage: Example
Task:
Set 10 GiB of HugePages on the system.
Formula:
vm.nr_hugepages = (Memory in KiB)/Hugepagesize
Calculation:
10 GiB = 10485760 KiB
Hugepagesize = 2048 KiB
vm.nr_hugepages = 10485760/2048
= 5120

9. HugePage:
How to determine if HugePages is disable or
enabled on the server ?
1) Check the value of HugePages_Total field
in /proc/meminfo file.
2) Check the value in /proc/sys/vm/nr_hugepages
file.
3) Check the value of vm.nr_hugepages in
/etc/sysctl.conf file.

10. HugePage:
● If the value is "0" (zero) then it means that HugePages
is "disable" on the system.
Eg:
# grep ­i HugePages_Total /proc/meminfo
HugePages_Total: 0
# cat /proc/sys/vm/nr_hugepages
0
# grep nr_hugepages /etc/sysctl.conf
vm.nr_hugepages = 0

11. HugePage:
● If the value is (!0) non­zero then it means that
HugePages is "enabled" on the system.
# grep ­i HugePages_Total /proc/meminfo
HugePages_Total: 1024
# cat /proc/sys/vm/nr_hugepages
1024
# grep nr_hugepages /etc/sysctl.conf
vm.nr_hugepages = 1024

12. HugePage: /proc/meminfo
$ grep ­i "HugePage" /proc/meminfo
HugePages_Total: 7176
HugePages_Free: 7176
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
Where;
HugePages_Total is the size of the pool of huge pages.
HugePages_Free is the number of huge pages in the pool that are
not yet allocated.
Hugepagesize is the size of single huge page.

13. HugePage: /proc/meminfo
$ grep ­i "HugePage" /proc/meminfo
HugePages_Total: 7176
HugePages_Free: 7176
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
Where;
HugePages_Total is the size of the pool of huge pages.
HugePages_Free is the number of huge pages in the pool that are
not yet allocated.
Hugepagesize is the size of single huge page.

14. HugePage:
Calculation:
Total memory allocated by Hugepages = HugePages_Total x Hugepagesize
= 7176 x 2048
= 14696448 MiB
= 14696448/1048576
= 14.01 GiB

15. HugePage:
Advantages:
● HugePages are never swapped out when the system
is under memory pressure.
● It reduces the kernel's memory management
overhead since it will be managing fewer pages.
(Improves Performance)
● The kernel virtual memory manager utilizes Low
Memory ­ fewer pages to manage means it will
consume less low memory.

16. HugePage:
Dis­advantages:
● The system reboot is required after allocating the
HugePages.
● The memory reserved by HugePages can only be
utilized by process(es) which are Hugepages aware.
● The memory pages for "HugePages" are kept apart at
boot­time of the system, and cannot be used for
other memory allocations, even if they are free.

17. Questions ?