Your SlideShare is downloading. ×
Transparent Hugepages in RHEL 6
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Transparent Hugepages in RHEL 6

20,106
views

Published on

Introduction to linux memory management and advantages of using huge pages for certain applications, and final wrap-up with benefits of Transparent hugepages available in RHEL 6.

Introduction to linux memory management and advantages of using huge pages for certain applications, and final wrap-up with benefits of Transparent hugepages available in RHEL 6.

Published in: Technology, Education

2 Comments
21 Likes
Statistics
Notes
No Downloads
Views
Total Views
20,106
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
0
Comments
2
Likes
21
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Transparent Hugepages in RHEL 6 Raghu Udiyar 22 July 2011
  • 2. Transparent Huge Pages
    • Written by Andrea Arcangeli (Red Hat kernel developer)
    • 3. Merged into upstream kernel 2.6.38
    • 4. Available in RHEL 6.0 on GA
  • 5.
    • A page is the smallest unit of memory that can be manipulated
    • 6. Memory is divided into pages of equal size
    Transparent Huge Pages Pages Memory
  • 7. Page : Memory Management
    • Page size varies with the architecture, x86 = 4KiB
    • 8. For other arch's : ia64 = 8KiB, ppc64 = 4,8 and 64KiB
    4KiB Page x86
  • 9. Page : Memory Management Physical Memory
    • Aka : RAM – memory installed on the system e.g. 2GiB
    • 10. Programs never work directly with physical memory
    • 11. A physical page is called a Page Frame
    Virtual Memory
    • Not real memory : 32bit – 4Gb, 64bit – more then 128T
    • 12. Programs always allocate virtual memory
    • 13. A virtual page is called simply a Page
  • 14. Page : Virtual and Physical
    • Each page is mapped to a page frame using tables
    Physical Virtual Page Page Frame
  • 15. Page : Memory Management Process 2 Process 1
  • 16. Page : Memory Management
    • Virtual to Physical address translations are stored in page tables
    • 17. These page tables are stored in memory
    • 18. Looking up these page tables is a processes called a
    • 19. Page Walk
  • 20. Page : Simplified view of the Page Table Virtual address
      Multiple tables are involved in a Page Walk
  • 21. Page : Memory Management
    • Every memory access requires a Page Walk
    • 22. Hardware assisted but it is still expensive
    • 23. Typically, takes 10-100 cpu clock cycles
    • 24. The obvious optimization is to cache the results
  • 25. Page : TLB - Translation look-aside buffer TLB caches Virtual->Physical Translations
      • Check TLB first
      • 26. If TLB hit, return address
      • 27. If TLB miss, do a Page Walk and store translation in TLB
      • Using TLB, increases performance by as much as 15%
      • 28. Look up takes 0.5 – 1 clock cycl e
  • 29. Page : TLB - Translation look-aside buffer
    • CAM (content addressable memory) is used that is quite expensive to manufacture and have other limitations
    • 30. But they are very fast
    • 31. Typically, a TLB can only hold 8 to 1024 entries
    • 32. Plus, TLB cache has to be flushed on every context switch
    • 33. Therefore, TLB is a scarce resource and every attempt has to be made to maximize its usage
  • 34. Page : Large Memory Applications
    • Applications touching greater then 10Gb
    • 35. 4KiB page size produces a large overhead to manage these pages
    • 36. Example :
    • 37. 17GiB Oracle SGA with 1000 Connection :
      • ~4.4 million pages
      • 38. Page table size : 16GiB
    • TLB can only hold at most 1024 entries
    • 39. BIG performance penalty
  • 40. Page : Questions?
    • Q1 : How can page walk work, if we don't know where the page tables are (as they are stored in memory) ?
    • 41. Ans : The address of the root table is stored in a special CR3 register.
  • 42.
      Huge Pages are pages of larger sizes e.g. on x86 it's 2MiB
    Transparent Huge Pages X 512 4 KiB 2 MiB
  • 43. Huge Pages : Benefits
    • Now each TLB entry can effectively store 512 normal pages
    • 44. Therefore, Increases TLB hit ratio
    • 45. Decrease in number of pages to manage
    • 46. Previous Oracle example :
    • 47. Using huge pages :
      • 17GiB SGA uses 17*1024/2 = 8704 Huge Pages (compared to ~4.4 million! Normal pages)
      • 48. Page Table size? ~33MiB (1000 processes)
      • 49. (compared to 16GiB)
    • Huge improvement!
  • 50. Huge Pages : Configuration
    • Set “vm.nr_hugepages = 10” in /etc/sysctl.conf
    • 51. To verify :
    • 52. # cat /proc/meminfo | grep Huge
    • 53. HugePages_Total: 10
    • 54. HugePages_Free: 10
    • 55. HugePages_Rsvd: 0
    • 56. Hugepagesize: 2048 kB
    • 57. A reboot may be required if not enough contiguous physical pages are available.
    • 58. Refer to : https://access.redhat.com/kb/docs/DOC-56553
  • 59. Huge Pages : Issues
    • Huge pages have to be reserved beforehand
    • 60. Not available for normal applications
    • 61. Explicit application support is required
    • 62. Cannot be swapped to disk – locked in memory
    • 63. Only really used by large database applications, such as Oracle and Sybase
  • 64. Huge Pages : Questions?
    • Why can't we have all normal pages as huge pages i.e. Increase from 4KiB to 2MiB
    • 65. Ans : This causes fragmentation.
        • Small page size causes large management overhead
        • 66. Large page size causes internal and external fragmentation
    • What we need is a best of both worlds
  • 67.
    • This builds up on the limitations of normal Huge pages
    • 68. THP is enabled by default for all applications
    • 69. Transparent to the application – No configuration required
    • 70. All applications benefit from using Huge pages
    • 71. Up to 10% performance improvement
    • 72. Therefore, we successfully have a mixed or hybrid page size scenario
    Transparent Huge Pages
  • 73. THP: How does it work?
    • Hugepages are allocated whenever possible
    • 74. Example : 2MiB memory requests
    • 75. THP are swappable, by breaking a hugepage to 4KiB pages
    • 76. A kernel thread Khugepaged will scan running processes and attempt to switch normal pages with Hugepages
    • 77. THP tries to maximizes Huge page use
    • 78. Kernel itself maps it's memory using hugepages
  • 79. THP: Work in Progress
    • Only enabled for anonymous memory, for now
    • 80. Page cache and shared memory still in progress
    • 81. To check THP usage on RHEL 6 or F14 and above :
    • 82. # grep AnonHugePages /proc/meminfo
    • 83. AnonHugePages: 632832 kB
  • 84. THP : Questions?
    • Refer for more details :
    • 85. https://access.redhat.com/kb/docs/DOC-49562
  • 86. Thank You Raghu Udiyar 22 July 2011