Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Алексей Лесовский "Тюнинг Linux для баз данных. "

539 views

Published on

В любом проекте всегда есть данные которые важно не потерять. И если проект перешагнул размеры домашнего, то наверняка там уже используется что-то более надежное чем текстовые файлы на диске. Скорее всего это какая-то конкретная база данных. Учитывая важную роль баз данных к ним предъявляют жесткие требования производительности и стабильности. Отдельной особенностью является и операционная система на которой работает база данных. Опыт и практика подсказывает что это будет Linux - одна из наиболее популярных на сегодняшний день операционных систем. Прогрессивное развитие в сочетании с гибкостью и стабильностью делает Linux хорошим кандидатом в качестве платформы для баз данных (да и не только). Однако настройки по-умолчанию в большинстве дистрибутивов предназначены для широкого круга задач и потребностей. В этом докладе я расскажу о том, на какие настройки следует обратить внимание и как они влияют на работу СУБД. Как сделать работу СУБД более предсказуемой, как в простых условиях, так и в условиях непостоянной или растущей нагрузки.

Published in: Education
  • Be the first to comment

Алексей Лесовский "Тюнинг Linux для баз данных. "

  1. 1. About PostgreSQL DBA. Linux system administrator. PostgreSQL-Consulting.com: ● 24/7 support. ● Audit, performance optimizations. ● Consulting and Training. ● Monitoring and Emergency. ● Capacity planning. Slides: https://goo.gl/awmZ2H
  2. 2. Agenda RDBMS on Linux, why? Databases and Resources. OS subsystems. CPU, Process scheduling, Power saving policies. Memory, VM, NUMA, Huge pages. Storage, File Systems, Input/Output. Other misc.
  3. 3. Why Linux? Linux is a good choice: ● Active development & Community support. ● A lot of features & Fast implementation. ● Stable & Mature & Durable.
  4. 4. Databases & Resources Concurrency Query speed Sort, group, hash,... OS page cache DB buffer pool Local process cache DB data files Transaction Log Cold start CPU Memory Storage
  5. 5. Databases & Resources CPU Scheduling NUMA Power Saving Virtual Memory NUMA Huge Pages File Systems Storage I/O CPU Memory Storage
  6. 6. Resources CPU scheduler. Virtual memory and NUMA. Huge pages. File systems. Storage IO. Power saving policy. Others.
  7. 7. CPU scheduling CPU scheduler responsible for proper processes planning: Sysctl: ● kernel.sched_migration_cost_ns = 5000000 (default: 500000). ● kernel.sched_autogroup_enabled = 0 (default: 1). http://www.postgresql.org/message-id/50E4AAB1.9040902@optionshouse.com http://kernelnewbies.org/Linux_2_6_38#head-59575a6aeafa38490226a560ee02de89829a5b20
  8. 8. CPU scheduling CPU scheduler responsible for proper processes planning: Sysctl: ● kernel.sched_migration_cost_ns = 5000000 (default: 500000). ● kernel.sched_autogroup_enabled = 0 (default: 1). http://www.postgresql.org/message-id/50E4AAB1.9040902@optionshouse.com http://kernelnewbies.org/Linux_2_6_38#head-59575a6aeafa38490226a560ee02de89829a5b20 Be aware on Ubuntu: 12.04 #1055222 and 14.04 #1422016. Use noautogroup kernel param instead of sysctl.conf.
  9. 9. Virtual Memory What is it? Allocator, Caching, Dirty pages and Writeback.
  10. 10. Virtual Memory
  11. 11. Virtual Memory Sysctl: vm.dirty_background_ratio & vm.dirty_ratio = disable it. vm.dirty_background_bytes & vm.dirty_bytes = depends on ... RAID cache size, 64MB/128MB otherwise
  12. 12. Virtual Memory Out-of-memory & OOM-Killer Sysctl: vm.swappiness = 1 (default: 60)
  13. 13. NUMA S — Socket C — CPU core M — Memory bank
  14. 14. NUMA BIOS: enable memory node interleaving. Kernel boot: numa=off. numactl utility. Sysctl: ● vm.zone_reclaim_mode = 0 (default: 0). ● kernel.numa_balancing = 0 (default: 0).
  15. 15. Huge Pages Huge pages vs. Transparent huge pages. Huge pages are supported by many RDBMS. Always disable transparent huge pages.
  16. 16. Huge Pages Huge pages vs. Transparent huge pages. Huge pages are supported by many RDBMS. Always disable transparent huge pages. /etc/rc.local: ● echo never > /sys/kernel/mm/transparent_hugepage/enabled ● echo never > /sys/kernel/mm/transparent_hugepage/defrag
  17. 17. Filesystems Ext3 vs Ext4 vs XFS: what is better? Filesystem Barriers.
  18. 18. Filesystems Ext3 vs Ext4 vs XFS: what is better? Filesystem Barriers. Disable Write Cache: ● hdparm -W0 /dev/device ● MegaCli64 -LDSetProp -DisDskCache -Lall -aALL
  19. 19. Filesystems Ext3 vs Ext4 vs XFS: what is better? Filesystem Barriers. Disable Write Cache: ● hdparm -W0 /dev/device ● MegaCli64 -LDSetProp -DisDskCache -Lall -aALL Hardware RAID + BBU = barrier=0 (disable). Software RAID = barrier=1 (enable).
  20. 20. Filesystems Ext3 vs Ext4 vs XFS: what is better? Filesystem Barriers. Disable Write Cache: ● hdparm -W0 /dev/device ● MegaCli64 -LDSetProp -DisDskCache -Lall -aALL Hardware RAID + BBU = barrier=0 (disable). Software RAID = barrier=1 (enable). Enterprise SSD with Power Loss Protection = barrier=0 (disable).
  21. 21. Storage IO SATA/SAS vs SSD. IO elevators.
  22. 22. Storage IO SATA/SAS vs SSD. IO elevators: ● noop: SSD, PCIe SSD, hi-end storages. ● deadline: RAID, SATA/SAS. ● cfq: good default. ● none (multi-queue block IO): SSD, PCIe SSD.
  23. 23. Storage IO SATA/SAS vs SSD. IO elevators: ● noop: SSD, PCIe SSD, hi-end storages. ● deadline: RAID, SATA/SAS. ● cfq: good default. ● none (multi-queue block IO): SSD, PCIe SSD. # echo 'elevator_name' > /sys/block/<device>/queue/scheduler kernel boot: elevator=<name> /sys/block/*/queue/: rotational, rq_affinity, read_ahead_kb
  24. 24. Power Saving Policy Drivers: acpi_cpufreq vs. intel_pstate. scaling_governor.
  25. 25. Power Saving Policy Drivers: acpi_cpufreq vs. intel_pstate. scaling_governor: ● /sys/devices/system/cpu/cpuX/cpufreq/scaling_available_governors ● /sys/devices/system/cpu/cpuX/cpufreq/scaling_governor
  26. 26. Power Saving Policy Drivers: acpi_cpufreq vs. intel_pstate. scaling_governor: ● /sys/devices/system/cpu/cpuX/cpufreq/scaling_available_governors ● /sys/devices/system/cpu/cpuX/cpufreq/scaling_governor acpi_cpufreq + performance. intel_pstate + powersave.
  27. 27. Misc: Clocksources What is clocksource? acpi_pm vs. hpet vs. tsc. /sys/devices/system/clocksource/clocksource0/available_clocksource. /sys/devices/system/clocksource/clocksource0/current_clocksource.
  28. 28. Summary Linux is a good choice for RDBMS: Modern, Universal, Flexible, Stable. Adapt Linux for your workloads. Test → Change → Test → Commit/Rollback.
  29. 29. Questions? Alexey Lesovsky lesovsky@pgco.me PostgreSQL-Consulting.com: Data maintenance at its best https://postgresql-consulting.com

×