Download as PDF, PPTX
Uploaded byMind The Firebird
SuperServer in Firebird 3
The document discusses changes to the architecture and implementation of the Super Server in Firebird 3. Key points include: - The Super Server now uses a single process with multiple threads to handle connections instead of cooperative multitasking with a single active thread. - Fine-grained locking is used for the shared page cache, with individual latches for each page buffer, improving parallelism but increasing synchronization costs. - Benchmarks show significant performance improvements for multi-threaded workloads in Firebird 3 compared to earlier versions thanks to better utilization of multiple CPU cores by the Super Server.
More Related Content
![[若渴]Study on Side Channel Attacks and Countermeasures](https://cdn.slidesharecdn.com/ss_thumbnails/studyonsidechannelattacksandcountermeasures-180120164619-thumbnail.jpg?width=640&height=640&fit=bounds)
![[若渴計畫] Studying ASLR^cache](https://cdn.slidesharecdn.com/ss_thumbnails/studyingaslrcache-170416163739-thumbnail.jpg?width=640&height=640&fit=bounds)
![[B5]memcached scalability-bag lru-deview-100](https://cdn.slidesharecdn.com/ss_thumbnails/b5memcached-scalability-baglru-deview-100-120919013502-phpapp02-thumbnail.jpg?width=640&height=640&fit=bounds)
SuperServer in Firebird 3
- 1. Super ServerSuper Server inFirebird 3in Firebird 3
- 2.
- 3. Prague 2014Prague 2014Firebird 3Firebird 33 Introduction into Firebird3 ● Firebird3 files ● Root directory – network server (firebird.exe) – client library (fbclient.dll) – utilities (gbak, gfix, isql, etc) – configuration files (*.conf) – security database (security3.fdb) – support libraries (icu*.dll, ib_util.dll, MSVC CRT dll's) – support files (firebird.msg, icu*.ddat, license.txt)
- 4. Prague 2014Prague 2014Firebird 3Firebird 34 Introduction into Firebird3 ● Firebird3 files ● bin sub-directory – yes, no more bin sub-directory on Windows ● intl sub-directory – internalization support library (fbintl.dll, fbintl.conf) ● plugins sub-directory – engine itself (engine12.dll) – trace plugin (fbtrace.dll) – auth plugins (srp.dll, legacy_auth.dll, legacy_usermanager.dll) – UDR engine plugin ● udf sub-directory – standard UDF libraries
- 5. Prague 2014Prague 2014Firebird 3Firebird 35 Introduction into Firebird3 ● Unified network listener ● single network listener firebird.exe – no more fbserver.exefb_inet_server.exe ● launch new process for incoming connection – process mode: by default ● launch new thread for incoming connection – threading mode: add switch -m
- 6. Prague 2014Prague 2014Firebird 3Firebird 36 Introduction into Firebird3 ● Unified client library and unified engine library ● No more fbclient.dllfbembed.dll – single client: fbclient.dll ● used for remote and embedded access – single engine: engine12.dll ● used as Super Server and Classic Server ● SuperClassic mode of engine defined by two new firebird.conf settings: – SharedDatabase – SharedCache
- 7. Prague 2014Prague 2014Firebird 3Firebird 37 Introduction into Firebird3 ● Classic Server – network server in process mode (firebird -a) – SharedDatabase = true – SharedCache = false ● Super Server – network server in threading mode (firebird -a -m) – SharedDatabase = false – SharedCache = true ● Super Classic (obsolete?) – network server in threading mode (firebird -a -m) – SharedDatabase = true – SharedCache = false
- 8. Prague 2014Prague 2014Firebird 3Firebird 38 Introduction into Firebird3 ● Embedded – need both fbclient.dll and pluginsengine12.dll ● Embedded in Classic mode – SharedDatabase = true – SharedCache = false ● Embedded in Super mode – SharedDatabase = false – SharedCache = true
- 9. Prague 2014Prague 2014Firebird 3Firebird 39 Super Server ● Main difference from Classic ● All attachments are handled by the single process ● Shared common data – metadata cache – page cache – transaction inventory cache ● Hand-made scheduler: cooperative multitasking – single scheduler, single active thread in process – threads voluntary switched control to each other ● Metadata still synchronized by LM ● Page cache not used LM – individual “latches” for page buffers
- 10. Prague 2014Prague 2014Firebird 3Firebird 310 Super Server before Firebird3 ● Pluses ● shared caches - used much less memory than lot of Classic processes ● single active thread - almost no need to synchronize concurrent access to in-memory structures ● Minuses ● single active thread - no way to use more than one CPUcore – SMP is not possible ● cooperative multitasking is very far from fair scheduling – “heavy” query in one attachment could get more CPU and slowdown queries in another attachments
- 11. Prague 2014Prague 2014Firebird 3Firebird 311 Super Server in Firebird3 ● On the way to the true Super Server ● shared page cache ● shared TIP cache ● metadata cache is not shared (private per attachment) ● no more custom scheduler, all threads works in parallel – SMP is supported now ● new synchronization schema
- 12. Prague 2014Prague 2014Firebird 3Firebird 312 Page Cache ● Page buffers ● used to cache contents of database file pages – descriptor of page buffer, data buffer itself ● Control structures ● Hash table – used to fast search for page buffer by given page number ● LRU queue – defines page buffer eviction policy ● Precedence graph – used to implement “careful write” policy ● Dirty pages queue – what to write on commit
- 13. Prague 2014Prague 2014Firebird 3Firebird 313 Page Cache ● Synchronization ● individual latch for every page buffer – latch is a lightweight RW-lock – used always (despite of SharedCache setting) ● individual Lock Manager's lock for every page buffer – used in SharedCache = false (Classic) mode only ● separate RW-lock for every control structure – hash table – LRU queue – precedence graph – dirty queue
- 14. Prague 2014Prague 2014Firebird 3Firebird 314 Page Cache ● Fine-grained synchronization is not free ! ● Every page access consists of following steps ● fetch page buffer – read contents from disk, if necessary ● access (use) page buffer – read record data, for example ● release page buffer
- 15. Prague 2014Prague 2014Firebird 3Firebird 315 Page Cache ● Cost of page access in Super Server, before v3 ● fetch page – find buffer in hash table – if buffer is not used - increment use_count – else - voluntary switch to another thread and wait while current buffer owner granted us access ● release page – decrement use_count – grant access to waiting tread(s) ● Minimal cost is two changes of use_count – very, very cheap
- 16. Prague 2014Prague 2014Firebird 3Firebird 316 Page Cache ● Cost of page access in Super Server, in v3 ● fetch page – acquire lock for hash table – find buffer in hash table – release lock for hash table – acquire latch for page buffer ● interlocked increment of latch state, ● or wait while current latch owner granted us access ● release page – release latch for page buffer ● interlocked decrement of latch state – grant access to waiting tread(s) ● Minimal cost is four interlocked operations
- 17. Prague 2014Prague 2014Firebird 3Firebird 317 Benchmark, single thread read Not cached Firebird File System 0 10 20 30 40 50 60 70 80 90 100 83 7 10 92 10 13 Firebird 2.5 Firebird 3.0 Time,sec SELECT COUNT(*) FROM STOCK 10'000'000 rows Memory buffers = 500'000 Reads from disk to cache = 418'604 ' not cached Reads from disk to cache = 0 ' cached by Firebird Reads from disk to cache = 418'822 ' cached by File System Fetches from cache = 20'837'432
- 18. Prague 2014Prague 2014Firebird 3Firebird 318 Benchmark, single thread read Not cached Firebird File System 0 20 40 60 80 100 120 104 15 17 101 18 20 Firebird 2.5 Firebird 3.0 Time,sec SELECT COUNT(DISTINCT S_W_ID), COUNT(DISTINCT S_I_ID) FROM STOCK 10'000'000 rows Memory buffers = 500'000 Reads from disk to cache = 418'604 ' not cached Reads from disk to cache = 0 ' cached by Firebird Reads from disk to cache = 418'822 ' cached by File System Fetches from cache = 20'837'432
- 19. Prague 2014Prague 2014Firebird 3Firebird 319 Benchmark, multithreaded read ● Multithreaded read only benchmark ● Table STOCK have 10'000'000 records – 100 warehouses, 100'000 items in each ● Each reader reads random items in own warehouse ● We don't test network subsystem – client application executes procedure, which selects a number of random items (100 by default) ● We don't test IO subsystem – before each test series we ensure whole table STOCK is fully cached by filesysem
- 20. Prague 2014Prague 2014Firebird 3Firebird 320 1 2 4 8 16 32 64 0 50 100 150 200 250 300 350 400 450 414 416 411 408 403 384 365377 382 359 217 168 154 142 Super Server, page cache 2048 Firebird 2.5 Firebird 3.0 Threads Count Benchmark, multithreaded read 1 2 4 8 16 32 64 0 50 100 150 200 250 300 350 400 450 Threads Count
- 21. Prague 2014Prague 2014Firebird 3Firebird 321 Benchmark, multithreaded read 1 2 4 8 16 32 64 0 100 200 300 400 500 600 700 Threads Count 1 2 4 8 16 32 64 0 100 200 300 400 500 600 700 414 416 411 408 403 384 365377 382 359 217 168 154 142 390 616 491 324 219 190 176 Super Server, page cache 2048 Firebird 2.5 FB 3.0.0.31369 FB 3.0.0.31385 Threads Count
- 22. Prague 2014Prague 2014Firebird 3Firebird 322 Benchmark, multithreaded read 1 2 4 8 16 32 64 0 100 200 300 400 500 600 700 Threads Count 1 2 4 8 16 32 64 0 100 200 300 400 500 600 700 414 416 411 408 403 384 365377 382 359 217 168 154 142 371 612 538 466 383 323 291 Super Server, page cache 2048 Firebird 2.5 FB 3.0.0.31369 FB3 hash table patch Threads Count
- 23. Prague 2014Prague 2014Firebird 3Firebird 323 1 2 4 8 16 32 64 0 100 200 300 400 500 600 700 350 527 620 569 561 535 513 330 476 572 550 533 498 451 Super Classic, page cache 256 Firebird 2.5 Firebird 3.0 Threads Count Benchmark, multithreaded read 1 2 4 8 16 32 64 0 100 200 300 400 500 600 700 Threads Count
- 24. Prague 2014Prague 2014Firebird 3Firebird 324 Benchmark, multithreaded read 1 2 4 8 16 32 64 0 500 1000 1500 2000 873 867 833 811 803 799 787766 1208 1794 1735 1714 1641 1541 Super Server, page cache 500'000 Firebird 2.5 Firebird 3.0 Threads Count 1 2 4 8 16 32 64 0 500 1000 1500 2000 Threads Count
- 25. Prague 2014Prague 2014Firebird 3Firebird 325 1 2 4 8 16 32 64 0 5000 10000 15000 20000 25000 30000 5271 6745 10660 11091 10452 10429 9946 8081 12732 15519 15786 12465 9013 7947 11437 18390 25631 23647 19203 15275 10469 2.5 Super Server 2.5 Super Classic 3.0 Super Server Terminals tpmC Benchmark, TPCC
- 26. Prague 2014Prague 2014Firebird 3Firebird 326 Benchmark, TPCC 1 2 4 8 16 32 64 0 5000 10000 15000 20000 25000 30000 2.5 Super Server 2.5 Super Classic 3.0 Super Server Terminals tpmC
- 27. Prague 2014Prague 2014Firebird 3Firebird 327 Questions ?Questions ? Firebird official web site Firebird tracker THANK YOU FOR ATTENTIONTHANK YOU FOR ATTENTION hvlad@users.sf.net