Super ServerSuper Server
in Firebird 3in Firebird 3

Sponsors !

Prague 2014Prague 2014 Firebird 3Firebird 33
Introduction into Firebird3
● Firebird3 files
● Root directory
– network server (firebird.exe)
– client library (fbclient.dll)
– utilities (gbak, gfix, isql, etc)
– configuration files (*.conf)
– security database (security3.fdb)
– support libraries (icu*.dll, ib_util.dll, MSVC CRT dll's)
– support files (firebird.msg, icu*.ddat, license.txt)

Prague 2014Prague 2014 Firebird 3Firebird 34
Introduction into Firebird3
● Firebird3 files
● bin sub-directory
– yes, no more bin sub-directory on Windows
● intl sub-directory
– internalization support library (fbintl.dll, fbintl.conf)
● plugins sub-directory
– engine itself (engine12.dll)
– trace plugin (fbtrace.dll)
– auth plugins (srp.dll, legacy_auth.dll, legacy_usermanager.dll)
– UDR engine plugin
● udf sub-directory
– standard UDF libraries

Prague 2014Prague 2014 Firebird 3Firebird 35
Introduction into Firebird3
● Unified network listener
● single network listener firebird.exe
– no more fbserver.exefb_inet_server.exe
● launch new process for incoming connection
– process mode: by default
● launch new thread for incoming connection
– threading mode: add switch -m

Prague 2014Prague 2014 Firebird 3Firebird 36
Introduction into Firebird3
● Unified client library and unified engine library
● No more fbclient.dllfbembed.dll
– single client: fbclient.dll
● used for remote and embedded access
– single engine: engine12.dll
● used as Super Server and Classic Server
● SuperClassic mode of engine defined by two new
firebird.conf settings:
– SharedDatabase
– SharedCache

Prague 2014Prague 2014 Firebird 3Firebird 37
Introduction into Firebird3
● Classic Server
– network server in process mode (firebird -a)
– SharedDatabase = true
– SharedCache = false
● Super Server
– network server in threading mode (firebird -a -m)
– SharedDatabase = false
– SharedCache = true
● Super Classic (obsolete?)
– network server in threading mode (firebird -a -m)
– SharedDatabase = true
– SharedCache = false

Prague 2014Prague 2014 Firebird 3Firebird 38
Introduction into Firebird3
● Embedded
– need both fbclient.dll and pluginsengine12.dll
● Embedded in Classic mode
– SharedDatabase = true
– SharedCache = false
● Embedded in Super mode
– SharedDatabase = false
– SharedCache = true

Prague 2014Prague 2014 Firebird 3Firebird 39
Super Server
● Main difference from Classic
● All attachments are handled by the single process
● Shared common data
– metadata cache
– page cache
– transaction inventory cache
● Hand-made scheduler: cooperative multitasking
– single scheduler, single active thread in process
– threads voluntary switched control to each other
● Metadata still synchronized by LM
● Page cache not used LM
– individual “latches” for page buffers

Prague 2014Prague 2014 Firebird 3Firebird 310
Super Server before Firebird3
● Pluses
● shared caches - used much less memory than lot of Classic
processes
● single active thread - almost no need to synchronize
concurrent access to in-memory structures
● Minuses
● single active thread - no way to use more than one CPUcore
– SMP is not possible
● cooperative multitasking is very far from fair scheduling
– “heavy” query in one attachment could get more CPU and
slowdown queries in another attachments

Prague 2014Prague 2014 Firebird 3Firebird 311
Super Server in Firebird3
● On the way to the true Super Server
● shared page cache
● shared TIP cache
● metadata cache is not shared (private per attachment)
● no more custom scheduler, all threads works in parallel
– SMP is supported now
● new synchronization schema

Prague 2014Prague 2014 Firebird 3Firebird 312
Page Cache
● Page buffers
● used to cache contents of database file pages
– descriptor of page buffer, data buffer itself
● Control structures
● Hash table
– used to fast search for page buffer by given page number
● LRU queue
– defines page buffer eviction policy
● Precedence graph
– used to implement “careful write” policy
● Dirty pages queue
– what to write on commit

Prague 2014Prague 2014 Firebird 3Firebird 313
Page Cache
● Synchronization
● individual latch for every page buffer
– latch is a lightweight RW-lock
– used always (despite of SharedCache setting)
● individual Lock Manager's lock for every page buffer
– used in SharedCache = false (Classic) mode only
● separate RW-lock for every control structure
– hash table
– LRU queue
– precedence graph
– dirty queue

Prague 2014Prague 2014 Firebird 3Firebird 314
Page Cache
● Fine-grained synchronization is not free !
● Every page access consists of following steps
● fetch page buffer
– read contents from disk, if necessary
● access (use) page buffer
– read record data, for example
● release page buffer

Prague 2014Prague 2014 Firebird 3Firebird 315
Page Cache
● Cost of page access in Super Server, before v3
● fetch page
– find buffer in hash table
– if buffer is not used - increment use_count
– else - voluntary switch to another thread and wait while current
buffer owner granted us access
● release page
– decrement use_count
– grant access to waiting tread(s)
● Minimal cost is two changes of use_count
– very, very cheap

Prague 2014Prague 2014 Firebird 3Firebird 316
Page Cache
● Cost of page access in Super Server, in v3
● fetch page
– acquire lock for hash table
– find buffer in hash table
– release lock for hash table
– acquire latch for page buffer
● interlocked increment of latch state,
● or wait while current latch owner granted us access
● release page
– release latch for page buffer
● interlocked decrement of latch state
– grant access to waiting tread(s)
● Minimal cost is four interlocked operations

Prague 2014Prague 2014 Firebird 3Firebird 317
Benchmark, single thread read
Not cached Firebird File System
0
10
20
30
40
50
60
70
80
90
100
83
7
10
92
10 13
Firebird 2.5
Firebird 3.0
Time,sec
SELECT COUNT(*) FROM STOCK
10'000'000 rows
Memory buffers = 500'000
Reads from disk to cache = 418'604 ' not cached
Reads from disk to cache = 0 ' cached by Firebird
Reads from disk to cache = 418'822 ' cached by File System
Fetches from cache = 20'837'432

Prague 2014Prague 2014 Firebird 3Firebird 318
Benchmark, single thread read
Not cached Firebird File System
0
20
40
60
80
100
120
104
15 17
101
18 20
Firebird 2.5
Firebird 3.0
Time,sec
SELECT COUNT(DISTINCT S_W_ID), COUNT(DISTINCT S_I_ID) FROM STOCK
10'000'000 rows
Memory buffers = 500'000
Reads from disk to cache = 418'604 ' not cached
Reads from disk to cache = 0 ' cached by Firebird
Reads from disk to cache = 418'822 ' cached by File System
Fetches from cache = 20'837'432

Prague 2014Prague 2014 Firebird 3Firebird 319
Benchmark, multithreaded read
● Multithreaded read only benchmark
● Table STOCK have 10'000'000 records
– 100 warehouses, 100'000 items in each
● Each reader reads random items in own warehouse
● We don't test network subsystem
– client application executes procedure, which selects a number
of random items (100 by default)
● We don't test IO subsystem
– before each test series we ensure whole table STOCK is fully
cached by filesysem

Prague 2014Prague 2014 Firebird 3Firebird 320
1 2 4 8 16 32 64
0
50
100
150
200
250
300
350
400
450 414 416 411 408 403
384
365377 382
359
217
168
154 142
Super Server, page cache 2048 Firebird 2.5
Firebird 3.0
Threads
Count
Benchmark, multithreaded read
1 2 4 8 16 32 64
0
50
100
150
200
250
300
350
400
450
Threads
Count

Prague 2014Prague 2014 Firebird 3Firebird 321
Benchmark, multithreaded read
1 2 4 8 16 32 64
0
100
200
300
400
500
600
700
Threads
Count
1 2 4 8 16 32 64
0
100
200
300
400
500
600
700
414 416 411 408 403 384 365377 382
359
217
168 154 142
390
616
491
324
219
190 176
Super Server, page cache 2048 Firebird 2.5
FB 3.0.0.31369
FB 3.0.0.31385
Threads
Count

Prague 2014Prague 2014 Firebird 3Firebird 322
Benchmark, multithreaded read
1 2 4 8 16 32 64
0
100
200
300
400
500
600
700
Threads
Count
1 2 4 8 16 32 64
0
100
200
300
400
500
600
700
414 416 411 408 403 384 365377 382
359
217
168 154 142
371
612
538
466
383
323
291
Super Server, page cache 2048 Firebird 2.5
FB 3.0.0.31369
FB3 hash table patch
Threads
Count

Prague 2014Prague 2014 Firebird 3Firebird 323
1 2 4 8 16 32 64
0
100
200
300
400
500
600
700
350
527
620
569 561
535
513
330
476
572
550 533
498
451
Super Classic, page cache 256 Firebird 2.5
Firebird 3.0
Threads
Count
Benchmark, multithreaded read
1 2 4 8 16 32 64
0
100
200
300
400
500
600
700
Threads
Count

Prague 2014Prague 2014 Firebird 3Firebird 324
Benchmark, multithreaded read
1 2 4 8 16 32 64
0
500
1000
1500
2000
873 867 833 811 803 799 787766
1208
1794 1735 1714
1641
1541
Super Server, page cache 500'000 Firebird 2.5
Firebird 3.0
Threads
Count
1 2 4 8 16 32 64
0
500
1000
1500
2000
Threads
Count

Prague 2014Prague 2014 Firebird 3Firebird 325
1 2 4 8 16 32 64
0
5000
10000
15000
20000
25000
30000
5271
6745
10660
11091
10452 10429
9946
8081
12732
15519 15786
12465
9013
7947
11437
18390
25631
23647
19203
15275
10469
2.5 Super Server
2.5 Super Classic
3.0 Super Server
Terminals
tpmC
Benchmark, TPCC

Prague 2014Prague 2014 Firebird 3Firebird 326
Benchmark, TPCC
1 2 4 8 16 32 64
0
5000
10000
15000
20000
25000
30000 2.5 Super Server
2.5 Super Classic
3.0 Super Server
Terminals
tpmC

Prague 2014Prague 2014 Firebird 3Firebird 327
Questions ?Questions ?
Firebird official web site
Firebird tracker
THANK YOU FOR ATTENTIONTHANK YOU FOR ATTENTION
hvlad@users.sf.net