Resolving Firebird
Performance Problems
© IBSurgeon
www.ib-aid.com

Usually big production database runs at
• Firebird 2.5
• Classic or SuperClassic architecture
• They can scale accross multiple CPU/cores!
• In Firebird 3 SuperServer is the better choice…
• but now let’s talk about 2.5 Classic and SuperClassic

What are performance indicators to
monitor in Firebird 2.5?
1. Lock table parameters
2. Monitoring record versions and garbage collection
3. Transactions
1. Long running transactions
2. Rolled back transactions
4. Temporary files – quantity and size
5. Connections, SQL queries and transactions

LOCK TABLE MONITORING

fb_inet_serverfb_inet_server
How lock table works
fb_inet_server
Firebird
Page cache Page cache Page cache
Page X
How to resolve a
write conflict when
changing the same
page?

How lock table works
fb_inet_server fb_inet_server fb_inet_server
mapped
Lock table
mapped mapped
shared memory
All requests to access
internal database objects go
through lock table

Why should we monitor lock table
parameters?
• Lock table is a critical part of Classic and SuperClassic
architectures
• Access to shared objects is implemented through locks in
Lock table…
Analysis of lock table parameters is the easiest
way to reveal problems with concurrent access

Lock table analysis - raw
• Fb_lock_print –d <database_name>
• LOCK_HEADER BLOCK
• Version: 17, Active owner: 0, Length: 6291456, Used: 5517236
• Flags: 0x0001
• Enqs: 10906251, Converts: 58907, Rejects: 22373, Blocks:
210859
• Deadlock scans: 5841, Deadlocks: 0, Scan interval: 10
• Acquires: 13636997, Acquire blocks: 558879, Spin count: 0
• Mutex wait: 4.1%
• Hash slots: 2003, Hash lengths (min/avg/max): 2/ 11/ 26
• Remove node: 0, Insert queue: 0, Insert prior: 0
• Owners (107): forward: 26696, backward: 5517140
• Free owners: *empty*
• Free locks (1630): forward: 3878196, backward: 2264580
• Free requests (793): forward: 5412916, backward: 1906516
• Lock Ordering: Enabled

Lock table analysis – where to look
• Fb_lock_print –d <database_name>
• LOCK_HEADER BLOCK
• Version: 17, Active owner: 0, Length: 6291456, Used: 5517236
• Flags: 0x0001
• Enqs: 10906251, Converts: 58907, Rejects: 22373, Blocks:
210859
• Deadlock scans: 5841, Deadlocks: 0, Scan interval: 10
• Acquires: 13636997, Acquire blocks: 558879, Spin count: 0
• Mutex wait: 4.1%
• Hash slots: 2003, Hash lengths (min/avg/max): 2/ 11/ 26
• Remove node: 0, Insert queue: 0, Insert prior: 0
• Owners (107): forward: 26696, backward: 5517140
• Free owners: *empty*
• Free locks (1630): forward: 3878196, backward: 2264580
• Free requests (793): forward: 5412916, backward: 1906516
• Lock Ordering: Enabled

Thresholds and recommendations
Essential values: Firebird.conf params to tune:
Length: 6291456 LockMemSize - default is 1Mb
Set at least:
Cache_pages * max_connections_count
* 100
Hash slots: 2003, Hash lengths
(min/avg/max): 2/ 11/ 26
LockHashSlots = 1009
Prime number. Set x20, like 20011
Mutex wait: 4.1% No parameter to adjust. More than 10%
could be a problem, check hardware and
concurrency settings
Owners (107) Number of connections for server
Use optimized Firebird configuration files from IBSurgeon:
http://ib-aid.com/en/optimized-firebird-configuration/

How to monitor lock table
• 1) Command prompt (on the server only), run every N min
fb_lock_print -d E:OLTP-EMULoltp30.fdb
• 2) Alerts and automatic
recommendations

Monitoring record versions and
garbage collection in Firebird

What is a record version?
TR50 TR60
UPDATE
set data =200read
N Tx Data
1 10 100
60 200
...
read
write
Different transactions can see different versions of
the record.

Why many record versions are bad
The more
versions record
has, the more
scan operations
server does to
find necessary
version
After data load or
restore – no
versions

• When UPDATE changes indexed fields, indices also must
be updated, and - UPDATE does not update keys in the
index, it adds new keys for new record version!
• DELETE does not delete index keys!
N Tx Data
1 10 100
60 200
...
Index key
= 100
Index key
= 200
Why many record versions are bad

What to monitor?
Database statistics
• Database statistics shows information about Record
Versions and Max Versions
• Taking database statistics can be long!

How to monitor
1. With gstat command line tool
2. With HQbird Firebird Database Analyst
More details: https://www.youtube.com/watch?v=_KZqXcgwN98

Goal of every Firebird developer is
to avoid creating versions for records!
Not only because server slowly reads
multiple record versions, but also
because of
GARBAGE COLLECTION

What is garbage and how it is collected
• When record versions become obsolete and non-
interested by any transaction, it is considered as
garbage and need to be cleaned.
• Garbage collection is a process of removing
unnecessary records versions
• It can be cooperative (Classic or SuperClassic) or
background (SuperServer)

Why should we monitor garbage
collection?
• It consumes resources.
• We should locate and fix those parts of the
application(s) which produce many record
versions which increase an intensity of GC

How to monitor GC?
• 1) Manual SQL queries to MON$ tables
• 2) HQbird Firebird MonLogger

How to decrease number of garbage
versions?
1. Use as short write transactions as possible
2. For read-only operations use (if possible)
transactions READ COMMITTED READ-ONLY
If you see a lot of record versions, and you are not
sure how to fix it, consider to order our Database
Optimization service:
http://ib-aid.com/en/firebird-interbase-performance-
optimization-service/

Monitoring transactions in
Firebird

Why to monitor transactions?
Transactions are key indicators of versioning
processes in Firebird database.
2 main problems
1. Long-running [writeable] transactions
2. Rolled back transactions

Gstat –h gives us snaphot of transactions
Database header page information:
Flags 0
Checksum 12345
Generation 1564
Page size 4096
ODS version 10.1
Oldest transaction 10009
Oldest active 20001
Oldest snapshot 20001
Next transaction 25007
Bumped transaction 1
Sequence number 0
Next attachment ID 1
Implementation ID 16
Shadow count 0
Page buffers 3000
Next header page 0
Database dialect 1
Creation date Nov 1, 2015 15:55:55

Long-running transactions
• All transactions have sequential numbers from 1 to…
• Oldest Active Transaction – currently active transaction
with the minimal number
Oldest transaction 10009
Oldest active 20001
Oldest snapshot 20001
Next transaction 25007
Interval NEXT – OAT shows number of potentially active
transactions – server cannot clean versions created by
transactions with numbers > OAT

How to track long running active transactions
in Firebird?
• 1. Manual tracking of gstat –h output (check every N
minutes)
• 2. Alerts from HQbird FBDataGuard

How to fix long running active
transactions in Firebird?
It is possible to find who is running long-
running transaction and CANCEL it
1. Manual query to MON$ tables
2. HQbird Firebird MON$Logger (GUI)

Rolled back transactions
• When some transaction is marked in transaction inventory
as rolled back, it prevents record versions beeing
cleaned by collective or background garbage collection
Oldest transaction 10009
Oldest active 20001
Oldest snapshot 20001
Next transaction 25007
Interval Oldest Snapshot – Oldest Interesting
determines the need for sweep

Sweep
• Sweep reads whole database from the beginning
to the end and cleans garbage record versions
• Sweep is necessary when OST-OIT interval
becomes big

How to make sweep
• Autosweep
• by default 20000
• Starts immediately when interval > threshold
• It causes slowness at unpredicted moments
• Manual sweep
• Scheduled sweep during the inactivity period of
time
• Can be run manually with gfix –sweep or
scheduled in HQbird FBDataGuard

Sweep must be controlled!
• If sweep did not succeed to align
transaction markers, it can happen due to
the serious problem/corruption!
• HQbird FBDataGuard checks sweep status

TEMP FILES

Temp files
• Temp files are created in the default temp folder
or in folders specified by TempDirectories in
firebird.conf
• Actually they are written to the disk only if size of
all sort files is more than TempCacheLimit
parameter
• It is better to have sorting in memory

How to track Temp files
1. Manually check size and quantity of temp files
(fb_sortxxx) in all temp folders
2. HQbird FBDataGuard monitors temp files size and
quantity, and sends warnings if necessary

How to move temp files to memory
• Increase TempCacheLimit parameter
• Warnings:
• At Classic TempCacheLimit is allocated for each
process
• 32bit processes have 2Gb limitation of memory to
address
Use optimized configuration files from IBSurgeon and
HQbird!
http://ib-aid.com/en/optimized-firebird-configuration/

Monitor connections,
transactions, and SQL queries

Why to monitor Firebird connections?
• Number of connections should be constantly monitored:
• Peaks of connections
• Firebird Classic/SuperClassic consume RAM as
Number_Of_Connections x Page Buffers
• In case of peaks can consume all RAM and go to swap – slowness!
• Application can do more than 1 connection to the database
• By mistake
• By design
• In case of connection pool – adjust the size of the pool and its
increment
• Suspicious connections
• Long running connections and connections from developer tools
• Connections from non-standard applications

How to monitor connections
1. Manually run SELECT count(*) FROM
MON$ATTACHMENTS
2. Use HQbird
1. FBDataGuard constantly monitors number of connections and
send alerts if their number grows too much
2. MonLogger can be used to go deep and get detailed information
about connections: memory, IO, garbage collection

Transactions
• Yes, again!
• Besides monitoring of intervals between transactions
markers (Next-OAT, OIT-OST), also we should check
these important things, which can affect Firebird
performance:
1. Transaction’s parameters (isolation level, wait/nowait)
2. Transactions flows and their relation with business
logic
3. Transaction-level statistics (IO, time to live)

Transaction’s parameters
• Transactions parameters must correspond to business
logic.
• Several often mistakes
• Non-modyfing queries (SELECT) are started in read-write mode
• Should be in READ ONLY
• Snapshot are used for operational queries, instead of read-
commited
• It should be used only for queries where consistency is really required
(snapshots)

Transactions flows mistakes
• Transaction (other then read-committed read only) never
ends, expires due to connection closing, and leads to
ROLLBACK
• Only 1 sentence in the transaction
• For example, refreshing query with timer:
• Quickly exhausts 2 billion limit of transactions
• it should be run in permanent read committed read only transaction
• Using the only read-write transaction for all operations
• Better split read-only and write operations

How to track transactions flows and
transactions parameters? - FBScanner
HQbird FBScanner allows to track transactions
flows: parameters, containing queries, plans, etc.

How to track transactions flows and
transactions parameters? - PerfMon
• HQbird Firebird Performance Monitor (PerfMon) uses
TraceAPI to capture all engine events – connections,
queries, transactions, stored procedures, triggers,
execution plans, execution statistics
• Works correctly only at Firebird 2.5.3+ versions

Quick check of problems with transactions
• HQbird MonLogger gives a fast and detailed overview of
active transactions in the system

HQbird tools for SQL and transactions
monitoring and optimization
HQbird FBScanner
+ works as a proxy, non-invasive tool, can be installed on
separate server and track only selected workstations
— cannot track stored procedures and triggers
HQbird PerfMon
+ works with TraceAPI, can monitor everything
— TraceAPI significantly supresses performance
HQbird MonLogger
+ Great ad-hoc tool, fast
— Captures only snapshots, lack of important information

IBSurgeon Firebird Optimization service
We can increase the performance of your database
http://ib-aid.com/en/firebird-interbase-performance-
optimization-service/

Thank you and don’t forget these links:
IBSurgeon optimized configuration files
http://ib-aid.com/en/optimized-firebird-configuration/
Firebird Hardware Guide
http://ib-aid.com/download/docs/Firebird_Hardware_Guide_IBSurgeon.pdf
Contact us: support@ib-aid.com