Resolving Firebird performance problems

53,305 views
52,399 views

Published on

In this presentation we consider how to resolve Firebird performance problems: what Firebird database parameters we need to monitor and how we need to tune Firebird configuration and adjust client applications.

Published in: Software, Technology, Sports
1 Comment
28 Likes
Statistics
Notes
  • I cannot understand sheet 4: '• Should we use OS cache? #FileSystemCacheThreshold = 65536 – use OS only cache #FileSystemCacheThreshold = 65536 – do not use OS cache'
    I think it has to be '• Should we use OS cache? #FileSystemCacheThreshold = 65536 – use OS only cache #FileSystemCacheThreshold = 0 – do not use OS cache'
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
53,305
On SlideShare
0
From Embeds
0
Number of Embeds
1,896
Actions
Shares
0
Downloads
573
Comments
1
Likes
28
Embeds 0
No embeds

No notes for slide

Resolving Firebird performance problems

  1. 1. Resolving Firebird Performance Problems © IBSurgeon www.ib-aid.com
  2. 2. Usually big production database runs at • Firebird 2.5 • Classic or SuperClassic architecture • They can scale accross multiple CPU/cores! • In Firebird 3 SuperServer is the better choice… • but now let’s talk about 2.5 Classic and SuperClassic
  3. 3. What are performance indicators to monitor in Firebird 2.5? 1. Lock table parameters 2. Monitoring record versions and garbage collection 3. Transactions 1. Long running transactions 2. Rolled back transactions 4. Temporary files – quantity and size 5. Connections, SQL queries and transactions
  4. 4. LOCK TABLE MONITORING
  5. 5. fb_inet_serverfb_inet_server How lock table works fb_inet_server Firebird Page cache Page cache Page cache Page X How to resolve a write conflict when changing the same page?
  6. 6. How lock table works fb_inet_server fb_inet_server fb_inet_server mapped Lock table mapped mapped shared memory All requests to access internal database objects go through lock table
  7. 7. Why should we monitor lock table parameters? • Lock table is a critical part of Classic and SuperClassic architectures • Access to shared objects is implemented through locks in Lock table… Analysis of lock table parameters is the easiest way to reveal problems with concurrent access
  8. 8. Lock table analysis - raw • Fb_lock_print –d <database_name> • LOCK_HEADER BLOCK • Version: 17, Active owner: 0, Length: 6291456, Used: 5517236 • Flags: 0x0001 • Enqs: 10906251, Converts: 58907, Rejects: 22373, Blocks: 210859 • Deadlock scans: 5841, Deadlocks: 0, Scan interval: 10 • Acquires: 13636997, Acquire blocks: 558879, Spin count: 0 • Mutex wait: 4.1% • Hash slots: 2003, Hash lengths (min/avg/max): 2/ 11/ 26 • Remove node: 0, Insert queue: 0, Insert prior: 0 • Owners (107): forward: 26696, backward: 5517140 • Free owners: *empty* • Free locks (1630): forward: 3878196, backward: 2264580 • Free requests (793): forward: 5412916, backward: 1906516 • Lock Ordering: Enabled
  9. 9. Lock table analysis – where to look • Fb_lock_print –d <database_name> • LOCK_HEADER BLOCK • Version: 17, Active owner: 0, Length: 6291456, Used: 5517236 • Flags: 0x0001 • Enqs: 10906251, Converts: 58907, Rejects: 22373, Blocks: 210859 • Deadlock scans: 5841, Deadlocks: 0, Scan interval: 10 • Acquires: 13636997, Acquire blocks: 558879, Spin count: 0 • Mutex wait: 4.1% • Hash slots: 2003, Hash lengths (min/avg/max): 2/ 11/ 26 • Remove node: 0, Insert queue: 0, Insert prior: 0 • Owners (107): forward: 26696, backward: 5517140 • Free owners: *empty* • Free locks (1630): forward: 3878196, backward: 2264580 • Free requests (793): forward: 5412916, backward: 1906516 • Lock Ordering: Enabled
  10. 10. Thresholds and recommendations Essential values: Firebird.conf params to tune: Length: 6291456 LockMemSize - default is 1Mb Set at least: Cache_pages * max_connections_count * 100 Hash slots: 2003, Hash lengths (min/avg/max): 2/ 11/ 26 LockHashSlots = 1009 Prime number. Set x20, like 20011 Mutex wait: 4.1% No parameter to adjust. More than 10% could be a problem, check hardware and concurrency settings Owners (107) Number of connections for server Use optimized Firebird configuration files from IBSurgeon: http://ib-aid.com/en/optimized-firebird-configuration/
  11. 11. How to monitor lock table • 1) Command prompt (on the server only), run every N min fb_lock_print -d E:OLTP-EMULoltp30.fdb • 2) Alerts and automatic recommendations
  12. 12. Monitoring record versions and garbage collection in Firebird
  13. 13. What is a record version? TR50 TR60 UPDATE set data =200read N Tx Data 1 10 100 60 200 ... read write Different transactions can see different versions of the record.
  14. 14. Why many record versions are bad The more versions record has, the more scan operations server does to find necessary version After data load or restore – no versions
  15. 15. • When UPDATE changes indexed fields, indices also must be updated, and - UPDATE does not update keys in the index, it adds new keys for new record version! • DELETE does not delete index keys! N Tx Data 1 10 100 60 200 ... Index key = 100 Index key = 200 Why many record versions are bad
  16. 16. What to monitor? Database statistics • Database statistics shows information about Record Versions and Max Versions • Taking database statistics can be long!
  17. 17. How to monitor 1. With gstat command line tool 2. With HQbird Firebird Database Analyst More details: https://www.youtube.com/watch?v=_KZqXcgwN98
  18. 18. Goal of every Firebird developer is to avoid creating versions for records! Not only because server slowly reads multiple record versions, but also because of GARBAGE COLLECTION
  19. 19. What is garbage and how it is collected • When record versions become obsolete and non- interested by any transaction, it is considered as garbage and need to be cleaned. • Garbage collection is a process of removing unnecessary records versions • It can be cooperative (Classic or SuperClassic) or background (SuperServer)
  20. 20. Why should we monitor garbage collection? • It consumes resources. • We should locate and fix those parts of the application(s) which produce many record versions which increase an intensity of GC
  21. 21. How to monitor GC? • 1) Manual SQL queries to MON$ tables • 2) HQbird Firebird MonLogger
  22. 22. How to decrease number of garbage versions? 1. Use as short write transactions as possible 2. For read-only operations use (if possible) transactions READ COMMITTED READ-ONLY If you see a lot of record versions, and you are not sure how to fix it, consider to order our Database Optimization service: http://ib-aid.com/en/firebird-interbase-performance- optimization-service/
  23. 23. Monitoring transactions in Firebird
  24. 24. Why to monitor transactions? Transactions are key indicators of versioning processes in Firebird database. 2 main problems 1. Long-running [writeable] transactions 2. Rolled back transactions
  25. 25. Gstat –h gives us snaphot of transactions Database header page information: Flags 0 Checksum 12345 Generation 1564 Page size 4096 ODS version 10.1 Oldest transaction 10009 Oldest active 20001 Oldest snapshot 20001 Next transaction 25007 Bumped transaction 1 Sequence number 0 Next attachment ID 1 Implementation ID 16 Shadow count 0 Page buffers 3000 Next header page 0 Database dialect 1 Creation date Nov 1, 2015 15:55:55
  26. 26. Long-running transactions • All transactions have sequential numbers from 1 to… • Oldest Active Transaction – currently active transaction with the minimal number Oldest transaction 10009 Oldest active 20001 Oldest snapshot 20001 Next transaction 25007 Interval NEXT – OAT shows number of potentially active transactions – server cannot clean versions created by transactions with numbers > OAT
  27. 27. How to track long running active transactions in Firebird? • 1. Manual tracking of gstat –h output (check every N minutes) • 2. Alerts from HQbird FBDataGuard
  28. 28. How to fix long running active transactions in Firebird? It is possible to find who is running long- running transaction and CANCEL it 1. Manual query to MON$ tables 2. HQbird Firebird MON$Logger (GUI)
  29. 29. Rolled back transactions • When some transaction is marked in transaction inventory as rolled back, it prevents record versions beeing cleaned by collective or background garbage collection Oldest transaction 10009 Oldest active 20001 Oldest snapshot 20001 Next transaction 25007 Interval Oldest Snapshot – Oldest Interesting determines the need for sweep
  30. 30. Sweep • Sweep reads whole database from the beginning to the end and cleans garbage record versions • Sweep is necessary when OST-OIT interval becomes big
  31. 31. How to make sweep • Autosweep • by default 20000 • Starts immediately when interval > threshold • It causes slowness at unpredicted moments • Manual sweep • Scheduled sweep during the inactivity period of time • Can be run manually with gfix –sweep or scheduled in HQbird FBDataGuard
  32. 32. Sweep must be controlled! • If sweep did not succeed to align transaction markers, it can happen due to the serious problem/corruption! • HQbird FBDataGuard checks sweep status
  33. 33. TEMP FILES
  34. 34. Temp files • Temp files are created in the default temp folder or in folders specified by TempDirectories in firebird.conf • Actually they are written to the disk only if size of all sort files is more than TempCacheLimit parameter • It is better to have sorting in memory
  35. 35. How to track Temp files 1. Manually check size and quantity of temp files (fb_sortxxx) in all temp folders 2. HQbird FBDataGuard monitors temp files size and quantity, and sends warnings if necessary
  36. 36. How to move temp files to memory • Increase TempCacheLimit parameter • Warnings: • At Classic TempCacheLimit is allocated for each process • 32bit processes have 2Gb limitation of memory to address Use optimized configuration files from IBSurgeon and HQbird! http://ib-aid.com/en/optimized-firebird-configuration/
  37. 37. Monitor connections, transactions, and SQL queries
  38. 38. Why to monitor Firebird connections? • Number of connections should be constantly monitored: • Peaks of connections • Firebird Classic/SuperClassic consume RAM as Number_Of_Connections x Page Buffers • In case of peaks can consume all RAM and go to swap – slowness! • Application can do more than 1 connection to the database • By mistake • By design • In case of connection pool – adjust the size of the pool and its increment • Suspicious connections • Long running connections and connections from developer tools • Connections from non-standard applications
  39. 39. How to monitor connections 1. Manually run SELECT count(*) FROM MON$ATTACHMENTS 2. Use HQbird 1. FBDataGuard constantly monitors number of connections and send alerts if their number grows too much 2. MonLogger can be used to go deep and get detailed information about connections: memory, IO, garbage collection
  40. 40. Transactions • Yes, again! • Besides monitoring of intervals between transactions markers (Next-OAT, OIT-OST), also we should check these important things, which can affect Firebird performance: 1. Transaction’s parameters (isolation level, wait/nowait) 2. Transactions flows and their relation with business logic 3. Transaction-level statistics (IO, time to live)
  41. 41. Transaction’s parameters • Transactions parameters must correspond to business logic. • Several often mistakes • Non-modyfing queries (SELECT) are started in read-write mode • Should be in READ ONLY • Snapshot are used for operational queries, instead of read- commited • It should be used only for queries where consistency is really required (snapshots)
  42. 42. Transactions flows mistakes • Transaction (other then read-committed read only) never ends, expires due to connection closing, and leads to ROLLBACK • Only 1 sentence in the transaction • For example, refreshing query with timer: • Quickly exhausts 2 billion limit of transactions • it should be run in permanent read committed read only transaction • Using the only read-write transaction for all operations • Better split read-only and write operations
  43. 43. How to track transactions flows and transactions parameters? - FBScanner HQbird FBScanner allows to track transactions flows: parameters, containing queries, plans, etc.
  44. 44. How to track transactions flows and transactions parameters? - PerfMon • HQbird Firebird Performance Monitor (PerfMon) uses TraceAPI to capture all engine events – connections, queries, transactions, stored procedures, triggers, execution plans, execution statistics • Works correctly only at Firebird 2.5.3+ versions
  45. 45. Quick check of problems with transactions • HQbird MonLogger gives a fast and detailed overview of active transactions in the system
  46. 46. HQbird tools for SQL and transactions monitoring and optimization HQbird FBScanner + works as a proxy, non-invasive tool, can be installed on separate server and track only selected workstations — cannot track stored procedures and triggers HQbird PerfMon + works with TraceAPI, can monitor everything — TraceAPI significantly supresses performance HQbird MonLogger + Great ad-hoc tool, fast — Captures only snapshots, lack of important information
  47. 47. IBSurgeon Firebird Optimization service We can increase the performance of your database http://ib-aid.com/en/firebird-interbase-performance- optimization-service/
  48. 48. Thank you and don’t forget these links: IBSurgeon optimized configuration files http://ib-aid.com/en/optimized-firebird-configuration/ Firebird Hardware Guide http://ib-aid.com/download/docs/Firebird_Hardware_Guide_IBSurgeon.pdf Contact us: support@ib-aid.com

×