Life with big Firebird databases

5,882 views
5,653 views

Published on

There are more and more companies have big Firebird databases, from 100Gb till 1Tb. Maintenance and optimization tasks for such databases are different from small, and database administrators need take into account several important things about big Firebird databases.

Published in: Technology
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,882
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
65
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Life with big Firebird databases

  1. 1. Life with big Firebird databases Alexey Kovyazin www.IBSurgeon.com
  2. 2. 2 • Tools and consulting • Platinum Sponsor of Firebird Project • Founded in 2002: 12 years of Firebird and InterBase recoveries and consulting • Based in Moscow, Russia • Firebase is our Brazil partner (since 2006)
  3. 3. Big databases 3 Series of performance tests with databases 9Gb... 30Gb... 1.7Tb http://ib-aid.com/en/articles/firebird-performance-degradation-tests-myths-and-truth/
  4. 4. Non-optimized performance 4 results
  5. 5. 5 Optimized test results
  6. 6. 6 What we did • SuperServer → Classic or SuperClassic • Tuned Firebird config parameters Optimized config are available here: http://ib-aid.com/en/optimized-firebird-configuration/
  7. 7. 7 The difference between big and small databases • Small – Optimize your SQLs – In case of any trouble do backup/restore (gbak) • Big – Where is the problem? – Why it happens? – Backup/restore takes 1 days... Will it solve the problem or it will be waste of time?
  8. 8. 8 Maintenance tasks • Backups • Performance monitoring – Transactions – SQLs and plans – Database structure
  9. 9. 9 Backups • Usual backup: new backup rewrites old backup Firebird database New backup Old backup
  10. 10. 10 Wrong! • If backup fails, there will be no good old backup, no new backup, and corrupted database Firebird database New backup Old backup
  11. 11. 11 Correct schema Firebird database New backup Finished backup Old backup Database space for backups: (Number_of_stored_backups+1)*Size
  12. 12. 12 Gbak vs Nbackup • NBACKUP + fast - does not check data - not reliable till 2.5.2 - delta problems • Gbak + checks data - slow (and speed degrades) - test restore needed
  13. 13. 13 Big databases backup schema Requirements ● delta monitoring ● health monitoring Requirements: ● gbak backup ● test restore Production server Backup server Nbackup gbak
  14. 14. 14 Big databases backup schema FBDataGuard 1 FBDataGuard 2 Production server Backup server Nbackup gbak
  15. 15. 15 FBDataGuard • Backups (in correct way) • Health checks of the database • Performance parameters monitoring – Transactions – Users, Indices, database statistics • Corruption monitoring • Email alerts
  16. 16. 16 Health monitoring • Critical metadata check and backup – Can be used for recovery with FirstAID • Database limitations checks – Transactions limits, format limits, space limits • Errors – Firebird.log
  17. 17. 17 Indices maintenance • Indices health – Idea of statistics – Recalculation of indices statistics • Manual • Automatic
  18. 18. 18 Demo
  19. 19. 19 Performance monitoring
  20. 20. 20 Tasks and Tools • SQL queries and plans: FBScanner and FBPerfMon • Transactions: FBDataGuard (alerts, database statistics) and IBTM (visualization) • Database structure analysys: IBAnalyst
  21. 21. 21 Recommended setup for monitoring Firebird Production server Users FBScanner remote server Administrator has: 1) FBScanner Analyzer 2) MON$Logger 3) TraceAPI 4) IBTM 5) IBAnalyst MON$ TraceAPI FBDataGuard & Trace API
  22. 22. Examples of problem resolution 22 Long running active transaction — prevent performance problem 1.Administrator got email from FBDataGuard about transaction gap 2.IBTM shows long running active transaction 3.MON$ logger identified source of transaction, and admin disconnected client
  23. 23. 23 Transactions monitoring • Transaction markers (NEXT, OIT, OST, OAT) dynamics is good indicator of problems – Long running active transactions (stuck OAT) – Forced Rollback (stuck OIT) – Sweep and autosweep success
  24. 24. 24
  25. 25. MON$ 25
  26. 26. Examples of problem resolution 26 Bad performance — fix performance problem – User reported problem with performance in application A – Quick check with MON$logger does not show the problem – Admin analysed logs from FBScanner (permanent monitoring) and found bad query, reported to developer, with SQL plan and fresh gstat statistics from FBDataGuard – Developer checked gstat and plan, found that new index is needed – Admin applied index, performance is good again
  27. 27. 27 SQL Queries and Plans • For all Firebird versions — FBScanner – Works as a proxy – Connection string changes required – Can be installed on remote server • For Firebird 2.5 — FBPerfMon – Mon$ support – TraceAPI support
  28. 28. Trace 28
  29. 29. 29 FBScanner Log
  30. 30. Database structure 30
  31. 31. 31 Conclusion • To sleep well and live long you need: – Relialble backups – Continious monitoring and alerts – Quick identification of problem – Full details how to fix it
  32. 32. 32 Backups Health monitoring Typical tasks Indices maintenance Transactions dynamics SQL Queries & plans Database structure analysys Database
  33. 33. 33 FBDataGuard IBTM FBScanner & FBPerfMon IBAnalyst
  34. 34. 34 Only at FDD 2014 IBSurgeon Optimization Pack 5-in-1 (FBDataGuard+FBScanner+FBPerfMon +IBTM +IBAnalyst) • EUR 299 R$400 per server • EUR 1290 R$850 Unlimited Subscription 1 year • EUR 349 R$400 FirstAID 50 database
  35. 35. 35 • Thank you! • Questions? Web: www.ib-aid.com www.ibsurgeon.com Email: support@ib-aid.com

×