This document describes the design and rationale of a database layer for the SAINT software. It addresses problems with using different SQL dialects and limitations of data types across databases. The solution involves creating extended Qt SQL drivers that generate SQL queries without hardcoding actual SQL. This allows manipulating database structures and using different databases without code changes. It also describes a unified database selection UI and classes for managing database connections and transactions.

1. April 20, 2011 Qt SQL module SAINT adaptations & workarounds André Somers | 1

2. André Somers | 2 | SAINT database layer : Design and Rationale Database use case Situation: All data stored in Relational Database, one database foreachstudy OriginallyusingMS Access Making database independent; first target MySql Problem: SQL dialects Different names and limitationsfor data types Solution: No actual SQL code in code Query textsgeneratedby (extended) SQL driverforQt

3. André Somers | 3 | SAINT database layer : Design and Rationale Qt & Databases Qt drivers do not support DDL by default No way to generate SQL to manipulate data structures Solution: custom extension to Qt SQL driver model Qt SQL drivers are plugins New drivers subclassed from standard driver (so they can still be used where you’d otherwise use other drivers), but have an extended base to allow for more functionality.

4. André Somers | 4 | SAINT database layer : Design and Rationale SAINT extended Qt DB drivers Standard Qt SAINT extended drivers QSqlDriver SqlDriverExt QODBCDriver AccessDriver QMySqlDriver MySqlDriver Other driver Future other drivers

5. André Somers | 5 | SAINT database layer : Design and Rationale SAINT extended Qt DB drivers 2 //createtable (simplified) QSqlRecordrecord; foreach (QSqlField field, table->fieldList) { record.append(field); } foreach(QStringrelationName, table->fkRelations) { Relation* rel = m_relations.value(relationName); QSqlFieldfkField(QString("%1_%2") .arg(tableName(rel->tableOne)) .arg(rel->fieldOne), QVariant::UInt); fkField.setLength(30); record.append(fkField); } sql= m_sqlDriver->sqlStatement(SqlDriverExt::CreateTableStatement, table->name, record); QSqlQueryquery(sql, m_db); Construct object-based representation of the desired table structure SqlDriverExt instance One of the new sqlStatement types Use representation to create SQL

6. New DDL statement types CreateTableStatement AlterTableAddFieldStatement AlterTableDropFieldStatement AlterTableModifyFieldStatement DropTableStatement CreateViewStatement DropViewStatement CreateIndexStatement DropDatabaseStatement

7. Other added features Create-if-not-exists Use database templates List databases on a server RAII class for transactions Return information on installed drivers: Human readable name Icon Server or file based Default settings for UI (like standard port number) Workarounds for some smaller Qt SQL issues

8. Unifying the UI Goals: Unified way to select a database independent of the type Keep history of recently used databases for easy re-selecting Re-usable components for selection for integration in all tools Treat all the information on a database as a single unit of information Avoid special-casing for different types of databases

9. Resulting UI André Somers | 9 | SAINT database layer: design and rationale Drop-down box for easy re-selecting recently used databases Access to more databases List is provided by proxy on custom model Icons representing database type Unique alias for each used database

10. Resulting UI (cont) André Somers | 10 | SAINT database layer: design and rationale Tabs to switch between already used databases and other databases Sortable Filter on alias Human-readable dates and times of last use

11. Resulting UI (cont) André Somers | 11 | SAINT database layer: design and rationale Drop down with discovered database drivers File selector dialog Recently used files Database alias (auto suggested)

12. Resulting UI (cont) André Somers | 12 | SAINT database layer: design and rationale Existing databases on selected server

13. DatabaseDescription class Properties QString driverName QString fileName QString host QString hostAlias int port QString user QString password QString databaseName QString alias QDateTime lastAccess Methods bool exists() bool isValid() State state() QSqlDatabase sqlDatabase(const Qstring&) SqlDriverExt::DriverInfo driverInfo() bool dropDatabase() Operators QDataStream& operator<<(QDataStream&, const DatabaseDescription&) QDataStream& operator>>(QDataStream&, DatabaseDescription&)

14. Transactions Perfect candidate for RAII Creating an instance on a QSqlDatabase begins a transaction Explicitly call commit() method to mark successful end of transaction If goes out of scope before commit is called, it tries a roll-back. André Somers | 14 | SAINT database layer: design and rationale

15. Transaction RAII class classSqltransaction { public: explicitSqlTransaction(QSqlDriver* driver); ~SqlTransaction(); QSqlDriver* driver() {return m_driver;} inlineboolinTransaction() {return m_inTransaction;} inlineboolactive() {return m_inTransaction;} boolrollBack(); boolcommit(); boolcommitOrRollBack(); boolsupported(); private: //... }; André Somers | 15 | SAINT database layer: design and rationale

16. Succes? Almost all database-dependent code has been removed from the code that is not part of the database layer presented. Still some pieces of code like this: Separate database template files are still needed All in all: pretty good, and certainly workable André Somers | 16 | SAINT database layer: design and rationale

17. Remaining problems Documentation in Qt is lacking on this Approach is quite low-level. Higher level would be nice Query construction primitives are very basic: No sorting Only = operator in WHERE statements No limiting Not async André Somers | 17 | SAINT database layer: design and rationale

18. Thanks Thank you for your attention. Questions & Discussion: What issues remain for you? André Somers | 18 | SAINT database layer: design and rationale

19. André Somers | 19 | SAINT database layer: design and rationale

20. André Somers | 20 | SAINT database layer: design and rationale

21. Bio-energy, 2008-2009, 8414 publications, approx. 187400 links, in Gephi using Force Atlas. Image: E. Horlings.