HailDB A NoSQL API direct to InnoDB Stewart Smith
What is HailDB?
What is InnoDB?
ACID database enigne
Commonly found in MySQL
Originally was separate
Been in MySQL “forever”
HailDB and InnoDB have...
Features you'd expect
Exposed through SQL
Really an interface to an InnoDB API
InnoDB internal API
Complex and sometimes MySQL specific
HailDB history
Embedded InnoDB
A couple of releases...
Abandoned?
Enter HailDB
Completely free and open source project
http://www.haildb.com
Launchpad
$ bzr branch lp:haildb
Tarball releases
Packaged in Ubuntu Natty
Packaged in drizzle-developers PPA
Also RPMs
(Currently) no Win32 release
Source compatible with embedded_innodb
libhaildb.so
haildb.h
Pandora-build provides autoconf foo
What HailDB Provides
Cursor based API
Transactions
Key lookups
Index scans
DIY Joins
Updates
Inserts
Deletes
Use cases
Embedded, fast, concurrent,  relational database
Not arbitrary queries
Not SQL
High-Performance Storage Services Using HailDB and Java Ballroom E, 2:50pm Thursday
Engine for SQL DB
Using the API
ib_init()
ib_cfg_set_int()
ib_cfg_set_bool_on|off()
ib_cfg_set_text()
Set callbacks
ib_startup(“barracuda”)
ib_shutdown(flag)
DDL
ib_database_create() ib_database_drop()
ib_table_create()
ib_table_drop()
ib_table_truncate()
Transactions
ib_trx_begin()
ib_trx_commit() ib_trx_rollback()
ib_savepoint_take() ib_savepoint_release() ib_savepoint_rollback()
Cursors
ib_cursor_open_table()
ib_cursor_close()
ib_cursor_reset()
Position a cursor, perform row operations
Tuples
Represents columns from table record or secondary index
ib_tpl_t tpl = ib_clust_read_tuple_create(cursor)
ib_col_set_value()
ib_col_get_value()
Examples
Full Table Scan
ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_rea...
ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_rea...
ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_rea...
ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_rea...
ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_rea...
ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_rea...
ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_rea...
ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_rea...
ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_rea...
Insert a row
ib_err_t err; ib_tpl_t tpl; tpl= ib_clust_read_tuple_create(crsr); /* INSERT … VALUES (“a”, “b”, 100); */ ib_col_set_value...
ib_err_t err; ib_tpl_t tpl; tpl= ib_clust_read_tuple_create(crsr); /* INSERT … VALUES (“a”, “b”, 100); */ ib_col_set_value...
ib_err_t err; ib_tpl_t tpl; tpl= ib_clust_read_tuple_create(crsr); /* INSERT … VALUES (“a”, “b”, 100); */ ib_col_set_value...
ib_err_t err; ib_tpl_t tpl; tpl= ib_clust_read_tuple_create(crsr); /* INSERT … VALUES (“a”, “b”, 100); */ ib_col_set_value...
ib_err_t err; ib_tpl_t tpl; tpl= ib_clust_read_tuple_create(crsr); /* INSERT … VALUES (“a”, “b”, 100); */ ib_col_set_value...
ib_err_t err; ib_tpl_t tpl; tpl= ib_clust_read_tuple_create(crsr); /* INSERT … VALUES (“a”, “b”, 100); */ ib_col_set_value...
Updating a row
Delete
ib_tuple_delete()
WHERE clause
Table statistics
HailDB
Questions?
Upcoming SlideShare
Loading in …5
×

HailDB: A NoSQL API Direct to InnoDB

6,527 views

Published on

HailDB is a project that maintains InnoDB as a shared library (forked from embedded_innodb). It provides an API direct to InnoDB without any pesky SQL in front of it. This session is an introduction to HailDB, followed by an introduction to programming it.

- Overview of InnoDB/HailDB
- Overview of what HailDB provides
- use cases
- Examples of usage
- API examples

Published in: Technology
0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,527
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
57
Comments
0
Likes
10
Embeds 0
No embeds

No notes for slide
  • Talk to me!
  • ACID compliance Transactions, savepoints, crash recovery, concurrency etc
  • Row buffer format etc
  • Row buffer format etc
  • Clean, easy to use API, shared library. Based on early InnoDB Plugin
  • Clean, easy to use API, shared library. Based on early InnoDB Plugin
  • Oracle certainly doesn't like to announce things.
  • So I created HailDB. Take existing Embedded InnoDB code. Fix bugs, add features. Update InnoDB based on.
  • So I created HailDB. Take existing Embedded InnoDB code. Fix bugs, add features. Update InnoDB based on.
  • Karmic, Lucid and Maverick
  • I don't personally use RPM based distros... but there are people making RPMs
  • Karmic, Lucid and Maverick
  • Since we've cleaned up what symbols are being exported, we've bumped the so version Added APIs Existing apps, change what you link against, header name, and recompile.
  • Since we've cleaned up what symbols are being exported, we've bumped the so version Added APIs Existing apps, change what you link against, header name, and recompile.
  • Since we've cleaned up what symbols are being exported, we've bumped the so version Added APIs Existing apps, change what you link against, header name, and recompile.
  • Detect which HailDB version is present etc.
  • Even where you may have connected to an external RDBMS. Need concurrency that sqlite doesn't provide Need less overhead than InnoDB in MySQL
  • That's what SQL is for
  • Use a SQL database. HailDB can be used to build a SQL database though.
  • Like the HailDB engine in Drizzle. Our work-in-progress next gen way to interface between innobase (HailDB) and the DB kernel
  • Call first. Sets up a few internal things.
  • Used to set configuration options. HailDB does not provide config file parsing or command line option parsing.
  • Log messages (defaults to printing to stderr) trx_is_interrupted (can interrupt threads stuck on locks) Panic handler (elegantly shut down the rest of your app on corruption)
  • Starts InnoDB. Reads tablespaces, does recovery etc.
  • The various innodb shutdown options NORMAL, NO_IBUF_MERGE, NO_BUFPOOL_FLUSH
  • Schema lock. Shared for read, exclusive for write.
  • Creates the directories for a database. Where file-per-table ibd files go. Otherwise, not really that useful :)
  • Create a table. Done inside a transaction. Get DDL lock. Schema specified by API calls, not SQL. No foreign keys (yet)
  • You can do several DDL operations in the same DDL transaction. You can even (within limits) do transactions on other tables. DFE in DDL txn
  • Same as TRUNCATE TABLE in SQL
  • Simple calls for begin/commit/rollback.
  • Supports isolation levels
  • Rather obvious. You can also query the state of the transaction, or in deadlock, ib_trx_release() if txn has been rolled back by HailDB
  • Named savepoints. Maps exactly to what you'd expect from SQL usage. In future we'll have a lightweight “statement” savepoint
  • Can have a cursor not yet attached to a transaction.
  • There's also using_id() methods.
  • When you're done
  • Lets you reuse the data sturctures, increased perf
  • Don't really need for INSERT. Do need for UPDATE and DELETE of course.
  • So can lookup the primary key from a secondary index without having to read the clustered index
  • To read/write row in clustered index. Other types for searching secondary index, redaing row from secondary index
  • Used to set values. e.g. doing an UPDATE or INSERT row operation.
  • A suite of calls both for specific data types (such as 64bit int) and pointer to it in memory (useful for BLOBs).
  • A suite of calls both for specific data types (such as 64bit int) and pointer to it in memory (useful for BLOBs).
  • A little more involved. Read a row, copy it to a new tuple, update that tuple, call ib_cursor_update_row()
  • On a positioned cursor
  • Search an index (or do a full table scan), examining rows until you find the ones you're looking for.
  • Same ones used by MySQL and Drizzle optimisers. Likely not too useful in most HailDB using apps.
  • And that's HailDB. A rather simple interface to an advanced transactional engine.
  • HailDB: A NoSQL API Direct to InnoDB

    1. 1. HailDB A NoSQL API direct to InnoDB Stewart Smith
    2. 2. What is HailDB?
    3. 3. What is InnoDB?
    4. 4. ACID database enigne
    5. 5. Commonly found in MySQL
    6. 6. Originally was separate
    7. 7. Been in MySQL “forever”
    8. 8. HailDB and InnoDB have...
    9. 9. Features you'd expect
    10. 10. Exposed through SQL
    11. 11. Really an interface to an InnoDB API
    12. 12. InnoDB internal API
    13. 13. Complex and sometimes MySQL specific
    14. 14. HailDB history
    15. 15. Embedded InnoDB
    16. 16. A couple of releases...
    17. 17. Abandoned?
    18. 18. Enter HailDB
    19. 19. Completely free and open source project
    20. 20. http://www.haildb.com
    21. 21. Launchpad
    22. 22. $ bzr branch lp:haildb
    23. 23. Tarball releases
    24. 24. Packaged in Ubuntu Natty
    25. 25. Packaged in drizzle-developers PPA
    26. 26. Also RPMs
    27. 27. (Currently) no Win32 release
    28. 28. Source compatible with embedded_innodb
    29. 29. libhaildb.so
    30. 30. haildb.h
    31. 31. Pandora-build provides autoconf foo
    32. 32. What HailDB Provides
    33. 33. Cursor based API
    34. 34. Transactions
    35. 35. Key lookups
    36. 36. Index scans
    37. 37. DIY Joins
    38. 38. Updates
    39. 39. Inserts
    40. 40. Deletes
    41. 41. Use cases
    42. 42. Embedded, fast, concurrent, relational database
    43. 43. Not arbitrary queries
    44. 44. Not SQL
    45. 45. High-Performance Storage Services Using HailDB and Java Ballroom E, 2:50pm Thursday
    46. 46. Engine for SQL DB
    47. 47. Using the API
    48. 48. ib_init()
    49. 49. ib_cfg_set_int()
    50. 50. ib_cfg_set_bool_on|off()
    51. 51. ib_cfg_set_text()
    52. 52. Set callbacks
    53. 53. ib_startup(“barracuda”)
    54. 54. ib_shutdown(flag)
    55. 55. DDL
    56. 56. ib_database_create() ib_database_drop()
    57. 57. ib_table_create()
    58. 58. ib_table_drop()
    59. 59. ib_table_truncate()
    60. 60. Transactions
    61. 61. ib_trx_begin()
    62. 62. ib_trx_commit() ib_trx_rollback()
    63. 63. ib_savepoint_take() ib_savepoint_release() ib_savepoint_rollback()
    64. 64. Cursors
    65. 65. ib_cursor_open_table()
    66. 66. ib_cursor_close()
    67. 67. ib_cursor_reset()
    68. 68. Position a cursor, perform row operations
    69. 69. Tuples
    70. 70. Represents columns from table record or secondary index
    71. 71. ib_tpl_t tpl = ib_clust_read_tuple_create(cursor)
    72. 72. ib_col_set_value()
    73. 73. ib_col_get_value()
    74. 74. Examples
    75. 75. Full Table Scan
    76. 76. ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_read_row(crsr, tpl); /* Handle locking and timeout errors */ ib_tuple_read...(); err= ib_cursor_next(crsr); /* Handle locking and timeout errors */ tpl= ib_tuple_clear(tpl); } ib_tuple_delete(tpl);
    77. 77. ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_read_row(crsr, tpl); /* Handle locking and timeout errors */ ib_tuple_read...(); err= ib_cursor_next(crsr); /* Handle locking and timeout errors */ tpl= ib_tuple_clear(tpl); } ib_tuple_delete(tpl);
    78. 78. ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_read_row(crsr, tpl); /* Handle locking and timeout errors */ ib_tuple_read...(); err= ib_cursor_next(crsr); /* Handle locking and timeout errors */ tpl= ib_tuple_clear(tpl); } ib_tuple_delete(tpl);
    79. 79. ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_read_row(crsr, tpl); /* Handle locking and timeout errors */ ib_tuple_read...(); err= ib_cursor_next(crsr); /* Handle locking and timeout errors */ tpl= ib_tuple_clear(tpl); } ib_tuple_delete(tpl);
    80. 80. ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_read_row(crsr, tpl); /* Handle locking and timeout errors */ ib_tuple_read...(); err= ib_cursor_next(crsr); /* Handle locking and timeout errors */ tpl= ib_tuple_clear(tpl); } ib_tuple_delete(tpl);
    81. 81. ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_read_row(crsr, tpl); /* Handle locking and timeout errors */ ib_tuple_read...(); err= ib_cursor_next(crsr); /* Handle locking and timeout errors */ tpl= ib_tuple_clear(tpl); } ib_tuple_delete(tpl);
    82. 82. ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_read_row(crsr, tpl); /* Handle locking and timeout errors */ ib_tuple_read...(); err= ib_cursor_next(crsr); /* Handle locking and timeout errors */ tpl= ib_tuple_clear(tpl); } ib_tuple_delete(tpl);
    83. 83. ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_read_row(crsr, tpl); /* Handle locking and timeout errors */ ib_tuple_read...(); err= ib_cursor_next(crsr); /* Handle locking and timeout errors */ tpl= ib_tuple_clear(tpl); } ib_tuple_delete(tpl);
    84. 84. ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_read_row(crsr, tpl); /* Handle locking and timeout errors */ ib_tuple_read...(); err= ib_cursor_next(crsr); /* Handle locking and timeout errors */ tpl= ib_tuple_clear(tpl); } ib_tuple_delete(tpl);
    85. 85. Insert a row
    86. 86. ib_err_t err; ib_tpl_t tpl; tpl= ib_clust_read_tuple_create(crsr); /* INSERT … VALUES (“a”, “b”, 100); */ ib_col_set_value(tpl, 0, “a”, strlen(“a”)); ib_col_set_value(tpl, 1, “b”, strlen(“b”)); ib_tuple_write_i32(tpl, 2, 100); err= ib_cursor_insert_row(crsr, tpl); /* Handle errors. e.g. duplicate key */ ib_tuple_delete(tpl);
    87. 87. ib_err_t err; ib_tpl_t tpl; tpl= ib_clust_read_tuple_create(crsr); /* INSERT … VALUES (“a”, “b”, 100); */ ib_col_set_value(tpl, 0, “a”, strlen(“a”)); ib_col_set_value(tpl, 1, “b”, strlen(“b”)); ib_tuple_write_i32(tpl, 2, 100); err= ib_cursor_insert_row(crsr, tpl); /* Handle errors. e.g. duplicate key */ ib_tuple_delete(tpl);
    88. 88. ib_err_t err; ib_tpl_t tpl; tpl= ib_clust_read_tuple_create(crsr); /* INSERT … VALUES (“a”, “b”, 100); */ ib_col_set_value(tpl, 0, “a”, strlen(“a”)); ib_col_set_value(tpl, 1, “b”, strlen(“b”)); ib_tuple_write_i32(tpl, 2, 100); err= ib_cursor_insert_row(crsr, tpl); /* Handle errors. e.g. duplicate key */ ib_tuple_delete(tpl);
    89. 89. ib_err_t err; ib_tpl_t tpl; tpl= ib_clust_read_tuple_create(crsr); /* INSERT … VALUES (“a”, “b”, 100); */ ib_col_set_value(tpl, 0, “a”, strlen(“a”)); ib_col_set_value(tpl, 1, “b”, strlen(“b”)); ib_tuple_write_i32(tpl, 2, 100); err= ib_cursor_insert_row(crsr, tpl); /* Handle errors. e.g. duplicate key */ ib_tuple_delete(tpl);
    90. 90. ib_err_t err; ib_tpl_t tpl; tpl= ib_clust_read_tuple_create(crsr); /* INSERT … VALUES (“a”, “b”, 100); */ ib_col_set_value(tpl, 0, “a”, strlen(“a”)); ib_col_set_value(tpl, 1, “b”, strlen(“b”)); ib_tuple_write_i32(tpl, 2, 100); err= ib_cursor_insert_row(crsr, tpl); /* Handle errors. e.g. duplicate key */ ib_tuple_delete(tpl);
    91. 91. ib_err_t err; ib_tpl_t tpl; tpl= ib_clust_read_tuple_create(crsr); /* INSERT … VALUES (“a”, “b”, 100); */ ib_col_set_value(tpl, 0, “a”, strlen(“a”)); ib_col_set_value(tpl, 1, “b”, strlen(“b”)); ib_tuple_write_i32(tpl, 2, 100); err= ib_cursor_insert_row(crsr, tpl); /* Handle errors. e.g. duplicate key */ ib_tuple_delete(tpl);
    92. 92. Updating a row
    93. 93. Delete
    94. 94. ib_tuple_delete()
    95. 95. WHERE clause
    96. 96. Table statistics
    97. 97. HailDB
    98. 98. Questions?

    ×