Your SlideShare is downloading. ×
  • Like
HailDB: A NoSQL API Direct to InnoDB
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

HailDB: A NoSQL API Direct to InnoDB

  • 5,751 views
Published

HailDB is a project that maintains InnoDB as a shared library (forked from embedded_innodb). It provides an API direct to InnoDB without any pesky SQL in front of it. This session is an introduction …

HailDB is a project that maintains InnoDB as a shared library (forked from embedded_innodb). It provides an API direct to InnoDB without any pesky SQL in front of it. This session is an introduction to HailDB, followed by an introduction to programming it.

- Overview of InnoDB/HailDB
- Overview of what HailDB provides
- use cases
- Examples of usage
- API examples

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
5,751
On SlideShare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
53
Comments
0
Likes
9

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Talk to me!
  • ACID compliance Transactions, savepoints, crash recovery, concurrency etc
  • Row buffer format etc
  • Row buffer format etc
  • Clean, easy to use API, shared library. Based on early InnoDB Plugin
  • Clean, easy to use API, shared library. Based on early InnoDB Plugin
  • Oracle certainly doesn't like to announce things.
  • So I created HailDB. Take existing Embedded InnoDB code. Fix bugs, add features. Update InnoDB based on.
  • So I created HailDB. Take existing Embedded InnoDB code. Fix bugs, add features. Update InnoDB based on.
  • Karmic, Lucid and Maverick
  • I don't personally use RPM based distros... but there are people making RPMs
  • Karmic, Lucid and Maverick
  • Since we've cleaned up what symbols are being exported, we've bumped the so version Added APIs Existing apps, change what you link against, header name, and recompile.
  • Since we've cleaned up what symbols are being exported, we've bumped the so version Added APIs Existing apps, change what you link against, header name, and recompile.
  • Since we've cleaned up what symbols are being exported, we've bumped the so version Added APIs Existing apps, change what you link against, header name, and recompile.
  • Detect which HailDB version is present etc.
  • Even where you may have connected to an external RDBMS. Need concurrency that sqlite doesn't provide Need less overhead than InnoDB in MySQL
  • That's what SQL is for
  • Use a SQL database. HailDB can be used to build a SQL database though.
  • Like the HailDB engine in Drizzle. Our work-in-progress next gen way to interface between innobase (HailDB) and the DB kernel
  • Call first. Sets up a few internal things.
  • Used to set configuration options. HailDB does not provide config file parsing or command line option parsing.
  • Log messages (defaults to printing to stderr) trx_is_interrupted (can interrupt threads stuck on locks) Panic handler (elegantly shut down the rest of your app on corruption)
  • Starts InnoDB. Reads tablespaces, does recovery etc.
  • The various innodb shutdown options NORMAL, NO_IBUF_MERGE, NO_BUFPOOL_FLUSH
  • Schema lock. Shared for read, exclusive for write.
  • Creates the directories for a database. Where file-per-table ibd files go. Otherwise, not really that useful :)
  • Create a table. Done inside a transaction. Get DDL lock. Schema specified by API calls, not SQL. No foreign keys (yet)
  • You can do several DDL operations in the same DDL transaction. You can even (within limits) do transactions on other tables. DFE in DDL txn
  • Same as TRUNCATE TABLE in SQL
  • Simple calls for begin/commit/rollback.
  • Supports isolation levels
  • Rather obvious. You can also query the state of the transaction, or in deadlock, ib_trx_release() if txn has been rolled back by HailDB
  • Named savepoints. Maps exactly to what you'd expect from SQL usage. In future we'll have a lightweight “statement” savepoint
  • Can have a cursor not yet attached to a transaction.
  • There's also using_id() methods.
  • When you're done
  • Lets you reuse the data sturctures, increased perf
  • Don't really need for INSERT. Do need for UPDATE and DELETE of course.
  • So can lookup the primary key from a secondary index without having to read the clustered index
  • To read/write row in clustered index. Other types for searching secondary index, redaing row from secondary index
  • Used to set values. e.g. doing an UPDATE or INSERT row operation.
  • A suite of calls both for specific data types (such as 64bit int) and pointer to it in memory (useful for BLOBs).
  • A suite of calls both for specific data types (such as 64bit int) and pointer to it in memory (useful for BLOBs).
  • A little more involved. Read a row, copy it to a new tuple, update that tuple, call ib_cursor_update_row()
  • On a positioned cursor
  • Search an index (or do a full table scan), examining rows until you find the ones you're looking for.
  • Same ones used by MySQL and Drizzle optimisers. Likely not too useful in most HailDB using apps.
  • And that's HailDB. A rather simple interface to an advanced transactional engine.

Transcript

  • 1. HailDB A NoSQL API direct to InnoDB Stewart Smith
  • 2. What is HailDB?
  • 3. What is InnoDB?
  • 4. ACID database enigne
  • 5. Commonly found in MySQL
  • 6. Originally was separate
  • 7. Been in MySQL “forever”
  • 8. HailDB and InnoDB have...
  • 9. Features you'd expect
  • 10. Exposed through SQL
  • 11. Really an interface to an InnoDB API
  • 12. InnoDB internal API
  • 13. Complex and sometimes MySQL specific
  • 14. HailDB history
  • 15. Embedded InnoDB
  • 16. A couple of releases...
  • 17. Abandoned?
  • 18. Enter HailDB
  • 19. Completely free and open source project
  • 20. http://www.haildb.com
  • 21. Launchpad
  • 22. $ bzr branch lp:haildb
  • 23. Tarball releases
  • 24. Packaged in Ubuntu Natty
  • 25. Packaged in drizzle-developers PPA
  • 26. Also RPMs
  • 27. (Currently) no Win32 release
  • 28. Source compatible with embedded_innodb
  • 29. libhaildb.so
  • 30. haildb.h
  • 31. Pandora-build provides autoconf foo
  • 32. What HailDB Provides
  • 33. Cursor based API
  • 34. Transactions
  • 35. Key lookups
  • 36. Index scans
  • 37. DIY Joins
  • 38. Updates
  • 39. Inserts
  • 40. Deletes
  • 41. Use cases
  • 42. Embedded, fast, concurrent, relational database
  • 43. Not arbitrary queries
  • 44. Not SQL
  • 45. High-Performance Storage Services Using HailDB and Java Ballroom E, 2:50pm Thursday
  • 46. Engine for SQL DB
  • 47. Using the API
  • 48. ib_init()
  • 49. ib_cfg_set_int()
  • 50. ib_cfg_set_bool_on|off()
  • 51. ib_cfg_set_text()
  • 52. Set callbacks
  • 53. ib_startup(“barracuda”)
  • 54. ib_shutdown(flag)
  • 55. DDL
  • 56. ib_database_create() ib_database_drop()
  • 57. ib_table_create()
  • 58. ib_table_drop()
  • 59. ib_table_truncate()
  • 60. Transactions
  • 61. ib_trx_begin()
  • 62. ib_trx_commit() ib_trx_rollback()
  • 63. ib_savepoint_take() ib_savepoint_release() ib_savepoint_rollback()
  • 64. Cursors
  • 65. ib_cursor_open_table()
  • 66. ib_cursor_close()
  • 67. ib_cursor_reset()
  • 68. Position a cursor, perform row operations
  • 69. Tuples
  • 70. Represents columns from table record or secondary index
  • 71. ib_tpl_t tpl = ib_clust_read_tuple_create(cursor)
  • 72. ib_col_set_value()
  • 73. ib_col_get_value()
  • 74. Examples
  • 75. Full Table Scan
  • 76. ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_read_row(crsr, tpl); /* Handle locking and timeout errors */ ib_tuple_read...(); err= ib_cursor_next(crsr); /* Handle locking and timeout errors */ tpl= ib_tuple_clear(tpl); } ib_tuple_delete(tpl);
  • 77. ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_read_row(crsr, tpl); /* Handle locking and timeout errors */ ib_tuple_read...(); err= ib_cursor_next(crsr); /* Handle locking and timeout errors */ tpl= ib_tuple_clear(tpl); } ib_tuple_delete(tpl);
  • 78. ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_read_row(crsr, tpl); /* Handle locking and timeout errors */ ib_tuple_read...(); err= ib_cursor_next(crsr); /* Handle locking and timeout errors */ tpl= ib_tuple_clear(tpl); } ib_tuple_delete(tpl);
  • 79. ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_read_row(crsr, tpl); /* Handle locking and timeout errors */ ib_tuple_read...(); err= ib_cursor_next(crsr); /* Handle locking and timeout errors */ tpl= ib_tuple_clear(tpl); } ib_tuple_delete(tpl);
  • 80. ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_read_row(crsr, tpl); /* Handle locking and timeout errors */ ib_tuple_read...(); err= ib_cursor_next(crsr); /* Handle locking and timeout errors */ tpl= ib_tuple_clear(tpl); } ib_tuple_delete(tpl);
  • 81. ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_read_row(crsr, tpl); /* Handle locking and timeout errors */ ib_tuple_read...(); err= ib_cursor_next(crsr); /* Handle locking and timeout errors */ tpl= ib_tuple_clear(tpl); } ib_tuple_delete(tpl);
  • 82. ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_read_row(crsr, tpl); /* Handle locking and timeout errors */ ib_tuple_read...(); err= ib_cursor_next(crsr); /* Handle locking and timeout errors */ tpl= ib_tuple_clear(tpl); } ib_tuple_delete(tpl);
  • 83. ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_read_row(crsr, tpl); /* Handle locking and timeout errors */ ib_tuple_read...(); err= ib_cursor_next(crsr); /* Handle locking and timeout errors */ tpl= ib_tuple_clear(tpl); } ib_tuple_delete(tpl);
  • 84. ib_tpl_t tpl= ib_clust_read_tuple_create(crsr); err= ib_cursor_first(crsr); while (err == DB_SUCCESS) { err= ib_cursor_read_row(crsr, tpl); /* Handle locking and timeout errors */ ib_tuple_read...(); err= ib_cursor_next(crsr); /* Handle locking and timeout errors */ tpl= ib_tuple_clear(tpl); } ib_tuple_delete(tpl);
  • 85. Insert a row
  • 86. ib_err_t err; ib_tpl_t tpl; tpl= ib_clust_read_tuple_create(crsr); /* INSERT … VALUES (“a”, “b”, 100); */ ib_col_set_value(tpl, 0, “a”, strlen(“a”)); ib_col_set_value(tpl, 1, “b”, strlen(“b”)); ib_tuple_write_i32(tpl, 2, 100); err= ib_cursor_insert_row(crsr, tpl); /* Handle errors. e.g. duplicate key */ ib_tuple_delete(tpl);
  • 87. ib_err_t err; ib_tpl_t tpl; tpl= ib_clust_read_tuple_create(crsr); /* INSERT … VALUES (“a”, “b”, 100); */ ib_col_set_value(tpl, 0, “a”, strlen(“a”)); ib_col_set_value(tpl, 1, “b”, strlen(“b”)); ib_tuple_write_i32(tpl, 2, 100); err= ib_cursor_insert_row(crsr, tpl); /* Handle errors. e.g. duplicate key */ ib_tuple_delete(tpl);
  • 88. ib_err_t err; ib_tpl_t tpl; tpl= ib_clust_read_tuple_create(crsr); /* INSERT … VALUES (“a”, “b”, 100); */ ib_col_set_value(tpl, 0, “a”, strlen(“a”)); ib_col_set_value(tpl, 1, “b”, strlen(“b”)); ib_tuple_write_i32(tpl, 2, 100); err= ib_cursor_insert_row(crsr, tpl); /* Handle errors. e.g. duplicate key */ ib_tuple_delete(tpl);
  • 89. ib_err_t err; ib_tpl_t tpl; tpl= ib_clust_read_tuple_create(crsr); /* INSERT … VALUES (“a”, “b”, 100); */ ib_col_set_value(tpl, 0, “a”, strlen(“a”)); ib_col_set_value(tpl, 1, “b”, strlen(“b”)); ib_tuple_write_i32(tpl, 2, 100); err= ib_cursor_insert_row(crsr, tpl); /* Handle errors. e.g. duplicate key */ ib_tuple_delete(tpl);
  • 90. ib_err_t err; ib_tpl_t tpl; tpl= ib_clust_read_tuple_create(crsr); /* INSERT … VALUES (“a”, “b”, 100); */ ib_col_set_value(tpl, 0, “a”, strlen(“a”)); ib_col_set_value(tpl, 1, “b”, strlen(“b”)); ib_tuple_write_i32(tpl, 2, 100); err= ib_cursor_insert_row(crsr, tpl); /* Handle errors. e.g. duplicate key */ ib_tuple_delete(tpl);
  • 91. ib_err_t err; ib_tpl_t tpl; tpl= ib_clust_read_tuple_create(crsr); /* INSERT … VALUES (“a”, “b”, 100); */ ib_col_set_value(tpl, 0, “a”, strlen(“a”)); ib_col_set_value(tpl, 1, “b”, strlen(“b”)); ib_tuple_write_i32(tpl, 2, 100); err= ib_cursor_insert_row(crsr, tpl); /* Handle errors. e.g. duplicate key */ ib_tuple_delete(tpl);
  • 92. Updating a row
  • 93. Delete
  • 94. ib_tuple_delete()
  • 95. WHERE clause
  • 96. Table statistics
  • 97. HailDB
  • 98. Questions?