Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Triggers in MongoDB

7,260 views

Published on

A database trigger is a stored procedure that is executed when specific actions occur within a database. Triggers fit perfectly on a relational schema (foreign keys) and are implemented as a built-in functionality on popular relational database like MySQL.

MongoDB does not have any support for triggers, mainly due to the lack of support for foreign keys. Even if it usually considered an antipattern, there are use cases in MongoDB that benefit from a partially-relational schema. The lack of triggers is an obstacle for a partially-relational schema but there can be workarounds for simulating trigger behavior.

This presentation will guide you through different ways to implement triggers in MongoDB. We will cover the topics streams, tailable cursors, and hooks. We will demonstrate coding examples for each topic and we will explain pros and cons of each implementation.

Published in: Data & Analytics
  • Login to see the comments

  • Be the first to like this

Triggers in MongoDB

  1. 1. Triggers in MongoDB Antonios Giannopoulos and Jason Terpko DBA’s @ Rackspace/ObjectRocket linkedin.com/in/antonis/ | linkedin.com/in/jterpko/ 1
  2. 2. Introduction www.objectrocket.com 2 Antonios Giannopoulos Jason Terpko
  3. 3. Overview • What is a Trigger • Why is Useful • Application Triggers • Oplog Tailing • 3.6+ Streams • Use Cases www.objectrocket.com 3
  4. 4. What is a trigger? www.objectrocket.com 4 A trigger is a database object that is associated with a table. It will be activated when a defined action is executed for the table. Actions usually are: - INSERT - UPDATE - DELETE It can be invoked before or after the event.
  5. 5. MySQL trigger definition www.objectrocket.com 5
  6. 6. Trigger example (BEFORE) www.objectrocket.com 6 *For Deposits an AFTER event trigger is preferred J
  7. 7. Trigger example (BEFORE) www.objectrocket.com 7
  8. 8. Trigger example (AFTER) www.objectrocket.com 8
  9. 9. Trigger example (AFTER) www.objectrocket.com 9
  10. 10. Foreign Keys www.objectrocket.com 10 Foreign Keys may also considered as triggers, but less flexible
  11. 11. MongoDB Application Triggers • Implementation Options • Considerations • Examples www.objectrocket.com 11
  12. 12. Application Based Triggers www.objectrocket.com 12 • Maintained internally • Compatibility • Flexibility • Added Development Cycles Self-Implemented Open-Source Package • Publically maintained • Contributions and tested by a larger user base • Potentially restricted to specific versions, client and server
  13. 13. Self-Implemented (Before) www.objectrocket.com 13
  14. 14. Self-Implemented (After) www.objectrocket.com 14
  15. 15. Python (mongotriggers) www.objectrocket.com 15
  16. 16. Node.js (mongo-oplog) www.objectrocket.com 16
  17. 17. Oplog Tailing • The oplog • Tailable cursors • Replica Set • Sharded cluster • Advantages/Disadvantages/Co nsiderations www.objectrocket.com 17
  18. 18. The oplog www.objectrocket.com 18 Capped collection (FIFO) Fixed Size - oplogSizeMB (5% of free space is the default) Located under local.oplog.rs Holds every CRUD operation (Insert/Update/Delete/Commands) MongoDB Replication Idempotent by design
  19. 19. Anatomy of the local.oplog.rs www.objectrocket.com 19
  20. 20. Anatomy of the local.oplog.rs www.objectrocket.com 20
  21. 21. Anatomy of the local.oplog.rs www.objectrocket.com 21 ts: timestamp of the oplog entry t: election "term" h: unique hash v: version of the oplog op: Type of operation (insert/update/delete/commands) ns: Database & collection affected o: The new state of the document after performing the change o2: Contain the _id field of the affected document ui: Collection’s UUID wall: timestamp of the oplog entry fromMigrate : Sparse field, is true when the operation comes from balancing
  22. 22. Tailable cursor www.objectrocket.com 22 Equivalent to the tail Unix command with the -f option Remains open after the client exhausts the results in the initial cursor Ideal for capped collections (where indexes are not practical) MongoDB replication uses tailable cursors to read the oplog Initial scan is expensive – It scans the entire collection Available on the vast majority of drivers
  23. 23. Trigger implementation www.objectrocket.com 23
  24. 24. Trigger implementation www.objectrocket.com 24
  25. 25. Using Replica Set www.objectrocket.com 25 Primary’s and Secondary's oplog are identical A tailable cursor is enough Trigger action must loop through the primary
  26. 26. Using Sharded Cluster www.objectrocket.com 26 s1 s2 Oplog is not visible thought the mongos A tailable cursor per shard Adjust to topology changes Must filter balancer events Security issues Trigger action must loop through the mongos
  27. 27. Considerations www.objectrocket.com 27 Only AFTER INSERT/UPDATE/DELETE supported Rollback an operation Handle nodes rollbacks (Replica set state) Avoid replay the same operations Keep up with the replication pace (Dedicated members using replica set tags)
  28. 28. Considerations www.objectrocket.com 28 Apply before commit/commit errors (w:majority or w:n, n>1) Preserve the order of operations – sharded clusters only Filter out migration events (fromMigrate) – sharded clusters only Filter out update events on Orphans – sharded clusters only
  29. 29. Change Streams • Topology • Change Events • Examples • Sharded Clusters • Considerations www.objectrocket.com 29
  30. 30. What is a Change Stream? www.objectrocket.com 30 s1 s2 3) Resumable 2) Driver Supported 1) Real-time 4) Secure 5) Synchronized 6) Flexible (1) (2) (3) (4) (5) (6) X X
  31. 31. Event Document (Insert) www.objectrocket.com 31
  32. 32. Event Document (Update) www.objectrocket.com 32
  33. 33. Event Document (Update - Full) www.objectrocket.com 33
  34. 34. Event Document (Replace) www.objectrocket.com 34
  35. 35. Event Document (Remove) www.objectrocket.com 35
  36. 36. Event Document (Invalidate) www.objectrocket.com 36
  37. 37. Python Stream – watch() www.objectrocket.com 37
  38. 38. Flexible (Pipeline Stages) www.objectrocket.com 38 $match, $addFields $match, $project
  39. 39. Sharded Clusters www.objectrocket.com 39 s1 s2 3) Cold Shard(s) 2) Rate of Change 1) Coordination 4) Geographical Dist. 5) Orphans (1) (2) (3) (4) (5)
  40. 40. Additional Considerations www.objectrocket.com 40 • Majority Read Concern / Engine • Replica Member Availability / Arbiters • Oplog Size • Collection Dropped or Renamed (Invalidation) • Large Documents and Maximum Document Size
  41. 41. Use Case • Shard Key Analysis www.objectrocket.com 41
  42. 42. Use Cases www.objectrocket.com 42 Auditing Event Scheduler Caching Selective Replication Disaster Recovery …
  43. 43. Mongo to Mongo connector www.objectrocket.com 43 Reads the oplog on source and replays on destination source dest Tailable cursor Use Cases • Selective replication • Multisource replication • Disaster recovery • Heterogeneous Replication
  44. 44. Use Case: Shard Key Analysis www.objectrocket.com 44 Define a shard key is challenging: • Can break your application • Not an easy task to revert it (requires downtime) Shard Key limitations (two out of many): • Shard Key Value in a Document is Immutable. • NULL values are not allowed
  45. 45. Use case: Shard Key Analysis www.objectrocket.com 45 Query system.profile • Requires extra room as default is 1MiB • Adds overhead (extra writes) • Snapshot of operations Oplog or Streams • Already present • No extra overhead • Covers bigger duration
  46. 46. Shard Key Analysis - Oplog www.objectrocket.com 46
  47. 47. Shard Key Analysis - Streams www.objectrocket.com 47
  48. 48. Questions? www.objectrocket.com 48
  49. 49. Rate My Session www.objectrocket.com 49
  50. 50. www.objectrocket.com 50 We’re Hiring! Looking to join a dynamic & innovative team? https://www.objectrocket.com/careers/ or email careers@objectrocket.com
  51. 51. Thank you! Address: 401 Congress Ave Suite 1950 Austin, TX 78701 Support: 1-800-961-4454 Sales: 1-888-440-3242 www.objectrocket.com 51

×