MongoDB, Event Sourcing & Spark
•
3 likes•1,112 views
The document discusses event sourcing and how it can be used to model state changes as a sequence of events. It provides examples of how event sourcing can track balance changes for users by logging a series of credit and debit events over time. It also discusses how event sourcing allows reconstructing past application states and tracking state evolution by replaying the full sequence of events.
Recommended
More Related Content
What's hot
What's hot (7)
Similar to MongoDB, Event Sourcing & Spark
Similar to MongoDB, Event Sourcing & Spark (20)
More from Bryan Reinero
Recently uploaded
Recently uploaded (20)
MongoDB, Event Sourcing & Spark
- 1. Architecting the Internet of Things @blimpyacht
- 5. 5
- 7. Event Sourcing THE BEST, UBIQUITOUS PATTERN YOU’VE NEVER HEARD OF
- 13. +4 -2 +5 +7 -4 +9 -3 -6 +2 +1 +1 -5 t(12) = 9 Event Sourcing
- 14. +4 -2 +5 +7 -4 +9 -3 -6 +2 +1 +1 -5 t(5) = 10 Event Sourcing
- 15. +4 -2 +5 +7 -4 +9 -3 -6 +2 +1 +1 -5 t(9) = 12 Event Sourcing
- 19. Res Gestae Divi Augusti
- 24. 2ndaryPrimary
- 25. 2ndaryPrimary
- 26. 2ndaryPrimary
- 28. +4 -2 +5 +7 -4 +9 -3 -6 +2 +1 +1 -5 t(4) = 14 Event Sourcing t(8) = 10 t(12) = 9
- 29. +4 -2 +5 +7 -4 +9 -3 -6 +2 +1 +1 -5 t(4) = 14 Event Sourcing t(8) = 10 t(12) = 9 t’(4) = -2 t’(8) = -17 t’(12) = -22
- 31. { _id: { ts: 1213084687000, taxi: "abboip" }, loc: { geometry: [ -122.39488, 37.75134 ] }, fare: 0, ts: ISODate("2008-06-10T07:58:07Z") }
- 33. { _id: ObjectId("56a4215a46778b69e08ff8ff"), taxi: "abboip", start: ISODate("2008-05-17T14:51:10Z"), route: { type: "LineString", coordinates: [ [ -122.39724, 37.74977 ], [ -122.40619, 37.74896 ], [ -122.41335, 37.74831 ], [ -122.414, 37.75157 ], [ -122.41438, 37.75552 ] ] }, end: ISODate("2008-05-17T14:55:58Z") }
- 35. Data Modeling Domain Events Domain Objects
- 37. Accounts Payable
- 42. CQRS
- 45. CRUD vs. CQRS
- 46. CRUD equals CQRS
- 48. WRITE READ Variant Read Models: VIEWS READ READ READ READ
- 51. Where it Gets Tricky
- 54. Consistent Replay +4 -2 +5 +7 -4 +9 -3 -6 +2 +1 +1 -5 t(5) = 10 Exchange Rate Server 10 / 0.88 = 8.82€
- 55. Consistent Replay +4 -2 +5 +7 -4 +9 -3 -6 +2 +1 +1 -5 t(5) = 10 Exchange Rate Server 10 / 0.87 = 8.70€
- 57. Where it Gets Trickier Still
- 67. Gaining Insight
- 73. MongoDB + Spark
- 74. Level Setting
- 92. executor Worker Node executor Worker Node Master Java Driver Hadoop Connector Driver Application
- 94. Transformations Parellelize = x t(x) = x’ t(x’) = x’’
- 95. Transformations filter( func ) union( func ) intersection( set ) distinct( n ) map( function )
- 96. Action f(x’’) = yParellelize = x t(x) = x’ t(x’) = x’’
- 97. Actions collect() count() first() take( n ) reduce( function )
- 98. Lineage f(x’’) = yParellelize = x t(x) = x’ t(x’) = x’’
- 100. Transform Transform ActionParallelize Transform Transform ActionParallelize Transform Transform ActionParallelize Transform Transform ActionParallelize Transform Transform ActionParallelize Lineage
- 101. Transform Transform ActionParallelize Transform Transform ActionParallelize Transform Transform ActionParallelize Transform Transform ActionParallelize Transform Transform ActionParallelize Lineage
- 102. Transform Transform ActionParallelize Transform Transform ActionParallelize Transform Transform ActionParallelize Transform Transform ActionParallelize Transform Transform ActionParallelize Lineage http://www.blimpyacht. com/2016/02/03/a-visual-guide-to- the-spark-hadoop-ecosystem/
- 104. Spark Configuration Configuration conf = new Configuration(); conf.set( "mongo.job.input.format", "com.mongodb.hadoop.MongoInputFormat” ); conf.set( "mongo.input.uri", "mongodb://localhost:27017/db.collection” );
- 105. Spark Context JavaPairRDD<Object, BSONObject> documents = context. newAPIHadoopRDD( conf, MongoInputFormat.class, Object.class, BSONObject.class );
- 106. Spark Submit /usr/local/spark-1.5.1/bin/spark-submit --class com.mongodb.spark.examples.DataframeExample --master local Examples-1.0-SNAPSHOT.jar
- 108. JavaRDD<Message> messages = documents.map ( new Function<Tuple2<Object, BSONObject>, Message>() { public Message call(Tuple2<Object, BSONObject> tuple) { BSONObject header = (BSONObject)tuple._2.get("headers"); Message m = new Message(); m.setTo( (String) header.get("To") ); m.setX_From( (String) header.get("From") ); m.setMessage_ID( (String) header.get( "Message-ID" ) ); m.setBody( (String) tuple._2.get( "body" ) ); return m; } } );
- 111. THE FUTURE AND BEYOND THE INFINITE
- 112. Spark Connector
- 113. Aggregation Filters $match | $project | $group
- 114. Data Locality mongos
- 115. THANKS! @blimpyacht