#MongoDBSydney




Roadmap
Stephen Steneker (stennie@10gen.com)
Support Engineer, 10gen Australia
Recent Release History
   1.8                 2.0                2.2         2.4
 March ‘11           Sept ‘11           Aug ‘12     Winter ‘13



Journaling      Index enhancements    Aggregation
Sharding and    to improve size and   Framework
Replica set     performance           Multi-Data Center
enhancements    Authentication with   Deployments
Spherical geo   sharded clusters      Improved
search          Replica Set           Performance and
                Enhancements          Concurrency
                Concurrency
                improvements
2.2 Overview
2.2 Release August 2012
• Concurrency: yielding + db level locking

• New aggregation framework

• TTL Collections

• Improved free list implementation

• Tag aware sharding

• Read Preferences

• http://docs.mongodb.org/manual/release-notes/2.2/
Yielding and DB Locking
• Improved yielding on page fault
• Breaking down the global level lock
   – Lock per Database in 2.2

• Writes on secondaries applied in batches
Aggregation Framework
• Pipeline model (a bit like unix pipes)
   – Like a "group by"
   – Operators
      • $project, $group, $match, $limit, $skip, $unwind, $sort
   – Expressions
      • Logical Expressions: $and, $not, $or, $cmp ...
      • Math Expressions: $add, $divide, $mod ...
      • String Expressions: $strcasecmp, $substr, $toLower ...
      • Date/Time Expressions: $dayOfMonth, $hour...
      • Multi-Expressions: $ifNull, $cond

• Use Cases: Real-time / inline analytics
Time to Live (TTL)
• Auto expire data out of a collection
• Must be on a date datatype
• Single value is evaluated
• Use Cases: data retention, cache expiration

db.events.ensureIndex(
 { "timestamp": 1 },
 { expireAfterSeconds: 3600 } )
Tag Aware Sharding
• Distribute data based on a Tag
• Use Cases: Locality for Data by Data Center
sh.addShardTag("shard0000", "dc-emea")

sh.addTagRange("mydb.users",
  { country: "uk"}, { country: "ul"},
  "dc-emea"
);

sh.addTagRange("mydb.users",
  { country: "by"},{ country: "bz"},
  "dc-emea"
);
Read Preferences
• Mode
  – PRIMARY, PRIMARY_PREFERRED
  – SECONDARY, SECONDARY_PREFERRED
  – NEAREST

• Tag Sets
  – Uses Replica Set tags
  – Passed Tag is used to find matching members
2.4 and Beyond
2.4 Highlights
• Security
   – SASL, Kerberos, Additions to privileges and auditing

• Hash-based Sharding

• Geospatial Indexing: query intersecting polygons

• Aggregation framework: faster and more features

• V8, background secondary indexing, replica set flapping

• Distribute non-sharded collections throughout cluster

• MMS running in your own data center (separate)
Ongoing Work
• Collection / Extent level locking
• Field level authorization
• Auditing
• Full-text query
What can I do?
• MongoDB is a community driven product
  – Use the product
  – Log feature requests
  – Vote for existing features

• Join Sydney MongoDB User Group
  – http://www.meetup.com/Sydney-MongoDB-User-group
After Party at Slip Inn on the Terrace
              Insert After Party Map Here

MongoDB Roadmap

  • 1.
  • 2.
    Recent Release History 1.8 2.0 2.2 2.4 March ‘11 Sept ‘11 Aug ‘12 Winter ‘13 Journaling Index enhancements Aggregation Sharding and to improve size and Framework Replica set performance Multi-Data Center enhancements Authentication with Deployments Spherical geo sharded clusters Improved search Replica Set Performance and Enhancements Concurrency Concurrency improvements
  • 3.
  • 4.
    2.2 Release August2012 • Concurrency: yielding + db level locking • New aggregation framework • TTL Collections • Improved free list implementation • Tag aware sharding • Read Preferences • http://docs.mongodb.org/manual/release-notes/2.2/
  • 5.
    Yielding and DBLocking • Improved yielding on page fault • Breaking down the global level lock – Lock per Database in 2.2 • Writes on secondaries applied in batches
  • 6.
    Aggregation Framework • Pipelinemodel (a bit like unix pipes) – Like a "group by" – Operators • $project, $group, $match, $limit, $skip, $unwind, $sort – Expressions • Logical Expressions: $and, $not, $or, $cmp ... • Math Expressions: $add, $divide, $mod ... • String Expressions: $strcasecmp, $substr, $toLower ... • Date/Time Expressions: $dayOfMonth, $hour... • Multi-Expressions: $ifNull, $cond • Use Cases: Real-time / inline analytics
  • 7.
    Time to Live(TTL) • Auto expire data out of a collection • Must be on a date datatype • Single value is evaluated • Use Cases: data retention, cache expiration db.events.ensureIndex( { "timestamp": 1 }, { expireAfterSeconds: 3600 } )
  • 8.
    Tag Aware Sharding •Distribute data based on a Tag • Use Cases: Locality for Data by Data Center sh.addShardTag("shard0000", "dc-emea") sh.addTagRange("mydb.users", { country: "uk"}, { country: "ul"}, "dc-emea" ); sh.addTagRange("mydb.users", { country: "by"},{ country: "bz"}, "dc-emea" );
  • 9.
    Read Preferences • Mode – PRIMARY, PRIMARY_PREFERRED – SECONDARY, SECONDARY_PREFERRED – NEAREST • Tag Sets – Uses Replica Set tags – Passed Tag is used to find matching members
  • 10.
  • 11.
    2.4 Highlights • Security – SASL, Kerberos, Additions to privileges and auditing • Hash-based Sharding • Geospatial Indexing: query intersecting polygons • Aggregation framework: faster and more features • V8, background secondary indexing, replica set flapping • Distribute non-sharded collections throughout cluster • MMS running in your own data center (separate)
  • 12.
    Ongoing Work • Collection/ Extent level locking • Field level authorization • Auditing • Full-text query
  • 13.
    What can Ido? • MongoDB is a community driven product – Use the product – Log feature requests – Vote for existing features • Join Sydney MongoDB User Group – http://www.meetup.com/Sydney-MongoDB-User-group
  • 14.
    After Party atSlip Inn on the Terrace Insert After Party Map Here

Editor's Notes

  • #14 Disclaimer: We’re sharing what we’re working on. Things change, priorities may change, and we won’t release a feature in the next release if it isn’t ready yet. So please handle this transparency carefully. We’ll be adding more details. You can watch SERVER tickets to see as components go in.
  • #15 A highlight of some key features in 2.4. . . . We’ll add more details and more items each month as we work towards a winter release.Security:SASL is a framework for authentication that helps decouple specific authentication mechanisms from client/server implementation. This framework will permit working with a variety of authentication mechanisms, initially we’ll build in kerberos. We may add others over time, but SASL implementation will make it much easier for you to add your own without having to implement a new client.Kerberos is quite common, so we’ll build that one in first.With additional authentication, we want to take a few steps to separate out activities authorized to various users. Separate read, read/write, security administration, database-specific (compact, validate, etc.), and server/cluster administration (fsync, log rotate, shutdown, create database, etc.). This is just an initial step in our authorization work.Hash-based shardingApply a hash function to a selected key as the shard key. Evenly spread documents in a sharded cluster. Evenly spread the work associated with queries in a sharded cluster. Will minimize migrations (should only happen when growing a cluster). Note: this is something you can do now, but not automatic.Geospatial index resolution: Talk about challenge of specifying some polygon and finding overlap with another polygon in a document, this becomes interesting for location-aware applications, intelligence community.Replica set flapping: avoid electing a new primary due to a falsely detecting that the current primary went down. Adding mechanisms to reduce false detections. This is good for heavy load and network issues/blips in a data center.