Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Big Data Day LA 2016/ NoSQL track - MongoDB 3.2 Goodness!!!, Mark Helmstetter, Principal Consulting Engineer, MongoDB

This talk explores the new features of MongoDB 3.2 such as $lookup, document validation rules, encryption-at-rest and tools like the BI Connector, OpsManager 2.0 and Compass.

  • Login to see the comments

  • Be the first to like this

Big Data Day LA 2016/ NoSQL track - MongoDB 3.2 Goodness!!!, Mark Helmstetter, Principal Consulting Engineer, MongoDB

  1. 1. MongoDB 3.2 Goodness MarkHelmstetter
  2. 2. MongoDB 3.2 – a BIG Release Hash-Based Sharding Roles Kerberos On-Prem Monitoring 2.2 2.4 2.6 3.0 3.2 Agg. Framework Location-Aware Sharding $out Index Intersection Text Search Field-Level Redaction LDAP & x509 Auditing Document Validation Fast Failover Simpler Scalability Aggregation ++ Encryption At Rest In-Memory Storage Engine BI Connector $lookup MongoDB Compass APM Integration Profiler Visualization Auto Index Builds Backups to File System Doc-Level Concurrency Compression Storage Engine API ≤50 replicas Auditing ++ Ops Manager
  4. 4. Dynamic Lookup Data Governance & Intelligence Schema Validation BI Connector
  5. 5. $lookup • Left-outer join – Includes all documents from the left collection – For each document in the left collection, find the matching documents from the right collection and embed them Left Collection Right Collection
  6. 6. $lookup db.leftCollection.aggregate( [{ $lookup: { from: “rightCollection”, localField: “leftVal”, foreignField: “rightVal”, as: “embeddedData” } }]) leftCollection rightCollection
  7. 7. Data Governance with Document Validation Implement data governance without sacrificing agility that comes from dynamic schema • Enforce data quality across multiple teams and applications • Use familiar MongoDB expressions to control document structure • Validation is optional and can be as simple as a single field, all the way to every field, including existence, data types, and regular expressions
  8. 8. Document Validation Example The example on the left adds a rule to the contacts collection that validates: • The year of birth is no later than 1994 • The document contains a phone number and / or an email address • When present, the phone number and email addresses are strings
  9. 9. 11 MongoDB Connector for BI Visualize and explore multi-dimensional documents using SQL-based BI tools. The connector does the following: • Provides the BI tool with the schema of the MongoDB collection to be visualized • Translates SQL statements issued by the BI tool into equivalent MongoDB queries that are sent to MongoDB for processing • Converts the results into the tabular format expected by the BI tool, which can then visualize the data based on user requirements
  10. 10. 12 Location & Flow of Data MongoDB BI Connector Mapping meta-data Application data {name: “Andrew”, address: {street:… }} DocumentTableAnalytics & visualization
  11. 11. 13 BI Connector - Data Mapping mongodrdl --host --port 27017 -d myDbName -o myDrdlFile.drdl mongobischema import myCollectionName myDrdlFile.drdl DRDL mongodrdl mongobischema PostgreSQL MongoDB- specific Foreign Data Wrapper
  12. 12. 14 BI Connector - Data Mapping DRDL file • Redact attributes • Use more appropriate types (sampling can get it wrong) • Rename tables (v1.1+) • Rename columns (v1.1+) • Build new views using MongoDB Aggregation Framework • e.g., $lookup to join 2 tables - table: homesales collection: homeSales pipeline: [] columns: - name: _id mongotype: bson.ObjectId sqlname: _id sqltype: varchar - name: address.county mongotype: string sqlname: address_county sqltype: varchar - name: address.nameOrNumber mongotype: int sqlname: address_nameornumber sqltype: varchar
  13. 13. Spark Connector
  15. 15. Storage Engines Operator Family Operators WiredTiger Default storage engine starting with MongoDB 3.2. Well-suited for both read and write intensive workloads and recommended for all new deployments. Document-level concurrency model and compression MMap The original MongoDB storage engine Performs well on workloads with high volumes of reads, in-place updates and limited document size growth. Collection-level concurrency and no compression In-Memory Retains data, indexes and oplog in-memory for more predictable data latencies. Encrypted Provides at-rest encryption. Key rotation and KMIP integration. AES256-CBC default encryption. AES256-GCM and FIPS mode also available.
  16. 16. More Workloads via Storage Engines In-Memory Encrypted
  18. 18. APM Integration Advanced Tools QueryProfiling &Tuning SchemaDiscovery &QueryBuilder
  19. 19. COMPASS DEMO
  21. 21. Next Steps • Download the Whitepaper – • Read the Release Notes – • Not yet ready for production but download and try! – • Detailed blogs – • Feedback – DISCLAIMER: MongoDB's product plans are for informational purposes only. MongoDB's plans may change and you should not rely on them for delivery of a specific feature at a specific time.
  22. 22. Questions