MongoDB Evenings Minneapolis
March 29, 2016 at VidkuHQ
Medtronic's MongoDB Journey
Jeff Lemmerman, Principal Software Engineer, Medtronic
Matthew Chimento, Engineering Manager, Medtronic
1. PRESENTED BY:
MATT CHIMENTO: ENGINEERING MANAGER
JEFF LEMMERMAN: PRINCIPAL SOFTWARE
ENGINEER
MEDTRONIC’S
JOURNEY WITH
MONGODB
2. 2
INTRODUCTION
Jeff Lemmerman
B.S. Physics, B.S. Astrophysics
University of Minnesota
MS Software Engineering
University of Minnesota
Pr. Software Engineer – Medtronic (2006-Present)
Matt Chimento
B.S. Computer Engineering
Kettering University
Master of Business Administration (2014)
University of Minnesota Carlson School of
Management
Engineering Manager – Medtronic (2006-2014),
Target (2014-2015), Medtronic (2015-Present)
3. 3
MEDTRONIC ENERGY AND COMPONENT CENTER
MECC est. 1976
Research, Development, and Manufacturing
Manufacture components for devices
Census – 1200 Employees
Plant Size – 190,000 Square Feet
40,000 Manufacturing
15,000 R&D Labs
38,000 Office
97,000 Common, Support, Warehouse
4. 4
MEDTRONIC USE CASES
Results API: RESTful API for storing results from automated test equipment
Component Database: Complete 360˚ View of the components we manufacture at
MECC
Tests: Storing test information for automated test equipment
Other Medtronic groups are using MongoDB differently
6. 6
RESULTS API: PROBLEM
VARIETY OF DATA INTENSIVE TESTS
Multivariate Analysis
Statistical Process Control
Waveform Measurements
Operational Dashboards
7. 7
HOW THE JOURNEY BEGAN
LEARNING NEW CONCEPTS AND TOOLS: WHAT CAN I BUILD?
9. TESTS:
PROBLEM
0.6M unique battery tests
Since 1976
Export as Analysis-Ready Data-Sets
Many thousand test channels for primary and
rechargeable batteries
Some tests extending >15 years
Each system has very different Test
Information
Commercial and Custom Systems
MECC Test Systems Data Management
Rechargeable Battery
Test Systems and
Ovens
Batteries on Test Boards
in Oven
9
11. 11
TEST INFORMATION – RELATIONAL DATABASE
Test Table System Specific Table(s)
Test Table Properties Table
Test Table
12. 12
TEST INFORMATION – RELATIONAL DATABASE
Test Table
Key Trade Offs For Flexibility –
Joining System Specific Tables
Querying and Processing Key/Value Table
Querying and Processing Structured Text Fields (JSON/XML)
15. 15
COMPONENT DATABASE - UPDATE
Full Material Consumption
Full Manufacture Step History
Summarized Data
Scrap Codes
15
16. 16
WHAT HAVE WE LEARNED IN 4 YEARS
MongoDB Training, Support, Documentation have remained excellent
Still struggling with standardization vs. flexibility
Responsibility for querying and processing data in MongoDB –
Application vs. MongoDB vs. 3rd Party Tools
Can be difficult to keep up with fast release cycle, new features -
C# Driver 2.x – methods, query builders/filters, Async methods
17. 17
WRITING CUSTOM SERIALIZATION IS TRICKY
Automapping functionality works in most cases
Can add control over how class properties are serialized
Add serialization information or override Serialize/Deserialize Methods
Our roles and how long we’ve been at the company, how long we’ve faced the problem
How the project came to be
The project team
Focus was on getting data in:
Needed to solve problem with Volume and Velocity: Whitepaper, and Variety: MongoDB provided Flexibility via flat schema
Focus was on getting data in:
Needed to solve problem with Volume and Velocity: Whitepaper, and Variety: MongoDB provided Flexibility via flat schema
Focus on web base query interface to minimize direct access
Generally know commonly used search patterns
Could implement a results repository with a data adapter for each data source, but still may not have all info needed to get results.
Give me all the results for this component? Look in each result repository and merge results together…all at reporting time!
Each row is a component and the columns are the things we know about each component
Difficult to store uncontrolled data formats
Scaling via big iron or custom data marts/partitioning schemes
Schema must be known at design time
Impedance mismatch with agile development and deployment techniques
Doesn’t map well to native language constructs
Data is optimized for storage
Data stored is very compact
Rigid schemas have led to powerful query capabilities (very complex queries, consequences of left/right/inner joins)
Generic data types make queries less effective
Robust ecosystem of tools, libraries, integrations
40+ years old!
5+ tables in a single Mongo document
20 Production Steps
30 Subcomponents
150 Facts