Testing with Mongo Orchestration
•Download as PPTX, PDF•
0 likes•913 views
"MongoDB focused over the last year on conforming to common APIs and algorithms across drivers. After doing so, we needed a way to validate our consistency. Mongo Orchestration is an HTTP server with a REST API for creating and managing MongoDB configurations on a single host. In this session, we will look at how the drivers team has used YAML to define common integration tests using Mongo Orchestration and how you can use the service to test your own code.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Testing with Mongo Orchestration
Similar to Testing with Mongo Orchestration (20)
More from MongoDB
More from MongoDB (20)
Recently uploaded
Recently uploaded (20)
Testing with Mongo Orchestration
- 2. Ruby Engineer on the drivers team. Based in Berlin. Work on the gems: bson mongo mongoid bson_ext mongo_kerberos @EmStolfo
- 4. MongoDB Drivers Interface to the server. 10 official drivers available. Many started as OS projects. History
- 5. No specifications. Divergent codebases and features. MongoDB Drivers Reality
- 6. Server “interface” was not consistent. MongoDB Drivers Reality
- 7. Organizations can experience 1 interface. Specifications guide design and provide documentation. Rewrites with collective knowledge. MongoDB Drivers Revamp
- 8. Server “interface” is a single product. MongoDB Drivers Revamp
- 9. Specifications Equally valuable to the drivers team as to the community.
- 10. Server Discovery and Monitoring Spec Logic required to make an application using MongoDB highly available.
- 12. Test requirements • Unit tests. • Integration tests. • Defined as “data”. • Language-agnostic. • Reproducible scenarios. Mongo Orchestration
- 13. Testing with Mongo Orchestration 1. Mongo Orchestration 2. Driver integration tests 3. How you can use MO
- 15. Define clusters using JSON. Manipulate clusters via RESTful API. Mongo Orchestration HTTP server providing REST interface to manage multiple MongoDB processes on the same machine. Implemented in python. Maintained by Luke Lovett (llvtt)
- 16. What about Automation!? • Is intended for testing. • Starts processes on one machine. • Allows fault injection. • Does not have a UI. • Has no protection against downtime. • Has one agent, so no resiliency. • Does not handle operation tasks. Mongo Orchestration.. !
- 17. Create different cluster topologies. Single Mongo Orchestration Replica Set Sharded Cluster
- 18. Setup
- 22. Define different installations in a config file mongo- orchestration.config
- 23. Set up a Replica Set
- 24. Benefits of MO • Reproducible test scenarios. • Abstracts differing configuration options across MongoDB versions. • Uniform interface regardless of OS. • Ability to define different locations of MongoDB installations for multi- version testing.
- 25. Multi-version testing Multi-topology testing SSL Authentication Used internally Drivers integration tests
- 26. No need for cluster managers test/tools/mongo_config.rb (cluster manager in 1.x ruby driver)
- 27. Jenkins testing
- 30. define Format - YAML Tests Scenarios Our approach: “DIY” testing
- 31. benefits Format: YAML • Describes data. • Can translate to actions. • Most languages can use a YAML parsing library. • Driver authors write a reusable harness. • Changes in the spec can be communicated via additional YAML tests or changes to existing ones.
- 32. Integration tests Test driver behavior. Input: cluster state Output: driver behavior Mongo Orchestration
- 33. Test harness (in Ruby) PR #584 [WIP] Integration tests using Mongo Orchestration
- 36. Harness structure 1. Resource (MO cluster) 2. Specification (parsed YAML) 3. Test (can be run) PR #584 [WIP] Integration tests using Mongo Orchestration
- 40. Restarted primary integration test rs/connection/primary-restarted.yml (I)
- 41. Restarted primary integration test rs/connection/primary-restarted.yml (II)
- 42. Restarted primary integration testrs/connection/primary-restarted.yml (III)
- 43. How you can use Mongo Orchestration
- 44. Install
- 45. Define different installations in a config file mongo- orchestration.config
- 46. start
- 47. setup/teardown in python Send requests
- 48. stop
- 49. General uses of MO • Test your application’s behavior during a failover. • Verify your application can reliably go into readonly mode. • Easily test your application with any configuration of MongoDB.
Editor's Notes
- "Code we have written, systems we have architected, and why they are the way that they are”
- I work on the drivers team Share some insight into how we work as a team and talk about a tool we have built that makes testing across drivers so easy and that you can use as well for testing.
- mongodb know for its great developer experience one benefit is that we offer drivers in 10 languages, with others in the community history: some of the drivers were developed as OS projects, then brought internal. Hired the authors http://www.mongodb.com/blog/post/announcing-next-generation-drivers-mongodb http://opensourcebridge.org/sessions/1580 When developers get started with MongoDB, they are delighted to discover that it supports the programming languages they love. For as long as the database has been around, we’ve offered at least eight official drivers that help to make developers productive quickly. Moving in lockstep with the server to support all major changes, these drivers ensure that you’re always able to take advantage of the latest features in the database. Over the years, each driver engineering team has designed its own approach to querying the database, providing fault tolerance, exposing database options, offering an asynchronous interface, and implementing dozens of other features. Each driver has also accumulated its fair share of technical debt, having been initially designed for a much simpler version of MongoDB. Meanwhile, as MongoDB has proliferated and matured, the number of organizations that use more than one driver has increased. Although each driver strives to be idiomatic in the language it supports, there is little reason why the core CRUD API needs to vary across drivers, and after several years, there was some divergence. We needed a fresh start, driven by more formal specifications and informed by all the learning we have collectively done since MongoDB was first introduced in 2009. We are excited to announce that in the next few weeks we will be rolling out new drivers for all languages that build on everything we've learned. Stay tuned for major releases from the Java, .NET, Python, Node.js, and Ruby teams. PHP and Perl will follow soon after. An updated C++ driver is also in development.
- mongodb know for its great developer experience one benefit is that we offer drivers in 10 languages, with others in the community history: some of the drivers were developed as OS projects, then brought internal. Hired the authors http://www.mongodb.com/blog/post/announcing-next-generation-drivers-mongodb http://opensourcebridge.org/sessions/1580 When developers get started with MongoDB, they are delighted to discover that it supports the programming languages they love. For as long as the database has been around, we’ve offered at least eight official drivers that help to make developers productive quickly. Moving in lockstep with the server to support all major changes, these drivers ensure that you’re always able to take advantage of the latest features in the database. Over the years, each driver engineering team has designed its own approach to querying the database, providing fault tolerance, exposing database options, offering an asynchronous interface, and implementing dozens of other features. Each driver has also accumulated its fair share of technical debt, having been initially designed for a much simpler version of MongoDB. Meanwhile, as MongoDB has proliferated and matured, the number of organizations that use more than one driver has increased. Although each driver strives to be idiomatic in the language it supports, there is little reason why the core CRUD API needs to vary across drivers, and after several years, there was some divergence. We needed a fresh start, driven by more formal specifications and informed by all the learning we have collectively done since MongoDB was first introduced in 2009. We are excited to announce that in the next few weeks we will be rolling out new drivers for all languages that build on everything we've learned. Stay tuned for major releases from the Java, .NET, Python, Node.js, and Ruby teams. PHP and Perl will follow soon after. An updated C++ driver is also in development.
- That means that there were no specs in the beginning Codebases diverged from the beginning, features differed, APIs different Making the server interface inconsistent http://www.mongodb.com/blog/post/announcing-next-generation-drivers-mongodb http://opensourcebridge.org/sessions/1580 When developers get started with MongoDB, they are delighted to discover that it supports the programming languages they love. For as long as the database has been around, we’ve offered at least eight official drivers that help to make developers productive quickly. Moving in lockstep with the server to support all major changes, these drivers ensure that you’re always able to take advantage of the latest features in the database. Over the years, each driver engineering team has designed its own approach to querying the database, providing fault tolerance, exposing database options, offering an asynchronous interface, and implementing dozens of other features. Each driver has also accumulated its fair share of technical debt, having been initially designed for a much simpler version of MongoDB. Meanwhile, as MongoDB has proliferated and matured, the number of organizations that use more than one driver has increased. Although each driver strives to be idiomatic in the language it supports, there is little reason why the core CRUD API needs to vary across drivers, and after several years, there was some divergence. We needed a fresh start, driven by more formal specifications and informed by all the learning we have collectively done since MongoDB was first introduced in 2009. We are excited to announce that in the next few weeks we will be rolling out new drivers for all languages that build on everything we've learned. Stay tuned for major releases from the Java, .NET, Python, Node.js, and Ruby teams. PHP and Perl will follow soon after. An updated C++ driver is also in development.
- That means that there were no specs in the beginning Codebases diverged from the beginning, features differed, APIs different Making the server interface inconsistent http://www.mongodb.com/blog/post/announcing-next-generation-drivers-mongodb http://opensourcebridge.org/sessions/1580 When developers get started with MongoDB, they are delighted to discover that it supports the programming languages they love. For as long as the database has been around, we’ve offered at least eight official drivers that help to make developers productive quickly. Moving in lockstep with the server to support all major changes, these drivers ensure that you’re always able to take advantage of the latest features in the database. Over the years, each driver engineering team has designed its own approach to querying the database, providing fault tolerance, exposing database options, offering an asynchronous interface, and implementing dozens of other features. Each driver has also accumulated its fair share of technical debt, having been initially designed for a much simpler version of MongoDB. Meanwhile, as MongoDB has proliferated and matured, the number of organizations that use more than one driver has increased. Although each driver strives to be idiomatic in the language it supports, there is little reason why the core CRUD API needs to vary across drivers, and after several years, there was some divergence. We needed a fresh start, driven by more formal specifications and informed by all the learning we have collectively done since MongoDB was first introduced in 2009. We are excited to announce that in the next few weeks we will be rolling out new drivers for all languages that build on everything we've learned. Stay tuned for major releases from the Java, .NET, Python, Node.js, and Ruby teams. PHP and Perl will follow soon after. An updated C++ driver is also in development.
- Organizations using more than more driver has increased. Increasing pressure to conform to a common API and set of features. Support team had trouble learning how to support drivers because they were too inconsistent. Couldn’t learn from each other’s mistakes or successes. Next gen drivers were released. Some were a rewrite, some were large improvements. Many authors have been working with MongoDB for years. Collective knowledge individually and across languages. No reason to have differing CRUD APIs. No reason to reinvent algorithms in each language for detecting the same server behavior and scenarios. Transitioning between languages should be easier for MongoDB users.
- That means that there were no specs in the beginning Codebases diverged from the beginning, features differed, APIs different Making the server interface inconsistent http://www.mongodb.com/blog/post/announcing-next-generation-drivers-mongodb http://opensourcebridge.org/sessions/1580 When developers get started with MongoDB, they are delighted to discover that it supports the programming languages they love. For as long as the database has been around, we’ve offered at least eight official drivers that help to make developers productive quickly. Moving in lockstep with the server to support all major changes, these drivers ensure that you’re always able to take advantage of the latest features in the database. Over the years, each driver engineering team has designed its own approach to querying the database, providing fault tolerance, exposing database options, offering an asynchronous interface, and implementing dozens of other features. Each driver has also accumulated its fair share of technical debt, having been initially designed for a much simpler version of MongoDB. Meanwhile, as MongoDB has proliferated and matured, the number of organizations that use more than one driver has increased. Although each driver strives to be idiomatic in the language it supports, there is little reason why the core CRUD API needs to vary across drivers, and after several years, there was some divergence. We needed a fresh start, driven by more formal specifications and informed by all the learning we have collectively done since MongoDB was first introduced in 2009. We are excited to announce that in the next few weeks we will be rolling out new drivers for all languages that build on everything we've learned. Stay tuned for major releases from the Java, .NET, Python, Node.js, and Ruby teams. PHP and Perl will follow soon after. An updated C++ driver is also in development.
- We were more rigorous about writing specifications. We open-sourced are specs for two main reasons. Ourselves Community Transparency with other authors, transparency with our users
- One spec we like to talk about is the SDAM spec. It’s particular useful for us to have a standard solution internally and to our users to understand how the driver reacts to changes in a cluster.
- That’s great but how do we inform each other about changes to the spec and how do we validate our own compliance?
- We wanted a way to test. Unit tests were fairly simple to write. Tested a specific part of the code. But how do we do integration tests? Requirements were data, language agnostic, reproducible scenarios. That’s where we realized we needed a service. We built Mongo Orchestration
- “I know what you’re thinking“ - question about automation
- automation doesn’t require a UI automation follows best practices for the order in which you should do ops ops: resize oplog
- Brighten the text color
- Why we chose YAML MongoDB config in YAML Most languages have a YAML parsing library
- change this
- Wrapper for a cluster resource in MO Define a base url Methods for creating the cluster, stopping the cluster, making requests to the cluster
- Specification represents a single YAML test definition Some test definitions have multiple phases, requiring actions and requests sent to the MO cluster to get it to a particular state. A number of tests validate the success or failure of the test.
- A single test can be one of three types. Example is a client operation and expected outcome.
- change this
- change this
- change this
- pip is a python package manager
- we saw this before… Used for multi-version testing
- Other examples used Ruby, this uses python. You’ll see some similarities