A talk around various development environment automations us and other ThoughtWorkers have seen and built on many different projects, and learnings around best practices. We've seen serious work put into this drastically increase the productivity of developers, and solve a lot of the problems that microservices can otherwise cause.

1. Laptop Devops
Daniel Lockman - Marcos Mercuri
Putting Modern Infrastructure Automation to Work for Local
Development

2. Who are you, folks?
Daniel Lockman
Lead developer at TW, started in NY.
Interests include fullstack dev, DevOps, and
data processing/engineering
Marcos Mercuri
Sr Software developer at TW. Been learning
how to make good software for more than
10 years, and still learning.

3. Why local development and
testing?
THE WHY THE HOW
THE WHAT
NOW
Featuring our friend the dev
workstation
How do these patterns fit
into the modern software
lifecycle?
What’s this all about?

4. But what’s the problem?

5. Your friendly neighborhood, the
test environment
Shared test environments have some significant problems in our experience:
● Contention over resources
● Difficulty of change attribution when things go wrong
● Special snowflake test environments
● “NOBODY TOUCH NOTHIN” syndrome
● The data... isn’t always the best

6. So…….. do it live?
Testing in production has a few clear advantages, and in certain cases can be a
simpler solution.
● You can be sure your code works against real external dependencies
● Doesn’t use additional test resources, less to maintain
● Works best for read-only dependencies
● Need to be very careful when manipulating data or environment for
testing purposes
Won’t be the focus of this presentation.

7. Why Local Development?

8. Wait! What is local development
exactly?
“The ability to develop software without external dependencies.”
● Not reliant on an internet connection
● Not reliant on software outside of the laptop
● Not sharing dependencies with other developers
● Ability to create meaningful business value without these dependencies

9. Some benefits
By writing code on my own computer and checking in “finished” work, I gain
some advantages.
● No internet needed!
● Independent sandboxed environment
● Can make potentially breaking changes without affecting others
● Full control of configuration
● Full access to debugging and logs without external noise
● See if my code actually works before committing to shared codebase
But aren’t some of these really testing…?

10. So what we really mean is testing
locally
Meaningful development locally means I need to confirm that my code actually
works:
● Calls external services correctly to orchestrate or for getting data
● Abides by contracts for services that depend on it
● Interacts correctly with tightly coupled infrastructure
○ Ex. Database, search, messaging middleware.
● Correctly handles reasonably prod-like test data or configuration

11. Principles of Local Development

12. ASAP + ASAP + ACAP
● As Soon As Possible
○ If I can test locally, I don’t need to wait to merge, deploy, etc
● As Sure As Possible
○ I can code on the airplane, and commit my changes, confident that it won’t break on
successive environments
● As Cheaply As Possible
○ I don't need extra environments
Shorten the feedback loop

13. But I have all these other tests!
Seems pointless...
● You still need to
○ Check it connects well with your infra
○ Connects well with the others dependencies in your machine
● Even with all your automated tests, there’s still room for manual testing
○ For exploratory testing
○ If you want to try some configurations tweaks
○ When a TDD test case is harder to write
There’s always going to be a scenario where you need to boot your service

14. So How Does This Work?

15. Introducing the “Developer
Workstation”

16. Setting up Our Own
Organization’s Services

17. The Problem
● Lots of different developer laptop environments
● Need to set things up consistently
● Lots of services and libraries that may conflict with local setup

18. Isolated Software Installs
● Vagrant
● Containers
○ We’re using docker, but there are other options
○ Will probably want some sort of orchestration to run these

19. common.yml

20. Dependencies on Infrastructure

21. Just Download Some Images?
● Often complicated setups or installation of 3rd party systems
● Configuration can widely change the behavior of the software
● Don’t want to write loads of custom scripts

22. More Hammers!
● Infrastructure Configuration/Orchestration Frameworks
○ Puppet / Chef / Ansible / Saltstack
● Fancier CI pipelines with image layers

25. Dependencies on External
Services
● How to deal with external dependencies
○ PACT for contract testing
○ Mountebank for stubbing over the wire

26. Example - Mountebank

27. How to manage my test data?
Features should be created with basic test datasets
● Flyway
○ Database schema versioning
○ Test data bootstrapping

28. Example - Flyway

29. Networking

30. Our Problems
● Local network environment can be very different than “real” environments
● Lots of services may have conflicting ports
● Orchestration heavy services may have lots of dependencies
○ Don’t want huge test configuration files

31. Our options
● Docker Networking (beware!)
○ More advanced with Kubernetes, Openshift, etc.
● Networking Middleware
○ Nginx, apache, consul
○ Bonus points if you’re already using it in production
● Full service discovery locally
● Hacking it together
○ This can get complicated quickly
○ Generally need custom scripts for each new service

32. Guidelines for Using this Magic

33. What do I want to test locally?
● End-to-end core business logic of service
● Interaction with tightly coupled dependencies
○ Database
○ Messaging system
● Contracts with other services or 3rd parties
● UI
● Bug reproduction
● Exploratory testing

34. What do I NOT want to test
locally?
● Don’t test performance
○ Too many things different from prod-like environment
● Don’t test the platform
● Depending on the size of your microservice architecture, you don’t want to
test all end-to-end features

35. Bringing it All Together

36. Local Development and CI/CD
Reinforce each other
● All services can be created and deployed programmatically
● Local development environment is a step away from creating CI
environments
● Investment in automation should be promoted upwards toward
production
○ Phoenixable environments and tracking of infrastructure changes is very important
for production
○ Test/local environments can be a special case of “real” environment infra-as-code

37. But do I even need CI anymore?
● YES
○ Local development is still only testing your changes
○ There will always be some things you can’t test locally
● BUT…
○ You can be a bit more liberal with your pipeline
○ Even with more overall steps, your pipeline will fail less frequently
■ Can focus on things that are harder to test locally
○ Average speed through the pipeline will increase

38. THANK YOU
For questions or suggestions:
Daniel Lockman - dlockman@thoughtworks.com
Marcos Mercuri - mmercuri@thoughtworks.com