SpringOne Platform 2017 Marcello de Sales, Intuit "Configuration management at Intuit has been reshaped over the last 18 months since the adoption of Spring Cloud Config Server. This work represents a breakthrough in configuration management practices that are changing how Intuit implements configuration management since the company’s inception over 20+ years ago. In essence, any application ranging from desktop and service monoliths started their migration to the cloud without breaking their own DNA: configuration was still part of the binary built on Continuous Integration to be deployed in different data centers. As a consequence, we were still facing the same old challenges: what happens when a new configuration change is required for the entire fleet on multiple private data centers and the cloud? The new answer lies in the adoption of the Spring Cloud Config Server as our One Intuit Configuration Service using the SaaS model, which represents a new shift from manual Operational changes to the simple Pull Requests on related Github Enterprise repositories. Needless to say, ranging from small internal services to the giants of TurboTax and Quickbooks that are used by millions of users worldwide, there are amazing results with the adoption of this Configuration practice and service such as the decreased time to change configuration from hours to minutes without involving Operations team while getting consistent configuration across a fleet of services. On the other hand, the strong adoption rate brought up a set of new challenges for us to support this new approach in the Enterprise: how to properly architect Spring Cloud Config to be deployed as a SaaS application in the Enterprise? how can we guarantee that users are pushing valid configuration properties to their repo? How can we help them debug their properties consistently, but without relying solely on Github Pull Requests? Finally, what if we need to replicate this solution for Mobile clients? Do we need to deploy hundreds of Configuration servers in the Cloud, and consequently, take the bite on cost? Overall, the solutions to the questions above are comprised of SaaS deployment of the Spring Cloud Config with some enterprise tweaks for security and performance. Then, we have created a Github Pre-receive hook called Spring Cloud Config Validator to validate user’s config repositories and a web application called Spring Cloud Config Inspector that helps users debug their config keys as associated values, secrets, etc. Lastly, our Spring Cloud Config Publisher solution allows users to use their applications to console a subset of their config properties from an Amazon S3 bucket that the publisher will be publishing to at every new valid commit.

1. Cloud Configuration Ecosystem at
Intuit
Marcello de Sales, Staff Software Engineer, Intuit
@marcellodesales
1

2. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Motivation
2

3. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Infrastructure
• 20+ year-old software startup
• Mix of legacy code with brand new code
• We declared our commitment to move to Amazon AWS
• Teams have accelerated the migration of current
• “Everything as a Service”
• Binaries and Configuration usually delivered together
• Teams, with shared confis, taking hours to change a single configuration
• Internal Service Directory
3

4. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Production Use Cases
4
TurboTax Apps
Binary + Configs

5. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Infrastructure – Don’t Touch it!
5

6. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Configuration Management
• It is NOT
• Operational System Settings
• It is Service/Application Level Configuration
• Business Logic Properties, Secrets
• Typically implemented as property files
6

7. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Requirements
7
• Cloud-ready standardization for agile changes with minimal impact
• 12-Factor: “strict separation of config from code”
• Reuse same binary for multiple environments
• Single Source of Truth
• Auditable / History
• Integration with Intuit Secrets Service

8. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Solutions
8
• Config Server + Client:
Backend Services
• Config Publisher:
Mobile Clients

9. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Config Service
• Consolidated clones as a Service in the Services Directory
• Spring Cloud Config + Config Repos in Github Enterprise
• Hardened: Security, Performance, Logging, Monitoring
• Performance: HTTP Conditional Req avoids round-trips to Github
• Github Enterprise integration: Auditable and Historical
• Aligns well with Git Flow processes
9

10. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Config Service Client
10
• Hardened: similar to config service
• Available as an Internal Shared library
• Augmented SpringBoot Client version
• Spring and Plain Java Clients also created internally
• Integration with Intuit Secrets Management through library
• Resilient: current state persists local and in Amazon S3

11. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Config Service Client
11
Individual
Team
Members
Config Repo
Retrieve
configs Fetch
Config
Files
Config Repo
Retrieve
Secrets
Save
Last
State

12. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Config Publisher
• Requirement: enable IOS and Android teams to quickly make shared config changes
• Separate Mobile traffic to Amazon S3
• Empower engineers to declare what they need to be published
• Maintain the same URL paths: as if S3 is a Spring Cloud Config Server
• https://configs.intuit.net/label/appName-profiles.ext
• https://appName-bucket. s3.amazonaws.com/label/appName-profiles.ext
• Provide security capabilities to help prevent man-in-the-middle attacks
12

13. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Config Publisher
13
Config Publisher
Amazon S3
Amazon CloudFront
Config Repo

14. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Config Publisher
14
Github
Webhook
Delivery ID

15. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Tools
15

16. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Feedback: difficult mindset shift
• Document the design model around the Spring Environment
• App Name, Profiles, Label
• Understanding the resolution of property values is hard
• Permutations of profiles sequence
• Values across different labels
16

17. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Intuit Spring Cloud Config Inspector
• Inspect a Configuration Repo and render the values
• Helps users figure out what are the current values deployed
• Debugging becomes a much easier task
• Embeddable or Standalone React App
• Currently Open-sourced as Apache 2.0
• https://github.com/intuit/intuit-spring-cloud-config-inspector
17

18. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Config Inspector
18

19. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Config Inspector: raw output
19

20. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Config Inspector: github parsing
20

21. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Config Inspector: diff across
21

22. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Config Inspector: diff across
22

23. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Config Inspector: github and config api logs
23

24. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Feedback: Users can break configs
• Manually updating configuration files is very fragile
• Config repo users can inject configuration problems
• Serious consequences for the Config Server
• Config Server does NOT serve broken configuration
• Client application quietly does NOT get updated
24

25. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Intuit Spring Cloud Config Validator
25
• Github Enterprise Pre-receive hook: Automatically validates configs pushed
• Manual commits pushed via CLI
• Changing configuration values in the Github Commit UI
• Commits from Pull Request
• Currently Open-sourced as Apache 2.0
• https://github.com/intuit/intuit-spring-cloud-config-validator

26. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Config Validator: config repo setup
26

27. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Config Validator: rejecting commits
27

28. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Future
28

29. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
What’s next?
• Config Server is a successful paradigm shift for Intuit
• Requires education about best practices for Config Design
• Easier migration of services to the cloud / containers
• Config Publisher empowered our mobile teams to scale in the cloud (serverless)
• Config changes from hours to minutes
• Delayed Open-source work
• Config Inspector and Config Validator helps driving productivity
29

30. Learn More. Stay Connected.
Cloud Configuration Ecosystem at Intuit, Thursday 11:50, Room 2012
https://github.com/intuit/intuit-spring-cloud-config-inspector
https://github.com/intuit/intuit-spring-cloud-config-validator
NEXT UP: A Journey Of Mobile, Microservices, and Speed to Market at the
World's Largest Home Improvement Retailer
30
#springone@s1p