From talk given at Spring One 2gx Dallas, 2014 Application configuration is an evolution. It starts as a hard-coded strings in your application and hopefully progresses to something external, such as a file or system property that can be changed without deployment. But what happens when other enterprise concerns enter the mix, such as audit requirements or access control around who can make changes? How do you maintain the consistency of values across too many application servers to manage at one time from a terminal window? The next step in the application configuration evolution is centralized configuration that can be accessed by your applications as they move through your various environments on their way to production. Such a service transfers the ownership of configuration from the last developer who touched the code to a well-versed application owner who is responsible for the configuration of the application across all environments. At Dealer.com, we have created one such solution that relies on Apache ZooKeeper to handle the storage and coordination of the configuration data and Spring to handle to the retrieval, creation and registration of configured objects in each application. The end result is a transparent framework that provides the same configured objects that could have been created using a Spring configuration, configuration file and property value wiring. This talk will cover both the why and how of our solution, with a focus on how we leveraged the powerful attributes of both Apache ZooKeeper and Spring to rid our application of local configuration files and provide a consistent mechanism for application configuration in our enterprise.

1. Centralized Application Configuration
with Spring and Apache ZooKeeper
By Ryan Gardner, Dealer.com
© 2014 SpringOne 2GX. All rights reserved. Do not distribute without permission.

2. How do you configure your applications?
2
http://b.socrative.com/login/student/
config2gx

3. Who is Dealer.com?
We make software that car dealers use
to fulfill their digital marketing vision.
3

4. The Remote Configuration Project
4

5. Some ways we had configured our applications
• Hardcoded values in code
• Properties file – per environment or merged
• Host files for database
• JNDI context files
5

6. Motivating factors
• Developer Efficiency
• Redeploying an application just to change a configuration is a drag
• Having to edit N config files whenever a single application changed is a
hassle
• Security Compliance
• Limit access to production databases
• Auditing and approval process for configuration changes
• Systems Engineering
• Can’t make certain changes without involving developers
6

7. Framework Development – Guns n Roses style
"Welcome to the jungle"
Thanks.
"We've got fun and games"
Cool.
"You're in the jungle"
We've established this
"You're gonna die!"
Wait what?
https://twitter.com/OhNoSheTwitnt/status/469838190141255680
7

8. High Level Overview
8

9. Three main components
9

10. 10

11. Key Concepts in the Remote Configuration
Configuration
A set of properties or values necessary to create an object.
Examples:
Database configuration, FTP endpoint configuration,
HTTP Proxy Factory Bean Configuration,
11

12. Tripoli – editing a configuration
12

13. Separation of environments
• We edit our configurations in one spot, but applications are only
able to retrieve configurations for the environment they are
running in
• Secrets – such as passwords or encryption keys – are encrypted
with an environment-specific key
• Tripoli has all the keys, each environment will only have a key specific to it
13

14. Configuration inheritance – avoiding copy & paste
By Environment
• Configuration can be set at a global level
and overridden at each environment
By Path
• Nodes added with /’s in the name will inherit data from nodes
above them.
• /remoting/core-services/UserLocator inherits values from
/remoting/core-services
14

15. Key Concepts in the Remote Configuration
Bindings
Identifying which configurations an application uses, and what
the application wants to call them
Example: An application that needs to talk to a certain database, look up users
from a remote service, and send data to a remote SFTP site would have bindings
such as:
jdbc/user-database is bound to the configuration called user-database-config
15

16. Tripoli – editing bindings
16

17. Creating objects, not properties*
• Configure once – use everywhere
• Avoid having to copy-and-paste boilerplate setup code
* properties are supported too
17

18. Behind the Scenes
18

19. Apache Zookeeper
• Hierarchical data registers
• Designed for high-throughput, low-latency, highly-available
• Nodes in zookeeper are called “znodes”
• Each path can stored data
• Designed for storing small amounts of data in the znodes
(KB, not MB)
• For more info:
• https://cwiki.apache.org/confluence/display/ZOOKEEPER/ProjectDescriptio
n
19

20. Where do we store this data?
20
• Versioned configuration in ZooKeeper as JSON
• In ZooKeeper znodes:
• /bindings/<binding name>
overrides:
• /bindings/<habitat>/<datacenter>/<binding name>
• /configurations/<configuration name>/

21. Talking to ZooKeeper
• Use Curator framework
• We use ACLs in ZooKeeper ensure apps can’t read data for
other environments
• We use a SASL config file on the machines to provide the
ZooKeeper credentials
21

22. Exhibitor – makes managing ZooKeeper easier
22

23. How do the objects get created?
• Two classes for each remotely-configurable object,
the config and the creator
• Configs use bean-validation annotations and a special annotation
on the config-field to explain what the config field does.
• This populates the tool-tips in the browser window and is used to ensure
that only valid entries are put into the fields
23

24. An example config class
24
@ConfigCategory("ftp”)
public class SpringIntegrationSftpSessionFactoryConfig extends Config {
@Required
@ConfigField(description = "The host name of the SFTP server.”)
private String host;
@Port
@ConfigField(description = "The port of the SFTP server. Defaults to 22.”)
private Integer port;
…

25. A config class (continued)
25
@Required
@Password
@ConfigField(description = "The private key used to establish the SFTP
connection.”)
private ConfigPassword privateKey;
@Password
@ConfigField(description = "The passphrase for the private key. Defaults to
empty string.”)
private ConfigPassword privateKeyPassphrase;

26. Creators take the config and return an object
26
public class ExampleObjectCreator extends
ObjectCreator<SomeConfig,ExampleObject> {
@Override
public ExampleObject create(SomeConfig) {
// do whatever is needed to create the object
return new ExampleObject();
}
}

27. Kinds of creators we have made
• Database connection (various connection pools)
• Mongo connection pools
• RPC remoting proxies (Spring HTTP Invoker, etc)
• REST resources
• Redis connections
• Properties / System properties
• FTP and SFTP connections
• Executor services
• RabbitMQ
• SOLR
• ElasticSearch
• … more
27

28. How do apps use this?
28

29. First pass – XML namespace parser
29
<beans … xmlns:remote-config=“http://www.dealer.com/schema/remote-config”
…>
<remote-config:lookup id="dataSource" name=”jdbc/my-datasource" />
<remote-config:remote-config-property-source id="myProps"
name="properties/my-props" />
</beans>

30. Second pass – Auto-config with XML
30
<beans … xmlns:remote-config=“http://www.dealer.com/schema/remote-config”
…>
…
<remote-config:auto-create />
…
</beans>

31. Third pass - @EnableRemoteConfig
31
@EnableRemoteConfig
public class ApplicationConfig {
// insert tweetable app here.
}

32. Accessing properties
32

33. Integrating remote properties into spring
• We create a PropertySource
• And we create a PropertySourcesPlaceholderConfigurer
33

34. Using properties via @Value
34
@Bean
public SomeBean someBean (@Value(“${some.value}”) someValue) {
return new SomeBean(someValue)
}
…
@Value(“${some.remote.property.value}”)
private String someValue;

35. Deeper dive – demo &
look at some of the code
35

36. Future plans & extensions for this
36

37. Questions?
37

38. 38
Learn More. Stay Connected
Tweet: “#s2gx talk about zookeeper blew my mind!
Thanks @ryebrye and @springcentral”
@springcentral | spring.io/video