The document discusses the operation of high-velocity organizations using Spring Cloud microservices, highlighting continuous delivery principles and challenges such as complex systems and customer expectations for quality and availability. It outlines key components of Spring Cloud microservices architecture, including service discovery, centralized monitoring, security, and circuit breakers, alongside coding examples for implementation. The document emphasizes the importance of a DevOps culture and automation in managing microservices effectively.

1. Operating a High Velocity Large Organization
with Spring Cloud Microservices
Noriaki (Nori) Tatsumi
Capital One

2. Who we are
2

3. Data Lake
Powered by C1 Data Intelligence Team
CC Image by Stanislav Sedov on Flickr

4. Continuous Delivery
Build, test, and release fast and frequently to operate high velocity organization
4
Prerequisite. Not luxury.

5. Continuous delivery principles
• Eliminate non-value added actions
• Release process must be repeatable and reliable
• Quality is built in
• Version everything
• Done = “Released”
• Small batches of features and experimentations
• Everyone is responsible
• Kaizen – Improve continuously
5

6. Our challenges
Category Examples
Complex systems Large code base and deployment
Implementation sensitivity
Test feedback speed
Many teams Collaboration of design & technology
Merge conflicts
Non functional qualities Security
High availability, reliability, maintainability
Same qualities across components/features
Compliance Traceability
Data lineage
Legacy and 3rd party
applications
Implementation of same qualities as the rest
Processes Time consuming reviews and approvals
6

7. Customer expectation
• Quality
• Availability
• Velocity - fast and frequent deliveries of features
But they don’t know about the technical
challenges
7

8. 8
Microservices architecture
http://martinfowler.com/articles/microservices.html
The twelve factor methodology
http://12factor.net/

9. 9
Intranet/VPN

10. 10

11. 11
Spring Cloud
For foundation of microservices architecture

12. Spring Cloud
• *Spring Cloud Config
• *Spring Cloud Netflix
• Spring Cloud Bus
• Spring Cloud for Cloud Foundry
• Spring Cloud Cloud Foundry Service
Broker
• Spring Cloud Cluster
• Spring Cloud Consul
• *Spring Cloud Security
• *Spring Cloud Sleuth
• Spring Cloud Data Flow
12
• Spring Cloud Stream
• Spring Cloud Stream Modules
• Spring Cloud Task
• Spring Cloud Zookeeper
• Spring Cloud for Amazon Web
Services
• Spring Cloud Connectors
• Spring Cloud Starters
• Spring Cloud CLI

13. Technology selection
• JVM-based
• Developer productivity
• Production support
• Spring Boot
• Opinionated view for developer productivity
• Production grade qualities
• Netflix OSS
• Proven microservices technology
13

14. Decoupling
14
• Work in parallel
• Quicker and smaller deploys
• Domain driven design
• Do one scope of things well

15. Decoupling
15
• Work in parallel
• Quicker and smaller deploys
• Domain driven design
• Do one scope of things well
• Technology stack agnostic

16. Decoupling
16
• Work in parallel
• Quicker and smaller deploys
• Domain driven design
• Do one scope of things well
• Technology stack agnostic
• Functions are shareable

17. Decoupling
17
• Work in parallel
• Quicker and smaller deploys
• Domain driven design
• Do one scope of things well
• Technology stack agnostic
• Functions are shareable
• Independent scalability by comp.
• Greater resiliency & availability
• Continuous delivery friendly

18. This looks hard to orchestrate
18

19. Service discovery (registry)
• Netflix Eureka (HashiCorp Consul, Apache Zookeeper)
• Locate services
• Load balancing
• Failover
• Resiliency
19

20. 20
<application>
<name>...</name>
<instance>
<instanceId>... </instanceId>
<hostName>... </hostName>
<app>...</app>
<ipAddr>...</ipAddr>
<status>UP</status>
<overriddenstatus>UNKNOWN</overriddenstatus>
<port enabled="false">...</port>
<securePort enabled="true">...</securePort>
<countryId>1</countryId>
<dataCenterInfo class="com.netflix.appinfo.AmazonInfo">
<name>Amazon</name>
<metadata>
<accountId>...</accountId>
<local-hostname>... </local-hostname>
<instance-id>...</instance-id>
<local-ipv4>...</local-ipv4>
<instance-type>...</instance-type>
<vpc-id>...</vpc-id>
<ami-id>...</ami-id>
<mac>...</mac>
<availability-zone>...</availability-zone>
</metadata>
</dataCenterInfo>
<leaseInfo>
<renewalIntervalInSecs>...</renewalIntervalInSecs>
<durationInSecs>...</durationInSecs>
…..

21. Service discovery (registry)
21
@SpringBootApplication
@EnableEurekaServer
public class Discovery {
public static void main(String[] args) {
SpringApplication.run(Discovery.class, args);
}
}

22. Service discovery (registry)
22
eureka:
instance:
appname: discovery
app-group-name: bedrock
lease-renewal-interval-in-seconds: 30 #sending heartbeats; 90 secs removes from registry
eureka.instance.preferIpAddress: false
home-page-url: http://${spring.cloud.client.hostname}:${server.port}/discovery
health-check-url:
http://${spring.cloud.client.hostname}:${management.port}${management.context-path}/health
status-page-url:
http://${spring.cloud.client.hostname}:${management.port}${management.context-path}/info
password: changeme
dashboard:
path: /discovery
client:
serviceUrl:
defaultZone: http://user:changeme@peer2:8761/eureka/
healthcheck:
enabled: true

23. Making your Spring Boot app discoverable
23
@SpringBootApplication
@EnableDiscoveryClient
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}

24. Making your Spring Boot app discoverable
24
protected static void registerShutdownHooks() {
Runtime.getRuntime().addShutdownHook(new Thread() {
@Override
public void run() {
LOG.info("Shutting down, unregister from discovery service!");
DiscoveryManager.getInstance().shutdownComponent();
}
});
…..
}

25. Making your Spring Boot app discoverable
25
@Bean
@Profile("aws")
public EurekaInstanceConfigBean eurekaInstanceAwsConfig(InetUtils inetUtils) {
LOG.info("Configuring this instance to be Amazon aware...");
EurekaInstanceConfigBean config = new EurekaInstanceConfigBean(inetUtils);
AmazonInfo info = AmazonInfo.Builder.newBuilder().autoBuild("eureka");
config.setDataCenterInfo(info);
return config;
}
Tip: Some Spring Cloud default configurations such as the Eureka instance ID
and the Eureka instance port doesn’t take effect when customizing
EurekaInstanceConfigBean. Set them explicitly.

26. Making your non-Spring Boot app
discoverable
Eureka Client (Java)
https://github.com/Netflix/eureka/wiki/Understanding-eureka-client-server-
communication
Eureka REST API (Polyglot)
https://github.com/Netflix/eureka/wiki/Eureka-REST-operations
Note: Omit “/v2” in URI
Sidecar/App Gateway (Polyglot)
Sits next to your app. Enables applications to be service discoverable and secure
without code modification.
26

27. Service discovery clients
• Netflix Eureka Client
• Spring Cloud
• Feign
• Spring RestTemplate
• Any HTTP client
27

28. Spring Boot Admin with Eureka
28
https://github.com/codecentric/spring-boot-admin

29. Spring Boot Admin with Eureka
29
@SpringBootApplication
@EnableAutoConfiguration
@EnableDiscoveryClient
@EnableAdminServer
public class Admin extends BaseSpringBootApplication {
public static void main(String[] args) {
SpringApplication.run(Admin.class, args);
}
}

30. Spring Boot Admin with Eureka
30
@Component
public class EurekaApplicationDiscoveryListener extends ApplicationDiscoveryListener {
@Autowired
public EurekaApplicationDiscoveryListener(DiscoveryClient discoveryClient, ApplicationRegistry registry) {
super(discoveryClient, registry);
ServiceInstanceConverter converter = new DefaultServiceInstanceConverter() {
@Override
public Application convert(ServiceInstance instance) {
EurekaDiscoveryClient.EurekaServiceInstance eurekaServiceInstance =
(EurekaDiscoveryClient.EurekaServiceInstance) instance;
final Application temp = super.convert(instance);
final Application converted = Application.create(temp)
.withHealthUrl(eurekaServiceInstance.getInstanceInfo().getHealthCheckUrl())
.withManagementUrl(eurekaServiceInstance.getInstanceInfo().getStatusPageUrl().replaceFirst("/info", ""))
.withServiceUrl(eurekaServiceInstance.getInstanceInfo().getHomePageUrl())
.build();
return converted;
}
};
setConverter(converter);
}
}

31. Spring Boot Admin with Component Auth
31
@Configuration
@AutoConfigureAfter({RevereseZuulProxyConfiguration.class})
protected static class ExtendedZuulProxyConfiguration extends
ZuulConfiguration {
@Bean
public ComponentAuthEnrichFilter componentAuthEnrichFilter() {
return new ComponentAuthEnrichFilter();
}
}

32. Centralized monitoring
32
@Component
public class MetricsShipper {
…..
public JSONObject composeMetrics() {
JSONObject metricsJson = new JSONObject();
metricsJson.put("timestamp", System.currentTimeMillis());
metricsJson.put("application", appName);
metricsJson.put("instance", eurekaInstanceConfig.getInstanceId());
metricsJson.put("status", healthEndpoint.invoke().getStatus().getCode());
Map<String, Object> metrics = metricsEndpoint.invoke();
for (String metricKey : metrics.keySet()) {
metricsJson.put(metricKey.replaceAll("[.]", "-"), metrics.get(metricKey));
}
return metricsJson;
}
@Scheduled(fixedRateString = "${monitoring.shipper.kafka.fixedRate}",
initialDelayString = "${monitoring.shipper.kafka.fixedRate}")
public void ship() { kafkaTarget.ship(composeMetrics()); }
}

33. Distributed tracing
33
http://cloud.spring.io/spring-cloud-sleuth/spring-cloud-sleuth.html

34. Distributed tracing
34
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-sleuth</artifactId>
</dependency>
service1.log:2016-02-26 11:15:47.561 INFO
[service1,2485ec27856c56f4,2485ec27856c56f4,true] 68058 --- [nio-8081-exec-1]
i.s.c.sleuth.docs.service1.Application : Hello from service1. Calling service2
-----
server.tomcat.accesslog.pattern=%h %l %u %t "%r" %s %b %D %I "%{x-b3-
spanid}i" "%{x-b3-traceid}i" "%{x-b3-parentspanid}i”
127.0.0.1 - - [24/May/2016:02:06:44 -0400] "POST
/elasticsearch/_msearch?ignore_unavailable=true&preference=1464070003866&ti
meout=0 HTTP/1.1" 200 148 7 http-nio-8444-exec-5 "27c37d6638ab6c87"
"150e347f81964ffd" "150e347f81964ffd"

35. Distributed tracing
35

36. Security
*Across all components in various languages
• SSO
• User authorization
• Component to component authentication and authorization
• CORS
• Appropriate routing
• Auditing and logging
• Insights and inspection
• Rate limiting
• A well known entry to platform
36

37. 37
Intranet/VPN

38. Edge gateway
38
@SpringBootApplication
@EnableAutoConfiguration
@EnableZuulProxy
public class EdgeGateway {
public static void main(String[] args) throws Exception {
SpringApplication.run(EdgeGateway.class, args);
}
}

39. Edge gateway - Routing
39
zuul.ignoredServices=*
zuul.routes.root.path=/
zuul.routes.root.url=forward:/redirect
zuul.routes.app1.path=/app1/**
zuul.routes.app1.serviceId=app1
zuul.routes.app1.stripPrefix=false
zuul.routes.app1.sensitive-headers=Cookie,Set-Cookie
zuul.routes.app2.path=/app2/**
zuul.routes.app2.url=https://app2:8443

40. Edge gateway - Authentication
• SAML 2.0
Sample: https://github.com/vdenotaris/spring-boot-security-saml-sample
• OAuth 2.0
Sample: https://github.com/royclarkson/spring-rest-service-oauth
40

41. Edge gateway - Filter
41
@Component
public class ComponentAuthEnrichFilter extends ZuulFilter {
….
@Override
public String filterType() {
return FilterType.pre.name();
}
@Override
public int filterOrder() {
return FilterOrder.BASIC_AUTH_ENRICH_PRE_FILTER.getOrder();
}
@Override
public boolean shouldFilter() {
return isFilterEnabled;
}
@Override
public Object run() {
RequestContext ctx = RequestContext.getCurrentContext();
ctx.addZuulRequestHeader("Authorization", authorizationHeaderValue);
return null;
}
}
• Enrich requests
• Inspect requests
• Collect stats
• Redirect
• Etc.

42. Sidecar / app gateway
• Discover service registry
• Authentication and authorization
• Inspect compliance
• Auditing and logging
• Distributed tracing
• Health check
• Monitoring
42

43. Sidecar / app gateway
43
DEMO TIME!

44. Sidecar / app gateway (@EnableSidecar)
44
server:
port: 5678
spring:
application:
name: sidecar
sidecar:
port: 8000
health-uri: http://localhost:8000/health.json

45. Sidecar / app gateway (Custom
@EnableZuulProxy)
45
@Component
public class SidecarHealthIndicator extends AbstractHealthIndicator {
@Override
protected void doHealthCheck(Health.Builder builder) throws Exception {
if (AppHeartbeatCheck.isHealthy()) {
builder.up();
} else {
builder.outOfService().withDetail("Failed attempts",
AppHeartbeatCheck.getFailedAttempts());
}
}
}
-----------
sidecar.healthcheck.fixedRate=5000
sidecar.healthcheck.maxAttempts=3
sidecar.healthcheck.url=http://localhost:5601
sidecar.healthcheck.expectedResponseCode=200

46. Circuit breaker
• Help reduce resources tied up in operations
which are likely to fail with fallback
• Avoid waiting on timeouts for the client
• Avoid putting loads on a struggling server
• Zuul uses Netflix Hystrix
46
http://martinfowler.com/bliki/CircuitBreaker.html

47. Circuit breaker
47
@SpringBootApplication
@EnableCircuitBreaker
public class Application {
public static void main(String[] args) {
new SpringApplicationBuilder(Application.class).web(true).run(args);
}
}
@Component
public class StoreIntegration {
@HystrixCommand(fallbackMethod = "defaultStores")
public Object getStores(Map<String, Object> parameters) {
//do stuff that might fail
}
public Object defaultStores(Map<String, Object> parameters) {
return /* something useful */;
}
}

48. Zuul gotchas
• No sticky session for your legacy and 3rd party apps that might need it
• WebSockets, Server-sent Events, HTTP2 not supported
48

49. Versioning
Artifacts - Semantic versioning
• MAJOR version when you make incompatible API changes
• MINOR version when you add functionality in a backwards-compatible manner
• PATCH version when you make backwards-compatible bug fixes
Configurations - Git revision hash
• All configurations applied via automation
49

50. Configuration
• There are more configurations to manage than monolith
• Spring Boot application properties (application.properties/yml)
• Spring Cloud bootstrap context (bootstrap.properties/yml)
• Parent context for main app (loaded before application properties)
• Loads configs from external properties (e.g. Spring Cloud Config Server)
• Encryption/decryption of sensitive properties
• Immutable infrastructure with automation
50

51. Other keys things…
• DevOps culture
• Good documentation for the microservices foundation for multiple teams to use
• Automate everything
51

52. Take away
• It’s not hard to get started with microservices… if you use Spring Cloud
• Start delivering faster and deploy more frequently
52

53. Learn More. Stay Connected.
Noriaki Tatsumi
noriaki.tatsumi@capitalone.com
https://www.linkedin.com
/company/capital-one
https://twitter.com
/capitalonejobs
https://github.com
/capitalone