There is a problem of finding the best instance of a service in distributed systems with dynamic configuration. Nowadays, there are many products for the configuration storage and service discovery. It should be mentioned at least Netflix Eureka, Consul, etc or good old Zookeeper. These products can keep and give configuration, manage service instances lifecycle and some of them even can be as dynamic DNS service. But main question is not about what instance may be called at the certain time. It is about what instance is better for call? This means that smart load balancing top on service discovery is required. Spring Cloud project allows to integrate these products to your project and provides powerful solutions for typical problems, that make cloud native services developing easier. This talk will review the internal structure of SpringCloud implementation of Client-Side Service Discovery and Client Load Balancing patterns. It also will include specific details of concrete implementations with examples from official libraries and the author’s own library.

1. Service Discovery
Больше, чем кажется

2. 2

3. 3

4. Что будет?
1. Новая реальность – новые
проблемы
2. Что такое Service Discovery и с
чем его едят
3. Как это сделано в Spring Cloud
4

5. А в чём именно проблема?
5

6. Database
JavaEE Cluster
Client
6
Так было когда-то

7. Big App
Static resource
Database
Remote EJB
7
Ресурсы и их типы в прошлом

8. Как было?
• Количество ресурсов мало
• Хост и порт, как правило,
статичны
• Конфигурация менялась редко
8

9. insert into resources_table values (key, endpoint)
9
Классическое решение проблемы

10. 10
Ещё одно решение
<providers>
<provider id="provider1"
url="socket://10.0.1.1:8080/?weight="42"/>
<provider id="provider2"
url="socket://10.0.1.2:8080/?weight="100"/>
</providers>

11. API
API instance
API instance
API instance
11
Путь к микросервисам

12. Some API
Another API
Instance 1
Another API
Instance 2
Another API
Instance 3
?
12
Кого же вызвать?

13. Статическая конфигурация
13
<providers>
<provider
id="provider{{ p.number }}"
url="socket://10.0.1.{{ p.number }}:8080/?weight="100"/>
</providers>

14. После набега
хипстеров
• Контейнеры, docker, облака
14

15. {
"id": "/hippo/rent-api",
"cpus": 1,
"mem": 1024,
"instances": 42,
"container": {
"docker": {
"image": "docker/hippo-rent-api:1.17.0",
"portMappings": [
{
"containerPort": 8080,
"servicePort": 0
}
]
}
}
}
Динамическая конфигурация (PAAS)
15

16. {
"id": "/hippo/rent-api",
"cpus": 1,
"mem": 1024,
"instances": 42,
"container": {
"docker": {
"image": "docker/hippo-rent-api:1.17.0",
"portMappings": [
{
"containerPort": 8080,
"servicePort": 0
}
]
}
}
}
Динамическая конфигурация (PAAS)
16

17. После набега
хипстеров
• Хост и порт, как правило,
заранее неизвестны
• Конфигурация меняется
постоянно, потому что
continuous delivery и прочий
DevOps
17

18. Service Discovery • Какой инстанс мы можем
вызвать?
18

19. 19
Сервис Фото
Простая задача
Сервис Группы
Связи Группа-
Пользователей
Сервис Пользователи
Сервис Альбомы Группы Обложка для фотоальбома

20. 5 parallel service calls
20
Простая задача

21. Client
call
S1 S2 S3 S4 S5
21

22. Client
call
S1 S2 S3 S4 S5
22

23. p = 0.99*
23
Простая статистика
* 0.99 не имеет отношения к реальности и взята для упрощения

24. P = p5 = (0.99)5 ≈ 0.95
p – вероятность успеха вызова одного сервиса
24
Получается не очень

25. Service Discovery
• Какой инстанс мы можем
вызвать?
• Какой инстанс ЛУЧШЕ
подходит для вызова?
25

26. P = p5 = (0.999)5 ≈ 0.995
p – вероятность успеха вызова одного сервиса
26
Хотим как-то так

27. Service Discovery
• Какой инстанс мы можем
вызвать?
• Какой инстанс ЛУЧШЕ
подходит для вызова?
• Как мы можем увеличить
вероятность успеха одиночного
вызова?
27

28. Client
call#1
S1 S2 S3 S4 S5
28
call#2
Speculative retry

29. Client
call#1
S1 S2 S3 S4 S5
29
call#2
Speculative retry

30. 30
Client
S1 S2 S3 S4 S5
Speculative retry
P = p2 = 1 - (1 - 0.99)2 ≈ 0.9999

31. Speculative Retry
• Надёжен в эксплуатации
• Все операции должны быть
идемпотентными
• Количество вызовов x2
31

32. S1 S2 S3 S4 S5
32

33. STATE
S1 S2 S3 S4 S5
S1 S2 S3 S4 S5
33

34. STATE
FILTER
S1 S2 S4
S1 S2 S3 S4 S5
S1 S2 S3 S4 S5
34

35. STATE
FILTER
S2 S4Other DC
S1 S2 S3 S4 S5
S1 S2 S3 S4 S5
35
S1

36. STATE
FILTER
S2 S4
S1 S2 S3 S4 S5
S1 S2 S3 S4 S5
?
36
S1

37. Service Service Service Service Service
STATE
FILTER
S2 S4
RULE
S1 S2 S3 S4 S5
S1 S2 S3 S4 S5
37
S1
S4

38. Service Service Service Service Service
STATE
FILTER
S2 S4
RULE
Client
S1 S2 S3 S4 S5
call
S1 S2 S3 S4 S5
38
S4

39. Service Service Service Service Service
STATE
FILTER
S2 S4
RULE
Client
S1 S2 S3 S4 S5
call
S1 S2 S3 S4 S5
39
S4
P

40. S1 S2 S3 S4 S5
?
40

41. Service Registry
• Database
• Properties File
• Environment variables
41

42. Service Registry
• Database
• Properties File
• Environment variables
42

43. Service Registry
• Netflix Eureka
• Consul
• etcd
• Zookeeper
• …
43

44. Service A
Service Registry
Service B Service C
register
44

45. Service A Service B Service C
health check
45
Service Registry

46. Service A Service B Service C
heartbeat
46
Service Registry

47. Service A Service B Service C
health check
47
Service Registry

48. Service A Service B Service C
health check
48
Service Registry

49. Service A Service B Service C
deregister
49
Service Registry

50. Service A Service B Service C
deregister
graceful shutdown
50
Service Registry

51. Leader
Master Master
Leader-Election
Quorum (n + 1)/2
Distributed Locks
51

52. Client/Slave
Leader
Master Master
Client/Slave Client/Slave
horizontal scaling
52

53. Service Registry
• Netflix Eureka, Consul, etcd,
Zookeeper
• ???
53

54. {
"id": "/hippo/rent-api",
"cpus": 1,
"mem": 1024,
"instances": 42,
"container": {
"docker": {
"image": "docker/hippo-rent-api:1.17.0",
"portMappings": [
{
"containerPort": 8080,
"servicePort": 0
}
]
}
}
}
Динамическая конфигурация (PAAS)
54

55. 55
Marathon
Marathon
Marathon
Mesos
Master
Mesos
Master
Mesos
Master
Mesos Slave
Mesos Slave

56. 56
Marathon
Marathon
Marathon
Mesos
Master
Mesos
Master
Mesos
Master
Mesos Slave
Mesos Slave

57. 57

58. curl http://marathon.master/v2/apps/{appId}
58
Получаем данные

59. 59
"tasks": [
{
"appId": "/test-marathon-app",
"host": "mesos-slave.dc2",
"id": "test-marathon-app.43ed8a70-…-0242ac110006",
"ipAddresses": [ {
"ipAddress": "172.17.0.10",
"protocol": "IPv4"
}],
"ports": [11786],
"state": "TASK_RUNNING",
"healthCheckResults": [{"alive": true}]
},
{
...
}
]

60. 60
"tasks": [
{
"appId": "/test-marathon-app",
"host": "mesos-slave.dc2",
"id": "test-marathon-app.43ed8a70-…-0242ac110006",
"ipAddresses": [ {
"ipAddress": "172.17.0.10",
"protocol": "IPv4"
}],
"ports": [11786],
"state": "TASK_RUNNING",
"healthCheckResults": [{"alive": true}]
},
{
...
}
]

61. 61
"tasks": [
{
"appId": "/test-marathon-app",
"host": "mesos-slave.dc2",
"id": "test-marathon-app.43ed8a70-…-0242ac110006",
"ipAddresses": [ {
"ipAddress": "172.17.0.10",
"protocol": "IPv4"
}],
"ports": [11786],
"state": "TASK_RUNNING",
"healthCheckResults": [{"alive": true}]
},
{
...
}
]

62. 62
"tasks": [
{
"appId": "/test-marathon-app",
"host": "mesos-slave.dc2",
"id": "test-marathon-app.43ed8a70-…-0242ac110006",
"ipAddresses": [ {
"ipAddress": "172.17.0.10",
"protocol": "IPv4"
}],
"ports": [11786],
"state": "TASK_RUNNING",
"healthCheckResults": [{"alive": true}]
},
{
...
}
]

63. “Service Registry”
• Netflix Eureka, Consul, etcd,
Zookeeper
• Kubernetes
• Mesos+Marathon
• DCOS
• Own solution
63

64. Spring Cloud
• Фреймворк для разработки
распределённых систем
• Реализует основные паттерны
• Имеет коннекторы к
различным третьим решениям
64

65. Client-Side Service Discovery
65

66. ...
@EnableDiscoveryClient
...
public class Application {
@Autowired
private DiscoveryClient discoveryClient;
public List<String> services() {
return discoveryClient.getServices();
}
}
66
Одна аннотация спасет мир

67. ...
@EnableDiscoveryClient
...
public class Application {
@Autowired
private DiscoveryClient discoveryClient;
public List<String> services() {
return discoveryClient.getServices();
}
}
67
Одна аннотация спасет мир

68. public interface DiscoveryClient {
String description();
ServiceInstance getLocalServiceInstance();
List<ServiceInstance> getInstances(String serviceId);
List<String> getServices();
}
68
Рекомендуемый интерфейс

69. 69
Рекомендуемый интерфейс
https://www.safaribooksonline.com/library/view/cloud-native-java/9781449374631/ch16.html
Spring Cloud provides the DiscoveryClient abstraction
to make it easy for clients to work with different types
of service registries.
Spring Cloud plugs the DiscoveryClient abstraction
into various parts of the stack, making its use almost
transparent.
The abstraction is simple enough that you
could adapt Spring Cloud to work with another service
registry if you’d like to. Conceptually,
the DiscoveryClient is read-only.

70. public interface DiscoveryClient {
String description();
@Deprecated
ServiceInstance getLocalServiceInstance();
List<ServiceInstance> getInstances(String serviceId);
List<String> getServices();
}
70
Рекомендуемый интерфейс

71. public interface DiscoveryClient {
String description();
ServiceInstance getLocalServiceInstance();
List<ServiceInstance> getInstances(String serviceId);
List<String> getServices();
}
71
Рекомендуемый интерфейс

72. public interface DiscoveryClient {
String description();
ServiceInstance getLocalServiceInstance();
List<ServiceInstance> getInstances(String serviceId);
List<String> getServices();
}
72
Рекомендуемый интерфейс

73. ...
@EnableDiscoveryClient
...
public class Application {
@Autowired
private DiscoveryClient discoveryClient;
public List<String> services() {
return discoveryClient.getServices();
}
}
73
Одна аннотация спасет мир

74. Spring Cloud Core
Eureka Consul Marathon
74

75. Боль реализации
• Разные конфигурации
подключения к реестрам
сервисов
• Специфические фичи,
зависящие от выбора реестра
сервисов
75

76. spring:
cloud:
marathon:
listOfServers: m1:8080,m2:8080,m3:8080
username: marathon
password: mesos
76
Marathon

77. spring:
cloud:
consul:
host: localhost
port: 8500
77
Но вот в Consul-е иначе
На каждом хосту есть consul-агент

78. spring:
cloud:
marathon:
listOfServers: m1:8080,m2:8080,m3:8080
username: marathon
password: mesos
78
Marathon. Авторизация#1

79. spring:
cloud:
marathon:
listOfServers: m1:8080,m2:8080,m3:8080
token: <api_token>
79
Marathon. Авторизация#2

80. Marathon AppId: /group/path/app
Spring Cloud serviceId: group.path.app
80
Marathon. Маппинг

81. Marathon AppId: /group/path/app
Spring Cloud serviceId: group.path.app
81
Marathon. Маппинг

82. Marathon AppId: /group/path/app
Spring Cloud serviceId: group.path.app
82
Невалидный hostname

83. @Override
public List<ServiceInstance> getInstances(String serviceId) {
return client.getAppTasks(ServiceIdConverter.convertToMarathonId(serviceId))
.getTasks()
.parallelStream()
.filter(task -> null == task.getHealthCheckResults() ||
task.getHealthCheckResults()
.stream()
.allMatch(HealthCheckResult::isAlive)
)
.map(task -> new DefaultServiceInstance(
ServiceIdConverter.convertToServiceId(task.getAppId()),
task.getHost(),
task.getPorts().stream().findFirst().orElse(0),
false
))
.collect(Collectors.toList());
}
83
Marathon. Имплементация

84. @Override
public List<ServiceInstance> getInstances(String serviceId) {
return client.getAppTasks(ServiceIdConverter.convertToMarathonId(serviceId))
.getTasks()
.parallelStream()
.filter(task -> null == task.getHealthCheckResults() ||
task.getHealthCheckResults()
.stream()
.allMatch(HealthCheckResult::isAlive)
)
.map(task -> new DefaultServiceInstance(
ServiceIdConverter.convertToServiceId(task.getAppId()),
task.getHost(),
task.getPorts().stream().findFirst().orElse(0),
false
))
.collect(Collectors.toList());
}
84
Marathon. Имплементация

85. @Override
public List<ServiceInstance> getInstances(String serviceId) {
return client.getAppTasks(ServiceIdConverter.convertToMarathonId(serviceId))
.getTasks()
.parallelStream()
.filter(task -> null == task.getHealthCheckResults() ||
task.getHealthCheckResults()
.stream()
.allMatch(HealthCheckResult::isAlive)
)
.map(task -> new DefaultServiceInstance(
ServiceIdConverter.convertToServiceId(task.getAppId()),
task.getHost(),
task.getPorts().stream().findFirst().orElse(0),
false
))
.collect(Collectors.toList());
}
85
Marathon. Имплементация

86. @Override
public List<ServiceInstance> getInstances(String serviceId) {
return client.getAppTasks(ServiceIdConverter.convertToMarathonId(serviceId))
.getTasks()
.parallelStream()
.filter(task -> null == task.getHealthCheckResults() ||
task.getHealthCheckResults()
.stream()
.allMatch(HealthCheckResult::isAlive)
)
.map(task -> new DefaultServiceInstance(
ServiceIdConverter.convertToServiceId(task.getAppId()),
task.getHost(),
task.getPorts().stream().findFirst().orElse(0),
false
))
.collect(Collectors.toList());
}
86
Marathon. Имплементация

87. @Override
public List<ServiceInstance> getInstances(String serviceId) {
return client.getAppTasks(ServiceIdConverter.convertToMarathonId(serviceId))
.getTasks()
.parallelStream()
.filter(task -> null == task.getHealthCheckResults() ||
task.getHealthCheckResults()
.stream()
.allMatch(HealthCheckResult::isAlive)
)
.map(task -> new DefaultServiceInstance(
ServiceIdConverter.convertToServiceId(task.getAppId()),
task.getHost(),
task.getPorts().stream().findFirst().orElse(0),
false
))
.collect(Collectors.toList());
}
87
Marathon. Имплементация

88. @Autowired
private DiscoveryClient discoveryClient;
@RequestMapping("/instances")
public List<ServiceInstance> instances() {
return discoveryClient.getInstances("someservice");
}
88
Marathon. Использование

89. Вся правда о
DiscoveryClient
• Не используется для
клиентской балансировки
• Участвует в формировании
гиперссылок, в health-check
эндпоинтах и в получении
списка сервисов в реализации
edge-сервера
89

90. LoadBalancer
• Основан на Netflix Ribbon
• В принципе может быть
использован вне стека Spring
Cloud
• В общем случае не нуждается в
Service Registry
90

91. Service Service Service Service Service
STATE
FILTER
S2 S4
RULE
S1 S2 S3 S4 S5
S1 S2 S3 S4 S5
91
Client
call S4

92. Service Service Service Service Service
S2 S4
S1 S2 S3 S4 S5
S1 S2 S3 S4 S5
ServerList
92
Client
call S4

93. Ribbon ServerList
• Static
• Configuration-based
• Discovery-based
93

94. test-service: #service name
ribbon: #namespace
listOfServers: host:8080,anotherHost:8081
ConfigurationBasedServerList
94

95. @Autowired
private LoadBalancerClient loadBalancer;
@RequestMapping("/url")
public String realUrl() throws IOException {
return loadBalancer.reconstructURI(
instance,
new URI("http://"+ serviceId +"/me")
).toString();
}
Будет заменено реальным хостом и портом
95

96. @Autowired
private LoadBalancerClient loadBalancer;
@RequestMapping("/url")
public String realUrl() throws IOException {
return loadBalancer.reconstructURI(
instance,
new URI("http://"+ serviceId +"/me")
).toString();
}
> curl service:8080/url
http://{host,anotherHost}:8080/me
96

97. DiscoveryBasedServerList
97
List<MarathonServer> extractServiceInstances(App app) {
return app.getTasks()
.stream()
.map(task -> {
Collection<HealthCheckResult> healthChecks =
null != task.getHealthCheckResults()
? task.getHealthCheckResults()
: new ArrayList<>();
return new MarathonServer(
task.getHost(),
task.getPorts().stream().findFirst().orElse(0),
healthChecks
).withZone(extractZoneFromHostname(task.getHost()));
})
.collect(Collectors.toList());
}

98. DiscoveryBasedServerList
98
List<MarathonServer> extractServiceInstances(App app) {
return app.getTasks()
.stream()
.map(task -> {
Collection<HealthCheckResult> healthChecks =
null != task.getHealthCheckResults()
? task.getHealthCheckResults()
: new ArrayList<>();
return new MarathonServer(
task.getHost(),
task.getPorts().stream().findFirst().orElse(0),
healthChecks
).withZone(extractZoneFromHostname(task.getHost()));
})
.collect(Collectors.toList());
}

99. DiscoveryBasedServerList
99
List<MarathonServer> extractServiceInstances(App app) {
return app.getTasks()
.stream()
.map(task -> {
Collection<HealthCheckResult> healthChecks =
null != task.getHealthCheckResults()
? task.getHealthCheckResults()
: new ArrayList<>();
return new MarathonServer(
task.getHost(),
task.getPorts().stream().findFirst().orElse(0),
healthChecks
).withZone(extractZoneFromHostname(task.getHost()));
})
.collect(Collectors.toList());
}

100. DiscoveryBasedServerList
100
List<MarathonServer> extractServiceInstances(App app) {
return app.getTasks()
.stream()
.map(task -> {
Collection<HealthCheckResult> healthChecks =
null != task.getHealthCheckResults()
? task.getHealthCheckResults()
: new ArrayList<>();
return new MarathonServer(
task.getHost(),
task.getPorts().stream().findFirst().orElse(0),
healthChecks
).withZone(extractZoneFromHostname(task.getHost()));
})
.collect(Collectors.toList());
}

101. Ribbon Server vs Service Instance
101
Service InstanсeRibbon Server
Разное представление одного и того же

102. Marathon. Ribbon-фишки
102
customer-service:
ribbon:
<your settings here>
MetaDataFilter:
API_VERSION: '!=V3' # not equal to V3
ENVIRONMENT: '==DEV' # equal to DEV;
CUSTOMER_ENTITY: V1 # equal to V1; default is equals

103. А жив ли мой
сервис?
• Что значит ”жив”?
• Как проверить живучесть?
103

104. S1 S2 S3 S4 S5
S2 S4
S1 S2 S3 S4 S5?
104
Server List
Load Balancer Client
S4

105. t t + 1
S1
S2
S3
105

106. t t + 1
S1
S2
S3
ServerList
S1
S2
S3
106

107. t t + 1
S1
S2
S3
ServerList
S1S1
S2
S3
107

108. t t + 1
S1
S2
S3
ServerList
S1 S1S1
S2
S3
108

109. t t + 1
S1
S2
S3
ServerList
S1 S1
t + 2
S1
S2
S3
109

110. S1 S2 S3 S4 S5
S2 S4
S1 S2 S3 S4 S5Ping ->
110
Server List
Load Balancer Client
S4

111. Pinger
(background thread)
LoadBalancer
background task
force if setServerList() is called
111

112. Pinger
(background thread)
LoadBalancer
background task
force if setServerList() is called
IPingStrategy
pingServers
112

113. Pinger
(background thread)
LoadBalancer
background task
force if setServerList() is called
IPingStrategy
pingServers
isAlive(server)
IPing
113

114. IPingStrategy SerialPingStrategy
int numCandidates = servers.length;
boolean[] results = new boolean[numCandidates];
for (int i = 0; i < numCandidates; i++) {
results[i] = false; /* Default answer is DEAD. */
if (ping != null) {
results[i] = ping.isAlive(servers[i]);
}
}
return results;
114

115. IPingStrategy SerialPingStrategy
int numCandidates = servers.length;
boolean[] results = new boolean[numCandidates];
for (int i = 0; i < numCandidates; i++) {
results[i] = false; /* Default answer is DEAD. */
if (ping != null) {
results[i] = ping.isAlive(servers[i]);
}
}
return results;
115

116. IPingStrategy SerialPingStrategy
int numCandidates = servers.length;
boolean[] results = new boolean[numCandidates];
for (int i = 0; i < numCandidates; i++) {
results[i] = false; /* Default answer is DEAD. */
if (ping != null) {
results[i] = ping.isAlive(servers[i]);
}
}
return results;
116

117. IPingStrategy SerialPingStrategy
int numCandidates = servers.length;
boolean[] results = new boolean[numCandidates];
for (int i = 0; i < numCandidates; i++) {
results[i] = false; /* Default answer is DEAD. */
if (ping != null) {
results[i] = ping.isAlive(servers[i]);
}
}
return results;
117
И что делать, если пинг слишком долгий?

118. NIWSDiscoveryPing
(Eureka)
IPing
instance.status.equals(
InstanceStatus.UP
)
ConsulPing
(Consul)
MarathonPing
(Marathon)
server.isPassingChecks()
server.isPassingChecks()
Кругом один обман J
118

119. IPingStrategy ?
Есть честная имплементация, но нет готовых её использований
IPing PingUrl
public boolean isAlive(Server server) {
URL url = constructUrl(server);
HttpResponse response = httpClient.execute(createReq(url));
return response.getStatusLine().getStatusCode() == 200;
}
119

120. А жив ли мой
сервис?
• Проверять на уровне
приложения дорого
• Доверяем тому, что приходит
из реестра сервисов
120

121. Отсеять плохое • Фильтры и их применение
121

122. S1 S2 S3 S4 S5
S2 S4
S1 S2 S3 S4 S5
Server List
Filter Chain ->
122
Load Balancer Client
S4

123. Cluster (Zone 1)
S1
S2
S3
Cluster (Zone 2)
S4
S5
S6
Client
(Zone 1)
123

124. Cluster (Zone 1)
S1
S2
S3
Cluster (Zone 2)
S4
S5
S6
Client
(Zone 1)
cheap call
124

125. Cluster (Zone 1)
S1
S2
S3
Cluster (Zone 2)
S4
S5
S6
Client
(Zone 1)
expensive call
125

126. Cluster (Zone 1)
S2
S3
Cluster (Zone 2)
S4
S5
S6
Client
(Zone 1)
unpredictable
S1
126

127. Cluster (Zone 1)
S1
S2
S3
Cluster (Zone 2)
S4
S5
S6
Client
(Zone 1)
more guaranteed result
127

128. ZoneAffinityFilter
getFilteredServers(servers)
LoadBalancerStats
getZoneSnaphot(servers)
128
DynamicServerListLoadBalancer

129. S1 S2 S3 S4 S5
S2 S4
S1 S2 S3 S4 S5LB Stats ->
Server List
Filter Chain
129
Load Balancer Client
S4
Server Stats

130. S1
S2
S3
Zone Snapshot
Server Stats
130

131. S1
S2
S3
Zone Snapshot
activeConnections
activeServers
circuitBreakerTrippedCount
131

132. ZoneAffinityFilter
LoadBalancerStats
enableZoneAffinity?
zoneAffinity:
maxLoadPerServer: 0.6
maxBlackOutServersPercentage: 0.8
minAvailableServers: 2
132

133. ZoneAffinityFilter
ZoneAffinityPredicate
исключаем инстансы
из других зон
133
zoneAffinity:
maxLoadPerServer: 0.6
maxBlackOutServersPercentage: 0.8
minAvailableServers: 2

134. Cluster (Zone 1)
S1
S2
S3
Cluster (Zone 2)
S4
S5
S6
Client
(Zone 1)
cheap call
134

135. ZoneAffinityFilter
ZoneAffinityPredicate
не фильтруем
135
zoneAffinity:
maxLoadPerServer: 0.6
maxBlackOutServersPercentage: 0.8
minAvailableServers: 2

136. Cluster (Zone 1)
S1
Cluster (Zone 2)
S4
S5
S6
Client
(Zone 1)
136
S2
S3

137. ZonePreferenceFilter
List servers = super.getFilteredServers();
List local = servers.filter(server ->
server.zone == zoneFromConfig
);
if (!local.isEmpty()) {
return local;
}
return servers;
super.getFilteredServers()
DynamicServerListLoadBalancer
137
default (local) zone
ZoneAffinityFilter

138. ZonePreferenceFilter
List servers = super.getFilteredServers();
List local = servers.filter(server ->
server.zone == zoneFromConfig
);
if (!local.isEmpty()) {
return local;
}
return servers;
super.getFilteredServers()
DynamicServerListLoadBalancer
138
ZoneAffinityFilter

139. Цепочка
фильтров
• Помогает увеличить шансы на
корректное исполнение
запроса
• Отсеивает «плохие» и «дорогие»
инстансы
139

140. Правило выбора • Последняя миля
140

141. S1 S2 S3 S4 S5
S2 S4
S1 S2 S3 S4 S5
?
141
Server List
Filter Chain
Rule
Load Balancer Client
S4

142. IRule
RoundRobin
ZoneAvoidance
default
Weighted Random
BestAvailability AvailabilityFilter
142

143. IRule
RoundRobin
ZoneAvoidance
Weighted Random
BestAvailability AvailabilityFiltering
143

144. upstream test-marathon-app {
server host1 weight=3;
server host2 weight=1;
server host3 weight=1;
}
Как правило вес предопределен
144
Типичный конфиг с весами

145. Weight Task
(background thread)
Weighted Rule
background task
LoadBalancerStats
calculate
serverWeight = summ(avgServerResponseTime)- avgServerResponseTime
145

146. Weight Task
(background thread)
Weighted Rule
background task
LoadBalancerStats
calculate
serverWeight = summ(avgServerResponseTime)- avgServerResponseTime
146
Больше время ответа – меньше вес

147. 147
Slow Instance Fast Instance
100 ms 10 ms

148. 148
Slow Instance Fast Instance
100 ms 10 ms5 000 requests

149. 149
Slow Instance Fast Instance
100 ms 10 ms5 000 requests
Round Robin Rule
Slow: 2500 and Fast: 2500
Average time: 70

150. 150
Slow Instance Fast Instance
100 ms 10 ms5 000 requests
Round Robin Rule
Slow: 2500 and Fast: 2500
Average time: 70

151. 151
Slow Instance Fast Instance
100 ms 10 ms5 000 requests
Round Robin Rule
Slow: 2500 and Fast: 2500
Average time: 70
Weighted Rule
Slow: 634 and Fast: 4366
Average time: 27

152. 152
Slow Instance Fast Instance
100 ms 10 ms1 000 requests
Round Robin Rule
Slow: 500 and Fast: 500
Average time: 70
Weighted Rule
?
?

153. 153
Slow Instance Fast Instance
100 ms 10 ms1 000 requests
Round Robin Rule
Slow: 500 and Fast: 500
Average time: 70
Weighted Rule
Slow: 500 and Fast: 500
Average time: 72

154. 154
Slow Instance Fast Instance
100 ms 10 ms1 000 requests
Round Robin Rule
Slow: 500 and Fast: 500
Average time: 70
Weighted Rule
Slow: 500 and Fast: 500
Average time: 72

155. test-service: #service name
ribbon: #namespace
ServerWeightTaskTimerInterval: 30000 #ms
Давайте разбираться
155

156. test-service: #service name
ribbon: #namespace
ServerWeightTaskTimerInterval: 30000 #ms
Weights [0.0, 0.0]
Давайте разбираться
156

157. test-service: #service name
ribbon: #namespace
ServerWeightTaskTimerInterval: 1000 #ms
Давайте разбираться
157

158. test-service: #service name
ribbon: #namespace
ServerWeightTaskTimerInterval: 1000 #ms
Weights [12.285123016785462, 120.30885719400065]
Давайте разбираться
158

159. IRule
RoundRobin
ZoneAvoidance
Weighted Random
BestAvailability AvailabilityFiltering
159

160. Best Available Rule LoadBalancerStats
calculate
chosen -> activeConnections == min(activeConnections)
160

161. Best Available Rule LoadBalancerStats
calculate
chosen -> activeConnections == min(activeConnections)
161
Меньше активных соединений -> больше шансов на успех

162. 162
Slow Instance Fast Instance
100 ms 10 ms1 000 requests
Round Robin Rule
Slow: 500 and Fast: 500
Average time: 70
Weighted Rule
Slow: 500 and Fast: 500
Average time: 72
Best Available Rule
Slow: 152 and Fast: 847
Average time: 38

163. IRule
RoundRobin
ZoneAvoidance
Weighted Random
BestAvailability AvailabilityFiltering
163

164. Availability Filtering
Rule
LoadBalancerStats
calculate
filtered -> activeConnections < maxActiveConnections
filtered -> !isCircuitBreakerTripped
Availability Predicate
filter
164
Почти как round-robin

165. IRule
RoundRobin
ZoneAvoidance
Weighted Random
BestAvailability AvailabilityFiltering
165

166. Zone Avoidance Rule Composite Predicate
filter
Zone Predicate Availability Predicate
166

167. Zone Avoidance Rule Composite Predicate
filter
Zone Predicate Availability Predicate
CompositePredicateBuilder
int minimalFilteredServers = 1
float minimalFilteredPercentage = 0
predicate.filter(servers)
167

168. Zone Avoidance Rule Composite Predicate
filter
Zone Predicate Availability Predicate
CompositePredicateBuilder
int minimalFilteredServers = 1
float minimalFilteredPercentage = 0
int count = predicate.filter(servers).size() //2
168

169. Zone Avoidance Rule Composite Predicate
filter
Zone Predicate Availability Predicate
CompositePredicateBuilder
int minimalFilteredServers = 1
float minimalFilteredPercentage = 0
int count = predicate.filter(servers).size() //2
count >= minimalFilteredServers
count >= minimalFilteredPercentage * count
169

170. Zone Avoidance Rule Composite Predicate
filter
Zone Predicate Availability Predicate
CompositePredicateBuilder
int minimalFilteredServers = 1
float minimalFilteredPercentage = 0
int count = predicate.filter(servers).size() //0
count >= minimalFilteredServers
count >= minimalFilteredPercentage * count
170

171. Zone Avoidance Rule Composite Predicate
filter
Zone Predicate Availability Predicate
next
171

172. triggeringLoadPerServerThreshold: 0.2
avoidZoneWithBlackoutPercetage: 0.99999d
worstZone -> loadPerServer == max(loadPerServer) &&
loadPerServer > triggeringLoadPerServerThreshold
zones.remove(worstZone)
Zone Avoidance Rule Composite Predicate
filter
Zone Predicate Availability Predicate
172

173. triggeringLoadPerServerThreshold: 0.2
avoidZoneWithBlackoutPercetage: 0.99999d
zoneToAvoid -> circuitBreakerTrippedCount / instanceCount >=
avoidZoneWithBlackoutPercetage
zones.remove(zoneToAvoid)
Zone Avoidance Rule Composite Predicate
filter
Zone Predicate Availability Predicate
173

174. Правила
• Ещё больше увеличивают
шансы найти «хороший»
инстанс
• По-умолчанию «хитрые»
правила не используются
174

175. Default Rule • Почему не BestAvailability?
175

176. Best Availability
• Не всегда предсказуем
• Использовать с осторожностью
176

177. Service Service Service Service Service
S2 S4
S2
LoadBalancerClient
S1 S2 S3 S4 S5
S1 S2 S3 S4 S5
New Service
register/deregister
177

178. Шаблоны
регистрации
• 3d-party
• Self-registration
178

179. Docker Engine
179
3d-party registration pattern

180. Docker Engine
Service
register/deregister
180
3d-party registration pattern

181. Docker Engine
Service
register/deregister
181
Registrator
emit event
3d-party registration pattern

182. Docker Engine
Service
register/deregister
182
Registrator
emit event
Service Registry
native event
3d-party registration pattern

183. Service Definition
183
deploy manifest
PaaS
Нативная регистрация (PaaS)

184. Service Definition
184
deploy manifest
PaaS
Нативная регистрация (PaaS)
S1 S2 S3 S4 S5
run services

185. Self-Registration
• Реализация (как всегда)
зависит от выбранного реестра
сервисов
• Не нужна для PaaS-решений
185

186. AutoServiceRegistration
LifecycleProcessor
autoStartup on refresh
startBeans on start
Service Registry
register
186

187. LifecycleProcessor
on RefreshScope event
stopBeans on stop
Service Registry
deregister
187
AutoServiceRegistration

188. Контейнеры • Docker etc.
188

189. Docker container
Internal IP External IP
189
?

190. > ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280stf0: flags=0<> mtu 1280
en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
en1: flags=963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX> mtu 1500
en2: flags=963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX> mtu 1500
p2p0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 2304
awdl0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1484
bridge0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
utun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1500
190

191. > ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280stf0: flags=0<> mtu 1280
en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
en1: flags=963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX> mtu 1500
en2: flags=963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX> mtu 1500
p2p0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 2304
awdl0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1484
bridge0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
utun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1500
Low index
191

192. spring:
cloud:
discovery:
lifecycle:
enabled: false
192
External Registrator

193. spring:
cloud:
inetutils:
ignored-interfaces:
- vbox*
- bridge*
- lo*
193

194. spring:
cloud:
inetutils:
ignored-interfaces:
- vbox*
- bridge*
- lo*
194

195. spring:
cloud:
inetutils:
preferred-interfaces:
- eth0
195

196. spring:
cloud:
discovery:
preferIpAddress: true (false)
196

197. Шаблоны
регистрации
• Выбирайте внешнюю
регистрацию сервисов, если
она идёт из коробки
• Саморегистрация имеет
подводные камни, на которые
можно натолкнуться
197

198. Выводы#1
• Облака, контейнеры,
динамическая привязка
ресурсов приводят к
необходимости решения
задачи обнаружения сервисов
• Но основная цель – это
предсказуемая и стабильная
работа всей системы в целом
198

199. Выводы#2
• В Spring Cloud-е много готовых
решений, но их использование
будет отличаться в
зависимости от выбранного
реестра сервисов
199

200. Выводы#3
• От нас многое прячут в недрах
конфигурации, и если
покопаться, то можно найти
интересные возможности
• Изучение базовых решений
(Netflix Ribbon. и т.д.) даст
возможность эффективнее
решать задачи
200

201. Выводы#4
• Выбирайте внешнюю
регистрацию сервисов, если
она идёт из коробки
• Саморегистрация имеет
подводные камни, на которые
можно натолкнуться
201

202. Ссылки
202
Spring Cloud (Service Registry)
https://github.com/spring-cloud/spring-cloud-netflix
https://github.com/spring-cloud/spring-cloud-consul
Spring Cloud (Paas)
https://github.com/aatarasoff/spring-cloud-marathon
https://github.com/fabric8io/spring-cloud-kubernetes

203. Почитать
203
Spring Cloud Marathon
http://developerblog.info/2017/04/19/marathon-spring-cloud-part1/
http://developerblog.info/2017/04/24/marathon-spring-cloud-part2/

204. 204