Spring Boot and Spring Cloud Inside NissanConnect at SPRING FEST '19
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
, . 8 1/ L C
9 2 88 9 2 8 0
/ 8 1 /1 DBL
. 1 8 .
, . 8 8 8 0
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
n Java 20
Swing UI )
n
Java
n 2019 4
&
n
2
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved. 3
e ea N 1
C
..
5
c 4 N3 2
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved. 4
e ea N 1
C
..
5
c 4 N3 2
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved. 5
n “CASE”
– C (Connected)
– A (Autonomous)
– S (Shared & Services)
– E (Electric)
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved. 6
https://www3.nissan.co.jp/connect/service_skyline/service.html
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
n
n to
n
n
n push
n
n
n
7
https://www3.nissan.co.jp/connect/service_skyline/service.html
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
https://www.amazon.co.jp/NISSAN-MOTOR-CO-LTD-LEAF/dp/B074KCT1SY
• EV
Alexa Skill
•
Alexa
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
https://play.google.com/store/apps/details?id=com.aqsmartphone.android.nissan_jp
•
• EV(Electric Vehicle)
EV
• NissanConnect
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
https://play.google.com/store/apps/details?id=jp.co.nissan.nissanconnect.ice
•
• ICE(Internal
Combustion Engine)
ICE
• NissanConnect (
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
n
–
–
– (E2E )
nWeb
–
– validation
12
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved. 13
e ea N 1
C
..
5
c 4 N3 2
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
System Overview (End to End)
14
Car GatewayService Layer Backends
(IS/IT) Telematics Service Provider
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
System Overview (Cloud)
15
Car Gateway
GenX
Alliance Service Layer
Backends
( Alliance Platform)
Nissan Service Layer
Backends
Nissan Backend For
Frontend
Car Gateway
GenY
Car Gateway
abstraction
IS/IT TSP
Integration
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
System Overview (Cloud)
16
Car Gateway
GenX
Alliance Service Layer
Backends
( Alliance Platform)
Nissan Service Layer
Backends
Nissan Backend For
Frontend
Car Gateway
GenY
Car Gateway
abstraction
IS/IT TSP
Integration
SpringSpring
Spring
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
System Overview (Cloud)
17
Car Gateway
GenX
Alliance Service Layer
Backends
( Alliance Platform)
Nissan Service Layer
Backends
Nissan Backend For
Frontend
Car Gateway
GenY
Car Gateway
abstraction
IS/IT TSP
Integration
SpringSpring
Spring
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
n PaaS
–Microsoft Azure(VM, Database, Monitor, CosmosDB)
–Pivotal Cloud Foundry( PCF)
–RabbitMQ
–Jenkins(Build, Unit Test, Code Check, Deployment…)
n Framework
–Java/Spring Boot
–Node.js/Express
18
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
Spring
nSpring Boot(Web, JPDA, Redis, RabbitMQ, Security, Actuator)
nSpring Cloud Sleuth
nSpring Cloud Stream
nSpring Cloud Netflix Zuul
nSpring Cloud Config, Spring Cloud Bus( )
nSpring Cloud Circuit Breaker( RC)
19
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved. 20
e ea N 1
C
..
5
c 4 N3 2
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
NissanConnect
n
– NissanConnect
–
21
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
NissanConnect ( )
n
–Alliance Platform
–
–NissanConnect EV
23
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
NissanConnect ( )
n
–Alliance Platform
– Car Gateway Alliance Platform
–Java / Spring Boot
24
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
NissanConnect ( )
25
Alliance Platform
(many microservices)
Car Gateway
AdaptationModule
Platform Spring Boot Microservices
Car Gateway
EV
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
NissanConnect ( )
n
–NissanConnect EV Alliance Platform
– CarGateway Alliance Platform
• contribution
• Car Gateway
–Alliance Platform
• breaking change
26
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
NissanConnect ( )
n
–
–NissanConnect EV
– ( IS/IT ) Integration
– Car Gateway
28
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
NissanConnect ( )
n
– IS/IT Integration
–Java / Spring Boot
–
–Alliance Platform (bug, breaking change…)
29
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
NissanConnect ( )
30
Alliance Platform
(many microservices)
Car Gateway
ICE
Nissan Backend
(monolith)
Zuul Proxy
IS/IT
Rest API
Global Zuul Proxy Integration Spring Web
/alliance proxy Spring Web /nissan
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
NissanConnect ( )
31
IS/IT
Spring Boot Application
HandlerIntercepter Spring Web Controller
ZuulFilter Zuul Proxy
Spring Web
Zuul
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
NissanConnect ( )
n
– IS/IT backend
– Spring Boot
• Alliance Platform Zuul Proxy
• API Spring Web
32
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
Spring Cloud Netflix Zuul
n
–
– API Gateway
– https://cloud.spring.io/spring-cloud-netflix/reference/html/#router-and-filter-zuul
n Maven dependency
n application.properties
33
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-zuul</artifactId>
</dependency>
#
zuul.routes.foo.path=/alliance/foo/v1/**
#
zuul.routes.foo.url=${foo.domain}/v1/**
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
Spring Cloud Netflix Zuul
n Spring Web Spring Cloud Netflix Zuul
n Timeout
34
@EnableZuulProxy // Zuul Proxy
@SpringBootApplication
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
zuul.host.connect-timeout-millis=5000
zuul.host.socket-timeout-millis=65000
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
Spring Cloud Netflix Zuul
n
– ZuulFilter ErrorController
– “filterZuulException”
35
@RestController
public class ZuulErrorController extends AbstractErrorController {
@Value("${error.path:/error}")
private String errorPath;
@Override
public String getErrorPath() {
return errorPath;
}
@RequestMapping(value = "${error.path:/error}", produces = "application/vnd.api+json;charset=UTF-8")
public ResponseEntity error(HttpServletRequest request) {
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
Spring Cloud Netflix Zuul
n
36
@Component
public class AllianceErrorFilter extends ZuulFilter {
@Override
public String filterType() {
return "post";
}
@Override
public boolean shouldFilter() {
RequestContext ctx = RequestContext.getCurrentContext();
if(ctx.getRequest().getRequestURI().contains("/alliance") && ctx.getResponse().getStatus() != 200) {
return true;
}
return false;
}
@Override
public Object run() {
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
NissanConnect ( )
n
–
–
38
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
NissanConnect ( )
n
–
–
– done →PJ
39
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
NissanConnect ( )
40
Alliance Platform
(many microservices) Car Gateway
Nissan
Backend For Frontend (BFF)
Nissan Service Layer
Backends
(divided as microservice)
Nissan Backend
(monolith)
ICE
PJ
IS/IT
backend PJ
microservice Spring Boot
API Gateway Backend For Frontend( BFF)
PJ
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
NissanConnect ( )
n
– Backend For Frontend
– Microservice
–
–
41
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
n Microservice
53
HTTP
HTTP
ServletFilter AutoConfiguration
HTTP
HTTP
RestTemplate ClientInterceptor AutoConfiguration
RabbitMQ RabbitTemplate Aspect AutoConfiguration
RabbitMQ SimpleRabbitListenerContainerFactory
Advice
RabbitListener
JavaConfig
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
n
– Microservice
– Spring Cloud Sleuth
– https://cloud.spring.io/spring-cloud-static/spring-cloud-sleuth/2.2.0.RELEASE/reference/html/
n Maven dependency
n Sample (logging)
– traceId: The ID of the latency graph that contains the span. ( )
– spanId: The ID of a specific operation that took place. ( )
54
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-sleuth</artifactId>
</dependency>
2019-12-18 14:30:56.901 INFO [bar,6bfd228dc00d216b,6bfd228dc00d216b,false] 23031 --- [nio-8081-exec-3] ...
2019-12-18 14:30:59.373 ERROR [bar,6bfd228dc00d216b,6bfd228dc00d216b,false] 23031 --- [nio-8081-exec-3] ...
2016-12-18 14:31:02.939 INFO [bar,46ab0d418373cbc9,46ab0d418373cbc9,false] 23031 --- [nio-8081-exec-4] ...
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved. 55
n
– Azure Monitor
• Top10
–
• micrometer(https://micrometer.io/)
• Microsoft app insight(Alliance Platform )
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
PCF / Spring Boot Actuator
n
– PCF deploy Spring Boot Spring Boot Actuator PCF App Manager UI
–
– PCF /cloudfoundryapplication
– /${service-context}/actuator/${endpoint} PCF
n
– https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-features.html
– https://docs.pivotal.io/platform/application-service/2-7/console/spring-boot-actuators.html
56
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
PCF / Spring Boot Actuator
n Tips
– spring-boot-maven-plugin build-info target jar
– PCF App Manager UI deploy
57
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>1.4.2.RELEASE</version>
<executions>
<execution>
<goals>
<goal>build-info</goal> <!– -->
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
PCF / Spring Boot Actuator
n Tips
– Deploy Spring Boot Servlet Context /cloudfoundryapplication servlet context
– https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-features.html#production-ready-cloudfoundry
58
@Bean
public TomcatServletWebServerFactory servletWebServerFactory() {
return new TomcatServletWebServerFactory() {
@Override
protected void prepareContext(Host host, ServletContextInitializer[] initializers) {
super.prepareContext(host, initializers);
StandardContext child = new StandardContext();
child.addLifecycleListener(new Tomcat.FixContextListener());
child.setPath("/cloudfoundryapplication");
ServletContainerInitializer initializer =
getServletContextInitializer(getContextPath());
child.addServletContainerInitializer(initializer, Collections.emptySet());
child.setCrossContext(true);
host.addChild(child);
}
};
}
private ServletContainerInitializer getServletContextInitializer(String contextPath) {
return (c, context) -> {
Servlet servlet = new GenericServlet() {
@Override
public void service(ServletRequest req, ServletResponse res) throws ServletException,
IOException {
ServletContext context = req.getServletContext().getContext(contextPath);
context.getRequestDispatcher("/cloudfoundryapplication").forward(req, res);
}
};
context.addServlet("cloudfoundry", servlet).addMapping("/*");
};
}
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
n
n
n PCF + Actuator
59
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
e ea N 1
C
..
60
5
c 4 N3 2
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
n
– Microservices Microservice
–
61
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
Auto Reconfiguration by Java Buildpack
n
–Spring Boot PCF deploy DB Datasource
– Spring Boot
62
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
Auto Reconfiguration by Java Buildpack
63
@Profile("cloud")
@Configuration
public class CloudFoundryConfig {
@Bean
public Cloud cloud() {
return new CloudFactory().getCloud();
}
@Bean
@ConfigurationProperties(prefix = "spring.datasource")
public DataSource dataSource() {
DataSource ds = cloud().getSingletonServiceConnector(DataSource.class, null);
return (ds instanceof DelegatingDataSource) ? ((DelegatingDataSource) ds).getTargetDataSource() : ds;
}
}
n
– Java Buildpack Auto Reconfiguration Datasource
– Cloud JavaConfig
– Datasource @ConfigurationProperties
– return
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
RestTemplate with Apache HttpClient
64
n
– RestTemplate java.net.HttpUrlConnection
HTTP
– HttpClient RestTemplate HttpClient
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
RestTemplate with Apache HttpClient
n
– HttpClient (PoolingHttpClientConnectionManager)
65
maxConnPerRoute
2
maxConnTotal
20
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
RestTemplate with Apache HttpClient
n LoadBalancer
– LB
(Azure LB 4min(240sec) )
– TCP/IP FIN HttpClient
Timeout
66
Spring Boot
Application LB
FIN
FIN
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
RestTemplate with Apache HttpClient
n
– HttpClientBuilder JavaConfig
– LB TimeToLive
– HTTP (Redis )
67
public ClosableHttpClient httpClient() {
return HttpClients.custom()
.setMaxConnPerRoute(routeMaxConnection) // set max connection
.setMaxConnTotal(totalMaxConnection) // set max connection
.setConnectionTimeToLive(180, TimeUnit.SECONDS) // set connection ttl to 180s
.evictExpiredConnections() // enable evict expired connection
.build();
}
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
n DB
n RedisTemaplate
68
(C) Copyright NISSAN MOTOR CO., LTD. 2019 All rights reserved.
n Microservices
n
n HTTP
n PCF Auto Reconfiguration
69