Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Event Driven Microservices with Spring Cloud Stream #jjug_ccc #ccc_ab3

11,743 views

Published on

Event Driven Microservices with Spring Cloud Stream JJUG CCC Fall 2016 #jjug_ccc #ccc_ab3

Published in: Technology

Event Driven Microservices with Spring Cloud Stream #jjug_ccc #ccc_ab3

  1. 1. ‹#›© 2016 Pivotal Software, Inc. All rights reserved. ‹#›© 2016 Pivotal Software, Inc. All rights reserved. Event Driven Microservices with Spring Cloud Stream Toshiaki Maki (@making) 2016-12-03 JJUG CCC 2016 Fall #jjug_ccc #ccc_ab3 http://bit.ly/making_ccc_a3 (source code)
  2. 2. © 2016 Pivotal Software, Inc. All rights reserved. Who am I ? • Toshiaki Maki (@making) http://blog.ik.am • Sr. Solutions Architect @Pivotal • Spring Framework enthusiast bit.ly/hajiboot2
  3. 3. © 2016 Pivotal Software, Inc. All rights reserved. Contents •Spring Cloud Stream (25min) •Advanced Topic (20min) •Spring Cloud Data Flow (2min) •Deploy Stream Apps to Cloud Foundry (3min)
  4. 4. © 2016 Pivotal Software, Inc. All rights reserved. Microservices Frontend Customer Service Email Service Point Service Post Service
  5. 5. © 2016 Pivotal Software, Inc. All rights reserved. Microservices Frontend Customer Service Email Service Point Service Post Service HTTP / REST?
  6. 6. © 2016 Pivotal Software, Inc. All rights reserved. Microservices Frontend Customer Service Email Service Point Service Post Service ABC Service XYZ Service
  7. 7. © 2016 Pivotal Software, Inc. All rights reserved. Microservices Frontend Customer Service Email Service Point Service Post Service ABC Service XYZ Service
  8. 8. © 2016 Pivotal Software, Inc. All rights reserved. Microservices Frontend Customer Service Email Service Point Service Post Service ABC Service XYZ Service
  9. 9. © 2016 Pivotal Software, Inc. All rights reserved. Microservices Frontend Customer Service Email Service Point Service Post Service ABC Service XYZ Service😩
  10. 10. © 2016 Pivotal Software, Inc. All rights reserved. Microservices Frontend Customer Service Email Service Point Service Post Service ABC Service XYZ Service😩 Orchestration Style
  11. 11. © 2016 Pivotal Software, Inc. All rights reserved. Read Failure Frontend Customer Service Email Service HTTP GET
  12. 12. © 2016 Pivotal Software, Inc. All rights reserved. Read Failure Frontend Customer Service Email Service 🔥 HTTP GET
  13. 13. © 2016 Pivotal Software, Inc. All rights reserved. Read Failure Frontend Customer Service Email Service 🔥 HTTP GET Circuit Breaker
  14. 14. © 2016 Pivotal Software, Inc. All rights reserved. Read Failure Frontend Customer Service Email Service 🔥 HTTP GET Circuit Breaker
  15. 15. © 2016 Pivotal Software, Inc. All rights reserved. Write Failure Frontend Customer Service Email Service HTTP POST
  16. 16. © 2016 Pivotal Software, Inc. All rights reserved. Write Failure 🔥 Frontend Customer Service Email Service HTTP POST
  17. 17. © 2016 Pivotal Software, Inc. All rights reserved. Write Failure 🔥 Frontend Customer Service Email Service 😲 HTTP POST
  18. 18. © 2016 Pivotal Software, Inc. All rights reserved. Event Driven MicroServices Frontend Customer Service Email Service Point Service Post Service
  19. 19. © 2016 Pivotal Software, Inc. All rights reserved. Event Driven MicroServices Frontend Customer Service Email Service Point Service Post Service
  20. 20. © 2016 Pivotal Software, Inc. All rights reserved. Event Driven MicroServices Frontend Customer Service Email Service Point Service Post Service Subscribe
  21. 21. © 2016 Pivotal Software, Inc. All rights reserved. Event Driven MicroServices Frontend Customer Service Email Service Point Service Post Service Subscribe Publish
  22. 22. © 2016 Pivotal Software, Inc. All rights reserved. Event Driven MicroServices Frontend Customer Service Email Service Point Service Post Service Subscribe ABC Service XYZ Service Publish
  23. 23. © 2016 Pivotal Software, Inc. All rights reserved. Event Driven MicroServices Frontend Customer Service Email Service Point Service Post Service Subscribe ABC Service XYZ Service Publish
  24. 24. © 2016 Pivotal Software, Inc. All rights reserved. Event Driven MicroServices Frontend Customer Service Email Service Point Service Post Service Subscribe ABC Service XYZ Service 🤓 Publish
  25. 25. © 2016 Pivotal Software, Inc. All rights reserved. Choreography Style https://www.thoughtworks.com/insights/blog/scaling-microservices-event-stream
  26. 26. ‹#›© 2016 Pivotal Software, Inc. All rights reserved. Spring Cloud Stream
  27. 27. © 2016 Pivotal Software, Inc. All rights reserved. Spring Cloud Stream • Event-driven microservice framework • Built on battle-tested components (Spring Boot / Spring Integration) • Opinionated primitives for streaming applications • Persistent Pub/Sub • Consumer Groups • Partitioning Support • Pluggable messaging middleware bindings source | processor | sink
  28. 28. © 2016 Pivotal Software, Inc. All rights reserved. Source | Sink Sink input Source output
  29. 29. © 2016 Pivotal Software, Inc. All rights reserved. Source | Processor | Sink Source output Processor output input Sink input
  30. 30. © 2016 Pivotal Software, Inc. All rights reserved. Spring Cloud Stream Applications Twitter Stream Cassandra java -jar twittersource.jar --server.port=8080 --consumerKey=XYZ --consumerSecret=ABC --spring.cloud.stream.bindings. output.destination=ingest Source Sink java -jar cassandrasink.jar --server.port=8081 --spring.cassandra.keyspace=tweet --spring.cloud.stream.bindings. input.destination=ingest
  31. 31. © 2016 Pivotal Software, Inc. All rights reserved. Spring Cloud Stream Applications Twitter Stream Cassandra java -jar twittersource.jar --server.port=8080 --consumerKey=XYZ --consumerSecret=ABC --spring.cloud.stream.bindings. output.destination=ingest Source Sink java -jar cassandrasink.jar --server.port=8081 --spring.cassandra.keyspace=tweet --spring.cloud.stream.bindings. input.destination=ingest Twitter Stream Cassandraingest
  32. 32. © 2016 Pivotal Software, Inc. All rights reserved. Message Binders • @EnableBinding • Binder Implementations • Production-Ready • Rabbit MQ • Apache Kafka • Experimental • JMS • Google PubSub
  33. 33. © 2016 Pivotal Software, Inc. All rights reserved. Programming Model (Sink) @SpringBootApplication
 @EnableBinding(Sink.class)
 public class DemoSinkApp { @StreamListener(Sink.INPUT)
 void receive(Message<String> message) {
 System.out.println("Received " + message.getPayload());
 }
 public static void main(String[] args) {
 SpringApplication.run(DemoSinkApp.class, args);
 }
 }
  34. 34. © 2016 Pivotal Software, Inc. All rights reserved. Programming Model (Sink) @SpringBootApplication
 @EnableBinding(Sink.class)
 public class DemoSinkApp { @StreamListener(Sink.INPUT)
 void receive(Message<String> message) {
 System.out.println("Received " + message.getPayload());
 }
 public static void main(String[] args) {
 SpringApplication.run(DemoSinkApp.class, args);
 }
 } public interface Sink { String INPUT = "input"; @Input(Sink.INPUT) SubscribableChannel input();
 }
  35. 35. © 2016 Pivotal Software, Inc. All rights reserved. Sink Properties (Sink) spring.cloud.stream.bindings.input.destination=demo-strm demo- strm input
  36. 36. © 2016 Pivotal Software, Inc. All rights reserved. Programming Model (Source) @SpringBootApplication @RestController @EnableBinding(Source.class)
 public class DemoSourceApp {
 @Autowired @Output(Source.OUTPUT) MessageChannel output; @GetMapping void send(@RequestParam String text) {
 output.send(MessageBuilder.withPayload(text).build());
 }
 public static void main(String[] args) {
 SpringApplication.run(DemoSourceApp.class, args);
 }
 }
  37. 37. © 2016 Pivotal Software, Inc. All rights reserved. Programming Model (Source) @SpringBootApplication @RestController @EnableBinding(Source.class)
 public class DemoSourceApp {
 @Autowired @Output(Source.OUTPUT) MessageChannel output; @GetMapping void send(@RequestParam String text) {
 output.send(MessageBuilder.withPayload(text).build());
 }
 public static void main(String[] args) {
 SpringApplication.run(DemoSourceApp.class, args);
 }
 } public interface Source { String OUTPUT = "output"; @Output(Source.OUTPUT) MessageChannel output();
 }
  38. 38. © 2016 Pivotal Software, Inc. All rights reserved. Programming Model (Source) @SpringBootApplication @RestController @EnableBinding(Source.class)
 public class DemoSourceApp {
 @Autowired Source source; @GetMapping void send(@RequestParam String text) {
 source.output() .send(MessageBuilder.withPayload(text).build());
 }
 public static void main(String[] args) {
 SpringApplication.run(DemoSourceApp.class, args);
 }
 }
  39. 39. © 2016 Pivotal Software, Inc. All rights reserved. Properties (Source) spring.cloud.stream.bindings.output.destination=demo-strm demo- strm Source output
  40. 40. © 2016 Pivotal Software, Inc. All rights reserved. Properties (Source) spring.cloud.stream.bindings.output.destination=demo-strm spring.cloud.stream.bindings.output.contentType=applicati on/json demo- strm Source output
  41. 41. © 2016 Pivotal Software, Inc. All rights reserved. Binder (RabbitMQ) <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-stream-rabbit</artifactId> </dependency> demo- strm
  42. 42. © 2016 Pivotal Software, Inc. All rights reserved. Binder (Apache Kafka) <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-stream-kafka</artifactId> </dependency> demo- strm
  43. 43. © 2016 Pivotal Software, Inc. All rights reserved. Sink Pipeline demo- strm input Source output source | sink
  44. 44. © 2016 Pivotal Software, Inc. All rights reserved. RabbitMQ demo-strm
  45. 45. © 2016 Pivotal Software, Inc. All rights reserved. RabbitMQ demo-strm Source
  46. 46. © 2016 Pivotal Software, Inc. All rights reserved. RabbitMQ demo-strm Topic Exchange demo-strm Source
  47. 47. © 2016 Pivotal Software, Inc. All rights reserved. RabbitMQ demo-strm Topic Exchange demo-strm Source
  48. 48. © 2016 Pivotal Software, Inc. All rights reserved. RabbitMQ demo-strm Topic Exchange demo-strm Source Sink
  49. 49. © 2016 Pivotal Software, Inc. All rights reserved. RabbitMQ demo-strm Topic Exchange demo-strm .anonymous.x Queue demo-strm Source Sink
  50. 50. © 2016 Pivotal Software, Inc. All rights reserved. RabbitMQ demo-strm Topic Exchange demo-strm .anonymous.x Queue demo-strm Source Sink
  51. 51. © 2016 Pivotal Software, Inc. All rights reserved. Programming Model (Processor) @SpringBootApplication
 @EnableBinding(Processor.class)
 public class DemoProcessorApp { @StreamListener(Processor.INPUT) @SendTo(Processor.OUTPUT)
 void receive(String text) {
 return "[[" + text + "]]";
 }
 public static void main(String[] args) {
 SpringApplication.run(DemoProcessorApp.class, args);
 }
 }
  52. 52. © 2016 Pivotal Software, Inc. All rights reserved. Properties (Processor) spring.cloud.stream.bindings.output.destination=my-source spring.cloud.stream.bindings.input.destination=my-source spring.cloud.stream.bindings.output.destination=my-proc spring.cloud.stream.bindings.input.destination=my-proc Source Processor Sink
  53. 53. © 2016 Pivotal Software, Inc. All rights reserved. Pipeline my- source Source output source | processor | sink Processor output input my-proc Sink input
  54. 54. © 2016 Pivotal Software, Inc. All rights reserved. Reactive API Support by Reactor @SpringBootApplication
 @EnableBinding(Processor.class)
 public class DemoProcessorRxApp { @StreamListener @Output(Processor.OUTPUT)
 public Flux<String> receive(@Input(Processor.INPUT) Flux<String> stream) {
 return stream.map(text -> "[[" + text + "]]");
 }
 public static void main(String[] args) {
 SpringApplication.run(DemoProcessorRxApp.class, args);
 }
 }
  55. 55. © 2016 Pivotal Software, Inc. All rights reserved. Reactive API Support by Reactor @StreamListener @Output(Processor.OUTPUT)
 public Flux<AverageData> receive(@Input(Processor.INPUT) Flux<SensorData> stream) {
 return stream.window(Duration.ofSecond(20), Duration.ofSecond(10)) .flatMap(win -> win.groupBy(sensor -> sensor.id)) .flatMap(group -> calcAverage(group));
 }
  56. 56. © 2016 Pivotal Software, Inc. All rights reserved. Stream Core Features •Persistent Pub-Sub •Consumer Group •Partitioning Support
  57. 57. © 2016 Pivotal Software, Inc. All rights reserved. Persistent Pub-Sub HTTP Average Top Ns1.http s1.ave Message Broker
  58. 58. © 2016 Pivotal Software, Inc. All rights reserved. Persistent Pub-Sub HTTP Average HDFS Top Ns1.http s1.ave Message Broker
  59. 59. © 2016 Pivotal Software, Inc. All rights reserved. Persistent Pub-Sub HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker
  60. 60. © 2016 Pivotal Software, Inc. All rights reserved. Persistent Pub-Sub HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker {"id":1, "temperature":38}
  61. 61. © 2016 Pivotal Software, Inc. All rights reserved. Persistent Pub-Sub HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker {"id":1, "temperature":38}{"id":1, "temperature":38}
  62. 62. © 2016 Pivotal Software, Inc. All rights reserved. Persistent Pub-Sub HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker {"id":1, "temperature":38} {"id":1, "temperature":38} {"id":1, "temperature":38}
  63. 63. © 2016 Pivotal Software, Inc. All rights reserved. Persistent Pub-Sub HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker {"id":1, "temperature": {"id":1, "temperature":38} {"id":1, "temperature":38}
  64. 64. © 2016 Pivotal Software, Inc. All rights reserved. HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker Average Average HDFS HDFS
  65. 65. © 2016 Pivotal Software, Inc. All rights reserved. HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker {"id":1, "temperature":38} Average Average HDFS HDFS
  66. 66. © 2016 Pivotal Software, Inc. All rights reserved. HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker {"id":1, "temperature":38}{"id":1, "temperature":38} Average Average HDFS HDFS
  67. 67. © 2016 Pivotal Software, Inc. All rights reserved. HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker {"id":1, "temperature":38} {"id":1, "temperature":38} Average Average HDFS HDFS {"id":1, "temperature":38} {"id":1, "temperature":38} {"id":1, "temperature":38} {"id":1, "temperature":38}
  68. 68. © 2016 Pivotal Software, Inc. All rights reserved. HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker {"id":1, "temperature":38} {"id":1, "temperature":38} Average Average HDFS HDFS {"id":1, "temperature":38} {"id":1, "temperature":38} {"id":1, "temperature":38} {"id":1, "temperature":38} 😩
  69. 69. © 2016 Pivotal Software, Inc. All rights reserved. Topic Exchange demo-strm demo-strm .anonymous.1 Queue
  70. 70. © 2016 Pivotal Software, Inc. All rights reserved. Topic Exchange demo-strm demo-strm .anonymous.1 Queue
  71. 71. © 2016 Pivotal Software, Inc. All rights reserved. Topic Exchange demo-strm demo-strm .anonymous.1 Queue demo-strm .anonymous.2
  72. 72. © 2016 Pivotal Software, Inc. All rights reserved. Topic Exchange demo-strm demo-strm .anonymous.1 Queue demo-strm .anonymous.2 Consumer Group
  73. 73. © 2016 Pivotal Software, Inc. All rights reserved. Topic Exchange demo-strm demo-strm .anonymous.1 Queue demo-strm .anonymous.2 spring.cloud.stream.bindings.<channelName>.group=ave Consumer Group
  74. 74. © 2016 Pivotal Software, Inc. All rights reserved. Consumer Group demo-strm demo-strm .ave QueueTopic Exchange
  75. 75. © 2016 Pivotal Software, Inc. All rights reserved. Consumer Group HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker Average Average HDFS HDFS group=ave group=hdfs
  76. 76. © 2016 Pivotal Software, Inc. All rights reserved. Consumer Group HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker Average Average HDFS HDFS group=ave group=hdfs {"id":1, "temperature":38}
  77. 77. © 2016 Pivotal Software, Inc. All rights reserved. Consumer Group HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker Average Average HDFS HDFS group=ave group=hdfs {"id":1, "temperature":38}{"id":1, "temperature":38}
  78. 78. © 2016 Pivotal Software, Inc. All rights reserved. Consumer Group HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker Average Average HDFS HDFS group=ave group=hdfs {"id":1, "temperature":38} {"id":1, "temperature":38}
  79. 79. © 2016 Pivotal Software, Inc. All rights reserved. Consumer Group HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker Average Average HDFS HDFS group=ave group=hdfs {"id":1, "temperature":38} {"id":1, "temperature":38} 🤓
  80. 80. © 2016 Pivotal Software, Inc. All rights reserved. Consumer Group HTTP Average HDFS Top N Fault Detection s1.http s1.ave Message Broker Average Average HDFS HDFS group=ave group=hdfs {"id":1, "temperature":38} {"id":1, "temperature":38} 🤓 consumer group subscriptions are durable 😁
  81. 81. © 2016 Pivotal Software, Inc. All rights reserved. Durability demo-strm demo-strm .ave QueueTopic Exchange
  82. 82. © 2016 Pivotal Software, Inc. All rights reserved. Durability demo-strm demo-strm .ave QueueTopic Exchange
  83. 83. © 2016 Pivotal Software, Inc. All rights reserved. Durability demo-strm demo-strm .ave Queue 🏀 Topic Exchange
  84. 84. © 2016 Pivotal Software, Inc. All rights reserved. Durability demo-strm demo-strm .ave Queue 🏀 Topic Exchange
  85. 85. © 2016 Pivotal Software, Inc. All rights reserved. Durability demo-strm demo-strm .ave Queue 🚑 🏀 Topic Exchange
  86. 86. © 2016 Pivotal Software, Inc. All rights reserved. Durability demo-strm demo-strm .ave Queue 🚑 🏀 Topic Exchange
  87. 87. © 2016 Pivotal Software, Inc. All rights reserved. Durability demo-strm demo-strm .ave Queue 🏀 Topic Exchange
  88. 88. © 2016 Pivotal Software, Inc. All rights reserved. Customer Service Email Service Point Service Post Service Source Sink
  89. 89. © 2016 Pivotal Software, Inc. All rights reserved. Customer Service Email Service Point Service Post Service Source Sink spring.cloud.stream.bindings.output.destination=customer spring.cloud.stream.bindings.output.contentType=applicatio n/json
  90. 90. © 2016 Pivotal Software, Inc. All rights reserved. Customer Service Email Service Point Service Post Service Subscribe Source Sink
  91. 91. © 2016 Pivotal Software, Inc. All rights reserved. Customer Service Email Service Point Service Post Service Subscribe Source Sink group=point-service group=email-service group=post-service
  92. 92. © 2016 Pivotal Software, Inc. All rights reserved. Customer Service Email Service Point Service Post Service Subscribe Source Sink group=point-service group=email-service group=post-service spring.cloud.stream.bindings.input.destination=customer spring.cloud.stream.bindings.input.group=point-service
  93. 93. © 2016 Pivotal Software, Inc. All rights reserved. Customer Service Email Service Point Service Post Service Subscribe Source Sink group=point-service group=email-service group=post-service spring.cloud.stream.bindings.input.destination=customer spring.cloud.stream.bindings.input.group=point-service spring.cloud.stream.bindings.input.destination=customer spring.cloud.stream.bindings.input.group=email-service
  94. 94. © 2016 Pivotal Software, Inc. All rights reserved. Customer Service Email Service Point Service Post Service Subscribe Source Sink group=point-service group=email-service group=post-service spring.cloud.stream.bindings.input.destination=customer spring.cloud.stream.bindings.input.group=point-service spring.cloud.stream.bindings.input.destination=customer spring.cloud.stream.bindings.input.group=email-service spring.cloud.stream.bindings.input.destination=customer spring.cloud.stream.bindings.input.group=post-service
  95. 95. © 2016 Pivotal Software, Inc. All rights reserved. customer Customer Service output Point Service input Email Service input Post Service input
  96. 96. © 2016 Pivotal Software, Inc. All rights reserved. customer customer .point- service Queue Consumer Group customer .email- service customer .post- service Topic Exchange
  97. 97. © 2016 Pivotal Software, Inc. All rights reserved. HTTP Average s1.http Average
  98. 98. © 2016 Pivotal Software, Inc. All rights reserved. HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
  99. 99. © 2016 Pivotal Software, Inc. All rights reserved. HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
  100. 100. © 2016 Pivotal Software, Inc. All rights reserved. HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
  101. 101. © 2016 Pivotal Software, Inc. All rights reserved. HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
  102. 102. © 2016 Pivotal Software, Inc. All rights reserved. HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
  103. 103. © 2016 Pivotal Software, Inc. All rights reserved. HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37} 😩
  104. 104. © 2016 Pivotal Software, Inc. All rights reserved. HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37} 😩 Partitioning Support(Stateful Stream)
  105. 105. © 2016 Pivotal Software, Inc. All rights reserved. HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37} 😩 spring.cloud.stream.bindings.<channelName>.producer.par titionKeyExpression=payload.id Partitioning Support(Stateful Stream)
  106. 106. © 2016 Pivotal Software, Inc. All rights reserved. Partitioning Support(Stateful Stream) HTTP Average s1.http Average
  107. 107. © 2016 Pivotal Software, Inc. All rights reserved. Partitioning Support(Stateful Stream) HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
  108. 108. © 2016 Pivotal Software, Inc. All rights reserved. Partitioning Support(Stateful Stream) HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
  109. 109. © 2016 Pivotal Software, Inc. All rights reserved. Partitioning Support(Stateful Stream) HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
  110. 110. © 2016 Pivotal Software, Inc. All rights reserved. Partitioning Support(Stateful Stream) HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
  111. 111. © 2016 Pivotal Software, Inc. All rights reserved. Partitioning Support(Stateful Stream) HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37}
  112. 112. © 2016 Pivotal Software, Inc. All rights reserved. Partitioning Support(Stateful Stream) HTTP Average s1.http Average {"id":1, "temperature":38} {"id":2, "temperature":41} {"id":2, "temperature":42} {"id":1, "temperature":37} 🤓
  113. 113. © 2016 Pivotal Software, Inc. All rights reserved. Test Support Source output Sink input
  114. 114. © 2016 Pivotal Software, Inc. All rights reserved. Test Support Source output Sink input TestSupportBinder
  115. 115. © 2016 Pivotal Software, Inc. All rights reserved. Test Support <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-stream-test-support</artifactId> <scope>test</scope> </dependency>
  116. 116. © 2016 Pivotal Software, Inc. All rights reserved. Unit Test (Sink) @RunWith(SpringRunner.class)
 @SpringBootTest(webEnvironment = WebEnvironment.NONE)
 public class DemoSinkAppTest { @Autowired Sink sink; @Rule public OutputCapture capture = new OutputCapture(); @Test public void testReceive() { sink.input() .send(MessageBuilder.withPayload("foo").build()); assertThat(capture.toString()) .isEqualsTo("Received foo");
 } }
  117. 117. © 2016 Pivotal Software, Inc. All rights reserved. Unit Test (Source) @RunWith(SpringRunner.class)
 @SpringBootTest(webEnvironment = WebEnvironment.NONE)
 public class DemoSourceAppTest { @Autowired DemoSourceApp app; @Autowired MessageCollector collector; @Autowired Source source; @Test public void testSend() { app.send("foo"); Message<String> message = collector .forChannel(source.output()).poll(); assertThat(message.getPayload()).isEqualsTo("foo"); 
 }}
  118. 118. ‹#›© 2016 Pivotal Software, Inc. All rights reserved. Advanced Topics
  119. 119. © 2016 Pivotal Software, Inc. All rights reserved. Advanced Topics •Multi Binding •Distributed Tracing •Error Handling •Consumer Driven Contract
  120. 120. © 2016 Pivotal Software, Inc. All rights reserved. Multi Bindings
  121. 121. © 2016 Pivotal Software, Inc. All rights reserved. customer Customer Service output Email Service input
  122. 122. © 2016 Pivotal Software, Inc. All rights reserved. customer Customer Service output Email Service input CustomerCreateEvent
  123. 123. © 2016 Pivotal Software, Inc. All rights reserved. customer Customer Service output Email Service input
  124. 124. © 2016 Pivotal Software, Inc. All rights reserved. customer Customer Service output Email Service input CustomerDeleteEvent
  125. 125. © 2016 Pivotal Software, Inc. All rights reserved. customer Customer Service output Email Service input CustomerDeleteEvent ClassCastException!!
  126. 126. © 2016 Pivotal Software, Inc. All rights reserved. create Email Service create- input delete- input Customer Service create- output delete- output delete CustomerCreateEvent CustomerDeleteEvent
  127. 127. © 2016 Pivotal Software, Inc. All rights reserved. public interface CustomerEventSource { String CREATE_OUTPUT = "create-output"; String DELETE_OUTPUT = "delete-output"; @Output(CustomerEventSource.CREATE_OUTPUT) MessageChannel createOutput(); @Output(CustomerEventSource.DELETE_OUTPUT) MessageChannel deleteOutput();
 } Multi Bindings
  128. 128. © 2016 Pivotal Software, Inc. All rights reserved. @Component
 public class CustomerService {
 @Autowired CustomerEventSource source; public void create(...) { source.createOutput().send(...); } public void delete() { source.deleteOutput().send(...); }
 } @SpringBootApplication @EnableBinding(CustomerEventSource.class)
 public class CustomerServiceApplication { /* ... */ }
  129. 129. © 2016 Pivotal Software, Inc. All rights reserved. @Component
 public class CustomerService {
 @Autowired CustomerEventSource source; public void create(...) { source.createOutput().send(...); } public void delete() { source.deleteOutput().send(...); }
 } @SpringBootApplication @EnableBinding(CustomerEventSource.class)
 public class CustomerServiceApplication { /* ... */ } spring.cloud.stream.bindings.create-output.destination =create spring.cloud.stream.bindings.delete-output.destination =delete
  130. 130. © 2016 Pivotal Software, Inc. All rights reserved. public interface CustomerEventSink { String CREATE_INPUT = "create-input"; String DELETE_INPUT = "delete-input"; @Input(CustomerEventSink.CREATE_INPUT) SubscribableChannel createInput(); @Input(CustomerEventSink.DELETE_INPUT) SubscribableChannel deleteInput();
 } Multi Bindings
  131. 131. © 2016 Pivotal Software, Inc. All rights reserved. @Component
 public class PointService { @StreamListener(CustomerEventSink.CREATE_INPUT) public void handleCreate(CustomerCreateEvent event) { } @StreamListener(CustomerEventSink.DELETE_INPUT) public void handleDelete(CustomerDeleteEvent event) { }
 } @SpringBootApplication @EnableBinding(CustomerEventSink.class)
 public class PointServiceApplication { /* ... */ }
  132. 132. © 2016 Pivotal Software, Inc. All rights reserved. @Component
 public class PointService { @StreamListener(CustomerEventSink.CREATE_INPUT) public void handleCreate(CustomerCreateEvent event) { } @StreamListener(CustomerEventSink.DELETE_INPUT) public void handleDelete(CustomerDeleteEvent event) { }
 } @SpringBootApplication @EnableBinding(CustomerEventSink.class)
 public class PointServiceApplication { /* ... */ } spring.cloud.stream.bindings.create-input.destination =create spring.cloud.stream.bindings.create-input.group =point-service spring.cloud.stream.bindings.delete-intput.destination =delete spring.cloud.stream.bindings.delete-input.group =point-service
  133. 133. © 2016 Pivotal Software, Inc. All rights reserved. create create .point- service Queue delete delete .point- service Topic Exchange
  134. 134. © 2016 Pivotal Software, Inc. All rights reserved. Frontend Customer Service Email Service Point Service Post Service Subscribe ABC Service XYZ Service
  135. 135. © 2016 Pivotal Software, Inc. All rights reserved. Frontend Customer Service Email Service Point Service Post Service Subscribe ABC Service XYZ Service 🕝
  136. 136. © 2016 Pivotal Software, Inc. All rights reserved. Frontend Customer Service Email Service Point Service Post Service Subscribe ABC Service XYZ Service🤔 🕝
  137. 137. © 2016 Pivotal Software, Inc. All rights reserved. Frontend Customer Service Email Service Point Service Post Service Subscribe ABC Service XYZ Service🤔 🕝 🕵 🕵 🕵 🕵 🕵 🕵🕵
  138. 138. © 2016 Pivotal Software, Inc. All rights reserved. Frontend Customer Service Email Service Point Service Post Service Subscribe ABC Service XYZ Service🤔 🕝 🕵 🕵 🕵 🕵 🕵 🕵🕵
  139. 139. © 2016 Pivotal Software, Inc. All rights reserved. Frontend Customer Service Email Service Point Service Post Service Subscribe ABC Service XYZ Service🤔 🕝 🕵 🕵 🕵 🕵 🕵 🕵🕵 TraceID, SpanID
  140. 140. © 2016 Pivotal Software, Inc. All rights reserved. Spring Cloud Sleuth • Distributed tracing solution for Spring Cloud • Interactions with external systems should be instrumented automatically • Capture data simply in logs, or by sending it to Zipkin via RestTemplate / Spring Cloud Stream / ...
  141. 141. © 2016 Pivotal Software, Inc. All rights reserved. Spring Cloud Sleuth Stream <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-stream-slueth</artifactId> </dependency>
  142. 142. © 2016 Pivotal Software, Inc. All rights reserved. Spring Cloud Sleuth Stream customer Customer Service output Email Service input
  143. 143. © 2016 Pivotal Software, Inc. All rights reserved. Spring Cloud Sleuth Stream customer Customer Service output Email Service input sleuth sleuth
  144. 144. © 2016 Pivotal Software, Inc. All rights reserved. Zipkin Stream Server <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-sleuth-zipkin-stream</artifactId> </dependency>
  145. 145. © 2016 Pivotal Software, Inc. All rights reserved. Zipkin Stream Server <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-sleuth-zipkin-stream</artifactId> </dependency> @SpringBootApplication
 @EnableZipkinStreamServer
 public class ZipkinStreamServer {
 public static void main(String[] args) {
 SpringApplication.run(DemoSinkApp.class, args);
 }
 }
  146. 146. © 2016 Pivotal Software, Inc. All rights reserved. Zipkin Stream Server customer Customer Service output Email Service input sleuth sleuth
  147. 147. © 2016 Pivotal Software, Inc. All rights reserved. Zipkin Stream Server customer Customer Service output Email Service input sleuth sleuth sleuth sleuth
  148. 148. © 2016 Pivotal Software, Inc. All rights reserved. Zipkin Stream Server customer Customer Service output Email Service input sleuth sleuth sleuth sleuth
  149. 149. © 2016 Pivotal Software, Inc. All rights reserved. Zipkin Stream Server customer Customer Service output Email Service input sleuth sleuth sleuth sleuth
  150. 150. © 2016 Pivotal Software, Inc. All rights reserved. Zipkin Stream Server customer Customer Service output Email Service input sleuth sleuth sleuth sleuth TraceID,SpanID TraceID,SpanID
  151. 151. © 2016 Pivotal Software, Inc. All rights reserved. Zipkin Stream Server customer Customer Service output Email Service input sleuth sleuth sleuth sleuth
  152. 152. © 2016 Pivotal Software, Inc. All rights reserved. Zipkin UI
  153. 153. © 2016 Pivotal Software, Inc. All rights reserved. Zipkin UI Trace
  154. 154. © 2016 Pivotal Software, Inc. All rights reserved. Zipkin UI Trace Span
  155. 155. © 2016 Pivotal Software, Inc. All rights reserved.
  156. 156. © 2016 Pivotal Software, Inc. All rights reserved. Error Handling •Depends on the message binder implementation •(Ex.) RabbitMQ binder routes the failed message to the Dead-Letter Queue(DLQ). No mechanism to handle DLQs.
  157. 157. © 2016 Pivotal Software, Inc. All rights reserved. Dead-Letter Queue Processing demo-strm Topic Exchange demo-strm .ave Source Sink RabbitMQBinder
  158. 158. © 2016 Pivotal Software, Inc. All rights reserved. Dead-Letter Queue Processing demo-strm Topic Exchange demo-strm .ave Source Sink DLX RabbitMQBinder
  159. 159. © 2016 Pivotal Software, Inc. All rights reserved. Dead-Letter Queue Processing demo-strm Topic Exchange demo-strm .ave Source Sink DLX RabbitMQBinder spring.cloud.stream.bindings.input.destination=demo-strm spring.cloud.stream.bindings.input.group=ave spring.cloud.stream.rabbit.bindings.input.consumer.autoBi ndDlq=true
  160. 160. © 2016 Pivotal Software, Inc. All rights reserved. Dead-Letter Queue Processing demo-strm Topic Exchange demo-strm .ave Source Sink DLX demo-strm .ave.dlq RabbitMQBinder spring.cloud.stream.bindings.input.destination=demo-strm spring.cloud.stream.bindings.input.group=ave spring.cloud.stream.rabbit.bindings.input.consumer.autoBi ndDlq=true
  161. 161. © 2016 Pivotal Software, Inc. All rights reserved. Dead-Letter Queue Processing demo-strm Topic Exchange demo-strm .ave Source Sink DLX demo-strm .ave.dlq RabbitMQBinder
  162. 162. © 2016 Pivotal Software, Inc. All rights reserved. Dead-Letter Queue Processing demo-strm Topic Exchange demo-strm .ave Source Sink DLX demo-strm .ave.dlq RabbitMQBinder
  163. 163. © 2016 Pivotal Software, Inc. All rights reserved. Dead-Letter Queue Processing demo-strm Topic Exchange demo-strm .ave Source Sink DLX demo-strm .ave.dlq RabbitMQBinder
  164. 164. © 2016 Pivotal Software, Inc. All rights reserved. Dead-Letter Queue Processing demo-strm Topic Exchange demo-strm .ave Source Sink DLX demo-strm .ave.dlq RabbitMQBinder 🔥
  165. 165. © 2016 Pivotal Software, Inc. All rights reserved. Dead-Letter Queue Processing demo-strm Topic Exchange demo-strm .ave Source Sink DLX demo-strm .ave.dlq RabbitMQBinder 🔥 Retry 3 times by default
  166. 166. © 2016 Pivotal Software, Inc. All rights reserved. Dead-Letter Queue Processing demo-strm Topic Exchange demo-strm .ave Source Sink DLX demo-strm .ave.dlq RabbitMQBinder 🔥 Retry 3 times by default
  167. 167. © 2016 Pivotal Software, Inc. All rights reserved.
  168. 168. © 2016 Pivotal Software, Inc. All rights reserved.
  169. 169. © 2016 Pivotal Software, Inc. All rights reserved.
  170. 170. © 2016 Pivotal Software, Inc. All rights reserved.
  171. 171. © 2016 Pivotal Software, Inc. All rights reserved.
  172. 172. © 2016 Pivotal Software, Inc. All rights reserved. Handling DLQ @Component
 public class DlqHander { @RabbitListener(queues = "customer.email-service.dlq") public void handle(Message event) { // Re-deliver if you want }
 }
  173. 173. © 2016 Pivotal Software, Inc. All rights reserved. DLQ Recovery Center 😛 https://github.com/making-demo-scst/dlq-recover-service
  174. 174. © 2016 Pivotal Software, Inc. All rights reserved. Trace everything
  175. 175. © 2016 Pivotal Software, Inc. All rights reserved. customer Customer Service output Point Service input Email Service input Post Service input
  176. 176. © 2016 Pivotal Software, Inc. All rights reserved. customer Customer Service output Point Service input Email Service input Post Service input
  177. 177. © 2016 Pivotal Software, Inc. All rights reserved. customer Customer Service output Point Service input Email Service input Post Service input
  178. 178. © 2016 Pivotal Software, Inc. All rights reserved. customer Customer Service output Point Service input Email Service input Post Service input
  179. 179. © 2016 Pivotal Software, Inc. All rights reserved. customer Customer Service output Point Service input Email Service input Post Service input
  180. 180. © 2016 Pivotal Software, Inc. All rights reserved. customer Customer Service output Point Service input Email Service input Post Service input 😗Breaking Change
  181. 181. © 2016 Pivotal Software, Inc. All rights reserved. customer Customer Service output Point Service input Email Service input Post Service input 😗Breaking Change
  182. 182. © 2016 Pivotal Software, Inc. All rights reserved. customer Customer Service output Point Service input Email Service input Post Service input 😗Breaking Change
  183. 183. © 2016 Pivotal Software, Inc. All rights reserved. customer Customer Service output Point Service input Email Service input Post Service input 😗Breaking Change
  184. 184. © 2016 Pivotal Software, Inc. All rights reserved. customer Customer Service output 😗Breaking Change 😡 😡 😡 😨
  185. 185. © 2016 Pivotal Software, Inc. All rights reserved. Consumer Driven Contracts •Consumer shares "expectation" with Producer via "Contract" (≈ DSL) •The contract violation should be detected by generated tests on the producer side.
  186. 186. © 2016 Pivotal Software, Inc. All rights reserved. Consumer Driven Contracts •Consumer shares "expectation" with Producer via "Contract" (≈ DSL) •The contract violation should be detected by generated tests on the producer side. ContractConsumer Producer
  187. 187. © 2016 Pivotal Software, Inc. All rights reserved. Flow of Consumer Driven Contracts Consumer Producer
  188. 188. © 2016 Pivotal Software, Inc. All rights reserved. Flow of Consumer Driven Contracts Consumer Producer Contract
  189. 189. © 2016 Pivotal Software, Inc. All rights reserved. Flow of Consumer Driven Contracts Consumer Producer Contract
  190. 190. © 2016 Pivotal Software, Inc. All rights reserved. Flow of Consumer Driven Contracts Consumer Producer Contract Acceptance Test
  191. 191. © 2016 Pivotal Software, Inc. All rights reserved. Flow of Consumer Driven Contracts Consumer Producer Contract Acceptance Test
  192. 192. © 2016 Pivotal Software, Inc. All rights reserved. Flow of Consumer Driven Contracts Consumer Producer Contract Acceptance Test ✅
  193. 193. © 2016 Pivotal Software, Inc. All rights reserved. Flow of Consumer Driven Contracts Consumer Producer Contract Acceptance Test Stub ✅
  194. 194. © 2016 Pivotal Software, Inc. All rights reserved. Flow of Consumer Driven Contracts Consumer Producer Contract Acceptance Test Stub ✅
  195. 195. © 2016 Pivotal Software, Inc. All rights reserved. Flow of Consumer Driven Contracts Consumer Producer Contract Acceptance Test Stub Unit Test ✅
  196. 196. © 2016 Pivotal Software, Inc. All rights reserved. Flow of Consumer Driven Contracts Consumer Producer Contract Acceptance Test Stub Unit Test ✅✅
  197. 197. © 2016 Pivotal Software, Inc. All rights reserved. Spring Cloud Contract •A CDC Solution for JVM apps (especially Spring) •Contract DSL using Groovy •Generates Acceptance test (JUnit or Spock) for producer •Generates Stub for consumer •WireMock Support for REST Test •Messaging Support (Spring Integration, Spring Cloud Stream and Apache Camel)
  198. 198. © 2016 Pivotal Software, Inc. All rights reserved. Contract DSL Contract.make { label 'create-customer' input { } outputMessage { sentTo('demo-strm') headers({header('Content-Type':'...')})
 body('''{"name":"@making"}''') }} shouldCreateCustomer.groovy
  199. 199. © 2016 Pivotal Software, Inc. All rights reserved. Contract DSL Contract.make { label 'create-customer' input { } outputMessage { sentTo('demo-strm') headers({header('Content-Type':'...')})
 body('''{"name":"@making"}''') }} shouldCreateCustomer.groovy Created by Consumer
  200. 200. © 2016 Pivotal Software, Inc. All rights reserved. Prepare Parent Test Class @RunWith(SpringRunner.class)
 @SpringBootTest(webEnvironment = WebEnvironment.NONE) @AutoConfigureMessageVerifier
 public abstract class MsgTestBase { @Autowired CustomerService service; protected void create() { service.create("@making"); 
 } }
  201. 201. © 2016 Pivotal Software, Inc. All rights reserved. Prepare Parent Test Class @RunWith(SpringRunner.class)
 @SpringBootTest(webEnvironment = WebEnvironment.NONE) @AutoConfigureMessageVerifier
 public abstract class MsgTestBase { @Autowired CustomerService service; protected void create() { service.create("@making"); 
 } } Created by Producer
  202. 202. © 2016 Pivotal Software, Inc. All rights reserved. Contract DSL Contract.make { label 'create-customer' input { triggeredBy('create()') } outputMessage { sentTo('demo-strm') headers({header('Content-Type':'...')})
 body('''{"name":"@making"}''') }} shouldCreateCustomer.groovy
  203. 203. © 2016 Pivotal Software, Inc. All rights reserved. Contract DSL Contract.make { label 'create-customer' input { triggeredBy('create()') } outputMessage { sentTo('demo-strm') headers({header('Content-Type':'...')})
 body('''{"name":"@making"}''') }} shouldCreateCustomer.groovy Updated by Producer
  204. 204. © 2016 Pivotal Software, Inc. All rights reserved. Generated Acceptance Test public class ContractVerifierTest extends MsgTestBase { // ... @Test public void validate_shouldCreateCustomer() { create(); ContractVerifierMessage res = verifierMessaging .receive("customer");
 assertThat(res).isNotNull(); DocumentContext parsedJson = JsonPath.parse( objectMapper.writeValueAsString(res.getPayload())); assertThatJson(parsedJson).field("name") .isEqualTo("@making");
 }}
  205. 205. © 2016 Pivotal Software, Inc. All rights reserved. Spring Cloud Contract Maven Plugin mvn spring-cloud-contract:generateTests mvn spring-cloud-contract:convert mvn spring-cloud-contract:generateStubs Acceptance Test WireMock stub file (only for REST) Stub jar file
  206. 206. © 2016 Pivotal Software, Inc. All rights reserved. Consumer Side Test @RunWith(SpringRunner.class)
 @SpringBootTest(webEnvironment = WebEnvironment.NONE) @AutoConfigureStubRunner(ids = "com.example:customer- service", workOffline = true)
 public class PointServiceConsumerTest { @Autowired StubTrigger stubTrigger; // ... @Test public void testCreateCustomer() { stubTrigger.trigger("create-customer"); // assert that the message is received ...
 } }
  207. 207. © 2016 Pivotal Software, Inc. All rights reserved. Check sample code •http://bit.ly/making_ccc_a3
  208. 208. © 2016 Pivotal Software, Inc. All rights reserved. (FYI) CQRS and Event Sourcing • https://spring.io/blog/2016/11/08/cqrs-and-event-sourcing- with-jakub-pilimon • https://github.com/pilloPl/event-source-cqrs-sample
  209. 209. ‹#›© 2016 Pivotal Software, Inc. All rights reserved. Data Microservices with Spring Cloud Data Flow
  210. 210. © 2016 Pivotal Software, Inc. All rights reserved. Data Microservices $ cat book.txt | tr ' ' '¥ ' | tr '[:upper:]' '[:lower:]' | tr -d '[:punct:]' | grep -v '[^a-z]‘ | sort | uniq -c | sort -rn | head
  211. 211. © 2016 Pivotal Software, Inc. All rights reserved. Data Microservices $ cat book.txt | tr ' ' '¥ ' | tr '[:upper:]' '[:lower:]' | tr -d '[:punct:]' | grep -v '[^a-z]‘ | sort | uniq -c | sort -rn | head
  212. 212. © 2016 Pivotal Software, Inc. All rights reserved. Data Microservices $ cat book.txt | tr ' ' '¥ ' | tr '[:upper:]' '[:lower:]' | tr -d '[:punct:]' | grep -v '[^a-z]‘ | sort | uniq -c | sort -rn | head Microservice for each data processing
  213. 213. © 2016 Pivotal Software, Inc. All rights reserved. Data Microservices $ cat book.txt | tr ' ' '¥ ' | tr '[:upper:]' '[:lower:]' | tr -d '[:punct:]' | grep -v '[^a-z]‘ | sort | uniq -c | sort -rn | head Microservice for each data processing
  214. 214. © 2016 Pivotal Software, Inc. All rights reserved. Data Microservices $ cat book.txt | tr ' ' '¥ ' | tr '[:upper:]' '[:lower:]' | tr -d '[:punct:]' | grep -v '[^a-z]‘ | sort | uniq -c | sort -rn | head Microservice for each data processing bound with Message Brokers
  215. 215. © 2016 Pivotal Software, Inc. All rights reserved. Data Microservices $ cat book.txt | tr ' ' '¥ ' | tr '[:upper:]' '[:lower:]' | tr -d '[:punct:]' | grep -v '[^a-z]‘ | sort | uniq -c | sort -rn | head Microservice for each data processing bound with Message Brokers
  216. 216. © 2016 Pivotal Software, Inc. All rights reserved. Data Microservices $ cat book.txt | tr ' ' '¥ ' | tr '[:upper:]' '[:lower:]' | tr -d '[:punct:]' | grep -v '[^a-z]‘ | sort | uniq -c | sort -rn | head Microservice for each data processing bound with Message Brokers on the modern platform such as Cloud Foundry
  217. 217. © 2016 Pivotal Software, Inc. All rights reserved. Data Microservices $ cat book.txt | tr ' ' '¥ ' | tr '[:upper:]' '[:lower:]' | tr -d '[:punct:]' | grep -v '[^a-z]‘ | sort | uniq -c | sort -rn | head Microservice for each data processing bound with Message Brokers on the modern platform such as Cloud Foundry
  218. 218. © 2016 Pivotal Software, Inc. All rights reserved. Data Microservices $ cat book.txt | tr ' ' '¥ ' | tr '[:upper:]' '[:lower:]' | tr -d '[:punct:]' | grep -v '[^a-z]‘ | sort | uniq -c | sort -rn | head Microservice for each data processing bound with Message Brokers on the modern platform such as Cloud Foundry
  219. 219. © 2016 Pivotal Software, Inc. All rights reserved. Spring Cloud Data Flow • Microservices-based Distributed Data Pipelines •Long Lived Stream Applications •Short Lived Task Applications
  220. 220. © 2016 Pivotal Software, Inc. All rights reserved. Spring Cloud Data Flow • Microservices-based Distributed Data Pipelines •Long Lived Stream Applications •Short Lived Task Applications Spring Cloud Stream
  221. 221. © 2016 Pivotal Software, Inc. All rights reserved. Spring Cloud Data Flow • Microservices-based Distributed Data Pipelines •Long Lived Stream Applications •Short Lived Task Applications Spring Cloud Stream Spring Cloud Task
  222. 222. © 2016 Pivotal Software, Inc. All rights reserved. Spring Cloud Data Flow • Microservices-based Distributed Data Pipelines •Long Lived Stream Applications •Short Lived Task Applications Spring Cloud Stream Spring Cloud Task Orchestration Layer
  223. 223. © 2016 Pivotal Software, Inc. All rights reserved. Check my slide • http://www.slideshare.net/makingx/data-microservices-with- spring-cloud-stream-task-and-data-flow-jsug-springday
  224. 224. ‹#›© 2016 Pivotal Software, Inc. All rights reserved. Deploy Stream Apps to Cloud Foundry
  225. 225. © 2016 Pivotal Software, Inc. All rights reserved. Cloud Foundry •https://www.cloudfoundry.org/ •Cloud Native Platform •OSS •Spring ❤ Cloud Foundry •Support Multi IaaS 
 (AWS, Azure, GCP, vSphere, OpenStack)
  226. 226. © 2016 Pivotal Software, Inc. All rights reserved. Cloud Foundry everywhere Public Cloud Foundry Private Cloud Foundry Development OSS ver Pivotal Cloud Foundry on Premise on Public Cloud
  227. 227. © 2016 Pivotal Software, Inc. All rights reserved. PCF Dev • https://docs.pivotal.io/pcf-dev • Cloud Foundry on your laptop • Included • Redis / RabbitMQ / MySQL • Spring Cloud Services • Install with cf dev start
  228. 228. © 2016 Pivotal Software, Inc. All rights reserved. Pivotal Web Services •https://run.pivotal.io/ •Public Cloud Foundry managed by Pivotal •$0.03/GB-hr (≈ ¥2200/GB-month) •$87 of free trial credit.
  229. 229. © 2016 Pivotal Software, Inc. All rights reserved. Deploy Spring Cloud Stream Apps cf push my-source my-source.jar --no-start cf bind-service my-source my-binder cf start my-source cf push my-sink my-sink.jar --no-start cf bind-service my-sink my-binder cf start my-sink # in case of PCF Dev cf create-service p-rabbitmq standard my-binder # in case of Pivotal Web Services cf create-service cloudamqp lemur my-binder
  230. 230. © 2016 Pivotal Software, Inc. All rights reserved. Sample Application http://bit.ly/making_ccc_a3 Pivotal Web Services / PCF Dev Frontend Customer Service Email Service Point Service Post ServiceZipkin Server MySQL SendGrid MySQL MySQL DLQ Recovery RabbitMQ HTTP
  231. 231. © 2016 Pivotal Software, Inc. All rights reserved. Tutorial https://github.com/Pivotal-Japan/spring-cloud-stream-tutorial
  232. 232. © 2016 Pivotal Software, Inc. All rights reserved. Thanks!! • https://cloud.spring.io/spring-cloud-stream/ • https://cloud.spring.io/spring-cloud-dataflow/ • https://cloud.spring.io/spring-cloud-sleuth/ • http://zipkin.io/ • https://cloud.spring.io/spring-cloud-contract/ • https://projects.spring.io/spring-amqp/ • https://run.pivotal.io/

×