Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Building Cloud-Native Data-Intensive Applications with Spring

50 views

Published on

SpringOne Platform 2018
Building Cloud-Native Data-Intensive Applications with Spring - Sabby Anandan and Soby Chako, Pivotal

Published in: Software
  • Be the first to comment

Building Cloud-Native Data-Intensive Applications with Spring

  1. 1. Building Cloud-Native Data-Intensive Applications with Spring Soby Chacko [@sobychacko] Sabby Anandan [@sabbyanandan]
  2. 2. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Disclaimer 2
  3. 3. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Safe Harbor Statement The following is intended to outline the general direction of Pivotal's offerings. It is intended for information purposes only and may not be incorporated into any contract. Any information regarding pre-release of Pivotal offerings, future updates or other planned modifications is subject to ongoing evaluation by Pivotal and is subject to change. This information is provided without warranty or any kind, express or implied, and is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions regarding Pivotal's offerings. These purchasing decisions should only be based on features currently available. The development, release, and timing of any features or functionality described for Pivotal's offerings in this presentation remain at the sole discretion of Pivotal. Pivotal has no obligation to update forward looking information in this presentation. 3
  4. 4. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Plot Events in the Enterprise Events and Data-intensive Workloads A Practical Take on Data-Intensive Applications in Cloud Q+A 4
  5. 5. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Events in the Enterprise 1 Day 15 Million Rides 3 Million Drivers 78 Countries 5
  6. 6. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Event-centric Thinking 6 Profound Customer Engagement
  7. 7. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Real-time Matching Efficient Pickups/ Drops Lower Prices Improved Ride Opportunities More Income More Tips Customer: Driver Convenient Costs Less Flexible Customer: Rider(s) 7
  8. 8. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Events in Reality Unpredictable Traffic Patterns Resource Contention Faults Rewind / Replay Ordering Guarantees Slow Processing 8
  9. 9. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Events in Reality Unpredictable Traffic Patterns Resource Contention Faults Rewind / Replay Ordering Guarantees Slow Processing 1 Day 15 Million Rides! 9
  10. 10. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Data Intensity Unpredictable Traffic Patterns Resource Contention Faults Rewind / Replay Ordering Guarantees Slow Processing 1 Day 15 Million Rides! Maintainability Scalability Portability Reliability 10
  11. 11. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ “We call an application data- intensive if data is its primary challenge—the quantity of data, the complexity of data, or the speed at which it is changing.” 11
  12. 12. !12
  13. 13. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ You have data from disparate systems 13
  14. 14. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 14 You have data of different types
  15. 15. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 15 You have data of varying speed, size and shape
  16. 16. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 16 You have data that evolves
  17. 17. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ But, we are not Uber! 🤔 17
  18. 18. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Every company must find ways to learn, adapt, and grow. 😀 18
  19. 19. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Spring Cloud Stream a event-driven microservices framework Pluggable Binder Implementations Stream Partitions Consumer Groups Message Headers Testing Framework Content-type Negotiation Imperative + Functional Programming Model public class TransferServiceImpl implements TransferService { public TransferServiceImpl(AccountRepository ar) { this.accountRepository = ar; } 19
  20. 20. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Spring Cloud Stream a event-driven microservices framework Pluggable Binder Implementations Opportunities: Same code; Same tests; Drop-in replacement for a variety of Message Brokers 20 Rabbit MQ Apache Kafka Google PubSub Amazon Kinesis Azure Event Hubs Solace
  21. 21. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Spring Cloud Stream a event-driven microservices framework Pluggable Binder Implementations Stream Partitions Consumer Groups Message Headers Testing Framework Content-type Negotiation Imperative + Functional Programming Model public class TransferServiceImpl implements TransferService { public TransferServiceImpl(AccountRepository ar) { this.accountRepository = ar; } 21
  22. 22. Demo #1 A practical take on Events in Data Intensive Applications. 22
  23. 23. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ :UserCreated :UserCreated :UserNameChanged :UserActivated 23
  24. 24. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Driver’s nearby Surge Pricing: Passengers headed to the same event Split fare with friends 24
  25. 25. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 25 ? John Doe User userCreated() userActivated() userDeactivated() Commands // passenger getUserByGeoLocation() // drivers getUsersNearby() Queries :Eve Append-only Log Application State Event Storming CQRS Event Sourcing Architecture Methods / Patterns
  26. 26. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 26 Use Case: Trending User Activity by Region
  27. 27. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Spring Cloud Stream + Kafka Streams 27 Bindings support not only for Channels, but also for Kafka Streams types. Hide the infrastructure and configuration overhead from the developers. Stateless and Stateful as options.
  28. 28. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Spring Cloud Stream + Kafka Streams Topic = foo Partition n Partition 1 Partition 0 Spring Cloud Stream App Instance 0 KTableKStream GlobalKTable 28 Spring Cloud Stream App Instance 1
  29. 29. Demo #2 Let’s interact with the Data Intensive Applications. 29
  30. 30. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 30 Use Case: Trending User Activity by Region on a Moving Window
  31. 31. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Unpredictable Traffic Patterns Resource Contention Faults Rewind / Replay Ordering Guarantees Slow Processing 1 Day 15 Million Rides! Maintainability Scalability Portability Reliability 31 Data Intensity
  32. 32. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Maintainability Build, test, and iterate as frequently as needed. CI/CD as first-class thinking for data-centric workloads. Data processing guarantees in the event of rolling-upgrades. Scalability Auto-scale up/down based on throughput demands.
 Linear throughput characteristics as you scale applications. Same app runs locally in the laptop or in any cloud platform where there’s JVM. PortabilityReliability Focus on the business logic and the unit-level, integration, and acceptance tests.
 Depend on platform runtime (Kubernetes or Cloud Foundry) for reliability and resiliency guarantees.
 32 Data Intensive Design
  33. 33. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 33 A toolkit for building data integration, real-time streaming, and batch data processing pipelines. WHAT IS SPRING CLOUD DATA FLOW?
  34. 34. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Data pipelines consist of Spring Boot apps, using Spring Cloud Stream for event-streaming or Spring Cloud Task for batch processes. Ready for Data Integration with >60 out-of-the- box streaming and batch Apps. DSL, GUI, and REST-APIs to build and orchestrate data pipelines onto platforms like Kubernetes and Cloud Foundry. Continuous delivery for streaming data pipelines using Spring Cloud Skipper. Cron-job scheduler for batch data pipelines using Spring Cloud Scheduler. 34 A toolkit for building data integration, real-time streaming, and batch data processing pipelines. WHAT IS SPRING CLOUD DATA FLOW?
  35. 35. Demo #3 Run all the things in the Cloud. 35
  36. 36. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 1 3 2 36 Putting it all together 1 2 3 Websocket Connection REST request/response REST request/response
  37. 37. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Next First-class support for Apps with Multiple Input & Output Channels (eg: KStream) Apps as Composable Functions Audit Trail: Who did what and when? 37
  38. 38. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Resources Spring Cloud Stream Samples | Gitter | StackOverflow Spring Cloud Data Flow Samples | Gitter | StackOverflow Demo: https://github.com/sabbyanandan/eda 38
  39. 39. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Q+A 39

×