Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
© 2016 Accenture. All Rights Reserved. 0
Architecting with the
Functional Paradigm for
High-Performance
Enterprise Applica...
1© 2016 Accenture. All Rights Reserved..
Contents
q Application Characteristics
q Technical Requirements
q Architecture Ou...
2© 2016 Accenture. All Rights Reserved..
The architecture approach described here applies to an application with the
follo...
3© 2016 Accenture. All Rights Reserved..
Technical Requirements
Provide reactive characteristics:
q Scalable, high-perform...
4© 2016 Accenture. All Rights Reserved..
Architecture Outline
The proposed architecture, based on the Lightbend stack, add...
5© 2016 Accenture. All Rights Reserved..
Challenges
q Need solid, pragmatic architecture patterns that address both busine...
6© 2016 Accenture. All Rights Reserved..
Software Structure – Function Flow Pattern
The Function Flow Pattern is a differe...
7© 2016 Accenture. All Rights Reserved..
Sample Function Flow Diagram
8© 2016 Accenture. All Rights Reserved..
Sample Asynchronous Non-Blocking Function Flow Module
// Preliminaries
val functi...
9© 2016 Accenture. All Rights Reserved..
q The atoms are pure functions, data access functions, and external interface cal...
10© 2016 Accenture. All Rights Reserved..
OO vs Functional Nesting of Calls
m0
m1
m2
m3
obj0 obj1 obj2 obj3
apply
apply
ap...
11© 2016 Accenture. All Rights Reserved..
Software Structure – Mapping of Functional to Technical
Designs
Functional desig...
12© 2016 Accenture. All Rights Reserved..
Advantages of the Function Flow Pattern
q Functional transparency
q Isolate tech...
13© 2016 Accenture. All Rights Reserved..
Taming Scala and the Lightbend Stack …
Scala can be used exclusively or in conce...
14© 2016 Accenture. All Rights Reserved..
… Taming Scala and the Lightbend Stack
q Establish and enforce Scala coding stan...
15© 2016 Accenture. All Rights Reserved..
Working with Non-Blocking Code
Unit testing and debugging non-blocking code
q Ne...
16© 2016 Accenture. All Rights Reserved..
Any large-scale application will need guidelines, patterns, utilities, and helpe...
17© 2016 Accenture. All Rights Reserved..
Architecture Pattern – Type-Safe Dependency Injection
Dependency injection is im...
18© 2016 Accenture. All Rights Reserved..
Architecture Pattern – Type-Safe Configuration
Access to configuration data shou...
19© 2016 Accenture. All Rights Reserved..
Combines Scala and Akka with battle-proven ActiveMQ to provide durable,
transact...
20© 2016 Accenture. All Rights Reserved..
Architecture Capability -- Sync Over Async
21© 2016 Accenture. All Rights Reserved..
Module Stereotypes
These are the different types of module that the system is ma...
22© 2016 Accenture. All Rights Reserved..
Building a Scala-Proficient Team
Having architecture and development leads with ...
23© 2016 Accenture. All Rights Reserved..
q Your Server as a Function, by Marius Eriksen, Senior Architect at Twitter –
ht...
Upcoming SlideShare
Loading in …5
×

Reactive summit functional_paradigm-20161005-unrestricted

54 views

Published on

Slides from the reactive summit.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Reactive summit functional_paradigm-20161005-unrestricted

  1. 1. © 2016 Accenture. All Rights Reserved. 0 Architecting with the Functional Paradigm for High-Performance Enterprise Applications by Paulo Villela and Hilda Lu October 5, 2016 v 1.0
  2. 2. 1© 2016 Accenture. All Rights Reserved.. Contents q Application Characteristics q Technical Requirements q Architecture Outline q Challenges q Software Architecture -- Function Flow Pattern q Taming Scala and the Lightbend Stack q Working with Non-Blocking Code q Architecture Guidelines, Patterns, and Utilities q Module Stereotypes q Building a Scala-Proficient Team
  3. 3. 2© 2016 Accenture. All Rights Reserved.. The architecture approach described here applies to an application with the following characteristics: q Functionality exposed as RESTful services q Highly complex business logic q Calls to multiple external systems, with varying availability and responsiveness profiles q High throughput: 300,000 requests per minute peak load q High availability, with 4-hour maintenance window allowable outside US business hours q Approximately 50/50 mix of database reads and writes, with very large data payload Application Characteristics
  4. 4. 3© 2016 Accenture. All Rights Reserved.. Technical Requirements Provide reactive characteristics: q Scalable, high-performance system, with efficient use of infrastructure resources q Provide predictable response times q Be resilient to failures and provide helpful responses in case of errors q If a external system we depend on is overwhelmed then we will push-back on requests to avoid further stress on the external system (“backpressure” / circuit-breaker) But don’t compromise on: q High development productivity, maintainability, and testability q Fast responsiveness to business changes q Operability and supportability q Cloud readiness q Ability to leverage plentiful developer skills and a proven enterprise software ecosystem
  5. 5. 4© 2016 Accenture. All Rights Reserved.. Architecture Outline The proposed architecture, based on the Lightbend stack, addresses the above requirements q Lightbend stack = Scala, possibly in combination with Java = Play / Akka HTTP = Akka actors q Execution architecture leveraging the functional paradigm = Functional programming principles applied at the architecture and design level = Modularization based on services implemented as flows of composable functions = Clean separation of business functions from architecture plumbing, factoring-out asynchronous and non-blocking aspects = Uses and complements the Lightbend stack to address the above requirements q A DevOps architecture tailored to the chosen technology stack = End-to-end automation, from source code management to operations monitoring = Leveraging best-of-breed tools like Git, Gradle, Jenkins, Ansible, Splunk = Enforcement of built-in quality and security from the get-go, with automated testing and static code scans
  6. 6. 5© 2016 Accenture. All Rights Reserved.. Challenges q Need solid, pragmatic architecture patterns that address both business and technical requirements q Technology stack complexity = Bewildering variety of ways to write applications with Scala and Lightbend stack = More ways to fail than with standard Java = Conventional wisdom not necessarily a good guide q Development and operations is much more challenging with non-blocking code than with traditional blocking code = Unit testing and debugging = Performance analysis and monitoring q Skilled Scala architects and developers with enterprise software delivery background are hard to find in the market
  7. 7. 6© 2016 Accenture. All Rights Reserved.. Software Structure – Function Flow Pattern The Function Flow Pattern is a different way to structure code, aligned with the functional paradigm q Is it really different? q Decompose services into flows q A flow module is a composite of functions which may be executed sequentially or in parallel q Clean separation of business functions from architecture plumbing q Separation of pure business functions from data access functions and external system call functions q Factor-out asynchronous and non-blocking aspects q Proactively manage dependences q This pattern is effective even for pure Java, but Scala makes it more natural
  8. 8. 7© 2016 Accenture. All Rights Reserved.. Sample Function Flow Diagram
  9. 9. 8© 2016 Accenture. All Rights Reserved.. Sample Asynchronous Non-Blocking Function Flow Module // Preliminaries val functionA = new FunctionA ... Val externalSvc1Ctx = new ExternalSvc1.CfgCtx(..., externalSvcEc, ...) val externalSvc1 = makeNice(new ExternalSvc1(externalSvc1Ctx), svc1Sentinel, svcTimeout) ... Val daf2Ctx = new Daf2.CfgCtx(..., databaseEc, ...) val daf2 = makeNice(new Daf2(daf2Ctx), databaseSentinel, dbTimeout) // Flow function def apply(x: T): Future[U] = { val result = for { _ <- Future.successful(()) xB = functionB(functionA(x)) fC1 = externalSvc1(functionC1(xB)) fC2 = externalSvc2(functionC2(xB)) xC1 <- fC1 xC2 <- fC2 xD = functionD(xC1, xC2) xE <- if (condY(xD)) daf2(xD) map functionE2 else daf1(xD) map functionE1 xF = functionF(xE) } yield xF result }
  10. 10. 9© 2016 Accenture. All Rights Reserved.. q The atoms are pure functions, data access functions, and external interface call functions q Flows may compose atoms and/or other flows Flow Composition
  11. 11. 10© 2016 Accenture. All Rights Reserved.. OO vs Functional Nesting of Calls m0 m1 m2 m3 obj0 obj1 obj2 obj3 apply apply apply apply flow f1 f2 f3 Object-OrientedFunctional
  12. 12. 11© 2016 Accenture. All Rights Reserved.. Software Structure – Mapping of Functional to Technical Designs Functional designs can be naturally mapped to technical designs, which in turn closely align with the code q Traditionally, flowcharts often used in functional designs to represent and decompose a business function q Flowcharts also a natural representation of functional composition or orchestration è use for technical design with FP q Boxes or sub-flows in a functional flowchart represent business sub-functions è mapped to functions in the technical flowchart q A complex function in a technical flowchart can be represented as a sub-flow q The technical flowchart can be refactored after the initial mapping q It is important to minimize the scope of functions with side-effects and to isolate them from pure functions
  13. 13. 12© 2016 Accenture. All Rights Reserved.. Advantages of the Function Flow Pattern q Functional transparency q Isolate technical complexity q Enable a simpler code structure q Focused function responsibilities / concerns, greater cohesion q Functional modules (business functions) are concerned with business logic only q Significantly reduce and simplify code inter-dependencies / coupling q Inherently testable q Align technical solution complexity to nonfunctional requirements q Ease of debugging q Minimize run-time surprises q Executable and testable code available early-on in the development lifecycle
  14. 14. 13© 2016 Accenture. All Rights Reserved.. Taming Scala and the Lightbend Stack … Scala can be used exclusively or in concert with Java. The Scala-Java mix can be tailored to the team's skills q The function-oriented software architecture is key to minimizing the complexity of the overall solution and individual modules (both Scala and Java) q Adhere to the functional paradigm, but with pragmatism q Use Scala for: = Orchestration of function composition (flow modules) = Domain objects = Architecture libraries and utilities = Unit testing = Functional testing q Other software components can be developed in Java or Scala = Business logic functions = Validation logic = Data access functions = Calls to external services
  15. 15. 14© 2016 Accenture. All Rights Reserved.. … Taming Scala and the Lightbend Stack q Establish and enforce Scala coding standards q Use Play or Akka HTTP only to expose REST service end-points q Use Akka actors to support certain key architecture capabilities q Do not use Akka actors to implement or directly support business functionality q Don’t feel compelled to take the advice of bloggers
  16. 16. 15© 2016 Accenture. All Rights Reserved.. Working with Non-Blocking Code Unit testing and debugging non-blocking code q Need to inspect future completions – patterns and/or support from ScalaTest q Special test kits available for Akka actors and Akka streams Log correlation for non-blocking code q Correlation of log entries across an end-to-end transaction is more important and also more difficult due to execution of a single request on multiple threads q Special techniques are required to enable logging context propagation across threads Performance monitoring of non-blocking code q Similar challenge to log correlation, but trickier as it is not straightforward to add up the pieces of asynchronous execution that make up an end-to-end response time q Requires one or more of the following and integration with reporting and alerting tools = Custom instrumentation utility calls in source code = Kamon tool = Lightbend monitoring tools
  17. 17. 16© 2016 Accenture. All Rights Reserved.. Any large-scale application will need guidelines, patterns, utilities, and helper libraries to provide common technical capabilities q Logging q Error-handling q Configuration q Dependency injection q Database access q Access control q External system access and integration patterns q etc. Architecture Guidelines, Patterns, and Utilities
  18. 18. 17© 2016 Accenture. All Rights Reserved.. Architecture Pattern – Type-Safe Dependency Injection Dependency injection is important for large enterprise applications. Keep it simple, lightweight, and type-safe. Do not use dependency injection frameworks q Dependency injection is important to enable parallel development and testing with large teams q Dependency injection frameworks introduce “magic” and complexity, obfuscation of wiring errors, and sometimes system startup overhead. They are not necessary with Scala q For Scala applications, use simple boot modules which are executed at start-up to create object/function instances in the appropriate sequence, injecting previously constructed objects into the constructors of subsequent ones as needed = Most wiring mistakes are detected at compile time, not at runtime
  19. 19. 18© 2016 Accenture. All Rights Reserved.. Architecture Pattern – Type-Safe Configuration Access to configuration data should be type-safe and application modules should not have a compilation dependence on other modules or libraries to access configuration data q In keeping with tradition, configuration data is externalized from the source code q Application code does not directly read configuration files q All configuration data required by a class is passed in through its constructor (typically as fields in case class instances) q All configuration data access within application code is type-safe q How does this work? Within boot modules (see section on dependency injection): = Configuration data is read from configuration files and converted to type-safe vals (which can be basic types or case classes) = The configuration vals are injected into the construction of objects/functions
  20. 20. 19© 2016 Accenture. All Rights Reserved.. Combines Scala and Akka with battle-proven ActiveMQ to provide durable, transactional, at-least-once messaging semantics Architecture Capability -- Fire & Forget
  21. 21. 20© 2016 Accenture. All Rights Reserved.. Architecture Capability -- Sync Over Async
  22. 22. 21© 2016 Accenture. All Rights Reserved.. Module Stereotypes These are the different types of module that the system is made up of. Each module stereotype has a well-defined purpose and constraints regarding other stereotypes it can call/access
  23. 23. 22© 2016 Accenture. All Rights Reserved.. Building a Scala-Proficient Team Having architecture and development leads with strong enterprise delivery experience and Scala skills is a strong predictor of project success. Skills can be built with the right approach q Need a software architect seasoned in Java enterprise delivery who is already comfortable with functional programming and Scala q Start with a small team. Project team selection is key = Quick learners with strong enterprise software development experience and knowledge of Scala or clear indications of aptitude to become proficient in Scala. = E.g., experience with other functional programming languages or expertise in JavaScript can be indicators of aptitude. q Scala “SMEs” can be dangerous if they lack of industrial-strength business software engineering experience. Some are just enjoying an early entrant advantage, not necessarily validated by successful delivery q One week of formal training (e.g., with a Lightbend instructor), with a project architect serving as part of the faculty = Customize training content, selecting topics to focus on, gloss over, or weed-out based on project needs, using examples from the project. q Follow initial training with instruction on the key architectural concepts = Decomposing and structuring the application according to the functional paradigm = The meaning and mechanics of the use of futures for non-blocking computing q Reinforce skills by attending training similar to Coursera’s Functional Programming Principles in Scala and Functional Programming Design in Scala q Start slowly, create high-quality app code that follows the architecture patterns and coding standards, to serve as a model for at-scale development. Stick with the functional paradigm, don’t regress into old habits q One more way to get Java developers comfortable with Scala is to use it for testing (ScalaTest, Gatling)
  24. 24. 23© 2016 Accenture. All Rights Reserved.. q Your Server as a Function, by Marius Eriksen, Senior Architect at Twitter – https://monkey.org/~marius/funsrv.pdf q Scala coding guidelines – https://github.com/pvillela/scala-style-guide q Play Framework: async I/O without the thread pool and callback hell – https://engineering.linkedin.com/play/play-framework-async-io-without-thread-pool-and- callback-hell References

×