Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Technology Division
Case study: Goldman Sachs
An enterprise-scale Workflow and Decision platform
June 26, 2018
Disclaimer:...
2Technology Division
Agenda
• Workflow Elements
• Building an enterprise-wide Workflow Engine platform on Camunda
• Decent...
3
Workflow Elements
Enterprise-scale Workflow Engine capabilities
4Technology Division
Workflow at an enterprise scale
Case study: Goldman Sachs
40,000 5,000 2,000,000 1,000 14,000
TOTAL U...
5Technology Division
Evolution of workflow at GS (Then)
Case study: Goldman Sachs
1. Operations task service
Central
Task ...
6Technology Division
Evolution of workflow at GS (Now)
Case study: Goldman Sachs
Workflow Engine
(Business 1)
Camunda proc...
7Technology Division
Model Driven Development Focus
Case study: Goldman Sachs
We Promote Model-Driven Development
Why?
• S...
8Technology Division
Workflow Elements
• Containerized version of Workflow Engine
• Deployable by appdev teams
• No core t...
9Technology Division
Evolution of workflow at GS (Soon)
Case study: Goldman Sachs
Workflow Engine
(Business 1)
Camunda
pro...
10
Decision Modelling
Orchestration of decisions using Camunda, and new developments
11Technology Division
A firm-wide Decision Services Platform
Case study: Goldman Sachs
REST
endpoint
Regional
HAProxy
rout...
12Technology Division
Decision orchestration with Camunda
Case study: Goldman Sachs
Decision Orchestration with Camunda
• ...
13Technology Division
Decision orchestration with Camunda
Case study: Goldman Sachs
The GS “Decision Flow” implementation
...
14Technology Division
An open-source Java execution library for DMN
Case study: Goldman Sachs
Native Java execution for DM...
15Technology Division
An open-source Java execution library for DMN
Case study: Goldman Sachs
Camunda DMN design environme...
16Technology Division
An open-source Java execution library for DMN
Case study: Goldman Sachs
Performance of jDMN-generate...
Upcoming SlideShare
Loading in …5
×

Richard Tarling, Managing Director and Randall Graebner, Senior Engineer from Goldman Sachs - Camunda Day NYC 2018

98 views

Published on

Goldman Sachs Workflow systems provide vital infrastructure to the operation of the firm, coordinating the execution and delivery of millions of business-critical tasks across the entire Firm (including trade facilitation, regulatory rule computation, exception processing coordination, personnel onboarding and thousands of other business processes).

In this presentation, you will learn how Goldman Sachs is leveraging Camunda software at the heart of two of its Workflow platforms:

Workflow Elements – The Camunda process execution engine is integrated with a task engine, task cache and entitlements cache, wrapped by REST APIs, to implement a “managed appliance” which integrates with Firm systems and orchestrates both human and system processes at enterprise scale.

Decision Services Platform – The Camunda process execution engine is leveraged in our in-memory Decision Flow engine. The engine integrates high-performance rule execution with data integration in an extension of the DMN standard. The platform leverages GS’s own open-source jDMN execution engine.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Richard Tarling, Managing Director and Randall Graebner, Senior Engineer from Goldman Sachs - Camunda Day NYC 2018

  1. 1. Technology Division Case study: Goldman Sachs An enterprise-scale Workflow and Decision platform June 26, 2018 Disclaimer: Views and opinions expressed are for informational purposes only. They do not constitute a recommendation by Goldman Sachs to buy, sell or hold any security. No part of this material may, without Goldman Sachs’s prior written consent, be (i) copied, photocopied or duplicated in any form, by any means, or (ii) distributed to any person that is an employee, officer, director, or authorized agent of the recipient Copyright Note: 2018 Goldman Sachs. All rights reserved.
  2. 2. 2Technology Division Agenda • Workflow Elements • Building an enterprise-wide Workflow Engine platform on Camunda • Decentralizing the platform through Workflow Elements • Decision Modeling • Orchestrating a Decision Services Platform using Camunda • New developments in Decision Modeling Agenda Case study: Goldman Sachs
  3. 3. 3 Workflow Elements Enterprise-scale Workflow Engine capabilities
  4. 4. 4Technology Division Workflow at an enterprise scale Case study: Goldman Sachs 40,000 5,000 2,000,000 1,000 14,000 TOTAL USERS (100% OF THE FIRM) DAILY USERS NEW ACTIVITIES DAILY PLATFORM DEVELOPERS UNIQUE ACTIVE WORKFLOWS
  5. 5. 5Technology Division Evolution of workflow at GS (Then) Case study: Goldman Sachs 1. Operations task service Central Task Service GS Operations Division External System External System … Create tasks Create tasks External BPM execution server Workflow jBPM library … 2. jBPM library for deploying BPM execution servers … External BPM execution server Workflow jBPM library Central Task Service GS Operations Division  Central task service for Operations  Challenges:  Volume  Management  Support • Core team built jBPM Library  BPMN used to standardize workflows • Appdev teams run own BPM server  must understand jBPM services
  6. 6. 6Technology Division Evolution of workflow at GS (Now) Case study: Goldman Sachs Workflow Engine (Business 1) Camunda process engine Workflow Engine (Business 2) Camunda process engine 3. Hosted Workflow Engine service built upon Camunda … Aggregation Service Web UI … … Workflow Engine (Business 1) Camunda process engine Central Task Service (Business 1) Central Task Service (Business 2) Workflow Engine (Business N) Camunda process engine Central Task Service (Business N) …
  7. 7. 7Technology Division Model Driven Development Focus Case study: Goldman Sachs We Promote Model-Driven Development Why? • Scalable Orchestration • Self Documenting • Promotes Business Partnerships • Available to whole Firm • Leverages Camunda capabilities Built an Integrated Design Tool • Workflow Designer Built on BPMN.IO • Integrates with SDLC • Bespoke Forms Designer / UI Builder • Coming: DMN Integration Above: Workflow Designer for BPMN modelling, BPMN.IO courtesy of Courtesy of camunda Services GmbH Above: Custom forms design integrated to the Workflow Designer
  8. 8. 8Technology Division Workflow Elements • Containerized version of Workflow Engine • Deployable by appdev teams • No core team needed Scaling to the cloud with Camunda Case study: Goldman Sachs Workflow Control Centre • UI to support engine provisioning • Metrics, logs, configuration • Appdev teams can self-manage engines Above: Self-service order for a new Workflow Elements instance Above: Teams are able to configure and manage their Elements instance independently
  9. 9. 9Technology Division Evolution of workflow at GS (Soon) Case study: Goldman Sachs Workflow Engine (Business 1) Camunda process engine Workflow Engine Camunda process engine and embedded task service 4. Workflow Elements Engine running with embedded Task Service Aggregation Service Web UI … … Workflow Engine (Business 1) Camunda process engine … Workflow Engine Camunda process engine and embedded task service Central Task Service (Business 1)
  10. 10. 10 Decision Modelling Orchestration of decisions using Camunda, and new developments
  11. 11. 11Technology Division A firm-wide Decision Services Platform Case study: Goldman Sachs REST endpoint Regional HAProxy routing Asia CN …CN CN … … … CN …CN CN EMEA CN …CN CN … … … CN …CN CN Americas CN …CN … … … CN …CN CN Apache Zookeeper Common decision repository Elasticsearch Audit Store Modelling Application CN = Compute Node DSP Compute Node Rules Engine 1 … Rules Engine N Orchestration External Data Integration Local repository REST I/face Summary • The GS Decision Services Platform (DSP) is a firm-wide execution platform for Decision Models • Models are stateless & declarative representations of business logic • Globally-distributed, horizontally-scalable and highly resilient • Approx. 160 million executions across ~2600 decision models per day • Rapid growth from 100k (2015)  2m (2016)  50m (2017)  160m (2018) daily executions • Leveraging Camunda process execution engine for orchestration CN
  12. 12. 12Technology Division Decision orchestration with Camunda Case study: Goldman Sachs Decision Orchestration with Camunda • Driven by increasing complexity of solutions and limitations of a purely declarative representation • E.g. Calculate X  Then use X to retrieve data Y from external source  Then continue execution using Y • But we have no concept of “before” or “after” • Manual coordination of these steps by the user would be unreliable and introduce back-and-forth latency • We required a solution to orchestrate these scenarios and chose to adopt the Camunda process engine Key considerations • Highly performant • Ease of integration • Intuitive design environment • Lightweight implementation • Doesn’t require the full expressive power of BPMN • Can be stateless; no persistence required • All executions can be considered atomic • Potential for customisation
  13. 13. 13Technology Division Decision orchestration with Camunda Case study: Goldman Sachs The GS “Decision Flow” implementation • We implemented a lightweight and restricted version of the Camunda process engine for orchestrating these decisions • Customisation was achieved through Camunda configuration and implementation against standard interfaces • Limited components – restricted to a subset of BPMN, all stateless components • In-memory H2 database – no stateful components so no need for communication with shared database • Custom stubbed components – for example, no object serialization required given lack of persistence • Integrated with existing Camunda-based design environment BPMN.IO courtesy of Courtesy of camunda Services GmbH
  14. 14. 14Technology Division An open-source Java execution library for DMN Case study: Goldman Sachs Native Java execution for DMN • We have been executing decisions within powerful & general-purpose rules engines • Scope of decision modelling expanding to other use cases, in particular very high-volume, low-latency scenarios • This drove the development of a purpose-built library for native Java code-generation and execution of DMN decision models • We are releasing this as an open-source library under the name “jDMN” Decision Modelling1 DMN XML2 Code-generated Java3
  15. 15. 15Technology Division An open-source Java execution library for DMN Case study: Goldman Sachs Camunda DMN design environment • We have adopted the Camunda DMN.js library to allow (initial, basic) rendering of these DMN models inside our own designer • Integrated with the execution platform; users can open their decision models and execute in bulk against the platform directly • Decision modelling still takes place within an external application • However we are exploring the capabilities of the DMN.js library and considering extensions to modelling support in the future BPMN.IO courtesy of Courtesy of camunda Services GmbH
  16. 16. 16Technology Division An open-source Java execution library for DMN Case study: Goldman Sachs Performance of jDMN-generated Java • Runtime of these generated Java models is 10-40x faster than our existing (and optimised) rules engine implementation Open-sourcing • We have released jDMN as an open-source library: https://github.com/goldmansachs/jdmn • jDMN has full support for DMN 1.1 and the soon-to-be-finalised DMN 1.2 standard, and we fully intend to support DMN 2.0 as it develops • Please feel free to use or contribute, and get in touch if you’d like to understand more about the project Existing rules engine jDMN Averageexecutiontime(ms)

×