In the context of a fast-paced competitive and regulated KYC market, Quicksign needed to rethink its platform. Our goal: quickly adjust to our customers' workflows and integration requirements, scale to support the growing usage and features of our services, be secured and resilient by design—while being fun to build on! Using Kafka Streams, the Camunda BPMN engine and some in-house developments, we built on Kafka and Kubernetes a fully reactive and streamed CQRS-ES microservices platform. It exposes, for each of our tenants’ processes and without involving any specific development, a bespoke and reactive REST API. We’ll share how we leveraged Kafka as the backbone of our architecture to fuel our microservices and Kafka Streams to implement the CQRS-ES architectural pattern which we believe is key to provide the strongest foundation for growth in the coming years.

1. 1
PARIS - 11 OCTOBRE 2018
Cédric Vidal
CTO
Rethinking digital onboarding: how
QuickSign's new microservices platform
leverages Kafka to massively scale
@cedricvidal

2. Kafka is the backbone of
microservices architecture

3. Why such a choice ?

4. ~500 million tweets per day
~1,3 million trace entries per day
We are not twitter ...

5. Who are we ?
● QuickSign is the European leader in digital
onboarding for financial services
● White label
● Now handling millions of digital subscriptions
per year
10 years experience
15 countries
+70% Transactions growth / year

6. The state of digital
subscription
● critical channel
● banks, insurances
● major challenge
● performance of the subscription

7. ● fast access to information
● fast decision
● automate the decision process
● push the data to the consumer
Realtime is key to the
decision process
realtime streamed
architectures

8. Artificial intelligence
● game changer
● the need for realtime dialog
● need to process a lot of data
● the machine can have doubts
● ask the human for confirmation

9. Traceability
● Regulated business
● Qualified signature ETSI 319 411
● Be able to prove conformance
● Non-repudiation

10. Data privacy and protection
● GDPR
● Personal data protection
● Multi tenant isolation
Full end to end
encryption !
No compromise !

11. Kafka Encryption
github.com/quicksign/kafka-encryption
code.quicksign.io/kafka-encryption
Apache 2
License
● application payload encryption
● GDPR request -> throw the key !

12. CQRS-ES with Kafka
user
Command Query Responsibility Segregation (Event Sourcing)
Aggregation Root
Store
direct modification
commands & queries
Controller
action
evaluates
Command
Event
EventEvent
Event Store
stores
produces
Command
Handler
reads
Event
EventEvent
Aggregation
Root
Event
Handler

13. Why Kafka ?
General observation
• Works for extremely massive big data use
cases
• Should handle easily our CQRS ES based
workloads …
Technical details of interest for CQRS-ES
• At least once guaranteed delivery
• Sequential processing per topic partition
• Sharding by design
• RPC like low latency
• Durability
• Replayability

14. Kafka Streams - The key
Technical details of interest for CQRS-ES
• Automatic offset tracking (inherited from Consumer
API)
• Embedded ephemeral disk based KV Store to persist
intermediate stream states and query stores
(Facebook’s RocksDB)
• Rich streaming DSL with operators (aggregates, joins,
…)
• Interactive Queries
Natural fit for Kubernetes’ stateless model
• a simple static void main which joins the Kafka Stream
cluster
• Very natural with the stateless Kubernetes deployment
model

15. All different
• Specific to each customer/product/country
• Business requirements
• Integration requirements
• Different maturity levels, business wise and technical wise

16. Progressive onboarding
• Start small
• Optimize progressively
• Gain maturity, business wise and technical wise

17. BPMN
• A process for each
customer/product
• State and transitions

18. The Kappa/CQRS-ES Architecture
REST APIWrite Interface Read Interface
Commands
History stream
History SPI
append only immutable log
Queries
Extremely fast read access
Streams
Pre-computed JSON
representations
Case µS
stream
processors
Avro

19. A reactively generated
REST API, crazy ?
● One API per customer
● API “generated” from BPMN
● Sync or Async
● Push or polling

20. Your turn!
Cédric Vidal
CTO
@cedricvidal

21. 21
PARIS - 11 OCTOBRE 2018