Nubank, Latin America’s first (and largest) cloud-native bank, has relied on AWS since day one. Operating in the cloud allows Nubank’s developers to create software that scales and quickly adapts to the changing needs of a complex market and a growing business. Nubank relies on services like Amazon EC2, Amazon DynamoDB, Amazon VPC, Amazon S3, and AWS CloudFormation to let 8.5 million customers make around 2.5 million purchases per day—all without a dedicated infrastructure team. Learn how Nubank’s fully automated, cell-based architecture allows the bank to provide the best customer experience while generating reliable audited financial records for regulators.

1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How Nubank isbuildinga
customer-obsessed Bank
Renan Capaverde
Director of Engineering
Nubank
F S V 2 0 1

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Agenda
• Nubank: An introduction
• Why create a bank from scratch in the cloud?
• Putting customer experience first
• Enabling developers to be customer obsessed
‒ Autonomy
‒ Reliability
‒ Scalability
‒ Velocity
• Lessons learned

3. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nubank:An introduction

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
WE STARTED HERE

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
TODAY

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
CVA Solution
“Nubank is the Brazilian’s favorite”

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
"With10 million customers,Nubankis the
largestonlinebankoutsideofAsia"

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Exponentialgrowth
0.0
3,000,000.0
6,000,000.0
9,000,000.0
12,000,000.0
June-2014 June-2015 June-2016 June-2017 June-2018 June-2019

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
BigBankshavebeenslowtoaddresscustomerfrustrations
89
78
69
15
7
-2
-11 -14
-27
Nubank Fintech 1 Fintech 2 Bank 1 Bank 2 Bank 3 Bank 4 Bank 5 Bank 6
NPS

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
270
1700
MICROSERVICES
EMPLOYEES
260ENGINEERS
40+
10M
3+SQUADS
CUSTOMERS
COUNTRIES

12. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Whycreatea bankfromscratch
in the cloud?

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
10%
22% 25% 29% 30%
45% 47%
352%
UK Chile USA Venezuela Colombia Peru Argentina Brasil
Highest interest rates for revolving
credit facilities worldwide …
Source: Proteste (Associação Brasileira de Defesa do Consumidor) – December 2017
https://www.proteste.org.br/institucional/imprensa/press-release/2017/brasileiro-paga-os-maiores-juros-do-mundo

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Big opportunity
Interest Rates & Fees
Customer experience
High

15. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Puttingcustomerexperiencefirst

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Technology Design Data Science Customer Experience
The four pillars of Nubank

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Completely digital, practical,
easy to use, with the best
service of the industry.
Credit card

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Integratedinthecreditcardthereis
asimple,intuitiveandcompletely
differentbenefitprogram.
Rewards

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Our version of a digital account.
Without concerns and without costs.
Developed for the modern days,
NuConta is the simplest and most
intelligent option in the market.
NuConta

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
MULTIPLE
MARKETS
Cannot tell you when ;-)

21. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Enablingdeveloperstobecustomerobsessed

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Wethinkofbankingasasoftwareengineeringproblem
Teams empowered to execute
independently, cradle to grave
autonomy
Rapidly evolving systems in small
increments
velocity
Carefully manage blast-radius and
time-to-fix for inevitable bugs
reliability
Build for the long term, scale out,
significant operating leverage
scalability

23. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Autonomy

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Engineering
Top of the line gas range
Sharp knives
Small batches
Sophisticated plate warming

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Security Operations
Center (SOC)
19 authorized personnel
Temperature normal
Order backlog normal
Behavior patterns normal

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Blue team
Non-skid floor
Safety hats and aprons
Food contamination risk
Segregated roles and access
control

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Red team
Propane tank
+
Sharp knife
+
Sparker
=
Profit

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
In-housesecurityteams
Blue Team Red Team SOC
Engineering Security SupportIntelligence
Physical Infra
• Don’t be an adversary
• Be part of the product
lifecycle
• Work closely with other
control functions
• Rotate team members

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Minimalpermissions(self-healing)
Fine-grained, just enough to accomplish work
Constant evolution
Lambdas
80+ official lambdas
Fine-grained control and orchestration of underlying systems
Management of accounts across providers
Integrations to Slack, OpsGenie, etc
Active monitoring (every n minutes)

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Minimalpermissions(self-healing)
Fine-grained, just enough to accomplish work
Constant evolution
IAM groups
100+ IAM groups for people
500+ IAM roles for machines
Access to specific operations on AWS services
Base permissions set, temporary escalation, automatic reaping
Lambdas

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Minimalpermissions(self-healing)
Fine-grained, just enough to accomplish work
Constant evolution
OAuth Scopes
IAM groups
~300 scopes in use
OAuth style, endpoint level granularity
Pre-approved, grants often contingent upon proof of
completed training (with tests!)
Restricted scopes and toxic combinations
Short-lived (expire), with longer lived refresh tokens for
rapid renewal
Auto-reaped scopes after inactivity
users++
Lambda
functions

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Microserviceshelpdividingthescopetoenhanceteam’sautonomy

34. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reliability

35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Security as code

36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
SecurityasCode:Rolecreation(before)
deploy
Internal Clojure project
wrapping cloud APIs
nucli
Internal cli to automate
operational workflows
AWS CloudFormation
template
Declarative, cohesive infrastructure
provisioning
Provisioning
Amazon EC2
IAM roles
Security groups
Load balancers
…

37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
SecurityasCode:Rolecreation(before)
copy/paste

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
SecurityasCode:Rolecreation(after)
deploy
Internal Clojure project
wrapping cloud APIs
nucli
Internal cli to automate
operational workflows
AWS CloudFormation
template
Declarative, cohesive
infrastructure provisioning
Amazon EC2
Security groups
Load balancers
…
IAM roles
Lambda
Robotic kitchen staff

39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
SecurityasCode:LambdaCI/CD
iam-policies
Internal repository for all things IAM
(with code reuse)
Lambda
Robotic kitchen staffpull requests CI/CD
lambda-automation
Lambda functions as code in git

40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Defenseindepth
Boundary defense is fundamental, but doesn’t address all attack vectors
Once the boundary has been compromised, it is necessary to defend subsequent layers
SERVICE A SERVICE B
mTLS
Modern ciphers & forward secrecy
Requests without certificates rejected at session layer
Device reputation scoring
Short-lived OAuth token grants endpoint-level scopes
Ubiquitous rate limits
Storage
Encryption at rest
Security groups per service
Specific IAM roles
Kafka
Digital signing of all messages
Sensitive topics envelope encrypted
Security groups
Office network
RADIUS + 802.11 authentication w/ certificates
Segregated subnets by function
AWS Session Manager for SSH

41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Pervasiveaudittrail
Set up for forensic analysis in advance
Multiple audit trails covering the same flows
Lambda
+
Redshift
All data from all production databases
(daily)
Including data provenance:
• Metadata associated with reified DB
transactions, incl. correlation ID,
user, service version
• Append-only (Datomic or Kafka)
• Automatically integrated to ETL for
high throughput querying
Splunk + S3
Unify all logs,
including all server
logs
Dashboards
Alarms
CloudTrail
VPC flow logs
HTTP reqs
CLI events
DNS reqs
Load balancers

42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
SecurityMonitoring:Dashboards
AWS CloudTrail Elastic Load
Balancing
AlertsVPC Flow Logs

43. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scalability

44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
70NODES PER CLUSTER
(M5.12XLARGE)
900PODS PER CLUSTER
1B
13
700MHTTP REQUESTS PER DAY
KUBERNETES CLUSTERS IN
PRODUCTION
KAFKA MESSAGES PER DAY

45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Shards
SERVICE 1
shard S0
SERVICE 3
SERVICE 2
SERVICE 1
shard S0
SERVICE 3
SERVICE 2
SERVICE 1
shard S0
SERVICE 3
SERVICE 2
Shard s0 Shard s2Shard s1

46. We offer many
services
acquisition
surrender
credit-card-accounts
peter-pan
sr-barriga
feed
billing
horadric
charging
griswold
mordor
fidc
double-entry
stevie
notification
next
bureau
bleach
shun-service
griswold
stormshield
idtrust-client
tyrion
tx-ranger
bonafont
ledger
russ-service
insulator
zedruu
falkor
deckard
conductor-client
line-items
dinowrangler
sommelier
wendy
big-mama
negociant
nexxera-client
auth
conductor-push
customerssmee
gemalto-client
chateau

47. mini startups inside the company

48. - reduce the impact of changes
- reduce the fear of change
- increases autonomy
- decreases conflicts between teams
- decreases the fear of going to prod
microservices

49. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Velocity

50. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Applications

51. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
400+
DEPLOYS PER WEEK

52. incremental
change review
tests
homologationproduction
monitoring

53. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
CI/CDtooling

54. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Infrastructure

55. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
deploy
Internal project that acts as
a wrapper of AWS and
Kubernetes API
definition
.edn files containing
definitions of all deployable
units with a declarative
format
AWS CloudFormation
Dynamic generation of AWS
CloudFormation templates
and Kubernetes templates

56. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
S1S0 S2
Global
V 1.0

57. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
S1S0 S2
Global
S1S0 S2
Global
V 1.0 V 2.0

58. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
S1S0 S2
Global
S1S0 S2
Global
V 1.0 V 2.0

59. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
S1S0 S2
Global
V 2.0

60. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lessonslearned

61. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Lessonslearned
• Customer obsession is a drive for better products, growth and better customer experience
• Technology should now focus in adding value to customers instead of being a cost center
• Microservices with a ton of automation help with scalability, reliability, velocity and autonomy
• Regulators expect audit trails, monitoring, risk mitigation and accountability
• This has nothing to do with Change-advisory boards and CYA culture
• Minimize the risk of deployments by using unit and integration testing, feature rollouts, canary deployments and
deploying frequent small changes instead of big batches
• Empower engineers. Automate bureaucracy and processes. Focus on learning and automation.
• Automate things so hard that even if someone intentionally wants to break things, it won’t be possible.

62. We’re hiring
https://nubank.workable.com
Sao Paulo, Brazil

63. Berlin, Germany
We’re hiring
https://nubank.workable.com

64. Mexico City, Mexico
We’re hiring
https://nubank.workable.com

65. Buenos Aires, Argentina
We’re hiring
https://nubank.workable.com

66. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Q&A

67. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Renan Capaverde
@apseyyyy