Cloud-native and with security integrated early in the software development process, Nubank is the largest digital bank in the world outside of Asia. Demand for higher levels of service and value, constantly evolving technology capabilities, and stringent regulatory requirements are all powerful forces reshaping retail banking. In this session, Nubank CTO Edward Wible discusses how the company mixes engineering culture, security philosophy and structure, automation, and integration with AWS security services. Learn how to leverage the day-to-day software development workflow for extensive security and maximum engineering throughput while minimizing the operational pain of running a large infrastructure.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How Nubank Automates Fine-Grained
Security with IAM, AWS Lambda, and
CI/CD
Edward Wible
Co-founder & CTO
Nubank
F S V 3 2 5

3. “Nubank is the largest digital bank in the
world outside of Asia”

4. CREDIT CARD
September 2014

5. BANK ACCOUNT
October 2017
13

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Growing quickly in a sensitive domain
Unique applications
18M+
Countries
198
Customers
5M+
Deploys per day
50
Microservices
180
Engineers
100+
0.0
1.3
2.5
3.8
5.0
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
FIGHT COMPLEXITY TO
EMPOWER PEOPLE
Core Purpose

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Be trustworthy
competence
reliability
how {
integrity
benevolence
why {

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Banking (and security) as a software engineering problem
Teams empowered to execute
independently, cradle to grave
autonomy
Rapidly evolving systems in
small increments
velocity
Carefully manage blast radius
and time-to-fix for inevitable bugs
reliability
Build for the long term, scale out,
significant operating leverage
scalability

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Web Services (AWS) tools are critical to achieving the
right balance
AWS Identity
and Access
Management
(IAM)
AWS
Lambda
AWS CloudTrail &
Amazon Virtual
Private Cloud
(Amazon VPC)

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Security principles

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Minimal permissions (self-healing)
Lambdas
Fine-grained, just enough to accomplish work
Constant evolution
• 80+ official Lambdas
• Fine-grained control and orchestration of underlying
systems
• Management of accounts across providers
• Integrations to Slack, OpsGenie, and more
• Active monitoring (every n minutes)

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
IAM groups
Minimal permissions (self-healing)
• 100+ IAM groups for people
• 500+ IAM roles for machines
• Access to specific operations on AWS services
• Base permissions set, temporary escalation, automatic
reaping
Lambdas
Fine-grained, just enough to accomplish work
Constant evolution

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
OAuth Scopes
IAM groups
Minimal permissions (self-healing)
users++
Lambdas
Fine-grained, just enough to accomplish work
Constant evolution
• ~300 scopes in use
• Oauth style, endpoint level granularity
• Pre-approved, grants often contingent upon
proof of completed training (with tests!)
• Restricted scopes and toxic combinations
• Short-lived (expire), with longer-lived refresh
tokens for rapid renewal
• Auto-reaped scopes after inactivity

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC flow
HTTP reqs
AWS CloudTrail
Lambdas
Nucli events
DNS
Load balancers
Pervasive audit trail
Set up for forensic analysis in advance
Multiple audit trails covering the same flows
+
Amazon Redshift
All data from all production
databases (daily)
Including data provenance:
• Metadata associated with reified
DB transactions, incl. correlation
ID, user, service version
• Append-only (Datomic or Kafka)
• Automatically integrated to ETL
for high throughput querying
Splunk + Amazon S3
Unify all logs,
including all server
logs
Dashboards
Alarms

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Defense in depth
Boundary defense is fundamental, but doesn’t address all attack vectors
Once the boundary has been compromised, it is necessary to defend subsequent layers
SERVICE A SERVICE B
mTLS
Modern ciphers & forward secrecy
Requests without certificates rejected at session layer
Device reputation scoring
Short-lived OAuth token grants endpoint-level scopes
Ubiquitous rate limits
Storage
Encryption at rest
Security groups per service
Specific IAM roles
Kafka
Digital signing of all messages
Sensitive topics envelope encrypted
Security groups
Office network
RADIUS + 802.11 authentication w/ certificates
Segregated subnets by function
AWS Session Manager for SSH

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
In-house security teams

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Engineering
Top-of-the-line gas range
Sharp knives
Small batches
Sophisticated plate warming

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Security Operations
Center (SOC)
19 authorized personnel
Temperature normal
Order backlog normal
Behavior patterns normal

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Blue team
Non-skid floor
Safety hats and aprons
Food contamination risk
Segregated roles and access
control

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Red team
Propane tank
+
Sharp knife
+
Sparker
=
Profit

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Blue Team Red Team SOC
Engineering
Security
Support
Intelligence
Physical Infra
In-house security teams
• Don’t be an adversary
• Be part of the product
lifecycle
• Work closely with other
control functions
• Rotate team members

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Security as code

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Security as code: Role creation (before)
Deploy
Internal Clojure project
wrapping cloud APIs
Nucli
Internal CLI to automate
operational workflows
AWS
CloudFormation
template
Declarative, cohesive
infrastructure provisioning
Provisioning
Amazon EC2
IAM roles
Security groups
Load balancers
…

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
copy/paste
Security as code: Role creation (before)

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Security as code: Role creation (after)
Deploy
Internal Clojure project
wrapping cloud APIs
Nucli
Internal cli to automate
operational workflows
AWS
CloudFormation
template
Declarative, cohesive
infrastructure provisioning
Amazon EC2
Security groups
Load balancers
…
IAM roles
Lambda
Robotic kitchen staff

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Security as code: Lambda CI/CD
IAM-policies
Internal repository for all things
IAM (with code reuse)
Lambda
Robotic kitchen staffpull requests CI/CD
Lambda-automation
Lambdas as code in Git

29. Nucli
“Automation is the Contingency Plan that
protects Software Systems from human failure”

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nucli: Granting OAuth scopes
nu security grant <scope> <user> --for=1hour …
2 Enforce user has permission to grant and not a self-grant
5 User can refresh token
3 Whitelist scope for user in auth service
4 Schedule scope revoke
1 Log event and alert via Slack

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nucli: IAM inline policies
nu security grant <username> s3 read <bucket> —for=20min …
3 Attach new inline policy for IAM user from pre-existing template
4 Schedule policy revoke (and eventually execute via Lambda)
1 Request & receive permission to read
2 Log event and alert via Slack

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nucli: Performance profiling
nu service flamegraph <service> <shard> …
2 Open SSH port in the right security group
5 Restore kernel parameter + close SSH port
3 Change kernel parameter for profiling within Docker container
4 Wait for data collection window, download SVG
1 Log event and alert via Slack

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nucli: Kafka maintenance
nu kafka increase-partitions <cluster> <topic> …
3 Open zookeeper port
5 Close zookeeper port, revoke permission
1 Ensure you are in the right IAM group
4
bin/kafka-topics.sh --alter --zookeeper zkurl:2181
--topic topic1 --partitions 4
2 Log event and alert via Slack

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why Nucli?
• Prevent people from doing manual task checklists
• Create leverage for security - Change once and everyone changes
behavior automatically
• Engineers will invent less-secure shortcuts unless provided with
secure shortcuts
• Make shortcuts robust to technology refresh cycles
• Over time, multi-step shell scripts become Lambdas

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Security monitoring

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Security monitoring: Dashboards
AWS CloudTrail Elastic Load
Balancing
AlertsVPC Flow Logs

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CloudTrail
ELB
Action!
e.g.: drop machine
thresholds & triggers
Splunk
realtime ingestion
Action!
e.g.: create new machine
declarative capacity requirements
Security monitoring: Automated response

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Technology / Layers 1 2 3 4 5 6 7
VPC - - - - - - -
ELB - - - - - - -
CloudTrail - - - - - - -
HTTPS - - - 3 - - -
SSH - - - - - - -
Kafka - - - - - - -
Security monitoring: Automated response

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fostering a security ownership mind-set

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Security ownership
Account management
No centralized login system
Automated onboarding &
offboarding; active correlation of
accounts across providers with Lambda
2FA and Yubikeys mandatory
Integrated logging and alerting
Account admins aren’t tool admins
Slack-based workflow for requesting
permissions
Change management
Secure design by co-creating with
embedded security team members
Pull request workflow, protected
master branch for more sensitive
repositories
Automated tests (including version
checks and other security scans) in
immutable build pipelines
Auditable manifests for every
deployable artifact containing ALL
versions used for a build
Engineering productivity team treats
CI/CD environment as security critical
IT management
Physical Office Network threat
modeling
RADIUS technology mapping and
managing employees to different
subnets and VLANs
Fully automated network
infrastructure
Automated employee machine
provisioning and maintenance
Nucli as the path of least resistance

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Summary
Security as code (Nucli)
Distributed security ownership
Decentralized change management
autonomy
Real-time log ingestion
Slack alerting
Monitoring / automated response
velocity
Minimal permissions
Menu instead of kitchen
Defense in depth
Audit trail
reliability
Distributed permissions granting
Automate all the things
Auto-reaping
In-house security team
scalability

42. Body Level One
Body Level Two
Body Level Three
Body Level Four
Body Level Five
Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

43. Please complete the
session survey in the
mobile app.
!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.