Platform Security
IRL
Chris Rutter
Java Developer > Security Champion >
Security Architect > Security Tech Lead
Payments, Banking & Government Transformations
Lead Security Engineer @ Clearbank
choss@outlook.com
Chris Rutter
Tips and techniques learned building security at scale on a
large software platform
General concepts for thinking about modern security
Benefits of approaches such as standardisation and
convention
Talk Aims
An integration oriented design approach emphasizing systematic reuse, for
developing complex products based upon platforms and compatible hardware
and software virtual components, intended to reduce development risks,
costs and time to market
+ Reduce security risks
Platform Based Architecture
https://en.wikipedia.org/wiki/Platform-based_design
Platform Based Architecture
Large Government Department
â—Ź 60 Development Pizza teams
â—Ź 1700 Total employees
â—Ź 1000 microservices
â—Ź 6 Platform Teams
â—Ź 1 Platform Security Team (4 people)
Lessons & Techniques
â—Ź Lead who understands security and tech
â—Ź Product owner who understands value and business impact
â—Ź Floating engineering resources
Security Team Structure
Platform Security Team Mandate
â—Ź Build security into platform development workflows
â—Ź Improve Access Management on platform
â—Ź Improve operational security capability of platform
â—Ź Help coordinate Live vulnerabilities and incidents
â—Ź Improve app team development security
â—Ź Own and push platform-wide security initiatives
Scalable and Valuable Security
Vulnerable Dependencies
Understand Mitigation Flow
Traditional Approach
X 50
Traditional Approach
â—Ź Investigation of each finding takes approx 30-60 mins
â—Ź Buy licences and on-board 50 champions to use tool
â—Ź Security team consulting with 50 teams
â—Ź Investigation robustness difficult to standardise
â—Ź Shifting Left
â—Ź Giving ownership to engineers
Practical Approach
POC scanning 10 teams
Focus on Shared Libs
â—Ź Platform Security
â—‹ Scan
â—‹ Investigate
â—‹ Triage
â—Ź Platform Applications Team
â—‹ Build updated shared libraries
â—‹ Test
â—‹ Publish new version
Jackson Deserialization Vulnerability (CVE 2017-7525):
Introduction: This vulnerability takes advantage of the ability of an attacker to force a server
to deserialize a compromised class which is known to be on a large number of class paths
and inject malicious input which can result in code execution.
Am I Vulnerable?: You are vulnerable if you use polymorphic typing feature anywhere in
your code. This can be configured in a few ways: @JsonTypeInfo, @JsonSubTypes or
mapper.enableDefaultTyping()
How can I remediate?: You must ensure that you globally configure ObjectMapper
disableDefaultTyping() and have no instances of @JsonTypeInfo, @JsonSubTypes
Sensible & Specific Remediation Advice
Basic Code Scanning Engine
Vulnerability Search Terms
XXE XMLInputFactory, TransformerFactory,
SchemaFactory, SAXTransformerFactory,
XMLReader
Jackson Deserialization @JsonTypeInfo, @JsonSubTypes,
mapper.enableDefaultTyping()
Logback ServerSocketReceiver
Block Releases After Timeframe
â—Ź In-depth analysis by security engineer for accurate diagnosis
â—Ź Upgrade path can scale out and is easily Enforced
â—Ź Small implementation effort, low load on security team, low
cost of scanning tool
â—Ź Lots of engineer time saved by centralising investigations
â—Ź 80% of value for 20% of the resources
Benefits
â—Ź Shift left but only if valuable and manageable
â—Ź Ensure shared functionality is centralised, then it can be
scanned less times and investigated more thouroughly
â—Ź Understand and test the full remediation flow
â—Ź Focus on value over comprehensiveness, how much risk
reduced for effort? Fight this fight with Auditors!
â—Ź Implement in value iterations like any other project
General Scanning Tool Concepts
â—Ź Granular database users for each application
â—Ź Controls around their scope of access and permissions
â—Ź Frictionless process to on-board new application
Granular Database Access Management
Access Management Requirements
â—Ź App is created and requires access to a database cluster
â—Ź App permissions must be defined (which DBs, tables etc.)
â—Ź Users created in DB and credentials stored securely
â—Ź Credentials provided to applications on deployment
â—Ź Permissions altered if data model changes
Granular Database Access Management
Traditional Approach #1 â—Ź Manual DB & User creation via
ticket
â—Ź Manual security review
â—Ź Manual storage of creds in
secrets manager
Slow & Tedious
Prone to error
Encourages larger DBs and wider
roles
Traditional Approach
â—Ź Extremely slow and involved process
â—Ź Requires full-time team of DBAs
â—Ź Encourages wider roles and larger DBs
â—Ź Significant chance of manual error
â—Ź Comprehensive review of all permissions
â—Ź Defined ownership of database management
Traditional Approach #2
https://imgflip.com/i/4gst9e
Standardise App Creation
Automate App Creation Lifecycle
Standardise Deployment Descriptors
Cater For Exceptions
â—Ź Legacy Apps
â—Ź True Snowflakes
â—Ź Automatically Applied
â—Ź Managed via PR
â—Ź Approval group on PR
Benefits
â—Ź All access by namespace convention
â—Ź Fast and automatic provisioning, new app could go to prod instantly
â—Ź Security controls built in, free and pre-approved
â—Ź Can be used to automate:
â—‹ Scaffolding with application libraries
â—‹ IAM access to Cloud PaaS resources
â—‹ Deployment pipeline creation
â—‹ DockerFile Creation
â—Ź Still allows for legacy or non-compliant patterns but makes this harder
Concepts
â—Ź Automate app creation lifecycle
â—Ź Standardise architectural patterns
â—Ź Provision apps to be secure by default
â—Ź Mechanism built and owned by platform teams
â—Ź Codified exceptions for legitimate use cases / legacy
Tiger Team Platform Project
â—Ź Cross cutting projects can be very slow
â—Ź A lot of time spent managing big bang migrations
â—Ź Pockets of knowledge
Tiger Team
â—Ź Architect for design continuity
â—Ź Engineers rotating from all necessary platform teams
Tiger Project Plan
â—Ź Small iterations through to production, cutting down on intra-team
backlog slow down, minimising big bang migrations and flushing out
integration issues
Traditional Security Review Approach
â—Ź Pen Test or architectural review at end of project
â—Ź No security input during development
â—Ź Project can not be released in iterations which will massively slow down
project / integration Review / Pen Test
Iterative Just-In-Time Approach
Review
/ Re-modelThreat Model
Review
/ Re-model
Finalise Review
Iterative Just-In-Time Approach
â—Ź Cross-team project to migrate to a new container hosting solution
Iterative Threat Model
â—Ź Iteration 1
New dummy service in private zone, no internet connection and no database
access
Network controls must be in place to isolate new apps from production.
Access controls in place so new deployer cannot deploy to existing prod
â—Ź Iteration 2
Dummy service with vpn-locked down internet access, no database access
VPN Whitelist must be securely implemented
â—Ź Iteration 3
Dummy service with vpn-locked down internet access and access to database
holding non-sensitive public data
Database credentials must be protected and no access given to other databases
Benefits
â—Ź Support much faster delivery of cross-cutting improvement projects
â—Ź Security team involved in entire project, understanding domain from the
outset
â—Ź System has security built-in from initial design
â—Ź Security considered an enabler
â—Ź Avoid expensive fixes based on late pen test or security review
Conclusions
â—Ź Standardisation and convention is the easiest and cheapest way to scale
securely
â—Ź Understand what your teams share, and what they truly need to
customise
â—Ź Create platform teams with concrete ownership to build and own cross-
cutting functionality (including security)
● Don’t be scared to take the 80%
â—Ź Use Cross-functional virtual teams to build faster and more securely
â—Ź Use just-in-time threat modelling to support rapid delivery

Platform Security IRL: Busting Buzzwords & Building Better

  • 1.
  • 2.
    Java Developer >Security Champion > Security Architect > Security Tech Lead Payments, Banking & Government Transformations Lead Security Engineer @ Clearbank choss@outlook.com Chris Rutter
  • 3.
    Tips and techniqueslearned building security at scale on a large software platform General concepts for thinking about modern security Benefits of approaches such as standardisation and convention Talk Aims
  • 4.
    An integration orienteddesign approach emphasizing systematic reuse, for developing complex products based upon platforms and compatible hardware and software virtual components, intended to reduce development risks, costs and time to market + Reduce security risks Platform Based Architecture https://en.wikipedia.org/wiki/Platform-based_design
  • 5.
  • 6.
    Large Government Department â—Ź60 Development Pizza teams â—Ź 1700 Total employees â—Ź 1000 microservices â—Ź 6 Platform Teams â—Ź 1 Platform Security Team (4 people) Lessons & Techniques
  • 7.
    â—Ź Lead whounderstands security and tech â—Ź Product owner who understands value and business impact â—Ź Floating engineering resources Security Team Structure
  • 8.
    Platform Security TeamMandate â—Ź Build security into platform development workflows â—Ź Improve Access Management on platform â—Ź Improve operational security capability of platform â—Ź Help coordinate Live vulnerabilities and incidents â—Ź Improve app team development security â—Ź Own and push platform-wide security initiatives Scalable and Valuable Security
  • 9.
  • 10.
  • 11.
  • 12.
    Traditional Approach â—Ź Investigationof each finding takes approx 30-60 mins â—Ź Buy licences and on-board 50 champions to use tool â—Ź Security team consulting with 50 teams â—Ź Investigation robustness difficult to standardise â—Ź Shifting Left â—Ź Giving ownership to engineers
  • 13.
  • 14.
    Focus on SharedLibs â—Ź Platform Security â—‹ Scan â—‹ Investigate â—‹ Triage â—Ź Platform Applications Team â—‹ Build updated shared libraries â—‹ Test â—‹ Publish new version
  • 15.
    Jackson Deserialization Vulnerability(CVE 2017-7525): Introduction: This vulnerability takes advantage of the ability of an attacker to force a server to deserialize a compromised class which is known to be on a large number of class paths and inject malicious input which can result in code execution. Am I Vulnerable?: You are vulnerable if you use polymorphic typing feature anywhere in your code. This can be configured in a few ways: @JsonTypeInfo, @JsonSubTypes or mapper.enableDefaultTyping() How can I remediate?: You must ensure that you globally configure ObjectMapper disableDefaultTyping() and have no instances of @JsonTypeInfo, @JsonSubTypes Sensible & Specific Remediation Advice
  • 16.
    Basic Code ScanningEngine Vulnerability Search Terms XXE XMLInputFactory, TransformerFactory, SchemaFactory, SAXTransformerFactory, XMLReader Jackson Deserialization @JsonTypeInfo, @JsonSubTypes, mapper.enableDefaultTyping() Logback ServerSocketReceiver
  • 17.
  • 18.
    â—Ź In-depth analysisby security engineer for accurate diagnosis â—Ź Upgrade path can scale out and is easily Enforced â—Ź Small implementation effort, low load on security team, low cost of scanning tool â—Ź Lots of engineer time saved by centralising investigations â—Ź 80% of value for 20% of the resources Benefits
  • 19.
    â—Ź Shift leftbut only if valuable and manageable â—Ź Ensure shared functionality is centralised, then it can be scanned less times and investigated more thouroughly â—Ź Understand and test the full remediation flow â—Ź Focus on value over comprehensiveness, how much risk reduced for effort? Fight this fight with Auditors! â—Ź Implement in value iterations like any other project General Scanning Tool Concepts
  • 20.
    â—Ź Granular databaseusers for each application â—Ź Controls around their scope of access and permissions â—Ź Frictionless process to on-board new application Granular Database Access Management
  • 21.
    Access Management Requirements â—ŹApp is created and requires access to a database cluster â—Ź App permissions must be defined (which DBs, tables etc.) â—Ź Users created in DB and credentials stored securely â—Ź Credentials provided to applications on deployment â—Ź Permissions altered if data model changes Granular Database Access Management
  • 22.
    Traditional Approach #1â—Ź Manual DB & User creation via ticket â—Ź Manual security review â—Ź Manual storage of creds in secrets manager Slow & Tedious Prone to error Encourages larger DBs and wider roles
  • 23.
    Traditional Approach â—Ź Extremelyslow and involved process â—Ź Requires full-time team of DBAs â—Ź Encourages wider roles and larger DBs â—Ź Significant chance of manual error â—Ź Comprehensive review of all permissions â—Ź Defined ownership of database management
  • 24.
  • 25.
  • 26.
  • 27.
  • 28.
    Cater For Exceptions â—ŹLegacy Apps â—Ź True Snowflakes â—Ź Automatically Applied â—Ź Managed via PR â—Ź Approval group on PR
  • 29.
    Benefits â—Ź All accessby namespace convention â—Ź Fast and automatic provisioning, new app could go to prod instantly â—Ź Security controls built in, free and pre-approved â—Ź Can be used to automate: â—‹ Scaffolding with application libraries â—‹ IAM access to Cloud PaaS resources â—‹ Deployment pipeline creation â—‹ DockerFile Creation â—Ź Still allows for legacy or non-compliant patterns but makes this harder
  • 30.
    Concepts â—Ź Automate appcreation lifecycle â—Ź Standardise architectural patterns â—Ź Provision apps to be secure by default â—Ź Mechanism built and owned by platform teams â—Ź Codified exceptions for legitimate use cases / legacy
  • 31.
    Tiger Team PlatformProject â—Ź Cross cutting projects can be very slow â—Ź A lot of time spent managing big bang migrations â—Ź Pockets of knowledge
  • 32.
    Tiger Team â—Ź Architectfor design continuity â—Ź Engineers rotating from all necessary platform teams
  • 33.
    Tiger Project Plan â—ŹSmall iterations through to production, cutting down on intra-team backlog slow down, minimising big bang migrations and flushing out integration issues
  • 34.
    Traditional Security ReviewApproach â—Ź Pen Test or architectural review at end of project â—Ź No security input during development â—Ź Project can not be released in iterations which will massively slow down project / integration Review / Pen Test
  • 35.
    Iterative Just-In-Time Approach Review /Re-modelThreat Model Review / Re-model Finalise Review
  • 36.
    Iterative Just-In-Time Approach â—ŹCross-team project to migrate to a new container hosting solution
  • 37.
    Iterative Threat Model â—ŹIteration 1 New dummy service in private zone, no internet connection and no database access Network controls must be in place to isolate new apps from production. Access controls in place so new deployer cannot deploy to existing prod â—Ź Iteration 2 Dummy service with vpn-locked down internet access, no database access VPN Whitelist must be securely implemented â—Ź Iteration 3 Dummy service with vpn-locked down internet access and access to database holding non-sensitive public data Database credentials must be protected and no access given to other databases
  • 38.
    Benefits â—Ź Support muchfaster delivery of cross-cutting improvement projects â—Ź Security team involved in entire project, understanding domain from the outset â—Ź System has security built-in from initial design â—Ź Security considered an enabler â—Ź Avoid expensive fixes based on late pen test or security review
  • 39.
    Conclusions ● Standardisation andconvention is the easiest and cheapest way to scale securely ● Understand what your teams share, and what they truly need to customise ● Create platform teams with concrete ownership to build and own cross- cutting functionality (including security) ● Don’t be scared to take the 80% ● Use Cross-functional virtual teams to build faster and more securely ● Use just-in-time threat modelling to support rapid delivery

Editor's Notes

  • #7 Security usually last ones to the party
  • #12 Shifting left, each team finds out very quickly
  • #13 Most dependency issues are not actually vulnerable based on usage patterns, but that takes a lot of time. I’ve seen projects like this take years, as infosec teams attempt to manage all of the same results
  • #16 Remediate immediately if this is the case
  • #17 Build basic scanning engine for vulnerable usage patterns Can be ran by sec team / hosted in Jenkins or CodeBuild Central location for Regexes
  • #19 INTRODUCE DEPENDENCY CHECKING Most of the time wasted is investigation!
  • #20 INTRODUCE DEPENDENCY CHECKING Most of the time wasted is investigation!
  • #24 Most dependency issues are not actually vulnerable based on usage patterns, but that takes a lot of time. I’ve seen projects like this take years, as infosec teams attempt to manage all of the same results