Bletchley is a home-grown decryption service that we built @square

1. Bletchley: dealing with HSM’s so you don’t
have to
@diogomonica • Square Security

2. Roadmap
‣ Square’s Service-Oriented Architecture
‣ Why do we need a decryption service?
‣ Our decryption service: Bletchley
‣ Bletchley’s architecture
‣ Use cases for Bletchley
‣ Conclusion

3. Square
‣ Mobile Payments Company.
‣ 1 Security Team.
‣ Infra: Java & Ruby, some Go.
‣ Moving > $15 billion annually.

4. Service
Oriented
Architecture
‣ Move fast!
‣ Loose component coupling.
‣ Independent scaling.
‣ Multiple languages.

5. ‣ Front ends
‣ User data
‣ Payments service
‣ Reader fulfillment
‣ TokenizationExample
Architecture

6. SOA
Security Goals
Establish Trust at Layer 7
‣ Authenticate and authorize every request
Protect Secrets
‣ Application secrets and customer data
Separate Concerns
‣ Principle of least privilege
Provide Common Security Infrastructure
‣ Get it right once, other services benefit

7. Security
Services
‣ Login Service:
verify user creds, create client cookies
‣ Token Service:
associates stable identifier with secret data
‣ Certificate Signing:
manages CAs
‣ Secret Management:
delivers secrets to other services
‣ Crypto Service:
offloaded crypto, manages keys

8. The Problem(s)
‣ Managing keys is hard.
‣ Infrastructure persists data aggressively.
‣ Crypto is hard ™
‣ Crypto can be expensive (CPU cycles && time && $$).

9. Why do we need
a decryption
service?
‣ Private Key centralization.
‣ Guaranteed key deletion.
‣ Get the code right, once.
‣ Crypto offloading.
‣ Database compromise requires an online attack.
‣ Hide the HSM complexity.

10. Bletchley

11. Assumptions
‣ We have a magic way to:
• Distribute secrets (e.g. private keys)
• Do strong S2S authentication

12. Our Solution:
Bletchley
‣ Very simple API.
‣ Issues public keys, decrypts with private keys.
‣ Supports strong key deletion.
‣ Backed by HSMs (nCipher).
• Hides the complexity/pain of dealing with
these things.

13. Bletchley API
‣ (publicKey, keyId) = createKey()
Bletchley Host
Service
createKey()
Bletchley Host
Service
(publicKey, keyId)
1
2

14. Bletchley API
‣ data = decrypt(keyId, blob)
Bletchley Host
Service
decrypt(keyId,
blob)
Bletchley Host
Service
data
1
2

15. Bletchley API
‣ success = deleteKey(keyId)
Bletchley Host
Service
deleteKey(KeyId)
Bletchley Host
Service
success2
1

16. Use Case 1:
External Partner
Square External Partner
{message}KprivBletchley Cluster
Money Moving App
Visa
{message}
1
23
4
Kpub
Kpriv

17. Bletchley
Architecture
‣ Several servers running the bletchley w/
access to HSMs
‣ Backed by a PG database
Bletchley Cluster
DATASHEET
SANbox9000Series
SANbox®
ProductFamily
Thenewlookforpowerful,easytomanagefabrics
TheSANbox9000istheflagshipintheSANboxlineoffabricswitches,intelligentstoragerouters,andstorageser-
vicesplatforms.Asindividualcomponents,everyQLogicSANboxdeliverstheadvantagesofabest-in-classproduct.
Workingtogetherasanintelligentnetworksolution,theyareeasytodeployandadministratorandtheymakeyour
SANperformbetter,too.That’swhytheentireQLogicSANboxlinewontheWindowsITPro“ReadersChoice”award.
Foryourswitchedfabric,youcancountonQLogicforexactlytherightswitch…fromthecore,tothedistribution
layer,totheedge.Forlow-costlocalandremoteserverconnectivity,QLogicIntelligentStorageRoutersboostutili-
zationwhiledrivingdowncostandcomplexity.Andforstoragevirtualization,theQLogicStorageServicesPlatform
offersnetwork-basedcommandandcontrolofyourheterogeneousstorage.Byvirtualizingstoragefromwithinthe
fabric,yougreatlysimplifymanagement.Moreimportantly,youensureanopenenvironmentthatcanaccommodate
multiplevendors,newsolutionsandfutureflexibility.
SANbox®
Thenewlookforpowerful,easytomanagefabrics
•SANbox9000StackableChassisSwitch
•SANbox8000StorageServicesPlatform
•SANbox6000IntelligentStorageRouter
•SANbox5000StackableSwitch
•SANbox1000FixedPortSwitch
database

18. Key Generation
‣ Each individual bletchley host generates keys
on it’s local HSM.
‣ The HSM uses files on disk to represent the
keys.
Bletchley Cluster
DATASHEET
SANbox9000Series
SANbox®
ProductFamily
Thenewlookforpowerful,easytomanagefabrics
TheSANbox9000istheflagshipintheSANboxlineoffabricswitches,intelligentstoragerouters,andstorageser-
vicesplatforms.Asindividualcomponents,everyQLogicSANboxdeliverstheadvantagesofabest-in-classproduct.
Workingtogetherasanintelligentnetworksolution,theyareeasytodeployandadministratorandtheymakeyour
SANperformbetter,too.That’swhytheentireQLogicSANboxlinewontheWindowsITPro“ReadersChoice”award.
Foryourswitchedfabric,youcancountonQLogicforexactlytherightswitch…fromthecore,tothedistribution
layer,totheedge.Forlow-costlocalandremoteserverconnectivity,QLogicIntelligentStorageRoutersboostutili-
zationwhiledrivingdowncostandcomplexity.Andforstoragevirtualization,theQLogicStorageServicesPlatform
offersnetwork-basedcommandandcontrolofyourheterogeneousstorage.Byvirtualizingstoragefromwithinthe
fabric,yougreatlysimplifymanagement.Moreimportantly,youensureanopenenvironmentthatcanaccommodate
multiplevendors,newsolutionsandfutureflexibility.
SANbox®
Thenewlookforpowerful,easytomanagefabrics
•SANbox9000StackableChassisSwitch
•SANbox8000StorageServicesPlatform
•SANbox6000IntelligentStorageRouter
•SANbox5000StackableSwitch
•SANbox1000FixedPortSwitch
database

19. Key Replication
‣ New keys are registered in the database
‣ Other bletchley hosts go to the original host
and retrieve it
Bletchley Cluster
DATASHEET
SANbox9000Series
SANbox®
ProductFamily
Thenewlookforpowerful,easytomanagefabrics
TheSANbox9000istheflagshipintheSANboxlineoffabricswitches,intelligentstoragerouters,andstorageser-
vicesplatforms.Asindividualcomponents,everyQLogicSANboxdeliverstheadvantagesofabest-in-classproduct.
Workingtogetherasanintelligentnetworksolution,theyareeasytodeployandadministratorandtheymakeyour
SANperformbetter,too.That’swhytheentireQLogicSANboxlinewontheWindowsITPro“ReadersChoice”award.
Foryourswitchedfabric,youcancountonQLogicforexactlytherightswitch…fromthecore,tothedistribution
layer,totheedge.Forlow-costlocalandremoteserverconnectivity,QLogicIntelligentStorageRoutersboostutili-
zationwhiledrivingdowncostandcomplexity.Andforstoragevirtualization,theQLogicStorageServicesPlatform
offersnetwork-basedcommandandcontrolofyourheterogeneousstorage.Byvirtualizingstoragefromwithinthe
fabric,yougreatlysimplifymanagement.Moreimportantly,youensureanopenenvironmentthatcanaccommodate
multiplevendors,newsolutionsandfutureflexibility.
SANbox®
Thenewlookforpowerful,easytomanagefabrics
•SANbox9000StackableChassisSwitch
•SANbox8000StorageServicesPlatform
•SANbox6000IntelligentStorageRouter
•SANbox5000StackableSwitch
•SANbox1000FixedPortSwitch
database
Bletchley Cluster
DATASHEET
SANbox9000Series
SANbox®
ProductFamily
Thenewlookforpowerful,easytomanagefabrics
TheSANbox9000istheflagshipintheSANboxlineoffabricswitches,intelligentstoragerouters,andstorageser-
vicesplatforms.Asindividualcomponents,everyQLogicSANboxdeliverstheadvantagesofabest-in-classproduct.
Workingtogetherasanintelligentnetworksolution,theyareeasytodeployandadministratorandtheymakeyour
SANperformbetter,too.That’swhytheentireQLogicSANboxlinewontheWindowsITPro“ReadersChoice”award.
Foryourswitchedfabric,youcancountonQLogicforexactlytherightswitch…fromthecore,tothedistribution
layer,totheedge.Forlow-costlocalandremoteserverconnectivity,QLogicIntelligentStorageRoutersboostutili-
zationwhiledrivingdowncostandcomplexity.Andforstoragevirtualization,theQLogicStorageServicesPlatform
offersnetwork-basedcommandandcontrolofyourheterogeneousstorage.Byvirtualizingstoragefromwithinthe
fabric,yougreatlysimplifymanagement.Moreimportantly,youensureanopenenvironmentthatcanaccommodate
multiplevendors,newsolutionsandfutureflexibility.
SANbox®
Thenewlookforpowerful,easytomanagefabrics
•SANbox9000StackableChassisSwitch
•SANbox8000StorageServicesPlatform
•SANbox6000IntelligentStorageRouter
•SANbox5000StackableSwitch
•SANbox1000FixedPortSwitch
database

20. Decryption
Authorization
‣ ACL could be stored in the Database
‣ On decryption request, verify if service
matches ACL
Service
DATASHEET
SANbox9000Series
SANbox®
ProductFamily
Thenewlookforpowerful,easytomanagefabrics
TheSANbox9000istheflagshipintheSANboxlineoffabricswitches,intelligentstoragerouters,andstorageser-
vicesplatforms.Asindividualcomponents,everyQLogicSANboxdeliverstheadvantagesofabest-in-classproduct.
Workingtogetherasanintelligentnetworksolution,theyareeasytodeployandadministratorandtheymakeyour
SANperformbetter,too.That’swhytheentireQLogicSANboxlinewontheWindowsITPro“ReadersChoice”award.
Foryourswitchedfabric,youcancountonQLogicforexactlytherightswitch…fromthecore,tothedistribution
layer,totheedge.Forlow-costlocalandremoteserverconnectivity,QLogicIntelligentStorageRoutersboostutili-
zationwhiledrivingdowncostandcomplexity.Andforstoragevirtualization,theQLogicStorageServicesPlatform
offersnetwork-basedcommandandcontrolofyourheterogeneousstorage.Byvirtualizingstoragefromwithinthe
fabric,yougreatlysimplifymanagement.Moreimportantly,youensureanopenenvironmentthatcanaccommodate
multiplevendors,newsolutionsandfutureflexibility.
SANbox®
Thenewlookforpowerful,easytomanagefabrics
•SANbox9000StackableChassisSwitch
•SANbox8000StorageServicesPlatform
•SANbox6000IntelligentStorageRouter
•SANbox5000StackableSwitch
•SANbox1000FixedPortSwitch
database
Bletchley
createKey() addPerm(keyId, service)
Service
DATASHEET
SANbox9000Series
SANbox®
ProductFamily
Thenewlookforpowerful,easytomanagefabrics
TheSANbox9000istheflagshipintheSANboxlineoffabricswitches,intelligentstoragerouters,andstorageser-
vicesplatforms.Asindividualcomponents,everyQLogicSANboxdeliverstheadvantagesofabest-in-classproduct.
Workingtogetherasanintelligentnetworksolution,theyareeasytodeployandadministratorandtheymakeyour
SANperformbetter,too.That’swhytheentireQLogicSANboxlinewontheWindowsITPro“ReadersChoice”award.
Foryourswitchedfabric,youcancountonQLogicforexactlytherightswitch…fromthecore,tothedistribution
layer,totheedge.Forlow-costlocalandremoteserverconnectivity,QLogicIntelligentStorageRoutersboostutili-
zationwhiledrivingdowncostandcomplexity.Andforstoragevirtualization,theQLogicStorageServicesPlatform
offersnetwork-basedcommandandcontrolofyourheterogeneousstorage.Byvirtualizingstoragefromwithinthe
fabric,yougreatlysimplifymanagement.Moreimportantly,youensureanopenenvironmentthatcanaccommodate
multiplevendors,newsolutionsandfutureflexibility.
SANbox®
Thenewlookforpowerful,easytomanagefabrics
•SANbox9000StackableChassisSwitch
•SANbox8000StorageServicesPlatform
•SANbox6000IntelligentStorageRouter
•SANbox5000StackableSwitch
•SANbox1000FixedPortSwitch
database
Bletchley
decrypt(keyId, blob) checkPerm(keyId, service)
1
2

21. Database Failure
‣ Decryptions become dependent on the
database for authorization
Bletchley Cluster
DATASHEET
SANbox9000Series
SANbox®
ProductFamily
Thenewlookforpowerful,easytomanagefabrics
TheSANbox9000istheflagshipintheSANboxlineoffabricswitches,intelligentstoragerouters,andstorageser-
vicesplatforms.Asindividualcomponents,everyQLogicSANboxdeliverstheadvantagesofabest-in-classproduct.
Workingtogetherasanintelligentnetworksolution,theyareeasytodeployandadministratorandtheymakeyour
SANperformbetter,too.That’swhytheentireQLogicSANboxlinewontheWindowsITPro“ReadersChoice”award.
Foryourswitchedfabric,youcancountonQLogicforexactlytherightswitch…fromthecore,tothedistribution
layer,totheedge.Forlow-costlocalandremoteserverconnectivity,QLogicIntelligentStorageRoutersboostutili-
zationwhiledrivingdowncostandcomplexity.Andforstoragevirtualization,theQLogicStorageServicesPlatform
offersnetwork-basedcommandandcontrolofyourheterogeneousstorage.Byvirtualizingstoragefromwithinthe
fabric,yougreatlysimplifymanagement.Moreimportantly,youensureanopenenvironmentthatcanaccommodate
multiplevendors,newsolutionsandfutureflexibility.
SANbox®
Thenewlookforpowerful,easytomanagefabrics
•SANbox9000StackableChassisSwitch
•SANbox8000StorageServicesPlatform
•SANbox6000IntelligentStorageRouter
•SANbox5000StackableSwitch
•SANbox1000FixedPortSwitch
database

22. keyID to the
rescue
‣ keyId = base64(key_alias|service1|
HMAC(key_alias, service1)
Bletchley Host
Service
decrypt(keyId,
blob)
Bletchley Host
Service
data
1
2

23. Decryption
Authorization
‣ Decryption authorization independent from
database
Service Bletchley
createKey(services)
1
newKeyId(services)
Service Bletchley
decrypt(keyId, blob)
2
decrypt(blob)
iff keyId.include?(service)

24. Key Deletion
‣ The key is marked for deletion in the DB
‣ All bletchley hosts securely delete it from disk
Service
DATASHEET
SANbox9000Series
SANbox®
ProductFamily
Thenewlookforpowerful,easytomanagefabrics
TheSANbox9000istheflagshipintheSANboxlineoffabricswitches,intelligentstoragerouters,andstorageser-
vicesplatforms.Asindividualcomponents,everyQLogicSANboxdeliverstheadvantagesofabest-in-classproduct.
Workingtogetherasanintelligentnetworksolution,theyareeasytodeployandadministratorandtheymakeyour
SANperformbetter,too.That’swhytheentireQLogicSANboxlinewontheWindowsITPro“ReadersChoice”award.
Foryourswitchedfabric,youcancountonQLogicforexactlytherightswitch…fromthecore,tothedistribution
layer,totheedge.Forlow-costlocalandremoteserverconnectivity,QLogicIntelligentStorageRoutersboostutili-
zationwhiledrivingdowncostandcomplexity.Andforstoragevirtualization,theQLogicStorageServicesPlatform
offersnetwork-basedcommandandcontrolofyourheterogeneousstorage.Byvirtualizingstoragefromwithinthe
fabric,yougreatlysimplifymanagement.Moreimportantly,youensureanopenenvironmentthatcanaccommodate
multiplevendors,newsolutionsandfutureflexibility.
SANbox®
Thenewlookforpowerful,easytomanagefabrics
•SANbox9000StackableChassisSwitch
•SANbox8000StorageServicesPlatform
•SANbox6000IntelligentStorageRouter
•SANbox5000StackableSwitch
•SANbox1000FixedPortSwitch
database
Bletchley
deleteKey(KeyId) markDelete(keyId)
Bletchley Cluster
DATASHEET
SANbox9000Series
SANbox®
ProductFamily
Thenewlookforpowerful,easytomanagefabrics
TheSANbox9000istheflagshipintheSANboxlineoffabricswitches,intelligentstoragerouters,andstorageser-
vicesplatforms.Asindividualcomponents,everyQLogicSANboxdeliverstheadvantagesofabest-in-classproduct.
Workingtogetherasanintelligentnetworksolution,theyareeasytodeployandadministratorandtheymakeyour
SANperformbetter,too.That’swhytheentireQLogicSANboxlinewontheWindowsITPro“ReadersChoice”award.
Foryourswitchedfabric,youcancountonQLogicforexactlytherightswitch…fromthecore,tothedistribution
layer,totheedge.Forlow-costlocalandremoteserverconnectivity,QLogicIntelligentStorageRoutersboostutili-
zationwhiledrivingdowncostandcomplexity.Andforstoragevirtualization,theQLogicStorageServicesPlatform
offersnetwork-basedcommandandcontrolofyourheterogeneousstorage.Byvirtualizingstoragefromwithinthe
fabric,yougreatlysimplifymanagement.Moreimportantly,youensureanopenenvironmentthatcanaccommodate
multiplevendors,newsolutionsandfutureflexibility.
SANbox®
Thenewlookforpowerful,easytomanagefabrics
•SANbox9000StackableChassisSwitch
•SANbox8000StorageServicesPlatform
•SANbox6000IntelligentStorageRouter
•SANbox5000StackableSwitch
•SANbox1000FixedPortSwitch
database

25. Key Rotation
‣ Service requests for new key
‣ Starts encrypting all new requests with new
key. Tries to decrypt all requests with both.
Service Bletchley
createKey(services)
1
keyId2 = newKeyId(services)
Service
addKey(keyId)
2
[ keyId1, keyId2 ]

26. Scaling
‣ Just add more hosts
Bletchley Cluster
DATASHEET
SANbox9000Series
SANbox®
ProductFamily
Thenewlookforpowerful,easytomanagefabrics
TheSANbox9000istheflagshipintheSANboxlineoffabricswitches,intelligentstoragerouters,andstorageser-
vicesplatforms.Asindividualcomponents,everyQLogicSANboxdeliverstheadvantagesofabest-in-classproduct.
Workingtogetherasanintelligentnetworksolution,theyareeasytodeployandadministratorandtheymakeyour
SANperformbetter,too.That’swhytheentireQLogicSANboxlinewontheWindowsITPro“ReadersChoice”award.
Foryourswitchedfabric,youcancountonQLogicforexactlytherightswitch…fromthecore,tothedistribution
layer,totheedge.Forlow-costlocalandremoteserverconnectivity,QLogicIntelligentStorageRoutersboostutili-
zationwhiledrivingdowncostandcomplexity.Andforstoragevirtualization,theQLogicStorageServicesPlatform
offersnetwork-basedcommandandcontrolofyourheterogeneousstorage.Byvirtualizingstoragefromwithinthe
fabric,yougreatlysimplifymanagement.Moreimportantly,youensureanopenenvironmentthatcanaccommodate
multiplevendors,newsolutionsandfutureflexibility.
SANbox®
Thenewlookforpowerful,easytomanagefabrics
•SANbox9000StackableChassisSwitch
•SANbox8000StorageServicesPlatform
•SANbox6000IntelligentStorageRouter
•SANbox5000StackableSwitch
•SANbox1000FixedPortSwitch
database

27. Use Case 2:
Internal File
Transfer
Square External Partner
{blob}Bletchley Cluster
File Transfer App
1
23
Kpriv
service1
{blob}Kpub
4
5
createKey(service1)

28. Use Case 2:
Internal File
Transfer
Square External Partner
Bletchley Cluster
File Transfer App
1
Kpriv
service1
{blob}Kpub
2
decrypt(keyID, {blob})
Kpub

29. Use Case 2:
Internal File
Transfer
Square External Partner
Bletchley Cluster
File Transfer App
1
Kpriv
service1
{blob}Kpub
decrypt(keyID, service1)

30. Use Case 3:
Downstream
Outage
Square
Customer
Bletchley Cluster
Money Moving App
Visa
1
2
{message}
{message}
Kpub
DATASHEET
SANbox9000Series
SANbox®
ProductFamily
Thenewlookforpowerful,easytomanagefabrics
TheSANbox9000istheflagshipintheSANboxlineoffabricswitches,intelligentstoragerouters,andstorageser-
vicesplatforms.Asindividualcomponents,everyQLogicSANboxdeliverstheadvantagesofabest-in-classproduct.
Workingtogetherasanintelligentnetworksolution,theyareeasytodeployandadministratorandtheymakeyour
SANperformbetter,too.That’swhytheentireQLogicSANboxlinewontheWindowsITPro“ReadersChoice”award.
Foryourswitchedfabric,youcancountonQLogicforexactlytherightswitch…fromthecore,tothedistribution
layer,totheedge.Forlow-costlocalandremoteserverconnectivity,QLogicIntelligentStorageRoutersboostutili-
zationwhiledrivingdowncostandcomplexity.Andforstoragevirtualization,theQLogicStorageServicesPlatform
offersnetwork-basedcommandandcontrolofyourheterogeneousstorage.Byvirtualizingstoragefromwithinthe
fabric,yougreatlysimplifymanagement.Moreimportantly,youensureanopenenvironmentthatcanaccommodate
multiplevendors,newsolutionsandfutureflexibility.
SANbox®
Thenewlookforpowerful,easytomanagefabrics
•SANbox9000StackableChassisSwitch
•SANbox8000StorageServicesPlatform
•SANbox6000IntelligentStorageRouter
•SANbox5000StackableSwitch
•SANbox1000FixedPortSwitch
Database
{message} Kpub 5
34
Kpriv

31. Use Case 3:
Downstream
Outage
Database
Square
Customer
Bletchley Cluster
Money Moving App
Visa
Kpub
12
3
{message}
DATASHEET
SANbox9000Series
SANbox®
ProductFamily
Thenewlookforpowerful,easytomanagefabrics
TheSANbox9000istheflagshipintheSANboxlineoffabricswitches,intelligentstoragerouters,andstorageser-
vicesplatforms.Asindividualcomponents,everyQLogicSANboxdeliverstheadvantagesofabest-in-classproduct.
Workingtogetherasanintelligentnetworksolution,theyareeasytodeployandadministratorandtheymakeyour
SANperformbetter,too.That’swhytheentireQLogicSANboxlinewontheWindowsITPro“ReadersChoice”award.
Foryourswitchedfabric,youcancountonQLogicforexactlytherightswitch…fromthecore,tothedistribution
layer,totheedge.Forlow-costlocalandremoteserverconnectivity,QLogicIntelligentStorageRoutersboostutili-
zationwhiledrivingdowncostandcomplexity.Andforstoragevirtualization,theQLogicStorageServicesPlatform
offersnetwork-basedcommandandcontrolofyourheterogeneousstorage.Byvirtualizingstoragefromwithinthe
fabric,yougreatlysimplifymanagement.Moreimportantly,youensureanopenenvironmentthatcanaccommodate
multiplevendors,newsolutionsandfutureflexibility.
SANbox®
Thenewlookforpowerful,easytomanagefabrics
•SANbox9000StackableChassisSwitch
•SANbox8000StorageServicesPlatform
•SANbox6000IntelligentStorageRouter
•SANbox5000StackableSwitch
•SANbox1000FixedPortSwitch
4
Kpriv

32. Disadvantages
‣ Cross-DC story is sad
‣ Tied to one vendor
‣ HSMs are hard to debug and support is bad.

33. Conclusions
‣ You should have a crypto service!
‣ Solves a lot of architectural problems.
‣ Get it right once.
‣ Save money by sharing HSM resources with multiple
applications.
‣ Not that hard to make HA

34. Thanks
@justincummins
@ebolten

35. @diogomonica diogo@squareup.com
https://squareup.com/careers/engineering