This document discusses the challenges of taking the open source Cloud Foundry platform to production. It identifies challenges in four areas: technology, organizational processes, culture, and processes. For technology, challenges include learning BOSH YAML, tailoring the deployment, migrating foundations, building and distributing buildpacks, and automating upgrades. Organizational challenges include hiring platform engineers and upskilling existing staff. Cultural challenges involve adopting an agile mindset and dealing constructively with failures. Process challenges include managing risks, supporting the platform through the community, fixing bugs, and giving back to open source.

1. The Challenges of Taking
Open Source Cloud
Foundry to Production
// Fabian Keller

2. Fabian Keller
Software Engineer @mimacom
Woodworker @127.0.0.1
# CLOUD PLATFORMS
# ARCHITECTURE
# DESIGN SYSTEMS
# DEVOPS
@_fabiankeller
www.fabian-keller.de
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 2

3. 2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 3

4. Why self-hosting at all?
There are some unique advantages
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 4
PRIVACY CONTROL ACCESS

5. The positions
There is more than yes or no
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 5
OSCF is easy.
It‘s the same technology.
There is no license cost attached.
OSCF is scary.
We don‘t have SLAs.
We‘re not an open source company.
This talk
Let‘s do this Better not do it

6. What are the challenges unique to
Open Source Cloud Foundry?
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 6

7. The Case
How we migrated the workload of our retailer
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 7
6
months to
go live
200
software
engineers
900
app
instances
in total
6
platform
engineers

8. The 4 Phases
we went through to take OSCF to production
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 8
Planning
Plan and create
required
infrastructure on
IaaS
Thriving
In
Concourse
we trust
Deploying
We can really do
this!
Operating
Mh, we need to get
this updated and
changed
Go Live Today

9. 2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 9
Technology Organisational
Processes Culture
The Challenges

10. 2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 10
Technology Organisational
Processes Culture
The Challenges

11. For developers, there is
no difference in using
OSCF!*
Same DX.
Same Operations.
Same Runtime.
*at least, there shouldn‘t be.
112019-09-11 / Cloud Foundry Summit Europe / Fabian Keller

12. So what‘s the difference?
It‘s just a different package for the same components
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 12
OSCF VendorYAML GUI
Packaging
Hiding Complexity

13. BOSH BOSH BOSH
Learning BOSH and all it‘s YAML
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 13

14. Tailoring the Deployment
Get inspired by the official operations files…
• Backup and Restore cf-deployment/operations/backup-and-restore/*.yml
• Log Management cf-deployment/operations/addons/enable-component-syslog.yml
• Monitoring bosh-deployment/hm/*.yml
• Resurrector by default
• Instance Sizing cf-deployment/operations/scale-to-one-az.yml
• High Availability by default
• Security several ops files for enabling TLS, rate limits, etc.
• Authentication cf-deployment/operations/test/add-oidc-provider.yml
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 14

15. Tailoring the Deployment
… and then write custom operations files
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 15
https://bosh.io/jobs/uaa?source=github.com/cloudfoundry/uaa-release&version=74.0.0#p%3duaa.ldap

16. Tailoring the Deployment
Dive deep to understand what happens
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 16

17. Tailoring the Deployment
Dive deep to understand what happens
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 17

18. Don’t be
afraid to read
the source
code!
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 18Photo by Jeremy Bishop on Unsplash

19. Migrating to Another Foundation
Leveraging route services to proxy traffic
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 19
CF-A
App App
App App
Load Balancer
CF-B
App App
App App
Before Migration
CF-A
App App
Migration
Proxy
Load Balancer
CF-B
App App
App App
During Migration
CF-A
App App
Migration
Proxy
Load Balancer
CF-B
App App
App App
After Migration

20. 2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 20
Technology Organisational
Processes Culture
The Challenges

21. Hiring
We‘re looking for a platform engineer
Required Skills
• With 10 years experience
operating a PaaS
• Who contributed to Cloud
Foundry
• Who used BOSH to run a
deployment
• Who knows Kubernetes, as it‘s our
next challenge
Required Skills
• Who has used a PaaS before
• Who knows Cloud Foundry
• Who has heard of BOSH before
• Who is interested in learning
Kubernetes
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 21

22. Upskilling
Invest in a culture of
learning and sharing
• Everyone must be able
to do everything
• Cloud Tooling
JSON / YAML / jq / Git / S3 /
Concourse / BOSH
• Pair & Mob
Programming
• Cloud Foundry
Architecture
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 22Photo by Aditya Romansa on Unsplash

23. Risk Management
Does OSCF have different risks compared to a vendor distribution?
• Same risks. Different Probability.
• Platform Breach: Equal Chances
• Platform Outage: Higher Chances
• Big difference: no risk outsourcing to a vendor!
• No guaranteed repair time
• No guaranteed security patches
• No help with disaster recovery
• What you can do: upskill & prepare!
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 23

24. 2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 24
Technology Organisational
Processes Culture
The Challenges

25. Buildpacks
Compiling from source…
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 25

26. Buildpacks
…and distributing accross environments
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 26

27. Upgrade Automation
Start small and iterate
• We‘re using GitOps and Concourse CI
• Pipeline pattern: upload releases à deploy à smoke test
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 27

28. Upgrade Automation Roadmap
So little time, so much to automate
• Automated release integration testing
• Download and bundle all releases
• Deploy to multiple environments with the exact same bits
• Optimize GitOps workflow
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 28

29. Support
Getting involved with the community
• CF Community: https://www.cloudfoundry.org/community/
• Slack: https://slack.cloudfoundry.org/
• GitHub: https://github.com/cloudfoundry
• Twitter: https://twitter.com/cloudfoundry
• Mailing Lists: https://lists.cloudfoundry.org/g/announce
• No SLAs
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 29

30. Bug Fixing & Patching
Learn Go
• Prepare to fix and patch things yourself
• Make a small patch in Go
• Build the BOSH release
• Verify the patch works
• Roll it out
• Give back to the community
• Companies are not set up for open source
• Open sourcing is not part of usual processes
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 30

31. 2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 31
Technology Organisational
Processes Culture
The Challenges

32. Agile
Adopting an agile mindset
• Delivering value, fast
• Maintaining and prioritizing a
backlog
• Daily Standup
• Pair / Mob programming
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 32
Photo by İrfan Simsar on Unsplash

33. Failure Happens
All the difference makes how you deal with it
• Failures by the platform team quickly affect hundreds of apps
• To err is human. No finger-pointing!
• Appreciate if someone did a good job!
• Be kind
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 33

34. Give back
• Share what you do
• Share how you do it
• Raise issues
• Contribute patches
• Get involved
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 34Photo by Tim Marshall on Unsplash

35. Cloud Foundry Operators Guide
See how we do things
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 35
https://cfops-guide.mimacom.com/ https://github.com/mimacom/cfops-guide

36. Retrospective
• Would we do it again? Hell yes!
• Enjoy the learning curve
• When to run OSCF
• If the PaaS is part of the business strategy
• Hybrid mode: critical vs. non-critical workloads
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 36

37. Thanks!
bosh -d audience deploy questions.yml
@_fabiankeller
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller
Photo by Bill Jelen on Unsplash
37

38. Credits
• Technology photo: Photo by Tim Mossholder on Unsplash
• Organisational photo: Photo by Tomoe Steineck on Unsplash
• Processes photo: Photo by Crystal Kwok on Unsplash
• Culture photo: Photo by Helena Lopes on Unsplash
• Icons: https://fontawesome.com
2019-09-11 / Cloud Foundry Summit Europe / Fabian Keller 38