Two weeks ago, we had the opportunity to present our research at the Rijksuniversiteit Groningen (RUG) in The 15th Workshop on Service-oriented Enterprise Architecture for Enterprise Engineering. It's been an exciting journey, and in about a month, our findings will be published in the workshop proceedings of EDOC 2023. Over the past six months, we've been dedicated to this research, and we would like to share our insights.
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
Paving the path towards platform engineering using a comprehensive reference model.pptx
1. Paving the path towards
platform engineering
using a comprehensive
reference model
by
Ruben van de Kamp
Kees C. Bakker
Zhiming Zhao
2. About Ruben
● Recent graduate Master Software
Engineering at the University of Amsterdam
● Senior Software Engineer
● Working on services, integrations,
automation and frontend
● Passionate about web development and
platform engineering
● TypeScript / Python
Databricks, NodeJS, Flask
Sports, Economy
Ruben van de Kamp
ruvdkamp@gmail.com
linkedin.com/in/ruben-vd-kamp
3. 3
About Kees
● Pathfinder & Software Engineer
for Wehkamp Tech
● Working on architecture, integrations,
automation and chatops
● Passionate about web, marketing &
automation
● C# / .NET Core
TypeScript+NodeJS, PySpark+Databricks
Techblog, History
Kees C. Bakker
@KeesTalksTech
KeesTalksTech.com
kbakker@wehkamp.nl
linkedin.com/in/keescbakker
4. 1. Who is Wehkamp?
2. How to implement platform engineering?
3. Case study
4. Conclusions & future work
4
Topics
7. over 2.500 brands
WE Fashion // Vingino //Mango // Tommy Hilfiger // Scotch & Soda // ONLY // Private Label
wehkamp home // HK living // Woood // Zuiver // Riverdale // House Doctor
>300.000
different
products
>560.000
daily
visitors
783 million
Sales
21/22
>11 million
sent
packages
>1.000
colleagues
80%
of all customers
shop mobile
72%
of our customers
are female
Wehkamp facts
Focus on: fashion // living // beauty / baby & kids
8. How we run www.wehkamp.nl
Focus on: delivery
334.000
avg. requests
per minute*
19.6 GiB
avg. bandwidth
per minute*
3
FE stacks: web,
IOS, Android
150
Micro-sites &
services
50+
engineers + POs
10
autonomous
DevOps teams
1
platform
team
It ain’t easy being fast
Not just a website. It’s an e-commerce platform!
* Thursday 2023-10-24
20:00 - 22:30 CEST
according to Cloudflare
Statistics Dashboard
10. 10
What ignited our research?
Feels like they are
describing what
we’re trying to do.
Platform as a self-
service product for
developers?
Source: thenewstack.io/how-is-platform-
engineering-different-from-devops-and-sre/
11. 11
What ignited our research?
Many of our
services have the
same basic features
in terms of
observability. We
must be able to
scaffold or provision
them, just like our
databases, right?
Source: engineering.atspotify.com/2020/08/
How-we-use-golden-paths-to-solve-
fragmentation-in-our-software-ecosystem
12. Definition | Research questions | Related work | Reference Model
How to implement
platform engineering?
13. ● Discipline to improve productivity and
efficiency across teams by standardizing
and centralizing tools by introducing a
platform.
● Instead of focussing on a team level, it
focuses on the organization level.
● Tries to solve the following challenges:
proliferation of tools
deployment difficulties
cloud expenses and
communication between teams 13
What is platform engineering?
14. Main research question
How can a software organization
effectively integrate platform engineering
using a comprehensive reference model?
14
Research Questions
RQ1
How to model platform
engineering in the context of a
software company?
RQ2
How to define a customized
platform engineering design
tailored to a specific organization?
RQ3
How to effectively construct a
technical platform engineering
implementation?
15. ● Scientific research → scarcity
15
How to implement platform engineering?
Related work
16. ● Scientific research → scarcity
● Industry related articles →
technical (commercial) approach
16
How to implement platform engineering?
Related work
17. ● Based on the Reference Model
of Open Distributed Processing
(ODP-RM)
17
How to implement platform engineering?
The Platform Engineering Reference Model (PE-RM)
26. ● Implementation of the platform
26
Platform Engineering Reference Model
Technology viewpoint
27. ● Implementation of the platform
● Brownfields vs greenfield
27
Platform Engineering Reference Model
Technology viewpoint
Can be: Grafana, Prometheus,
CloudWatch, Datadog, etc.
Can be: Jenkins, CircleCI,
GitHub Actions, etc.
Start with the tooling that is
currently used.
28. Conceptual design | Technical implementation | Experiments
How to validate
platform engineering?
30. 30
Conceptual design
Enterprise viewpoint
● Gap analysis
● Organizational structure is in place
● Need to redefine roles
● Missing consultation structures (1) Pathfinders: more
formally involved in
the engineering
lifecycle.
31. 31
Conceptual design
Enterprise viewpoint
● Gap analysis
● Organizational structure is in place
● Need to redefine roles
● Missing consultation structures (1) Pathfinders: more
formally involved in
the engineering
lifecycle
(2) Accountability on
platform changes should
be made more formal:
planning and delivery to
stakeholders
32. 32
Conceptual design
Enterprise viewpoint
● Gap analysis
● Organizational structure is in place
● Need to redefine roles
● Missing consultation structures (1) Pathfinders: more
formally involved in
the engineering
lifecycle.
(3) Main stakeholders of
the platform.
(2) Accountability on
platform changes should
be made more formal:
planning and delivery to
stakeholders.
33. 33
Conceptual design
Enterprise viewpoint
● Gap analysis
● Organizational structure is in place
● Need to redefine roles
● Missing consultation structures (1) Pathfinders: more
formally involved in
the engineering
lifecycle.
(2) Accountability on
platform changes should
be made more formal:
planning and delivery to
stakeholders.
(3) Main stakeholders of
the platform.
(4) New stakeholder for
golden path adaptations
which should implement
new platform features.
35. 35
Conceptual design
Informational viewpoint
(2) To enhance understanding
of the platform and productivity
of the development teams.
(1) Improve documentation
and facilitate communication
between platform and
development teams.
36. 36
Conceptual design
Informational viewpoint
(3) Boost productivity,
increase standardization and
best practices, resulting in
improved onboarding time
(2) To enhance understanding
of the platform and productivity
of the development teams.
(1) Improve documentation
and facilitate communication
between platform and
development teams.
41. 41
Conceptual design
Technology viewpoint
No developer portal as a central
starting point for self service
provisioning, docs, observability,
software catalog, etc.
The platform is not
treated as 1
integrated product.
42. By building a prototype
in which we can experiment.
How to validate
platform engineering?
49. 50
What experiments did we design?
Experiment 1
Does the platform improve
productivity?
Experiment 2
Is this something the developer
community wants?
Experiment 3
Is this doable in a
production environment?
performance
analysis
usability
study
expert
feedback
51. ● Use case
● Three tasks
1. Deploy react application to dev
2. Deploy nestjs application to prod
3. Gather logs and metrics of
application in production
Experiments
Performance analysis
52. ● Role
● Ease of use
● Platform task ratio
Experiments
Usability study
easy difficult
53. ● Role
● Ease of use
● Platform task ratio
● Likeable feature
● Recommendation
Experiments
Usability study
54. ● Role
● Platform design
● Ease of use
Experiments
Platform experts feedback
difficult easy
55. ● Role
● Platform design
● Ease of use
● Applicability in organization
● Integration other systems
● Recommendation
Experiments
Platform experts feedback
not applicable very applicable
57. ● PE-RM to model platform engineering in the context of an organization
● Conceptual design guided by the PE-RM
● Technical implementation guided by the PE-RM
● Expose the added value and practical implementation of platform engineering
● Complexity of modelling a methodology (example: accountability)
● Organizational depended decisions
58
Conclusions
Achievements
58. RQ1: How to model platform engineering in the
context of a software company?
● Platform Engineering Reference Model (PE-RM)
● Validation by case study
59
Conclusions
Research questions
59. RQ2: How to define a customized platform engineering
design tailored to a specific organization?
● Conceptual design guided by the PE-RM
60
Conclusions
Research questions
60. RQ3: How to effectively construct a technical platform
engineering implementation?
● Basic engineering platform implementation
● Validation with experiments
● Showcase added value of platform engineering
61
Conclusions
Research questions
61. Main research question: How can a software
organization effectively integrate platform engineering
using a comprehensive reference model?
● Platform Engineering Reference Model
● Case study validation
62
Conclusions
Research questions
62. ● Implement the reference model in a case study
to measure the outcomes
● Case studies within different types of
organizations
● NOTE: The field is new → changes could be
needed
63
Conclusions
Future work
63. Paving the path towards
platform engineering
using a comprehensive
reference model
by
Ruben van de Kamp
Kees C. Bakker
Zhiming Zhao
Editor's Notes
KEES
Paper is based on the master thesis of Ruben van de Kamp, investigating platform engineering at Wehkamp.
KEES
Recent graduate Master Software Engineering at the University of Amsterdam - - graduated with an honours
Have been working for Wehkamp almost 4 years (Product Information team)
KEES
Explain path finder:
Not only enterprise architecture
Also way of working
Turning strategy into delivery
KEES
First we’re going to give some context about Wehkamp and our engineering setup
Then we’re going to look at platform engineering
We’ll elaborate on our case study
And we’ll share the conclusions
KEES
KEES
WRG: Wehkamp Retail Group
3 webshops: Wehkamp, Kleertjes.com and Union River
KEES
Let’s focus on Wehkamp and some statistics
Quite an operation
KEES
KEES
How did we get here?
KEES
Article by The New Stack introducing Platform Engineering
Had not heard about it yet
It feels like a fit, we already use many tools in our container platform.
KEES
Golden paths feel like a fit
Again, the fit is there: observability (logs, metrics) are 100% shared.
Most services use a form of dependency injection and ORM that is not so different.
So the main question is: How can we implement platform engineering?
One way to start: get a graduate student to do some research! So here’s where Ruben comes in!
RUBEN
RUBEN
Before we dive into the research we first want to give a short explanation of what platform engineering is and why it is important
Platform to centralize and standardize tools
Focusses on organization instead of team level
RUBEN
Main focus:
How can platform engineering be implemented
In order to give answer: 3 questions:
How to model
How to design guided by PE-RM
How to implement guided by PE-RM
RUBEN
It can be concise: not a lot of research done.
We have focussed on what reference models there are related to other methodologies:
DevOps and Agile
Focusses on team level and therefore not sufficient for platform engineering
How to model a reference model:
ODP-RM -> later will be explained
RUBEN
Many articles talking on what they believe platform engineering is
White paper from humanitec -> reference architecture, but only focusses on technical implementation
RUBEN
ODP-RM → multiple viewpoints to give better understanding
RUBEN
Created with a iterative process where we interviewed different experts
RUBEN
First thing will be how the platform lifecycle will work, by introducing a platform it needs to have a lifecycle
Two starting positions: no adoption, new features
Lead to same lifecycle where platform is build
RUBEN
The existing development lifecycle will stay the same
Platform engineering will have impact on provision of application domain that will help in development lifecycle
RUBEN
First talk about platform team
Talk about development team and that they are stakeholders of the platform
Development guild → important for golden paths
Enterprise architects
RUBEN
Point out the platform version and golden paths
Development guilds are maintaining these golden paths
RUBEN
Explain that the development team will create an application design and based on that they can come up with platform feature requests
RUBEN
Operations can be separated into two functionalities: getting applications in production and keeping applications in production
Computational viewpoint is more difficult → can be different for each organisation
Golden paths are essential in the translation of these operations
The sequence diagram separates the different actors in planes which we will showcase in the engineering viewpoint
Diagram link: https://app.diagrams.net/#G13CGjLkFYjmYsTMrY3XqGA_1XHfFMPVkd
RUBEN
Based on the Humanitec reference architecture since the separation of planes are valid
Normally it is important to point out the objects and channels, but this way we can better translate into real world scenarios
Developer Portal: one stop shop for self service
RUBEN
It depends on the organization and what tools they already use
Brownfield project
More focussed on the standardization and guidelines on what would be most suitable in this platform
RUBEN
It depends on the organization and what tools they already use
Brownfield project
More focussed on the standardization and guidelines on what would be most suitable in this platform
KEES
Nice overview by Ruben
Let’s see how this matches at Wehkamp
First we’ll create a conceptual design
And than Ruben will guide us through a prototype to do some experiments so validate our model
KEES
Implementation of features
Do’s or don’t – guard against excessive feature
KEES
KEES
Now: users, eat what you get
PE: stakeholder with influence
KEES
Golden path as a reference architecture
Golden path as a jump start
No formal delivery, poor versioning of the platform
Individual tools are now versioned and communicated
Platform should be one integrated product
Measure effectiveness
Missing golden paths
Missing process to further do platform development from the dev team perspective
Again: it is all about individual tools
Less effort
Standardize on default settings
KEES
Let’s add the tools to the boxes of the technology viewpoint
Our tools fit very well within the model
KEES
No dev portal
KEES
The platform is not treated as 1 integrated product.
Ruben wants to do some experiments
We’re missing metrics
Let’s get them from our pipelines / GitHub
Ruben wants to do some experiments
Ruben wants to do some experiments
We’re missing metrics
Let’s get them from our pipelines / GitHub
RUBEN
RUBEN
We mainly used two open source tools from which we continued to implement the platform:
Otomi: PaaS solution → helps to setup a lot of the tooling, however the configurations of the tooling and the way it all interacts as one implementation in this research had to be done manually
Backstage: this is the main point of work, this linked everything together including templates etc.
RUBEN
In order to showcase the added value a platform will bring to the organization and development teams we have created a very based application lifecycle
You will start with creating your application domain:
Create repository
Pipelines
Maybe some boilerplating
You will develop your application
You want to update your domain by making deployment files
You want to monitor you application
RUBEN
We have created golden paths that will have only one input: the application name
It will do the following:
Create your repository
Create your CI/CD pipelines
Create your docker registry
Scaffold the application with best practices
RUBEN
When you want to deploy an application you can use a different golden path → setup deployment workload files
This will take care of the following
Create yaml files for argocd to deploy application
Setup the docker image to the latest version that has been stored
Update pipelines to automatically deploy when new version is released
RUBEN
Because the application are scaffolded in the golden paths they will all expose defaults metrics
Use predefined dashboards to see the metrics
RUBEN
Now that we have created a prototype we can do some experiments to see if it actually has any effects
RUBEN
We did three experiments:
Performance analysis: validate if it is actually helping developers
Usability study: is this platform something that the developers want
Expert feedback: is the implementation suitable within an organization
Performance analysis and usability study are done in one experiment
People from inside and outside the organization were asked
Before we did the experiments we asked the participants to give an estimation on the time it would take them to deploy an entire application to production in which they already have the tools available, they only have to make the entire devops lifecycle
Average: almost 7 hours
The participants are divided into two categories:
No tutorial: deploy react application to dev
Tutorial: deploy nestjs application to Prod, but also dev
Tutorial: gather logs and metrics of application in production
The average time it took to get an application to production was around 22 minutes which is much lower than the average of almost 7 hours.
After the tasks the participants were asked to fill in a questionnaire
Role → different type of roles to get feedback from different experienced people
Ease of use → accidentally reversed the scale, however the ease of use was pretty create, however the open questions suggests that the multiple ui’s made it more difficult
Based on open questions a tutorial and documentation is very helpful especially when onboarding new developers
Task ratio → the platform did almost all the tasks, which is exactly how the participants experienced the experiments
Likeable feature → backstage and the golden paths, since it gave the developers an easier time setup their infrastructure
Recommendation → everybody suggests to introduce this within their work
IMPORTANT: tell about the fact that we first did a demo and discussion to give them the most information before the questionnaire
Role → different types of roles, not on experience but more on the way they are located in the organization, different viewpoints
Platform design → the design of the platform was great, both technical as well as functional
Ease of use → they believe the platform is easy to use
Applicability → not always very high → this was expected since each organization could have their own way of implementing things or have an existing tool set
Integration with other systems → same thing, depends on the organization since green field projects is not likely
Recommend → everybody would recommend the platform however Otomi is not always the best solution
RUBEN
Now that we have explained to you what we have done it is time to wrap things up and summarize our research
RUBEN
Now we only have created a reference model and validated it with a conceptual design and a separate prototype
It would be better to implement it in a real organization and see the effect, but that will take time
Case studies within more organizations to see if is applicable to a range of organizations
The field is new, so it can change fast and therefore the reference model needs to be updated (if necessary)
Although we have tried to keep it as general as possible