SlideShare a Scribd company logo
Developer Experience at
The Guardian
Equal Experts, Sept 2021
Akash
Askoolum
he/him
Senior Software Engineer
akash.askoolum@theguardian.com
@akash1810
👋
What does the DevX team do?
A brief history
2015
Solely on AWS
(no more on premises servers)
2012
Move to AWS started
(mix of on premises and AWS)
May 1821
First Manchester Guardian
newspaper published
1999
Guardian Unlimited network
of websites launched
aws.amazon.com/solutions/case-studies/guardian
Autonomous
teams
● Dotcom team, building
theguardian.com
● Apps teams, building the iOS
and Android apps
● Reader Revenue teams,
building supporter and
contribution platforms
● Journalism teams, building
internal tools for Editorial
staff
News website of the year
theguardian.com/p/t6qvf
Digital revenues
theguardian.com/p/937j6
(some of) technologies used by the Engineering dept.
July 2020
Full time DevX team
formed
A brief history (continued)
2015
Solely on AWS
(no more on premises servers)
2012
Move to AWS started
(mix of on premises and AWS)
May 1821
First Manchester Guardian
newspaper published
1999
Guardian Unlimited network
of websites launched
aws.amazon.com/solutions/case-studies/guardian
What does the DevX team do?
3 areas of focus
1. Client side tools and
infrastructure
2. Security tools and
infrastructure
3. Operation tools and
infrastructure
Mission
Enable teams to focus on
delivering value at lightning
speed by streamlining
infrastructure management
Our Tools
Our DevX Tools
● AMIable - Amazon Machine Image (AMI) management tool
● AMIgo - An AMI bakery
● Anghammarad - Centralised notifications service
● CDK - Generic Guardian flavoured AWS CDK components
● Central ELK - Centralised logging platform
● dev-nginx - Tools to configure a local development nginx to proxy our applications and services
● DNS Validation - An AWS lambda that automatically creates ACM validation DNS records
● Grafana - Centralised metrics service
● Gu:who - answering: who are all these users in my GitHub org?
● Janus - Google-backed AWS account access
● master-to-main - A CLI to automate updating a GitHub repository master branch to main
● Prism - Tool for collecting live data about infrastructure so it can be easily queried by users and automated tooling
● PRout - Looks after your pull requests, tells you when they're live
● Repo Apocalypse - Archive old Github projects to S3
● RiffRaff - The Guardian's deployment platform
● Security HQ - Centralised security information for AWS accounts
● Source - The Guardian’s design system
● SSM Scala - ssh replacement: CLI program that wraps SSM's EC2 Run Command
● Strap - Bootstrap your macOS development system
● ...
Our DevX Tools
● AMIable - Amazon Machine Image (AMI) management tool
● AMIgo - An AMI bakery
● Anghammarad - Centralised notifications service
● CDK - Generic Guardian flavoured AWS CDK components
● Central ELK - Centralised logging platform
● dev-nginx - Tools to configure a local development nginx to proxy our applications and services
● DNS Validation - An AWS lambda that automatically creates ACM validation DNS records
● Grafana - Centralised metrics service
● Gu:who - answering: who are all these users in my GitHub org?
● Janus - Google-backed AWS account access
● master-to-main - A CLI to automate updating a GitHub repository master branch to main
● Prism - Tool for collecting live data about infrastructure so it can be easily queried by users and automated tooling
● PRout - Looks after your pull requests, tells you when they're live
● Repo Apocalypse - Archive old Github projects to S3
● RiffRaff - The Guardian's deployment platform
● Security HQ - Centralised security information for AWS accounts
● Source - The Guardian’s design system
● SSM Scala - ssh replacement: CLI program that wraps SSM's EC2 Run Command
● Strap - Bootstrap your macOS development system
● ...
😱😱😱😱
😱
Janus
When making API calls to AWS locally,
AWS credentials are required.
Developers do not have any IAM users
for our AWS accounts, instead we use
temporary credentials or short-term,
federated sessions.
Janus is our federated login system for
managing access to our myriad AWS
accounts.
This approach allows us to make our
infrastructure easier and more secure
to use.
Google-backed AWS account
access
github.com/guardian/janus-app
Janus
github.com/guardian/janus-app
SSM Scala
A command line tool that wraps AWS Systems Manager (SSM).
Used to execute commands on EC2 servers using EC2 Run command. It
provides the user with:
● an alternative to SSH for running commands
● standard SSH access using short lived RSA keys
We have removed direct SSH access (port 22) from applications and
replaced it with SSM Scala.
github.com/guardian/ssm-scala
Riff-Raff
Our applications primarily run
on EC2 or Lambda.
Riff-Raff is our deployment tool
and it allows us to use AWS
auto-scaling to achieve zero
downtime deploys.
Riff-Raff also records the
deployment history of each
project, and can be used to
schedule deploys.
The Guardian's deployment
platform
github.com/guardian/riff-raff
Riff-Raff
github.com/guardian/riff-raff
Riff-Raff
github.com/guardian/riff-raff
Source
Source is our Design System.
Written in React and Emotion, it
creates a consistent user and
developer experience across
*.theguardian.com and the apps.
It's expanding to support
internal tooling too.
The Guardian's design
system
github.com/guardian/source
Prism
Prism captures information about the infrastructure across our entire
estate in near real-time.
There is also a command line tool that uses the Prism API to locate
infrastructure.
github.com/guardian/prism
What’s next?
Areas of focus
For the Operations team
● Observability
● Runtime environment
● Configuration
● Deployment
Guardian Cloud Development Kit
Let’s talk about CloudFormation...
✅ Tracked in VCS
✅ Repeatable
❌ JSON / YAML
❌ JSON can become verbose
❌ YAML is sensitive to whitespace
❌ Long feedback loop
❌ Errors first seen at runtime (UPDATE_ROLLBACK_COMPLETE 😈)
❌ Can result in not following the principles of least privilege
❌ Encourages copy pasting
❌ Consistency?
❌ Following best practice?
Guardian CDK
✅ Tracked in VCS
✅ Repeatable
✅ Written in a strongly typed language (TypeScript)
✅ Built using AWS CDK
✅ Unit tested
✅ Encodes today’s best practices
✅ Observability
✅ Runtime environment
✅ Configuration
✅ Deployment
✅ Follows the principles of least privilege
✅ npm install @guardian/cdk@latest
github.com/guardian/cdk
Guardian CDK
github.com/guardian/cdk
Guardian CDK
● A set of Guardian flavoured AWS CDK constructs and patterns,
encoding today’s best practice on:
○ Observability
○ Runtime environment
○ Configuration
○ Deployment
● CloudFormation in TypeScript
● Tracked in VCS
● Improved consistency
● Not YAML or JSON 🎉
○ Strongly typed
○ Unit tested
github.com/guardian/cdk
What does the DevX team do?
A lot of our backend tooling
revolves around simplifying
AWS resource management
across multiple accounts.
Observation
Vision
We will collaboratively create and
maintain best in class standard patterns
for deployment, configuration, runtime
and observability.
Creating and maintaining apps that follow
these patterns will be quick, logical, free
of boilerplate and secure by default.
We will have worked with teams to
migrate the majority of applications to
our standard patterns.
Teams will be focussed on delivering
business value at high velocity as
infrastructure concerns have been
streamlined.
Thanks!
Questions?
🙏
akash.askoolum@theguardian.com
@akash1810
github.com/guardian
workforus.theguardian.com

More Related Content

What's hot

betterCode Workshop: Effizientes DevOps-Tooling mit Go
betterCode Workshop:  Effizientes DevOps-Tooling mit GobetterCode Workshop:  Effizientes DevOps-Tooling mit Go
betterCode Workshop: Effizientes DevOps-Tooling mit Go
QAware GmbH
 
Building a Python Serverless Applications with AWS Chalice - AWS Online Tech...
 Building a Python Serverless Applications with AWS Chalice - AWS Online Tech... Building a Python Serverless Applications with AWS Chalice - AWS Online Tech...
Building a Python Serverless Applications with AWS Chalice - AWS Online Tech...
Amazon Web Services
 
Serverless use cases with AWS Lambda - More Serverless Event
Serverless use cases with AWS Lambda - More Serverless EventServerless use cases with AWS Lambda - More Serverless Event
Serverless use cases with AWS Lambda - More Serverless Event
Boaz Ziniman
 
An Overview of Spinnaker
An Overview of SpinnakerAn Overview of Spinnaker
An Overview of Spinnaker
Pierre-Nicolas Durette
 
AWS Code{Commit,Deploy,Pipeline} (June 2016)
 AWS Code{Commit,Deploy,Pipeline} (June 2016) AWS Code{Commit,Deploy,Pipeline} (June 2016)
AWS Code{Commit,Deploy,Pipeline} (June 2016)
Julien SIMON
 
Integrating-Cloud-Development-Security-And-Operations.pdf
Integrating-Cloud-Development-Security-And-Operations.pdfIntegrating-Cloud-Development-Security-And-Operations.pdf
Integrating-Cloud-Development-Security-And-Operations.pdf
Amazon Web Services
 
Twelve Factor Serverless Applications
Twelve Factor Serverless ApplicationsTwelve Factor Serverless Applications
Twelve Factor Serverless Applications
Amazon Web Services
 
Securing your AWS Deployments with Spinnaker and Armory Enterprise
Securing your AWS Deployments with Spinnaker and Armory EnterpriseSecuring your AWS Deployments with Spinnaker and Armory Enterprise
Securing your AWS Deployments with Spinnaker and Armory Enterprise
DevOps.com
 
Continuous delivery and deployment on AWS
Continuous delivery and deployment on AWSContinuous delivery and deployment on AWS
Continuous delivery and deployment on AWS
Shiva Narayanaswamy
 
Kubernetes 101 and Fun
Kubernetes 101 and FunKubernetes 101 and Fun
Kubernetes 101 and Fun
QAware GmbH
 
From Docker Straight to AWS
From Docker Straight to AWSFrom Docker Straight to AWS
From Docker Straight to AWS
DevOps.com
 
Cloud Native Unleashed
Cloud Native UnleashedCloud Native Unleashed
Cloud Native Unleashed
QAware GmbH
 
All the Ops: DataOps with GitOps for Streaming data on Kafka and Kubernetes
All the Ops: DataOps with GitOps for Streaming data on Kafka and KubernetesAll the Ops: DataOps with GitOps for Streaming data on Kafka and Kubernetes
All the Ops: DataOps with GitOps for Streaming data on Kafka and Kubernetes
DevOps.com
 
Rein in Your Cloud Costs with Terraform and AWS Lambda
Rein in Your Cloud Costs with Terraform and AWS LambdaRein in Your Cloud Costs with Terraform and AWS Lambda
Rein in Your Cloud Costs with Terraform and AWS Lambda
Amanda MacLeod
 
Intro to js august 31
Intro to js august 31Intro to js august 31
Intro to js august 31
Thinkful
 
Crafting Kubernetes Operators
Crafting Kubernetes OperatorsCrafting Kubernetes Operators
Crafting Kubernetes Operators
Red Hat Developers
 
Large Scale Kubernetes on AWS at Europe's Leading Online Fashion Platform - A...
Large Scale Kubernetes on AWS at Europe's Leading Online Fashion Platform - A...Large Scale Kubernetes on AWS at Europe's Leading Online Fashion Platform - A...
Large Scale Kubernetes on AWS at Europe's Leading Online Fashion Platform - A...
Henning Jacobs
 
Netflix Open Source: Building a Distributed and Automated Open Source Program
Netflix Open Source:  Building a Distributed and Automated Open Source ProgramNetflix Open Source:  Building a Distributed and Automated Open Source Program
Netflix Open Source: Building a Distributed and Automated Open Source Program
aspyker
 
Spinnaker workshop
Spinnaker workshopSpinnaker workshop
Spinnaker workshop
Lee Xie
 
Pipelining DevOps with Jenkins and AWS
Pipelining DevOps with Jenkins and AWSPipelining DevOps with Jenkins and AWS
Pipelining DevOps with Jenkins and AWS
Jimmy Ray
 

What's hot (20)

betterCode Workshop: Effizientes DevOps-Tooling mit Go
betterCode Workshop:  Effizientes DevOps-Tooling mit GobetterCode Workshop:  Effizientes DevOps-Tooling mit Go
betterCode Workshop: Effizientes DevOps-Tooling mit Go
 
Building a Python Serverless Applications with AWS Chalice - AWS Online Tech...
 Building a Python Serverless Applications with AWS Chalice - AWS Online Tech... Building a Python Serverless Applications with AWS Chalice - AWS Online Tech...
Building a Python Serverless Applications with AWS Chalice - AWS Online Tech...
 
Serverless use cases with AWS Lambda - More Serverless Event
Serverless use cases with AWS Lambda - More Serverless EventServerless use cases with AWS Lambda - More Serverless Event
Serverless use cases with AWS Lambda - More Serverless Event
 
An Overview of Spinnaker
An Overview of SpinnakerAn Overview of Spinnaker
An Overview of Spinnaker
 
AWS Code{Commit,Deploy,Pipeline} (June 2016)
 AWS Code{Commit,Deploy,Pipeline} (June 2016) AWS Code{Commit,Deploy,Pipeline} (June 2016)
AWS Code{Commit,Deploy,Pipeline} (June 2016)
 
Integrating-Cloud-Development-Security-And-Operations.pdf
Integrating-Cloud-Development-Security-And-Operations.pdfIntegrating-Cloud-Development-Security-And-Operations.pdf
Integrating-Cloud-Development-Security-And-Operations.pdf
 
Twelve Factor Serverless Applications
Twelve Factor Serverless ApplicationsTwelve Factor Serverless Applications
Twelve Factor Serverless Applications
 
Securing your AWS Deployments with Spinnaker and Armory Enterprise
Securing your AWS Deployments with Spinnaker and Armory EnterpriseSecuring your AWS Deployments with Spinnaker and Armory Enterprise
Securing your AWS Deployments with Spinnaker and Armory Enterprise
 
Continuous delivery and deployment on AWS
Continuous delivery and deployment on AWSContinuous delivery and deployment on AWS
Continuous delivery and deployment on AWS
 
Kubernetes 101 and Fun
Kubernetes 101 and FunKubernetes 101 and Fun
Kubernetes 101 and Fun
 
From Docker Straight to AWS
From Docker Straight to AWSFrom Docker Straight to AWS
From Docker Straight to AWS
 
Cloud Native Unleashed
Cloud Native UnleashedCloud Native Unleashed
Cloud Native Unleashed
 
All the Ops: DataOps with GitOps for Streaming data on Kafka and Kubernetes
All the Ops: DataOps with GitOps for Streaming data on Kafka and KubernetesAll the Ops: DataOps with GitOps for Streaming data on Kafka and Kubernetes
All the Ops: DataOps with GitOps for Streaming data on Kafka and Kubernetes
 
Rein in Your Cloud Costs with Terraform and AWS Lambda
Rein in Your Cloud Costs with Terraform and AWS LambdaRein in Your Cloud Costs with Terraform and AWS Lambda
Rein in Your Cloud Costs with Terraform and AWS Lambda
 
Intro to js august 31
Intro to js august 31Intro to js august 31
Intro to js august 31
 
Crafting Kubernetes Operators
Crafting Kubernetes OperatorsCrafting Kubernetes Operators
Crafting Kubernetes Operators
 
Large Scale Kubernetes on AWS at Europe's Leading Online Fashion Platform - A...
Large Scale Kubernetes on AWS at Europe's Leading Online Fashion Platform - A...Large Scale Kubernetes on AWS at Europe's Leading Online Fashion Platform - A...
Large Scale Kubernetes on AWS at Europe's Leading Online Fashion Platform - A...
 
Netflix Open Source: Building a Distributed and Automated Open Source Program
Netflix Open Source:  Building a Distributed and Automated Open Source ProgramNetflix Open Source:  Building a Distributed and Automated Open Source Program
Netflix Open Source: Building a Distributed and Automated Open Source Program
 
Spinnaker workshop
Spinnaker workshopSpinnaker workshop
Spinnaker workshop
 
Pipelining DevOps with Jenkins and AWS
Pipelining DevOps with Jenkins and AWSPipelining DevOps with Jenkins and AWS
Pipelining DevOps with Jenkins and AWS
 

Similar to Developer Experience at the Guardian, Equal Experts Sept 2021

Infrastructure Is Code with the AWS Cloud Development Kit (DEV372) - AWS re:I...
Infrastructure Is Code with the AWS Cloud Development Kit (DEV372) - AWS re:I...Infrastructure Is Code with the AWS Cloud Development Kit (DEV372) - AWS re:I...
Infrastructure Is Code with the AWS Cloud Development Kit (DEV372) - AWS re:I...
Amazon Web Services
 
AWS Meetup Fort Lauderdale Re:invent Recap
AWS Meetup Fort Lauderdale Re:invent RecapAWS Meetup Fort Lauderdale Re:invent Recap
AWS Meetup Fort Lauderdale Re:invent Recap
Anthony Palmer
 
re:Invent Recap-AWSMeetup
re:Invent Recap-AWSMeetupre:Invent Recap-AWSMeetup
re:Invent Recap-AWSMeetup
CloudHesive
 
Continuous Integration and Deployment Best Practices on AWS (ARC307) | AWS re...
Continuous Integration and Deployment Best Practices on AWS (ARC307) | AWS re...Continuous Integration and Deployment Best Practices on AWS (ARC307) | AWS re...
Continuous Integration and Deployment Best Practices on AWS (ARC307) | AWS re...
Amazon Web Services
 
Continuous Deployment @ AWS Re:Invent
Continuous Deployment @ AWS Re:InventContinuous Deployment @ AWS Re:Invent
Continuous Deployment @ AWS Re:Invent
John Schneider
 
Cluster-as-code. The Many Ways towards Kubernetes
Cluster-as-code. The Many Ways towards KubernetesCluster-as-code. The Many Ways towards Kubernetes
Cluster-as-code. The Many Ways towards Kubernetes
QAware GmbH
 
AWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for GovernmentAWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for Government
Amazon Web Services
 
Secure Configuration and Automation Overview
Secure Configuration and Automation OverviewSecure Configuration and Automation Overview
Secure Configuration and Automation Overview
Amazon Web Services
 
Increase Speed and Agility with Amazon Web Services
Increase Speed and Agility with Amazon Web ServicesIncrease Speed and Agility with Amazon Web Services
Increase Speed and Agility with Amazon Web Services
Amazon Web Services
 
Increase Speed and Agility with Amazon Web Services
Increase Speed and Agility with Amazon Web ServicesIncrease Speed and Agility with Amazon Web Services
Increase Speed and Agility with Amazon Web Services
Amazon Web Services
 
CI/CD on AWS: Deploy Everything All the Time | AWS Public Sector Summit 2016
CI/CD on AWS: Deploy Everything All the Time | AWS Public Sector Summit 2016CI/CD on AWS: Deploy Everything All the Time | AWS Public Sector Summit 2016
CI/CD on AWS: Deploy Everything All the Time | AWS Public Sector Summit 2016
Amazon Web Services
 
Serverless Computing, AWS Way by SourceFuse Technologies
Serverless Computing, AWS Way by SourceFuse Technologies Serverless Computing, AWS Way by SourceFuse Technologies
Serverless Computing, AWS Way by SourceFuse Technologies
SourceFuse
 
Serverless Computing, AWS Way: SourceFuse Technologies
Serverless Computing, AWS Way: SourceFuse Technologies Serverless Computing, AWS Way: SourceFuse Technologies
Serverless Computing, AWS Way: SourceFuse Technologies
Sarmishtha Sinha
 
Cluster-as-code. The Many Ways towards Kubernetes
Cluster-as-code. The Many Ways towards KubernetesCluster-as-code. The Many Ways towards Kubernetes
Cluster-as-code. The Many Ways towards Kubernetes
QAware GmbH
 
Track 4 Session 4_ MAD02 MAD 04 如何藉由 CICD 流程管理容器化和無伺服器應用
Track 4 Session 4_ MAD02 MAD 04 如何藉由 CICD 流程管理容器化和無伺服器應用Track 4 Session 4_ MAD02 MAD 04 如何藉由 CICD 流程管理容器化和無伺服器應用
Track 4 Session 4_ MAD02 MAD 04 如何藉由 CICD 流程管理容器化和無伺服器應用Amazon Web Services
 
CI&CD on AWS - Meetup Roma Oct 2016
CI&CD on AWS - Meetup Roma Oct 2016CI&CD on AWS - Meetup Roma Oct 2016
CI&CD on AWS - Meetup Roma Oct 2016
Paolo latella
 
Serverless cat detector workshop - cloudyna 2017 (16.12.2017)
Serverless cat detector   workshop - cloudyna 2017 (16.12.2017)Serverless cat detector   workshop - cloudyna 2017 (16.12.2017)
Serverless cat detector workshop - cloudyna 2017 (16.12.2017)
Paweł Pikuła
 
DevOps, Microservices and Serverless Architecture
DevOps, Microservices and Serverless ArchitectureDevOps, Microservices and Serverless Architecture
DevOps, Microservices and Serverless Architecture
Mikhail Prudnikov
 
re:Invent Recap: Security Week at the SF Loft
re:Invent Recap: Security Week at the SF Loftre:Invent Recap: Security Week at the SF Loft
re:Invent Recap: Security Week at the SF Loft
Amazon Web Services
 
How to implement DevSecOps on AWS for startups
How to implement DevSecOps on AWS for startupsHow to implement DevSecOps on AWS for startups
How to implement DevSecOps on AWS for startups
Aleksandr Maklakov
 

Similar to Developer Experience at the Guardian, Equal Experts Sept 2021 (20)

Infrastructure Is Code with the AWS Cloud Development Kit (DEV372) - AWS re:I...
Infrastructure Is Code with the AWS Cloud Development Kit (DEV372) - AWS re:I...Infrastructure Is Code with the AWS Cloud Development Kit (DEV372) - AWS re:I...
Infrastructure Is Code with the AWS Cloud Development Kit (DEV372) - AWS re:I...
 
AWS Meetup Fort Lauderdale Re:invent Recap
AWS Meetup Fort Lauderdale Re:invent RecapAWS Meetup Fort Lauderdale Re:invent Recap
AWS Meetup Fort Lauderdale Re:invent Recap
 
re:Invent Recap-AWSMeetup
re:Invent Recap-AWSMeetupre:Invent Recap-AWSMeetup
re:Invent Recap-AWSMeetup
 
Continuous Integration and Deployment Best Practices on AWS (ARC307) | AWS re...
Continuous Integration and Deployment Best Practices on AWS (ARC307) | AWS re...Continuous Integration and Deployment Best Practices on AWS (ARC307) | AWS re...
Continuous Integration and Deployment Best Practices on AWS (ARC307) | AWS re...
 
Continuous Deployment @ AWS Re:Invent
Continuous Deployment @ AWS Re:InventContinuous Deployment @ AWS Re:Invent
Continuous Deployment @ AWS Re:Invent
 
Cluster-as-code. The Many Ways towards Kubernetes
Cluster-as-code. The Many Ways towards KubernetesCluster-as-code. The Many Ways towards Kubernetes
Cluster-as-code. The Many Ways towards Kubernetes
 
AWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for GovernmentAWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for Government
 
Secure Configuration and Automation Overview
Secure Configuration and Automation OverviewSecure Configuration and Automation Overview
Secure Configuration and Automation Overview
 
Increase Speed and Agility with Amazon Web Services
Increase Speed and Agility with Amazon Web ServicesIncrease Speed and Agility with Amazon Web Services
Increase Speed and Agility with Amazon Web Services
 
Increase Speed and Agility with Amazon Web Services
Increase Speed and Agility with Amazon Web ServicesIncrease Speed and Agility with Amazon Web Services
Increase Speed and Agility with Amazon Web Services
 
CI/CD on AWS: Deploy Everything All the Time | AWS Public Sector Summit 2016
CI/CD on AWS: Deploy Everything All the Time | AWS Public Sector Summit 2016CI/CD on AWS: Deploy Everything All the Time | AWS Public Sector Summit 2016
CI/CD on AWS: Deploy Everything All the Time | AWS Public Sector Summit 2016
 
Serverless Computing, AWS Way by SourceFuse Technologies
Serverless Computing, AWS Way by SourceFuse Technologies Serverless Computing, AWS Way by SourceFuse Technologies
Serverless Computing, AWS Way by SourceFuse Technologies
 
Serverless Computing, AWS Way: SourceFuse Technologies
Serverless Computing, AWS Way: SourceFuse Technologies Serverless Computing, AWS Way: SourceFuse Technologies
Serverless Computing, AWS Way: SourceFuse Technologies
 
Cluster-as-code. The Many Ways towards Kubernetes
Cluster-as-code. The Many Ways towards KubernetesCluster-as-code. The Many Ways towards Kubernetes
Cluster-as-code. The Many Ways towards Kubernetes
 
Track 4 Session 4_ MAD02 MAD 04 如何藉由 CICD 流程管理容器化和無伺服器應用
Track 4 Session 4_ MAD02 MAD 04 如何藉由 CICD 流程管理容器化和無伺服器應用Track 4 Session 4_ MAD02 MAD 04 如何藉由 CICD 流程管理容器化和無伺服器應用
Track 4 Session 4_ MAD02 MAD 04 如何藉由 CICD 流程管理容器化和無伺服器應用
 
CI&CD on AWS - Meetup Roma Oct 2016
CI&CD on AWS - Meetup Roma Oct 2016CI&CD on AWS - Meetup Roma Oct 2016
CI&CD on AWS - Meetup Roma Oct 2016
 
Serverless cat detector workshop - cloudyna 2017 (16.12.2017)
Serverless cat detector   workshop - cloudyna 2017 (16.12.2017)Serverless cat detector   workshop - cloudyna 2017 (16.12.2017)
Serverless cat detector workshop - cloudyna 2017 (16.12.2017)
 
DevOps, Microservices and Serverless Architecture
DevOps, Microservices and Serverless ArchitectureDevOps, Microservices and Serverless Architecture
DevOps, Microservices and Serverless Architecture
 
re:Invent Recap: Security Week at the SF Loft
re:Invent Recap: Security Week at the SF Loftre:Invent Recap: Security Week at the SF Loft
re:Invent Recap: Security Week at the SF Loft
 
How to implement DevSecOps on AWS for startups
How to implement DevSecOps on AWS for startupsHow to implement DevSecOps on AWS for startups
How to implement DevSecOps on AWS for startups
 

Recently uploaded

Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"
Donna Lenk
 
top nidhi software solution freedownload
top nidhi software solution freedownloadtop nidhi software solution freedownload
top nidhi software solution freedownload
vrstrong314
 
2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx
Georgi Kodinov
 
SOCRadar Research Team: Latest Activities of IntelBroker
SOCRadar Research Team: Latest Activities of IntelBrokerSOCRadar Research Team: Latest Activities of IntelBroker
SOCRadar Research Team: Latest Activities of IntelBroker
SOCRadar
 
Vitthal Shirke Microservices Resume Montevideo
Vitthal Shirke Microservices Resume MontevideoVitthal Shirke Microservices Resume Montevideo
Vitthal Shirke Microservices Resume Montevideo
Vitthal Shirke
 
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdf
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdfEnhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdf
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdf
Jay Das
 
First Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User EndpointsFirst Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User Endpoints
Globus
 
Quarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden ExtensionsQuarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden Extensions
Max Andersen
 
A Comprehensive Look at Generative AI in Retail App Testing.pdf
A Comprehensive Look at Generative AI in Retail App Testing.pdfA Comprehensive Look at Generative AI in Retail App Testing.pdf
A Comprehensive Look at Generative AI in Retail App Testing.pdf
kalichargn70th171
 
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Globus
 
Cyaniclab : Software Development Agency Portfolio.pdf
Cyaniclab : Software Development Agency Portfolio.pdfCyaniclab : Software Development Agency Portfolio.pdf
Cyaniclab : Software Development Agency Portfolio.pdf
Cyanic lab
 
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Globus
 
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.ILBeyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Natan Silnitsky
 
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
Juraj Vysvader
 
Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024
Paco van Beckhoven
 
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...
Anthony Dahanne
 
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
AI Pilot Review: The World’s First Virtual Assistant Marketing SuiteAI Pilot Review: The World’s First Virtual Assistant Marketing Suite
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
Google
 
Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus
 
Prosigns: Transforming Business with Tailored Technology Solutions
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns: Transforming Business with Tailored Technology Solutions
Prosigns: Transforming Business with Tailored Technology Solutions
Prosigns
 
Accelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with PlatformlessAccelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with Platformless
WSO2
 

Recently uploaded (20)

Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"
 
top nidhi software solution freedownload
top nidhi software solution freedownloadtop nidhi software solution freedownload
top nidhi software solution freedownload
 
2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx
 
SOCRadar Research Team: Latest Activities of IntelBroker
SOCRadar Research Team: Latest Activities of IntelBrokerSOCRadar Research Team: Latest Activities of IntelBroker
SOCRadar Research Team: Latest Activities of IntelBroker
 
Vitthal Shirke Microservices Resume Montevideo
Vitthal Shirke Microservices Resume MontevideoVitthal Shirke Microservices Resume Montevideo
Vitthal Shirke Microservices Resume Montevideo
 
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdf
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdfEnhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdf
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdf
 
First Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User EndpointsFirst Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User Endpoints
 
Quarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden ExtensionsQuarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden Extensions
 
A Comprehensive Look at Generative AI in Retail App Testing.pdf
A Comprehensive Look at Generative AI in Retail App Testing.pdfA Comprehensive Look at Generative AI in Retail App Testing.pdf
A Comprehensive Look at Generative AI in Retail App Testing.pdf
 
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
 
Cyaniclab : Software Development Agency Portfolio.pdf
Cyaniclab : Software Development Agency Portfolio.pdfCyaniclab : Software Development Agency Portfolio.pdf
Cyaniclab : Software Development Agency Portfolio.pdf
 
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
 
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.ILBeyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
 
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
 
Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024
 
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...
 
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
AI Pilot Review: The World’s First Virtual Assistant Marketing SuiteAI Pilot Review: The World’s First Virtual Assistant Marketing Suite
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
 
Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024
 
Prosigns: Transforming Business with Tailored Technology Solutions
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns: Transforming Business with Tailored Technology Solutions
Prosigns: Transforming Business with Tailored Technology Solutions
 
Accelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with PlatformlessAccelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with Platformless
 

Developer Experience at the Guardian, Equal Experts Sept 2021

  • 1. Developer Experience at The Guardian Equal Experts, Sept 2021
  • 3. What does the DevX team do?
  • 4. A brief history 2015 Solely on AWS (no more on premises servers) 2012 Move to AWS started (mix of on premises and AWS) May 1821 First Manchester Guardian newspaper published 1999 Guardian Unlimited network of websites launched aws.amazon.com/solutions/case-studies/guardian
  • 5. Autonomous teams ● Dotcom team, building theguardian.com ● Apps teams, building the iOS and Android apps ● Reader Revenue teams, building supporter and contribution platforms ● Journalism teams, building internal tools for Editorial staff
  • 6. News website of the year theguardian.com/p/t6qvf
  • 8. (some of) technologies used by the Engineering dept.
  • 9. July 2020 Full time DevX team formed A brief history (continued) 2015 Solely on AWS (no more on premises servers) 2012 Move to AWS started (mix of on premises and AWS) May 1821 First Manchester Guardian newspaper published 1999 Guardian Unlimited network of websites launched aws.amazon.com/solutions/case-studies/guardian
  • 10. What does the DevX team do?
  • 11. 3 areas of focus 1. Client side tools and infrastructure 2. Security tools and infrastructure 3. Operation tools and infrastructure
  • 12. Mission Enable teams to focus on delivering value at lightning speed by streamlining infrastructure management
  • 14. Our DevX Tools ● AMIable - Amazon Machine Image (AMI) management tool ● AMIgo - An AMI bakery ● Anghammarad - Centralised notifications service ● CDK - Generic Guardian flavoured AWS CDK components ● Central ELK - Centralised logging platform ● dev-nginx - Tools to configure a local development nginx to proxy our applications and services ● DNS Validation - An AWS lambda that automatically creates ACM validation DNS records ● Grafana - Centralised metrics service ● Gu:who - answering: who are all these users in my GitHub org? ● Janus - Google-backed AWS account access ● master-to-main - A CLI to automate updating a GitHub repository master branch to main ● Prism - Tool for collecting live data about infrastructure so it can be easily queried by users and automated tooling ● PRout - Looks after your pull requests, tells you when they're live ● Repo Apocalypse - Archive old Github projects to S3 ● RiffRaff - The Guardian's deployment platform ● Security HQ - Centralised security information for AWS accounts ● Source - The Guardian’s design system ● SSM Scala - ssh replacement: CLI program that wraps SSM's EC2 Run Command ● Strap - Bootstrap your macOS development system ● ...
  • 15. Our DevX Tools ● AMIable - Amazon Machine Image (AMI) management tool ● AMIgo - An AMI bakery ● Anghammarad - Centralised notifications service ● CDK - Generic Guardian flavoured AWS CDK components ● Central ELK - Centralised logging platform ● dev-nginx - Tools to configure a local development nginx to proxy our applications and services ● DNS Validation - An AWS lambda that automatically creates ACM validation DNS records ● Grafana - Centralised metrics service ● Gu:who - answering: who are all these users in my GitHub org? ● Janus - Google-backed AWS account access ● master-to-main - A CLI to automate updating a GitHub repository master branch to main ● Prism - Tool for collecting live data about infrastructure so it can be easily queried by users and automated tooling ● PRout - Looks after your pull requests, tells you when they're live ● Repo Apocalypse - Archive old Github projects to S3 ● RiffRaff - The Guardian's deployment platform ● Security HQ - Centralised security information for AWS accounts ● Source - The Guardian’s design system ● SSM Scala - ssh replacement: CLI program that wraps SSM's EC2 Run Command ● Strap - Bootstrap your macOS development system ● ... 😱😱😱😱 😱
  • 16. Janus When making API calls to AWS locally, AWS credentials are required. Developers do not have any IAM users for our AWS accounts, instead we use temporary credentials or short-term, federated sessions. Janus is our federated login system for managing access to our myriad AWS accounts. This approach allows us to make our infrastructure easier and more secure to use. Google-backed AWS account access github.com/guardian/janus-app
  • 18. SSM Scala A command line tool that wraps AWS Systems Manager (SSM). Used to execute commands on EC2 servers using EC2 Run command. It provides the user with: ● an alternative to SSH for running commands ● standard SSH access using short lived RSA keys We have removed direct SSH access (port 22) from applications and replaced it with SSM Scala. github.com/guardian/ssm-scala
  • 19. Riff-Raff Our applications primarily run on EC2 or Lambda. Riff-Raff is our deployment tool and it allows us to use AWS auto-scaling to achieve zero downtime deploys. Riff-Raff also records the deployment history of each project, and can be used to schedule deploys. The Guardian's deployment platform github.com/guardian/riff-raff
  • 22. Source Source is our Design System. Written in React and Emotion, it creates a consistent user and developer experience across *.theguardian.com and the apps. It's expanding to support internal tooling too. The Guardian's design system github.com/guardian/source
  • 23. Prism Prism captures information about the infrastructure across our entire estate in near real-time. There is also a command line tool that uses the Prism API to locate infrastructure. github.com/guardian/prism
  • 25. Areas of focus For the Operations team ● Observability ● Runtime environment ● Configuration ● Deployment
  • 27. Let’s talk about CloudFormation... ✅ Tracked in VCS ✅ Repeatable ❌ JSON / YAML ❌ JSON can become verbose ❌ YAML is sensitive to whitespace ❌ Long feedback loop ❌ Errors first seen at runtime (UPDATE_ROLLBACK_COMPLETE 😈) ❌ Can result in not following the principles of least privilege ❌ Encourages copy pasting ❌ Consistency? ❌ Following best practice?
  • 28. Guardian CDK ✅ Tracked in VCS ✅ Repeatable ✅ Written in a strongly typed language (TypeScript) ✅ Built using AWS CDK ✅ Unit tested ✅ Encodes today’s best practices ✅ Observability ✅ Runtime environment ✅ Configuration ✅ Deployment ✅ Follows the principles of least privilege ✅ npm install @guardian/cdk@latest github.com/guardian/cdk
  • 30. Guardian CDK ● A set of Guardian flavoured AWS CDK constructs and patterns, encoding today’s best practice on: ○ Observability ○ Runtime environment ○ Configuration ○ Deployment ● CloudFormation in TypeScript ● Tracked in VCS ● Improved consistency ● Not YAML or JSON 🎉 ○ Strongly typed ○ Unit tested github.com/guardian/cdk
  • 31. What does the DevX team do?
  • 32. A lot of our backend tooling revolves around simplifying AWS resource management across multiple accounts. Observation
  • 33. Vision We will collaboratively create and maintain best in class standard patterns for deployment, configuration, runtime and observability. Creating and maintaining apps that follow these patterns will be quick, logical, free of boilerplate and secure by default. We will have worked with teams to migrate the majority of applications to our standard patterns. Teams will be focussed on delivering business value at high velocity as infrastructure concerns have been streamlined.