This document discusses LinkedIn's approach to software development and continuous delivery. It describes how LinkedIn moved from a single code base to supporting diverse programming languages and frameworks as the company grew exponentially. LinkedIn developed a multiproduct approach using tools, services, and processes that are agnostic to the underlying technologies. This allows for pluggable implementations and continuous and automated delivery of code changes. The document outlines how LinkedIn delivers nearly 1,000 code changes per day through an automated pipeline that publishes changes typically within 20 minutes. It also discusses the importance of backwards compatible APIs, pushing feedback earlier, and incentivizing desired behaviors like craftsmanship.
2. The LinkedIn Story
There was a single, relatively homogenous code base
– Build from source, “spaghetti” dependency management
Java, Spring, Ant, …
JavaScript, HTML, JSPs, CSS, …
JARs, WARs, Jetty, Tomcat, …
3. Then everything went exponential
Number of developers, programming languages, build systems,
frameworks, lines of code, servers, users, page views, …
… and pretty much everything else
5. LinkedIn Software Development
We don’t ever want to be in the box
– Living on the bleeding edge, often defining it
– Technical experimentation and diversity encouraged
Cheap experimentation
– Pace of iteration
– Continuous Delivery
Automation: people are too slow
Build over buy
7. Our Answer: Multiproduct
Tools, services, interfaces and processes that are architected for a
diverse technology stack
Agnostic to version control, build system and programming
technology
Abstracts common software lifecycle tasks
– For example: “build”, “test”, “release”
– Provide a default implementation, allow users to override
8. Key Concepts
Elevate tooling from artifact to product level
Metadata ties the tooling together
Pluggable implementation of subsystems
Version management
Continuous and automated delivery
11. Continuous Delivery in Multiproduct
Continuous Delivery
– Automatically deliver every code change to “production”
Automated pipeline triggered on developer change
– No other developer action needed
Processing around 1,000 code changes a day
– Median time to artifact publish: ~20 minutes
11
12.
13. Keys to Continuous Delivery
Backwards compatible APIs
– Strong API contracts, implementation can change
– Culture and Enforcement
– Continuous Integration
– https://www.linkedin.com/pulse/find-seams-jens-pillgram-larsen
Push feedback earlier
– Feedback loops: post-mortems, root cause analysis, culture
– Developer can run same tests as CI (dev > qa > staging > prod)
Incentivize desired behavior
– Breakage -> learning and improvement
14.
15. Software Craftsmanship
You know it when you see it
– Documentation, testing, code quality, design, architecture, APIs
– Maintainability, extensibility
Is valued everywhere: production code, tests, build logic, process
Is a feeling; an emotion
Is necessary, but not sufficient
https://www.linkedin.com/pulse/inside-software-jens-pillgram-larsen
16. Deployment
Simple concept, hard problem
– Download an artifact and run it
Artifact
– Ensures “once and once only” build, validation and release
– Consistency
– Speed
– Caching
Separate code and configuration
Dynamic provisioning
17.
18.
19.
20. 12-24 Months
Connect Development to Delivery
– Continuous Task Execution
– Intelligent Developer Platform
– Infinite Scalability
http://www.slideshare.net/jenspillgram/software-engineering-in-the-continuous-age
Editor's Notes
Homogenous -> Heterogenous
Build vs buy, starting taking on more tasks internally. Started open-sourcing our own stuff to pay it forward.
“out of the box” thinking should make you ask why the heck you were in the box in the first place
Automation requires trust in the system
But there must be basic sanity checks: our operations team must be able to configure and operate the products at scale, the organization must maintain focus and prioritize, have to consider compliance (legal, government, ethical)
There’s constructive and destructive chaos. Multiproduct tries to define constructive chaos.