This document discusses LinkedIn's approach to software development and continuous delivery. It describes how LinkedIn moved from a single code base to supporting diverse programming languages and frameworks as the company grew exponentially. LinkedIn developed a multiproduct approach using tools, services, and processes that are agnostic to the underlying technologies. This allows for pluggable implementations and continuous and automated delivery of code changes. The document outlines how LinkedIn delivers nearly 1,000 code changes per day through an automated pipeline that publishes changes typically within 20 minutes. It also discusses the importance of backwards compatible APIs, pushing feedback earlier, and incentivizing desired behaviors like craftsmanship.

1. Software Development and Continuous Delivery
Vmware August 25th, 2015

2. The LinkedIn Story
 There was a single, relatively homogenous code base
– Build from source, “spaghetti” dependency management
 Java, Spring, Ant, …
 JavaScript, HTML, JSPs, CSS, …
 JARs, WARs, Jetty, Tomcat, …

3. Then everything went exponential
Number of developers, programming languages, build systems,
frameworks, lines of code, servers, users, page views, …
… and pretty much everything else

4. Hadoop

5. LinkedIn Software Development
 We don’t ever want to be in the box
– Living on the bleeding edge, often defining it
– Technical experimentation and diversity encouraged
 Cheap experimentation
– Pace of iteration
– Continuous Delivery
 Automation: people are too slow
 Build over buy

6. Constructive Chaos

7. Our Answer: Multiproduct
 Tools, services, interfaces and processes that are architected for a
diverse technology stack
 Agnostic to version control, build system and programming
technology
 Abstracts common software lifecycle tasks
– For example: “build”, “test”, “release”
– Provide a default implementation, allow users to override

8. Key Concepts
 Elevate tooling from artifact to product level
 Metadata ties the tooling together
 Pluggable implementation of subsystems
 Version management
 Continuous and automated delivery

9. Source
Control
Build System Deployment

11. Continuous Delivery in Multiproduct
 Continuous Delivery
– Automatically deliver every code change to “production”
 Automated pipeline triggered on developer change
– No other developer action needed
 Processing around 1,000 code changes a day
– Median time to artifact publish: ~20 minutes
11

13. Keys to Continuous Delivery
 Backwards compatible APIs
– Strong API contracts, implementation can change
– Culture and Enforcement
– Continuous Integration
– https://www.linkedin.com/pulse/find-seams-jens-pillgram-larsen
 Push feedback earlier
– Feedback loops: post-mortems, root cause analysis, culture
– Developer can run same tests as CI (dev > qa > staging > prod)
 Incentivize desired behavior
– Breakage -> learning and improvement

15. Software Craftsmanship
 You know it when you see it
– Documentation, testing, code quality, design, architecture, APIs
– Maintainability, extensibility
 Is valued everywhere: production code, tests, build logic, process
 Is a feeling; an emotion
 Is necessary, but not sufficient
 https://www.linkedin.com/pulse/inside-software-jens-pillgram-larsen

16. Deployment
 Simple concept, hard problem
– Download an artifact and run it
 Artifact
– Ensures “once and once only” build, validation and release
– Consistency
– Speed
– Caching
 Separate code and configuration
 Dynamic provisioning

20. 12-24 Months
 Connect Development to Delivery
– Continuous Task Execution
– Intelligent Developer Platform
– Infinite Scalability
http://www.slideshare.net/jenspillgram/software-engineering-in-the-continuous-age