Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Architecting for Continuous DeliveryRuss Barnett, Chief ArchitectJohn Esser, Director of Engineering ProductivityAncestry....
InfoQ.com: News & Community Site• 750,000 unique visitors/month• Published in 4 languages (English, Chinese, Japanese and ...
Presented at QCon San Franciscowww.qconsf.comPurpose of QCon- to empower software development by facilitating the spread o...
Background2
Growth at Ancestry.com• Subscribers have doubled in the past 3 years– (~1M to > 2M)• Page views have doubled in the past 3...
Continuous Deliveryis consistently and reliablyreleasing business value incrementsfastthrough automated build, test, confi...
Evolution to Continuous DeliveryAgileBoot Up(Scrum)EnterpriseAgileFrameworkAgileArchitectureStandardsContinuousDeliveryAdo...
Typical Impediments to Continuous Delivery• Cultural• Technical practices• Quality ownership• Infrastructure• Architectural
Limiting Factors• Pipeline serialized at integration– Errors that occurred in this stagestalled the pipeline• Stalls in in...
Everything was coupled!(Aka, large batch size)(It became known as the “big blob!”)8
9
Little’s Law𝑊𝑎𝑖𝑡 𝑇𝑖𝑚𝑒 =𝑄𝑢𝑒𝑢𝑒 𝑠𝑖𝑧𝑒𝑃𝑟𝑜𝑐𝑒𝑠𝑠𝑖𝑛𝑔 𝑅𝑎𝑡𝑒𝑄𝑢𝑒𝑢𝑒 𝑠𝑖𝑧𝑒 ∝ 𝐵𝑎𝑡𝑐ℎ 𝑠𝑖𝑧𝑒We can reduce wait time (cycle time)by reducing bat...
Problems with Large Batches• Increases cycle time• Increases variability non-linearly as 2n• Increases risk• Reduces effic...
Answer?Utilize small batches.12
Fluidity PrincipleLoose couplingbetween product systemsenables small batches“Once a product developer realizes that small ...
Fluidity PrincipleLoose couplingbetween product systemsenables small batches“Once a product developer realizes that small ...
Creating an Architecture for Agility15
Architectural Impediments• Cross-Component Coupling– Creates groups of systems that must be deployed together• Insufficien...
Architectural Methods for Removing Impediments• Partition into small single-responsibility components“There should only be...
Backward and Forward CompatibilityClientVersion 1ServiceVersion 1ServiceVersion 2ClientVersion 2DeploymentRollbackDeployme...
Enforcing Decoupled Components• Implementing standards is insufficient– Independent deployment forces some decoupling– Hig...
Improving Interface Verification20ServerClientIn ProcessClient/ServerServerClient• Remember when you could run yourentire ...
Interface Verification using Proxies and Stubs21StubProxyClientServerClient/ServerwithProxy/Stub• Verifies interface at co...
Managing a Complex Network of Services• Ancestry Scale– About 40 different teams– Over 300 separate application or service...
Ancestry Deep Status Check23Service1ServiceAServiceCServiceBApp 2Service2GetStatusMonitor App 1UnidentifiedClientUnregiste...
< 2 s95%< 4 sfail99.9%SLAService Level Agreements• Understand Business Expectations– Each application and service establis...
Verify Client IdentityCheck Circuit BreakerTrack Each Incoming RequestReport ComplianceIncoming RequestsCheck Circuit Brea...
Conclusion• Architecture affects agility and continuous deliverycapability as much or more than other factors.• Process an...
• Questions?Contact info:jesser@ancestry.com.rbarnett@ancestry.com.Ancestry is hiring in San Francisco and Utah27
Upcoming SlideShare
Loading in …5
×

Architecting for Continuous Delivery

818 views

Published on

Video and slides synchronized, mp3 and slide download available at http://bit.ly/18a23ES.

John Esser and Russell Barnett discuss Ancestry.com’s SOA implementation capable of supporting continuous delivery, architectural standards used, and how continuous delivery works for them. Filmed at qconsf.com.

John Esser is currently the Director of Engineering Productivity and Agile Development at Ancestry.com. He is the architect of Ancestry’s transformation to Agile development and continuous delivery. Russell Barnett has worked as Chief Architect for Ancestry.com for the last ten years. His passion is for building highly scalable, available, and agile architectures.

Published in: Technology
  • Be the first to comment

Architecting for Continuous Delivery

  1. 1. Architecting for Continuous DeliveryRuss Barnett, Chief ArchitectJohn Esser, Director of Engineering ProductivityAncestry.com
  2. 2. InfoQ.com: News & Community Site• 750,000 unique visitors/month• Published in 4 languages (English, Chinese, Japanese and BrazilianPortuguese)• Post content from our QCon conferences• News 15-20 / week• Articles 3-4 / week• Presentations (videos) 12-15 / week• Interviews 2-3 / week• Books 1 / monthWatch the video with slidesynchronization on InfoQ.com!http://www.infoq.com/presentations/ancestry-SOA-continuous-delivery
  3. 3. Presented at QCon San Franciscowww.qconsf.comPurpose of QCon- to empower software development by facilitating the spread ofknowledge and innovationStrategy- practitioner-driven conference designed for YOU: influencers ofchange and innovation in your teams- speakers and topics driving the evolution and innovation- connecting and catalyzing the influencers and innovatorsHighlights- attended by more than 12,000 delegates since 2007- held in 9 cities worldwide
  4. 4. Background2
  5. 5. Growth at Ancestry.com• Subscribers have doubled in the past 3 years– (~1M to > 2M)• Page views have doubled in the past 3 years– (~25M/day to ~50M/day)• Development head count has tripled– (100 to 300)• Feature throughput has dramatically increased3
  6. 6. Continuous Deliveryis consistently and reliablyreleasing business value incrementsfastthrough automated build, test, configuration anddeployment.What is the value of going fast?A LOT!
  7. 7. Evolution to Continuous DeliveryAgileBoot Up(Scrum)EnterpriseAgileFrameworkAgileArchitectureStandardsContinuousDeliveryAdoptionFuture –“Agile v2”Two year period (2010 – present)
  8. 8. Typical Impediments to Continuous Delivery• Cultural• Technical practices• Quality ownership• Infrastructure• Architectural
  9. 9. Limiting Factors• Pipeline serialized at integration– Errors that occurred in this stagestalled the pipeline• Stalls in integration inducedadditional problems• Increasing frequency of stalls– As number of developmentteams grew, frequency of stallsincreased7IntegrationSource ControlBuildUnit TestingSystem TestingDeploymentTeam A Team B Team Cerrorcausesstall
  10. 10. Everything was coupled!(Aka, large batch size)(It became known as the “big blob!”)8
  11. 11. 9
  12. 12. Little’s Law𝑊𝑎𝑖𝑡 𝑇𝑖𝑚𝑒 =𝑄𝑢𝑒𝑢𝑒 𝑠𝑖𝑧𝑒𝑃𝑟𝑜𝑐𝑒𝑠𝑠𝑖𝑛𝑔 𝑅𝑎𝑡𝑒𝑄𝑢𝑒𝑢𝑒 𝑠𝑖𝑧𝑒 ∝ 𝐵𝑎𝑡𝑐ℎ 𝑠𝑖𝑧𝑒We can reduce wait time (cycle time)by reducing batch sizewithout changing demand or capacity.10
  13. 13. Problems with Large Batches• Increases cycle time• Increases variability non-linearly as 2n• Increases risk• Reduces efficiency• Limited by its worst element.from Principles of Product Development Flow, Don Reinertsen11
  14. 14. Answer?Utilize small batches.12
  15. 15. Fluidity PrincipleLoose couplingbetween product systemsenables small batches“Once a product developer realizes that small batches aredesirable, they start adopting product architectures that permitwork to flow in small, decoupled batches.”from Principles of Product Development Flow, Don Reinertsen13
  16. 16. Fluidity PrincipleLoose couplingbetween product systemsenables small batches“Once a product developer realizes that small batches aredesirable, they start adopting product architectures that permitwork to flow in small, decoupled batches.”from Principles of Product Development Flow, Don Reinertsen14
  17. 17. Creating an Architecture for Agility15
  18. 18. Architectural Impediments• Cross-Component Coupling– Creates groups of systems that must be deployed together• Insufficient Rollback Capability– Causes teams to resort to cascading rollback• Poor Testing and Monitoring– Requires a long testing period– Lengthens feedback cycle– Allows quality problems to escape to and affect customers16
  19. 19. Architectural Methods for Removing Impediments• Partition into small single-responsibility components“There should only be one reason for a [component] to change”- Robert Martin• Decouple deployment of components– Separately deploy components– Remove order dependent deployment• Support Independent Rollback– Enforce strict backward and forward compatibility17
  20. 20. Backward and Forward CompatibilityClientVersion 1ServiceVersion 1ServiceVersion 2ClientVersion 2DeploymentRollbackDeploymentRollbackBackwardCompatibilityBackwardCompatibilityForwardCompatibilityForwardCompatibility• Server Backward Compatibility– Newer servers work with clientswritten to old interface• Server Forward Compatibility– Existing servers work with clientswritten to newer interface– Supports early client deployment• Client Backward Compatibility– Newer clients work with serversthat implement old interface– Supports server rollback• Client Forward Compatibility– Old clients work with servers thatimplement new interface
  21. 21. Enforcing Decoupled Components• Implementing standards is insufficient– Independent deployment forces some decoupling– High rate of deployment issues indicate remaining coupling• Improve integration testing– Verify backward and forward compatibility– Identify breaking changes quickly– Make writing integration tests easier19
  22. 22. Improving Interface Verification20ServerClientIn ProcessClient/ServerServerClient• Remember when you could run yourentire application in one process?• How do we get better interfaceverification with services?In ProcessThriftSOAP/WSDLRESTVerifiableDecoupledGoogleProtocol Buffers
  23. 23. Interface Verification using Proxies and Stubs21StubProxyClientServerClient/ServerwithProxy/Stub• Verifies interface at compile time• Isolates code from versioning issues• Easier to provide mock implementations• Can test backward and forwardcompatibility
  24. 24. Managing a Complex Network of Services• Ancestry Scale– About 40 different teams– Over 300 separate application or service systems– Stack is 5+ levels deep• Historical Diagnostics– Presented a client centric or top down view– Insufficient for identifying problems in a network of services• Solution: Deep Status Check– Components provide dynamic status information for eachclient and dependency– Report traverses dependencies up to a given depth22
  25. 25. Ancestry Deep Status Check23Service1ServiceAServiceCServiceBApp 2Service2GetStatusMonitor App 1UnidentifiedClientUnregisteredClientConnection ErrorDatabaseFileSystemLevel 1Level 2Level 3Level 4
  26. 26. < 2 s95%< 4 sfail99.9%SLAService Level Agreements• Understand Business Expectations– Each application and service establishes a contract with eachclient specifying the expected performance characteristics24• Methodology– Use percentilebuckets rather thanan average– Performance is acomponent ofavailability
  27. 27. Verify Client IdentityCheck Circuit BreakerTrack Each Incoming RequestReport ComplianceIncoming RequestsCheck Circuit BreakerProvide Fallback MechanismTrack Each Outgoing RequestReport ComplianceOutgoing RequestsService Level Agreement System• Compare incoming and outgoing SLA complianceService
  28. 28. Conclusion• Architecture affects agility and continuous deliverycapability as much or more than other factors.• Process and tool improvements alone are insufficient.• Good architecture techniques enable effectivecontinuous delivery at large scale.– Partition to single-responsibility components.– Decouple deployment– Support independent rollback– Improve testing and monitoring infrastructure26
  29. 29. • Questions?Contact info:jesser@ancestry.com.rbarnett@ancestry.com.Ancestry is hiring in San Francisco and Utah27

×