Download to read offline
](https://image.slidesharecdn.com/radicalarchitecturalchange-atlascamp2015-150625194235-lva1-app6891/75/AtlasCamp-2015-How-to-deliver-radical-architectural-change-without-the-customer-noticing-29-2048.jpg)
](https://image.slidesharecdn.com/radicalarchitecturalchange-atlascamp2015-150625194235-lva1-app6891/75/AtlasCamp-2015-How-to-deliver-radical-architectural-change-without-the-customer-noticing-30-2048.jpg)
Uploaded byAtlassian
AtlasCamp 2015: How to deliver radical architectural change without the customer noticing
1. Atlassian delivered radical architectural changes by migrating 60,000 virtual Linux containers and 120,000 application instances to a microservices architecture on their platform without major incidents. 2. They achieved this through careful planning including gradual rollouts, shadowing new services, and making data migrations idempotent. 3. End-to-end ownership of the changes by dedicated teams also helped ensure flexibility and minimized incidents during the migration.
More Related Content
Similar to AtlasCamp 2015: How to deliver radical architectural change without the customer noticing
AtlasCamp 2015: How to deliver radical architectural change without the customer noticing
- 1. How to Deliver RadicalArchitectural Change SIDNEY SHEK • ARCHITECT • ATLASSIAN • @SIDNEYSHEK …without the customer noticing! ROBIN FERNANDES • TEAM LEAD • ATLASSIAN • @REWBS
- 2. Source: BMW S1000RR. Dinner for RR. www.youtube.com/watch?v=-cM9S2AzU28
- 3. 22TB attachments moved 65mobjects Zero major incidents Done during business hours
- 4. 60,000 +Virtualised Linuxcontainers
- 5.
- 7. A solid platformfor all BlobStore
- 8.
- 9.
- 10.
- 11.
- 12. Logging and metrics Seamless deployment Resource management Monitoring and alerts + service descriptor .json}{service artifact { ... scaling: { min: 5 } loadBalancer: { timeout: 600 } ... } Heavy lifting by Atlassian PaaS
- 13.
- 15.
- 16. Stored at b10b (SHA-1of content) PUT att-1 BlobStore server S3 bucket Uploading to BlobStore
- 17. Stored at b10b (SHA-1of content) PUT att-1 ETAG b10b BlobStore server S3 bucket Uploading to BlobStore
- 18. Dynamo table Stored atb10b (SHA-1 of content) PUT att-1 ETAG b10b BlobStore server S3 bucket Logical Key Sequence Operation att-1 1 Insert b10b Uploading to BlobStore
- 19. Dynamo table Stored atb10b (SHA-1 of content) BlobStore server S3 bucket Logical Key Sequence Operation att-1 1 Insert b10b Deleting from BlobStore DELETE att-1 ETAG b10b
- 20. Dynamo table Stored atb10b (SHA-1 of content) BlobStore server S3 bucket Logical Key Sequence Operation att-1 1 Insert b10b att-1 2 Delete Deleting from BlobStore DELETE att-1 ETAG b10b 204 “OK”
- 21. Dynamo table Stored atb10b (SHA-1 of content) BlobStore server S3 bucket Logical Key Sequence Operation att-1 1 Insert b10b att-1 2 Delete Deleting from BlobStore DELETE att-1 ETAG b10b 204 “OK”
- 22. Dynamo table Stored atb10b (SHA-1 of content) BlobStore server S3 bucket Logical Key Sequence Operation att-1 1 Insert b10b att-1 2 Delete att-1 3 Insert b10b Restoring in BlobStore DELETE att-1 ETAG b10b
- 23.
- 24.
- 25. Use proper types. Foreverything.
- 26.
- 27. ‘Example’ based unittest = 1 manually written test case Property-based test = 100s auto-generated examples
- 28. Advanced types to simplifytesting of non-blocking code
- 29. Production code runsasynchronously new S3Storage[Future](bucketName)
- 30. Test code runssynchronously new S3Storage[Id](bucketName)
- 31.
- 32. Merge to master inStash
- 33. Merge to master inStash
- 34. Merge to master inStash Automated staged deployments to prod < 2 hours
- 35.
- 36.
- 37.
- 38. Gradually Increase theCriticality of the New Service Disk BlobStore 1 Disk BlobStore 2 Disk BlobStore 3 JIRA JIRA JIRA
- 39. Roll out toCohorts
- 40. Rolling Out Progressively DataCentre 1 AWS Customer instance in OpenVZ container (approx. 1500 per rack) Rack 1 … Rack N … … … Data Centre 2 Rack 1 … Rack N … … … Data Centre 3 Rack 1 … Rack N … … …
- 41. Step 1: AManageable Number of Instances Data Centre 1 AWS Rack 1 … Rack N … … … Data Centre 2 Rack 1 … Rack N … … … Data Centre 3 Rack 1 … Rack N … … …
- 42. Step 2: FullRacks Data Centre 1 AWS Rack 1 … Rack N … … … Data Centre 2 Rack 1 … Rack N … … … Data Centre 3 Rack 1 … Rack N … … …… …
- 43. Step 3: AllInstances Data Centre 1 AWS Rack 1 … Rack N… Data Centre 2 Rack 1 Rack N… Data Centre 3 Rack 1 Rack N… … … … … … … … …
- 44. Repeat For NextMode! Data Centre 1 AWS Rack 1 … Rack N… Data Centre 2 Rack 1 Rack N… Data Centre 3 Rack 1 Rack N… … … … … … … … …
- 45. Repeat For NextMode! Data Centre 1 AWS Rack 1 … Rack N… Data Centre 2 Rack 1 Rack N… Data Centre 3 Rack 1 Rack N… … … … … … … … …… …
- 46. Repeat For NextMode! Data Centre 1 AWS Rack 1 … Rack N… Data Centre 2 Rack 1 Rack N… Data Centre 3 Rack 1 Rack N… … …… ……………… …
- 47.
- 48.
- 49. Make your datamigration idempotent.
- 50.
- 51. Micros (PaaS) DevOps, AtlassianStyle One team owns the change, end-to-end. JIRA JIRA Attachment Manager BlobStore Plugin BlobStore Client BlobStore Server BlobStore Migration Plugin Support Scripts Monitoring Tooling Roll-out Tooling Roll-out Process Consumer Service Operational Tooling Processes Monitoring Process Incident Response Owned by the BlobStore team Data Deletion
- 52. 1. A microservicesarchitecture requires investment in infrastructure. 2. Safety needs to be built into the architecture and software from the onset 3. Plan for gradual roll-outs & data migrations 4. End-to-end ownership means architectural flexlibility and fewer / less severe incidents 5. Don’t underestimate the integration points – technical and people. Key Takeaways
- 53. aws-scala lib: http://bitbucket.org/atlassian/aws-scala event sourcinglib: http://bitbucket.org/atlassianlabs/eventsrc
- 54. How to deliverradical architectural change without the customer noticing Submit your feedback: go.atlassian.com/acarchitectural