Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Implementing Gannett's next-generation CMS with CQRS and event sourcing using Couchbase – Connect Silicon Valley 2017

414 views

Published on

Speaker: Justin Haugh, Principal Developer, Gannet Co., Inc.

When the limits of Gannett's existing content management platform started to become evident, a new solution was sought to better execute the mission of our content creators and editors. We needed to design a system that provided extremely fast responses to our front-end consumers, as well as a more responsive and informative content management system for our newsrooms. We elected to use Couchbase as our data storage solution. This is our journey.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Implementing Gannett's next-generation CMS with CQRS and event sourcing using Couchbase – Connect Silicon Valley 2017

  1. 1. Implementing Gannett's next generation CMS Using CQRS with Event Sourcing Supported by Couchbase
  2. 2. Who we are • We exist to get the right information, tools and guidance to people at the right time. • We serve communities of all kinds and the diverse individuals and businesses that form them. • Our media drives action, not passive consumption. It helps people and businesses make meaningful change.
  3. 3. Who we are • 108 news rooms across the country • ~3000 contributors • > 100 million unique visitors/month
  4. 4. Original Architecture
  5. 5. How we query for a site layout • 100+ stored procs • Some 400+ lines • Business logic • Not incld server side
  6. 6. Challenges • Slow response times for all consumers • Slow time to publish content – critical in our domain • Heavy caching for front end consumption (~16 minutes) • On-prem pet servers – difficulty scaling​ – protected by DBAs and Release Managers • Replication issues with SQL Server​ • Business logic everywhere • Lack of feedback from transactions
  7. 7. Goals • Migrate a legacy system to the cloud • Decrease publishing latency • Increase performance, reliability • Decrease cost • Consolidate and organize business logic
  8. 8. Guiding Principles • Not everything should be crammed into a relational model • Write it as you want to read it • Separate reads and writes to scale independently (CQRS) • Domain Driven Design for modeling behaviors • Event Driven Design for implementing behaviors
  9. 9. Why Couchbase? • Read performance, optimize for more reads than writes • Fault Tolerance • N1QL • NodeJS integration • Ease of scaling
  10. 10. A word about performance • Minimum required ~500 writes and 3000 reads/second • Successful Load Test – 10,000 writes/second – 30,000 reads/second – spikes of up to 330,000 reads​/second – 8 r3.xlarge nodes (Amazon) – 32 vCPU – 172 GB Memory Allocated
  11. 11. Solution • Messaging pipeline (Integration) • Presentation environment (Query) • Authoring environment (Command)
  12. 12. Messaging Pipeline
  13. 13. Presentation System
  14. 14. Authoring System
  15. 15. No software survives contact with reality • Problems with N1QL eventual consistency • Turns out a lot happens to our entities
  16. 16. Authoring System Phase 2
  17. 17. Put it all together
  18. 18. Presentation Couchbase • 5 nodes • r3.2xlarge • 8 vCPUs • 61GB RAM • 160GB SSD • 234GB total cache size • 2.4TB total storage size
  19. 19. Presentation Couchbase Buckets • Other Buckets – 2k gets/sec – 50 sets/sec – 500,000 items – cache miss ratio: ~0 – active resident: 100% – value ejection • Assets – 100k get/sec sustained – >150k get/sec burst – 30 set/sec sustained – 300 set/sec burst – 200 million items – cache miss ratio: < 0.5 – active resident: 9% – value ejection
  20. 20. Presentation API Metrics • 90th percentile response times – Site layout gets: ~170ms – Search (various APIs): 90–350ms – Analytics based APIs: 400-800ms • Median – Site layout gets: ~55-65ms – Search (various APIs): ~15-50ms – Analytics based APIs: ~300-600ms • Published content is surfaced within seconds
  21. 21. Authoring Couchbase • 8 nodes • r3.2xlarge • 8 vCPUs • 61GB RAM • 160GB SSD • 400GB total cache size • 23.4TB total storage size
  22. 22. Authoring Couchbase Buckets • Snapshot Store – 2k gets/sec – 330 sets/sec – 150 million items – Cache miss ratio: <1 – active resident: 3% – value ejection • Event Store – 16k gets/sec – 70 sets/sec – 350 million items – cache miss ratio: ~2 – active resident: 6% – full ejection
  23. 23. Authoring API Metrics • 90th percentile response times – Commands: 176ms – Gets: 319ms – Search: 771ms • Median – Commands: 26ms – Gets: 64ms – Search: 390ms
  24. 24. Lessons Learned • Strong, effective leadership is required • For new, large projects: have a kickoff! • Long-term roadmaps • Silo teams are poison • CQRS and DDD – live and breath the domain
  25. 25. Lessons Learned • TESTING!!! – Unit test – Integration test • Module integration • Application integration • System integration – Load test – Stress test – UI module tests
  26. 26. Next Steps • Complete the rollout to all markets • Complete feature development • Decommission legacy system • Further tuning on Authoring Couchbase
  27. 27. Next Steps Continued • Next event store implementation • Multi-region Robust HA • End to end transaction tracking • Domain Driven Design is challenging • Containerization of stateless applications
  28. 28. To be continued…

×