CCB12 Concur's journey to 1 billion Couchbase operations per day

6,427 views

Published on

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,427
On SlideShare
0
From Embeds
0
Number of Embeds
4,351
Actions
Shares
0
Downloads
11
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

CCB12 Concur's journey to 1 billion Couchbase operations per day

  1. 1. Couchbase BerlinOct 2012Drew Garner
  2. 2. Who am I? Director of Service Architecture Been working with Concur for 8 years – in various roles (Network, Storage, Servers, Datacenter, Service Management and now Architecture) Previously worked in Network and Systems consulting and services My groups are focused on innovation within our Stack in partnership with Development Architecture – specifically around Scale, Performance, Availability improvements We are also responsible for Monitoring and tools strategy within the stack© 2011 Concur, all rights reserved. Concur is a registered trademark of Concur Technologies, Inc.
  3. 3. Areas of Focus Performance Availability Scalability Improve performance by 20% Prevent Outages and when they Work to reduce the cost of each year happen reduce impact time handling new transactions Platform Security/ Compliance Geographically Dispersed Hybrid Cloud Maintain current compliance; Extend Concur Trust Platform Extend hosting footprint Connecting new apps and with less cost to outside of North America partners through a common the business (through utilizing public and private controlled API method automation and integration) clouds© 2011 Concur, all rights reserved. Concur is a registered trademark of Concur Technologies, Inc.
  4. 4. Background Concur provides Cloud based Business Automation software (SaaS) – particularly around Expense, Travel, and Invoice areas Grown by acquisition and massive demand for our services – 6 in last 10 years Acquisitions have resulted in a blended Architecture stack Monthly software releases with thousands of changes and bug fixes committed to production every month Constant Culture of change/growth and reinvention © 2011 Concur, all rights reserved. Concur is a registered trademark of Concur Technologies, Inc.
  5. 5. Flight Visualization© 2011 Concur, all rights reserved. Concur is a registered trademark of Concur Technologies, Inc.
  6. 6. Typical busy Day for us • 166 Million CDN Get Requests External Traffic • 4,400 Pages Served per second at peak • 85 Million External Web Requests Web Requests • 41 Million Expense MT Requests • 6 Million GDS Web Service Requests • 1.1 Billion Couchbase Calls Data Requests • 3.4 Billion SQL Queries • 800 GB of Web Traffic Pushed Network Data • 15 TB of Internal DC Traffic© 2011 Concur, all rights reserved. Concur is a registered trademark of Concur Technologies, Inc.
  7. 7. Our Journey with Couchbase We started out by replacing a legacy Caching sub-system with Couchbase – memcached systems (started in August 2010) This was a massive success and resulted in huge performance, scale and cost gains Spent this last year focused on expanding our Caching strategy that has increased our traffic from 100 Million hits to 1.2 Billion hits per day Now we are moving into utilizing more Couchbase full features and functionality Loading data into Couchbase from our Hadoop cluster to provide instant access to the data to our application© 2011 Concur, all rights reserved. Concur is a registered trademark of Concur Technologies, Inc.
  8. 8. Before and After Moved from 613.57ms to 238.67ms a 61% improvement© 2011 Concur, all rights reserved. Concur is a registered trademark of Concur Technologies, Inc.
  9. 9. Lessons learned A disruptive system like Couchbase was ultimately a big positive step – showed that we could trust a new Open Source based vendor Implementing two different clients (Java and .NET) and understanding how to deal with cache “staleness” was a learning curve that Couchbase helped us with Education was a continuous process and needs to be supported with data/metrics and testing and feedback to Ops and Development Finding monitoring partners to work with to extend the solution was key – server side monitoring they provided was great but we wanted the full picture© 2011 Concur, all rights reserved. Concur is a registered trademark of Concur Technologies, Inc.
  10. 10. Extrahop Screenshot© 2011 Concur, all rights reserved. Concur is a registered trademark of Concur Technologies, Inc.
  11. 11. Extrahop Screenshot2© 2011 Concur, all rights reserved. Concur is a registered trademark of Concur Technologies, Inc.
  12. 12. Extrahop Custom Metric Dashboard© 2011 Concur, all rights reserved. Concur is a registered trademark of Concur Technologies, Inc.
  13. 13. Splunk Screenshot© 2011 Concur, all rights reserved. Concur is a registered trademark of Concur Technologies, Inc.
  14. 14. What’s next for us with Couchbase? Continue to look for additional workloads to move to the system and away from our traditional SQL systems – our Session system is the big one Excited about capabilities in 2.0 – Multi-datacenter replication on the top of the list Looking for ways to tweak performance to gain even more speed - 10GE end to end networking, splitting cluster topology, layering virtual machines Evolving our Cache strategy to not just cache database outputs© 2011 Concur, all rights reserved. Concur is a registered trademark of Concur Technologies, Inc.
  15. 15. © 2011 Concur, all rights reserved. Concur is a registered trademark of Concur Technologies, Inc.

×