SlideShare a Scribd company logo
1 of 34
Download to read offline
Apache Cassandra at Target: 
Pioneering NoSQL 
in a Big Enterprise 
Dan Cundiff (@pmotch) 
Target
Context 
● Target’s API platform 
● mostly REST APIs 
● e.g. products, locations, inventory, etc. 
● consumers inside and outside of Target 
● wide variety of providing systems (legacy, in-house 
built, saas, packages, etc.)
Problems we needed to solve 
● slow providing systems 
● cost prohibitive to call directly 
● unable to scale from increased demand 
● need a place to aggregate data from multiple 
systems 
● some data wasn’t even in a database to 
begin with!
Barriers with existing tools, part 1 
● cost too much 
● process for traditional DBs wasn’t a fit 
● too few tools/vendors
Barriers with existing tools, part 2 
● RDBMS isn’t: 
○ distributed (multi-tenant) 
○ close to Guests (geographic distribution) 
○ distributed across our data centers 
○ distributed to the cloud!
Barriers with existing tools, part 3 
● lack of performance control 
○ process, not owning it all, flexibility on 
changes like indexing, etc 
● availability 
○ systems before had outages, downtime, 
etc. 
● not automate-able
Discovering the solution
Taking the idea back 
● i just went and talked to Pete and we 
decided to do it! 
● tried other things in the past 
● show results by trying; succeed or fail fast
Reasons trying was attractive, part 1 
● fit 80% of our need 
● years in development 
● rich C* dev ecosystem
Reasons trying was attractive, part 2 
● google-able 
● strong community 
● a company who would support it
Reasons trying was attractive, part 3 
● chef-able 
● aligned well with existing investments 
● simple pricing model
Barriers to adoption 
● enterprise IT; the nature of it 
● selling it 
● NoSQL for the first time 
● automation (was happening at the time; 
scary to do) 
● political
Challenges integrating 
● bulk loading data 
● keeping cassandra in sync 
● many systems not event driven 
● packaged software 
● limited ways to integrate with providing 
systems
Challenges of standing it up, part 1 
● early distributed system (new to teams) 
● needed local disk (always used SAN before) 
● needed SSDs (always used spinning things) 
● existing config conflicts (backups, 
monitoring, raid, swap, etc) 
● use right sized server (don’t settle for what 
your infra friends give you by default)
Challenges of standing it up, part 2 
● full stack ownership 
● it’s new, don’t hand it off 
● support response is quick because we own it 
● you’re closest to the problem; you’re best 
suited to solve it 
● tuned to meet the needs of our APIs 
● data is modeled for API performance gains
Challenges of standing it up, part 3 
● skills supply is low (but getting better) 
● train your people 
● be wary of promises from consultants 
○ grill them on what they claim to know
Challenges of development, part 1 
● skills ramp up (data modeling, datastax 
driver, etc) 
● developers need to care 
○ encourage tweaking, research, make 
things better 
○ clients are equally as important to get the 
most out of C*
Challenges of development, part 2 
● mind shift from RDBMS 
● started with Astyanax; switched to DataStax 
driver 
○ DataStax supported 
○ newer features
Ops challenges, part 1 
● lots of machines; don’t config by hand 
● wrote Chef cookbooks 
● support people saw these odd servers and 
turned on things we disabled (like swap) 
● can’t use “legacy” testing, cassandra works 
differently; chaos stuff (turn off gossip, thrift, 
etc.)
Ops challenges, part 2 
● made logging awesome; we can see 
anything 
● utilized C* jmx interface to send data in real-time 
to Splunk 
● can correlate these events with the app tier 
(because app logs are in Splunk too!)
Ops challenges, part 3 
● useful mbeans: 
○ heap usage 
○ specific read/write latencies 
○ dropped reads/writes 
○ bloom filter ratios 
○ column count, size
Ops challenges, part 4 
● more useful mbeans: 
○ ss tables per read 
○ tombstones 
○ cache hits and ratios 
○ misbehaving queries (range slice)
Open source cookbook! 
● https://github.com/target/dse-cookbook 
● by Danny Parker 
● pull requests encouraged
Blog post on tuning 
● http://target.github.io/infrastructure/tuning-cassandra/ 
● by Danny Parker (@dcparker88)
Results, part 1 
● from n00bs to production ready = 2 months! 
○ infra, operation testing, app dev, and 
deployed! 
○ just in time before peak season 
● today our highest volume APIs depend on it
Results, part 2 
● growth (↑ functions + ↑ volume) = ~2000% 
● increased adoption of our APIs 
● C* unlocking things we couldn't do before 
● quick changes possible 
○ makes Agile possible 
○ gets us close to continuous delivery
Results, part 3 
● other teams are using it; more coming 
● sharing our cookbooks, lessons, etc. 
● opened the door to other distributed systems
Future, part 1 
● Use across more of our APIs 
● Remove remaining spinning disks
Future, part 2 
● move to cloud 
● automate full stack down to infra 
○ scale, quick geo-distribute, flexibility to 
tweak new infra settings, etc.
Future, part 3 
● get better at data modeling designs 
● less bulk loading 
○ remove compaction process overhead 
● weave in Spark, Kafka 
○ more event-based updates
Future, part crazy 
● Docker + Cassandra?
We’re hiring! 
Come talk to us
#CassandraSummit 
Dan Cundiff (@pmotch) 
Danny Parker (@dcparker88) 
Pete Guidarelli (@pguidarelli) 
Heather Mickman (@hmmickman)
Apache Cassandra at Target - Cassandra Summit 2014

More Related Content

Viewers also liked

Operating Model
Operating ModelOperating Model
Operating Modelrmuse70
 
Ceph Deployment at Target: Customer Spotlight
Ceph Deployment at Target: Customer SpotlightCeph Deployment at Target: Customer Spotlight
Ceph Deployment at Target: Customer SpotlightRed_Hat_Storage
 
Target Holding - Big Dikes and Big Data
Target Holding - Big Dikes and Big DataTarget Holding - Big Dikes and Big Data
Target Holding - Big Dikes and Big DataFrens Jan Rumph
 
Best buy strategic analysis (bb team) final
Best buy strategic analysis (bb team) finalBest buy strategic analysis (bb team) final
Best buy strategic analysis (bb team) finalRichard Chan, MBA
 
Strategic Design by Architecture and Organisation @ FINN.no - JavaZone 2016
Strategic Design by Architecture and Organisation @ FINN.no - JavaZone 2016Strategic Design by Architecture and Organisation @ FINN.no - JavaZone 2016
Strategic Design by Architecture and Organisation @ FINN.no - JavaZone 2016Sebastian Verheughe
 
Webinar | Target Modernizes Retail with Engaging Digital Experiences
Webinar | Target Modernizes Retail with Engaging Digital ExperiencesWebinar | Target Modernizes Retail with Engaging Digital Experiences
Webinar | Target Modernizes Retail with Engaging Digital ExperiencesDataStax
 
Target: Performance Tuning Cassandra at Target
Target: Performance Tuning Cassandra at TargetTarget: Performance Tuning Cassandra at Target
Target: Performance Tuning Cassandra at TargetDataStax Academy
 
Best buy-analysis
Best buy-analysisBest buy-analysis
Best buy-analysisTaposh Roy
 
Building a Multi-Region Cluster at Target (Aaron Ploetz, Target) | Cassandra ...
Building a Multi-Region Cluster at Target (Aaron Ploetz, Target) | Cassandra ...Building a Multi-Region Cluster at Target (Aaron Ploetz, Target) | Cassandra ...
Building a Multi-Region Cluster at Target (Aaron Ploetz, Target) | Cassandra ...DataStax
 
GWU Strategy Formulation & Implementation--Best Buy Case Study: Spring 2014
GWU Strategy Formulation & Implementation--Best Buy Case Study: Spring 2014GWU Strategy Formulation & Implementation--Best Buy Case Study: Spring 2014
GWU Strategy Formulation & Implementation--Best Buy Case Study: Spring 2014Lisa Fischer
 

Viewers also liked (12)

Operating Model
Operating ModelOperating Model
Operating Model
 
Ceph Deployment at Target: Customer Spotlight
Ceph Deployment at Target: Customer SpotlightCeph Deployment at Target: Customer Spotlight
Ceph Deployment at Target: Customer Spotlight
 
Hadoop for the Masses
Hadoop for the MassesHadoop for the Masses
Hadoop for the Masses
 
Target Holding - Big Dikes and Big Data
Target Holding - Big Dikes and Big DataTarget Holding - Big Dikes and Big Data
Target Holding - Big Dikes and Big Data
 
Best buy strategic analysis (bb team) final
Best buy strategic analysis (bb team) finalBest buy strategic analysis (bb team) final
Best buy strategic analysis (bb team) final
 
Strategic Design by Architecture and Organisation @ FINN.no - JavaZone 2016
Strategic Design by Architecture and Organisation @ FINN.no - JavaZone 2016Strategic Design by Architecture and Organisation @ FINN.no - JavaZone 2016
Strategic Design by Architecture and Organisation @ FINN.no - JavaZone 2016
 
Webinar | Target Modernizes Retail with Engaging Digital Experiences
Webinar | Target Modernizes Retail with Engaging Digital ExperiencesWebinar | Target Modernizes Retail with Engaging Digital Experiences
Webinar | Target Modernizes Retail with Engaging Digital Experiences
 
Target: Performance Tuning Cassandra at Target
Target: Performance Tuning Cassandra at TargetTarget: Performance Tuning Cassandra at Target
Target: Performance Tuning Cassandra at Target
 
Best buy
Best buyBest buy
Best buy
 
Best buy-analysis
Best buy-analysisBest buy-analysis
Best buy-analysis
 
Building a Multi-Region Cluster at Target (Aaron Ploetz, Target) | Cassandra ...
Building a Multi-Region Cluster at Target (Aaron Ploetz, Target) | Cassandra ...Building a Multi-Region Cluster at Target (Aaron Ploetz, Target) | Cassandra ...
Building a Multi-Region Cluster at Target (Aaron Ploetz, Target) | Cassandra ...
 
GWU Strategy Formulation & Implementation--Best Buy Case Study: Spring 2014
GWU Strategy Formulation & Implementation--Best Buy Case Study: Spring 2014GWU Strategy Formulation & Implementation--Best Buy Case Study: Spring 2014
GWU Strategy Formulation & Implementation--Best Buy Case Study: Spring 2014
 

Similar to Apache Cassandra at Target - Cassandra Summit 2014

DevOps at Obama for America(2012) and the DNC (DevOps Days NYC Jan 2013)
DevOps at Obama for America(2012) and the DNC (DevOps Days NYC Jan 2013)DevOps at Obama for America(2012) and the DNC (DevOps Days NYC Jan 2013)
DevOps at Obama for America(2012) and the DNC (DevOps Days NYC Jan 2013)John Schneider
 
Simply Business' Data Platform
Simply Business' Data PlatformSimply Business' Data Platform
Simply Business' Data PlatformDani Solà Lagares
 
Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...
Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...
Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...InfluxData
 
Project deep dive - Best practices.pptx
Project deep dive - Best practices.pptxProject deep dive - Best practices.pptx
Project deep dive - Best practices.pptxVIKASSINGH356734
 
Continuous Infrastructure First
Continuous Infrastructure FirstContinuous Infrastructure First
Continuous Infrastructure FirstKris Buytaert
 
A culture of Automation - Joe Smith - DevOpsDays Tel Aviv 2017
A culture of Automation - Joe Smith - DevOpsDays Tel Aviv 2017A culture of Automation - Joe Smith - DevOpsDays Tel Aviv 2017
A culture of Automation - Joe Smith - DevOpsDays Tel Aviv 2017DevOpsDays Tel Aviv
 
Technical Webinar: Patterns for Integrating Your Salesforce App with Off-Plat...
Technical Webinar: Patterns for Integrating Your Salesforce App with Off-Plat...Technical Webinar: Patterns for Integrating Your Salesforce App with Off-Plat...
Technical Webinar: Patterns for Integrating Your Salesforce App with Off-Plat...CodeScience
 
OSMC 2019 | How to improve database Observability by Charles Judith
OSMC 2019 | How to improve database Observability by Charles JudithOSMC 2019 | How to improve database Observability by Charles Judith
OSMC 2019 | How to improve database Observability by Charles JudithNETWAYS
 
Scaling Monitoring At Databricks From Prometheus to M3
Scaling Monitoring At Databricks From Prometheus to M3Scaling Monitoring At Databricks From Prometheus to M3
Scaling Monitoring At Databricks From Prometheus to M3LibbySchulze
 
Aws uk ug #8 not everything that happens in vegas stay in vegas
Aws uk ug #8   not everything that happens in vegas stay in vegasAws uk ug #8   not everything that happens in vegas stay in vegas
Aws uk ug #8 not everything that happens in vegas stay in vegasPeter Mounce
 
Building a custom cms with django
Building a custom cms with djangoBuilding a custom cms with django
Building a custom cms with djangoYann Malet
 
From 10 Deploys Per Year to 4 Per Day at DBS Bank: How Pivotal Platform Can R...
From 10 Deploys Per Year to 4 Per Day at DBS Bank: How Pivotal Platform Can R...From 10 Deploys Per Year to 4 Per Day at DBS Bank: How Pivotal Platform Can R...
From 10 Deploys Per Year to 4 Per Day at DBS Bank: How Pivotal Platform Can R...VMware Tanzu
 
Data Engineer's Lunch #85: Designing a Modern Data Stack
Data Engineer's Lunch #85: Designing a Modern Data StackData Engineer's Lunch #85: Designing a Modern Data Stack
Data Engineer's Lunch #85: Designing a Modern Data StackAnant Corporation
 
#lspe Building a Monitoring Framework using DTrace and MongoDB
#lspe Building a Monitoring Framework using DTrace and MongoDB#lspe Building a Monitoring Framework using DTrace and MongoDB
#lspe Building a Monitoring Framework using DTrace and MongoDBdan-p-kimmel
 
Preparing for Neo - Singapore OutSystems User Group October 2022 Meetup
Preparing for Neo - Singapore OutSystems User Group October 2022 MeetupPreparing for Neo - Singapore OutSystems User Group October 2022 Meetup
Preparing for Neo - Singapore OutSystems User Group October 2022 MeetupYashrajNayak4
 
CouchConf SF 2012 Lightning Talk - Operational Excellence
CouchConf SF 2012 Lightning Talk - Operational ExcellenceCouchConf SF 2012 Lightning Talk - Operational Excellence
CouchConf SF 2012 Lightning Talk - Operational ExcellenceLaine Campbell
 
Bootstrapping state in Apache Flink
Bootstrapping state in Apache FlinkBootstrapping state in Apache Flink
Bootstrapping state in Apache FlinkDataWorks Summit
 
Continuous Deployment
Continuous DeploymentContinuous Deployment
Continuous DeploymentSmartling
 
Building a Distributed & Automated Open Source Program at Netflix
Building a Distributed & Automated Open Source Program at NetflixBuilding a Distributed & Automated Open Source Program at Netflix
Building a Distributed & Automated Open Source Program at NetflixAll Things Open
 

Similar to Apache Cassandra at Target - Cassandra Summit 2014 (20)

DevOps at Obama for America(2012) and the DNC (DevOps Days NYC Jan 2013)
DevOps at Obama for America(2012) and the DNC (DevOps Days NYC Jan 2013)DevOps at Obama for America(2012) and the DNC (DevOps Days NYC Jan 2013)
DevOps at Obama for America(2012) and the DNC (DevOps Days NYC Jan 2013)
 
Simply Business' Data Platform
Simply Business' Data PlatformSimply Business' Data Platform
Simply Business' Data Platform
 
Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...
Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...
Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...
 
Project deep dive - Best practices.pptx
Project deep dive - Best practices.pptxProject deep dive - Best practices.pptx
Project deep dive - Best practices.pptx
 
Continuous Infrastructure First
Continuous Infrastructure FirstContinuous Infrastructure First
Continuous Infrastructure First
 
A culture of Automation - Joe Smith - DevOpsDays Tel Aviv 2017
A culture of Automation - Joe Smith - DevOpsDays Tel Aviv 2017A culture of Automation - Joe Smith - DevOpsDays Tel Aviv 2017
A culture of Automation - Joe Smith - DevOpsDays Tel Aviv 2017
 
Technical Webinar: Patterns for Integrating Your Salesforce App with Off-Plat...
Technical Webinar: Patterns for Integrating Your Salesforce App with Off-Plat...Technical Webinar: Patterns for Integrating Your Salesforce App with Off-Plat...
Technical Webinar: Patterns for Integrating Your Salesforce App with Off-Plat...
 
OSMC 2019 | How to improve database Observability by Charles Judith
OSMC 2019 | How to improve database Observability by Charles JudithOSMC 2019 | How to improve database Observability by Charles Judith
OSMC 2019 | How to improve database Observability by Charles Judith
 
Scaling Monitoring At Databricks From Prometheus to M3
Scaling Monitoring At Databricks From Prometheus to M3Scaling Monitoring At Databricks From Prometheus to M3
Scaling Monitoring At Databricks From Prometheus to M3
 
Aws uk ug #8 not everything that happens in vegas stay in vegas
Aws uk ug #8   not everything that happens in vegas stay in vegasAws uk ug #8   not everything that happens in vegas stay in vegas
Aws uk ug #8 not everything that happens in vegas stay in vegas
 
Building a custom cms with django
Building a custom cms with djangoBuilding a custom cms with django
Building a custom cms with django
 
From 10 Deploys Per Year to 4 Per Day at DBS Bank: How Pivotal Platform Can R...
From 10 Deploys Per Year to 4 Per Day at DBS Bank: How Pivotal Platform Can R...From 10 Deploys Per Year to 4 Per Day at DBS Bank: How Pivotal Platform Can R...
From 10 Deploys Per Year to 4 Per Day at DBS Bank: How Pivotal Platform Can R...
 
Data Engineer's Lunch #85: Designing a Modern Data Stack
Data Engineer's Lunch #85: Designing a Modern Data StackData Engineer's Lunch #85: Designing a Modern Data Stack
Data Engineer's Lunch #85: Designing a Modern Data Stack
 
#lspe Building a Monitoring Framework using DTrace and MongoDB
#lspe Building a Monitoring Framework using DTrace and MongoDB#lspe Building a Monitoring Framework using DTrace and MongoDB
#lspe Building a Monitoring Framework using DTrace and MongoDB
 
Preparing for Neo - Singapore OutSystems User Group October 2022 Meetup
Preparing for Neo - Singapore OutSystems User Group October 2022 MeetupPreparing for Neo - Singapore OutSystems User Group October 2022 Meetup
Preparing for Neo - Singapore OutSystems User Group October 2022 Meetup
 
CouchConf SF 2012 Lightning Talk - Operational Excellence
CouchConf SF 2012 Lightning Talk - Operational ExcellenceCouchConf SF 2012 Lightning Talk - Operational Excellence
CouchConf SF 2012 Lightning Talk - Operational Excellence
 
Bootstrapping state in Apache Flink
Bootstrapping state in Apache FlinkBootstrapping state in Apache Flink
Bootstrapping state in Apache Flink
 
Devops For Drupal
Devops  For DrupalDevops  For Drupal
Devops For Drupal
 
Continuous Deployment
Continuous DeploymentContinuous Deployment
Continuous Deployment
 
Building a Distributed & Automated Open Source Program at Netflix
Building a Distributed & Automated Open Source Program at NetflixBuilding a Distributed & Automated Open Source Program at Netflix
Building a Distributed & Automated Open Source Program at Netflix
 

More from Dan Cundiff

Governance to Guidance to Awesome Product - DOES 2018
Governance to Guidance to Awesome Product - DOES 2018Governance to Guidance to Awesome Product - DOES 2018
Governance to Guidance to Awesome Product - DOES 2018Dan Cundiff
 
How Target Made It Super Easy for Developers to Contribute to Open Source - L...
How Target Made It Super Easy for Developers to Contribute to Open Source - L...How Target Made It Super Easy for Developers to Contribute to Open Source - L...
How Target Made It Super Easy for Developers to Contribute to Open Source - L...Dan Cundiff
 
From No Git to 3000 GitHub Users and How to Keep Them Happy - GitHub Universe...
From No Git to 3000 GitHub Users and How to Keep Them Happy - GitHub Universe...From No Git to 3000 GitHub Users and How to Keep Them Happy - GitHub Universe...
From No Git to 3000 GitHub Users and How to Keep Them Happy - GitHub Universe...Dan Cundiff
 
How to Build APIs - MHacks 2016
How to Build APIs - MHacks 2016How to Build APIs - MHacks 2016
How to Build APIs - MHacks 2016Dan Cundiff
 
Why DevOps != the Wild West and How Embracing it Can Improve Security - RSA C...
Why DevOps != the Wild West and How Embracing it Can Improve Security - RSA C...Why DevOps != the Wild West and How Embracing it Can Improve Security - RSA C...
Why DevOps != the Wild West and How Embracing it Can Improve Security - RSA C...Dan Cundiff
 
Jenkins User Conference 2014
Jenkins User Conference 2014Jenkins User Conference 2014
Jenkins User Conference 2014Dan Cundiff
 
Splunk All the Things: Our First 3 Months Monitoring Web Service APIs - Splun...
Splunk All the Things: Our First 3 Months Monitoring Web Service APIs - Splun...Splunk All the Things: Our First 3 Months Monitoring Web Service APIs - Splun...
Splunk All the Things: Our First 3 Months Monitoring Web Service APIs - Splun...Dan Cundiff
 

More from Dan Cundiff (7)

Governance to Guidance to Awesome Product - DOES 2018
Governance to Guidance to Awesome Product - DOES 2018Governance to Guidance to Awesome Product - DOES 2018
Governance to Guidance to Awesome Product - DOES 2018
 
How Target Made It Super Easy for Developers to Contribute to Open Source - L...
How Target Made It Super Easy for Developers to Contribute to Open Source - L...How Target Made It Super Easy for Developers to Contribute to Open Source - L...
How Target Made It Super Easy for Developers to Contribute to Open Source - L...
 
From No Git to 3000 GitHub Users and How to Keep Them Happy - GitHub Universe...
From No Git to 3000 GitHub Users and How to Keep Them Happy - GitHub Universe...From No Git to 3000 GitHub Users and How to Keep Them Happy - GitHub Universe...
From No Git to 3000 GitHub Users and How to Keep Them Happy - GitHub Universe...
 
How to Build APIs - MHacks 2016
How to Build APIs - MHacks 2016How to Build APIs - MHacks 2016
How to Build APIs - MHacks 2016
 
Why DevOps != the Wild West and How Embracing it Can Improve Security - RSA C...
Why DevOps != the Wild West and How Embracing it Can Improve Security - RSA C...Why DevOps != the Wild West and How Embracing it Can Improve Security - RSA C...
Why DevOps != the Wild West and How Embracing it Can Improve Security - RSA C...
 
Jenkins User Conference 2014
Jenkins User Conference 2014Jenkins User Conference 2014
Jenkins User Conference 2014
 
Splunk All the Things: Our First 3 Months Monitoring Web Service APIs - Splun...
Splunk All the Things: Our First 3 Months Monitoring Web Service APIs - Splun...Splunk All the Things: Our First 3 Months Monitoring Web Service APIs - Splun...
Splunk All the Things: Our First 3 Months Monitoring Web Service APIs - Splun...
 

Recently uploaded

Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 

Recently uploaded (20)

Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 

Apache Cassandra at Target - Cassandra Summit 2014

  • 1. Apache Cassandra at Target: Pioneering NoSQL in a Big Enterprise Dan Cundiff (@pmotch) Target
  • 2. Context ● Target’s API platform ● mostly REST APIs ● e.g. products, locations, inventory, etc. ● consumers inside and outside of Target ● wide variety of providing systems (legacy, in-house built, saas, packages, etc.)
  • 3. Problems we needed to solve ● slow providing systems ● cost prohibitive to call directly ● unable to scale from increased demand ● need a place to aggregate data from multiple systems ● some data wasn’t even in a database to begin with!
  • 4. Barriers with existing tools, part 1 ● cost too much ● process for traditional DBs wasn’t a fit ● too few tools/vendors
  • 5. Barriers with existing tools, part 2 ● RDBMS isn’t: ○ distributed (multi-tenant) ○ close to Guests (geographic distribution) ○ distributed across our data centers ○ distributed to the cloud!
  • 6. Barriers with existing tools, part 3 ● lack of performance control ○ process, not owning it all, flexibility on changes like indexing, etc ● availability ○ systems before had outages, downtime, etc. ● not automate-able
  • 8. Taking the idea back ● i just went and talked to Pete and we decided to do it! ● tried other things in the past ● show results by trying; succeed or fail fast
  • 9. Reasons trying was attractive, part 1 ● fit 80% of our need ● years in development ● rich C* dev ecosystem
  • 10. Reasons trying was attractive, part 2 ● google-able ● strong community ● a company who would support it
  • 11. Reasons trying was attractive, part 3 ● chef-able ● aligned well with existing investments ● simple pricing model
  • 12. Barriers to adoption ● enterprise IT; the nature of it ● selling it ● NoSQL for the first time ● automation (was happening at the time; scary to do) ● political
  • 13. Challenges integrating ● bulk loading data ● keeping cassandra in sync ● many systems not event driven ● packaged software ● limited ways to integrate with providing systems
  • 14. Challenges of standing it up, part 1 ● early distributed system (new to teams) ● needed local disk (always used SAN before) ● needed SSDs (always used spinning things) ● existing config conflicts (backups, monitoring, raid, swap, etc) ● use right sized server (don’t settle for what your infra friends give you by default)
  • 15. Challenges of standing it up, part 2 ● full stack ownership ● it’s new, don’t hand it off ● support response is quick because we own it ● you’re closest to the problem; you’re best suited to solve it ● tuned to meet the needs of our APIs ● data is modeled for API performance gains
  • 16. Challenges of standing it up, part 3 ● skills supply is low (but getting better) ● train your people ● be wary of promises from consultants ○ grill them on what they claim to know
  • 17. Challenges of development, part 1 ● skills ramp up (data modeling, datastax driver, etc) ● developers need to care ○ encourage tweaking, research, make things better ○ clients are equally as important to get the most out of C*
  • 18. Challenges of development, part 2 ● mind shift from RDBMS ● started with Astyanax; switched to DataStax driver ○ DataStax supported ○ newer features
  • 19. Ops challenges, part 1 ● lots of machines; don’t config by hand ● wrote Chef cookbooks ● support people saw these odd servers and turned on things we disabled (like swap) ● can’t use “legacy” testing, cassandra works differently; chaos stuff (turn off gossip, thrift, etc.)
  • 20. Ops challenges, part 2 ● made logging awesome; we can see anything ● utilized C* jmx interface to send data in real-time to Splunk ● can correlate these events with the app tier (because app logs are in Splunk too!)
  • 21. Ops challenges, part 3 ● useful mbeans: ○ heap usage ○ specific read/write latencies ○ dropped reads/writes ○ bloom filter ratios ○ column count, size
  • 22. Ops challenges, part 4 ● more useful mbeans: ○ ss tables per read ○ tombstones ○ cache hits and ratios ○ misbehaving queries (range slice)
  • 23. Open source cookbook! ● https://github.com/target/dse-cookbook ● by Danny Parker ● pull requests encouraged
  • 24. Blog post on tuning ● http://target.github.io/infrastructure/tuning-cassandra/ ● by Danny Parker (@dcparker88)
  • 25. Results, part 1 ● from n00bs to production ready = 2 months! ○ infra, operation testing, app dev, and deployed! ○ just in time before peak season ● today our highest volume APIs depend on it
  • 26. Results, part 2 ● growth (↑ functions + ↑ volume) = ~2000% ● increased adoption of our APIs ● C* unlocking things we couldn't do before ● quick changes possible ○ makes Agile possible ○ gets us close to continuous delivery
  • 27. Results, part 3 ● other teams are using it; more coming ● sharing our cookbooks, lessons, etc. ● opened the door to other distributed systems
  • 28. Future, part 1 ● Use across more of our APIs ● Remove remaining spinning disks
  • 29. Future, part 2 ● move to cloud ● automate full stack down to infra ○ scale, quick geo-distribute, flexibility to tweak new infra settings, etc.
  • 30. Future, part 3 ● get better at data modeling designs ● less bulk loading ○ remove compaction process overhead ● weave in Spark, Kafka ○ more event-based updates
  • 31. Future, part crazy ● Docker + Cassandra?
  • 32. We’re hiring! Come talk to us
  • 33. #CassandraSummit Dan Cundiff (@pmotch) Danny Parker (@dcparker88) Pete Guidarelli (@pguidarelli) Heather Mickman (@hmmickman)