MyDrive Solutions: Case Study: Troubleshooting Production Issues as a Developer.

•

1 like•1,083 views

This talk will be a step by step walkthrough of a developer troubleshooting a real performance issue we had at MyDrive, from the very first steps diagnosing the symptoms, through looking at metric charts down to CQL queries, the Ruby CQL driver, and Ruby code profiling.

Technology

Case Study: Troubleshooting Production Issues as a Developer

© 2015. All Rights Reserved.
1 Introductions
2 The problem
3 Show me the code!
4 Conclusions
5 Q & A

Carlos Alonso - About me
3© 2015. All Rights Reserved.
• Spanish Londoner
• MSc at Salamanca University, Spain
• Software Engineer @ MyDrive Solutions
• Enjoying Cassandra since 2014
• Cassandra Certified Developer
• DataStax Cassandra MVP 2015
• @calonso / http://mrcalonso.com

MyDrive Solutions - About
4© 2015. All Rights Reserved.
• World leading driver profiling company.
• Using technology and data to understand
how to improve driving behaviour.
• Enjoying Cassandra since 2012
• Recently acquired by The Generali Group.
• @_MyDrive
• http://www.mydrivesolutions.com
• We are hiring!

@calonso @_MyDrive© 2015. All Rights Reserved. 5
Troubleshooting performance issues in production
as developer

@calonso @_MyDrive© 2015. All Rights Reserved. 6
trips.csv
driver_id start_time end_time start_location end_location
123456 2011-04-18
12:23:36 +0100
2011-04-18
14:11:22 +0100
-11.06238284,
-126.71490131
86.02409727,
72.13493018
…
import.rb
The program

$@calonso @_MyDrive© 2015. All Rights Reserved. 7 CREATE KEYSPACE drivers WITH replication = { 'class': 'SimpleStrategy', 'replication_factor': '2' }; • 2 environments • 3 nodes clusters • Cassandra 1.2.9 The Setup$

$@calonso @_MyDrive© 2015. All Rights Reserved. 8 CREATE TABLE trips ( driver_id int, end_time timestamp, end_location text, start_location text, start_time timestamp, PRIMARY KEY (driver_id, end_time) ) WITH bloom_filter_fp_chance=0.010000 AND caching='KEYS_ONLY' AND comment='' AND dclocal_read_repair_chance=0.000000 AND gc_grace_seconds=864000 AND read_repair_chance=0.100000 AND replicate_on_write='true' AND populate_io_cache_on_flush='false' AND compaction={'class': 'SizeTieredCompactionStrategy'} AND compression={'sstable_compression': 'SnappyCompressor'};$

@calonso @_MyDrive© 2015. All Rights Reserved. 9
Tools
Ruby-Prof https://github.com/ruby-prof/ruby-prof
Cassanity https://github.com/jnunemaker/cassanity

@calonso @_MyDrive
Conclusions
• Measure everything
• Metrics and monitoring let us know we were facing an
unexpected performance issue 
 
• Keep calm, read to the end
• On first profiling report I could have spotted that the issue was
on Cluster#connect but due to my eagerness to find a fix I
made a wrong assumption that meant more time (~2 days!).
© 2015. All Rights Reserved. 11

@calonso @_MyDrive
Thank the community
• We have great tools such as:
• Cassandra: Apache and DataStax
• Statsd, Graphite.
• RubyProf: All members of RubyProf Organisation 
• Special thanks to Patrick McFadin.
© 2015. All Rights Reserved. 12

You know Cassandra works and can solve a lot of problems, but then you try to design it into your application and things start falling apart. Stop! This is where we need to have some real talk. I've been helping organizations implement Cassandra for years. True story. I can help! It's easy to get lost in the details, but making the switch to Cassandra is a journey of many steps. This will be a system of the next 30 years so take your time, do it right and feel the happiness. It's all there for you. Your health and sanity will be intact and most importantly, your job will be better!

Battery Ventures: Simulating and Visualizing Large Scale Cassandra Deployments

DataStax Academy

Ebay: DB Capacity planning at eBay

DataStax Academy

eBay has developed comprehensive database capacity planning process with 20 years heavy usage of Oracle. With adoption of NoSQL technologies, we are working on to adopt same process with NoSQL, especially Cassandra. eBay has annual traffic peaks during Q4 holiday, we need to proactively review capacity needs and adjust to sail through peak time without any issue. With bench-marking test for each available SKU, we are able to meet business needs without over provisioning.

Capital One: Using Cassandra In Building A Reporting Platform

DataStax Academy

As a leader in the financial industry, Capital One applications generate huge amounts of data that require fast and accurate handling, storage and analysis. We are transforming how we report operational data to our internal users so that they can make quick and precise business decisions to serve our customers. As part of this transformation, we are building a new Go-based data processing framework that will enable us to transfer data from multiple data stores (RDBMS, files, etc.) to a single NoSQL database - Cassandra. This new NoSQL store will act as a reporting database that will receive data on a near real-time basis and serve the data through scorecards and reports. We would like to share our experience in defining this fast data platform and the methodologies used to model financial data in Cassandra.

PagerDuty: Span the WAN? Yes you can!

DataStax Academy

Most Cassandra usages take advantage of its exceptional performance and ability to handle massive data sets. At PagerDuty, we use Cassandra for entirely different reasons: to reliably manage mutable application states and to maintain durability requirements even in the face of full data center outages. We achieve this by deploying Cassandra clusters with hosts in multiple WAN-separated data centers, configured with per-data center replica placement requirements, and with significant application-level support to use Cassandra as a consistent datastore. Accumulating several years of experience with this approach, we've learned to accommodate the impact of WAN network latency on Cassandra queries, how to horizontally scale while maintaining our placement invariants, why asymmetric load is experienced by nodes in different data centers, and more. This talk will go over our workload and design goals, detail the resultant Cassandra system design, and explain a number of our unintuitive operational learnings about this novel Cassandra usage paradigm.

Target: Performance Tuning Cassandra at Target

DataStax Academy

Macy's: Changing Engines in Mid-Flight

DataStax Academy

This presentation recounts the story of Macys.com and Bloomingdales.com's migration from legacy RDBMS to NoSQL Cassandra in partnership with DataStax. One thing that differentiates this talk from others on Cassandra is Macy's philosophy of "doing more with less." You will see why we emphasize the performance tuning aspects of iterative development when you see how much processing we can support on relatively small configurations. This session will cover: 1) The process that led to our decision to use Cassandra 2) The approach we used for migrating from DB2 & Coherence to Cassandra without disrupting the production environment 3) The various schema options that we tried and how we settled on the current one. We'll show you a selection of some of our extensive performance tuning benchmarks, as well as how these performance results figured into our final schema designs. 4) Our lessons learned and next steps

The Last Pickle: Distributed Tracing from Application to Database

DataStax Academy

Monitoring provides information on system performance, however tracing is necessary to understand individual request performance. Detailed query tracing has been provided by Cassandra since version 1.2 and is invaluable when diagnosing problems. Although knowing what queries to trace and why the application makes them still requires deep technical knowledge. By merging Application tracing via Zipkin and Cassandra query tracing we automate the process and make it easier to identify and resolve problems. In this talk Mick Semb Wever, Team Member at The Last Pickle, will introduce Cassandra query tracing and Zipkin. He will then propose an extension that allows clients to pass a trace identifier through to Cassandra, and a way to integrate Zipkin tracing into Cassandra. Driving all this is the desire to create one tracing view across the entire system.

Netflix stores 98 percent of data related with streaming services: right from bookmarks, viewing history to billing and payment information. These services / applications simply desire highly available and scalable persistence solution to keep themselves running efficiently in a normal and disastrous situation. How does Netflix plan for capacity for it's new as well as existing services? In this talk, Arun Agrawal, Senior Software Engineer and Ajay Upadhyay, Cloud Data Architect @Netflix will talk about the capacity planning and capacity forecasting in cassandra world. We will take you through the science behind forecasting the short and long term usage and auto-scaling adequate capacity well before C* clusters reach their limit. This guarantees highly scalable and available persistence solution meeting our SLAs @ Netflix. About the Speakers ajay upadhyay Senior Database Engineer, Netflix Responsible for persistent layer at Netflix, part of CDE [Cloud Database Engineering] team. Working with application team, suggesting and guiding them with the best practices for various persistent layers provided by CDE team. Arun Agrawal Senior Software Engineer, Netflix Arun Agrawal is part of Cloud Database Engineering where they provide CAAS (Cassandra as a service). Ensuring smooth operations of service and finding innovative ways to reduce the management overheads of having CAAS.

Scylla Summit 2018: How Scylla Helps You to be a Better Application Developer

ScyllaDB

Tsinghua University: Two Exemplary Applications in China

DataStax Academy

In this talk, we will share the experiences of applying Cassandra with two real customers in China. In the first use case, we deployed Cassandra at Sany Group, a leading company of Machinery manufacturing, to manage the sensor data generated by construction machinery. By designing a specific schema and optimizing the write process, we successfully managed over 1.5 billion historical data records and achieved the online write throughput of 10k write operations per second with 5 servers. MapReduce is also used on Cassandra for valued-added services, e.g. operations management, machine failure prediction, and abnormal behavior mining. In the second use case, Cassandra is deployed in the China Meteorological Administration to manage the Meteorological data. We design a hybrid schema to support both slice query and time window based query efficiently. Also, we explored the optimized compaction and deletion strategy for meteorological data in this case.

Azure + DataStax Enterprise Powers Office 365 Per User Store

DataStax Academy

Cassandra Summit 2014: Apache Cassandra Best Practices at Ebay

DataStax Academy

Presenter: Feng Qu, Principal DBA at eBay Cassandra has been adopted widely at eBay in recent years and used by many end-user facing applications. I will introduce best practices we have built over the time around system design, capacity planning, deployment automation, monitoring integration, performance analysis and troubleshooting. I will also share our experience working with DataStax support to provide a highly available, highly scalable data store fitting into eBay infrastructure.

Expedia Group: Our Migration Journey from Apache Cassandra to ScyllaDB

ScyllaDB

Proofpoint: Fraud Detection and Security on Social Media

DataStax Academy

Social media has become the new frontier for cyber-attackers. The explosive growth of this new communications platform, combined with the potential to reach millions of people through a single post, has provided a low barrier for exploitation. In this talk, we will focus on how Cassandra is used to enable our fight against bad actors on social media. In particular, we will discuss how we use Cassandra for anomaly detection, social mob alerting, trending topics, and fraudulent classification. We will also speak about our Cassandra data models, integration with Spark Streaming, and how we use KairosDB for our time series data. Watch us don our superhero-Cassandra capes as we fight against the bad guys!

Apache Cassandra in the Cloud

Instaclustr

Building A Diverse Geo-Architecture For Cloud Native Applications In One Day

VMware Tanzu

Presenter: Ben Laplanche, Product Manager, Pivotal Cloud Foundry Companies turn to PaaS and Cloud Native Applications to gain agility and speed. To provide customer value, a fault tolerant infrastructure is essential. But what happens if an entire data center, region, or even country should go offline? Cassandra holds the key to keeping application state in sync through replication, whilst Pivotal Cloud Foundry provides easy deployment to multiple IaaS providers. It also comes complete with a managed service offering for DataStax Enterprise. This talk will discuss how this setup can be deployed in one day, including demonstrations and a walkthrough of the key concepts, approaches, and considerations.

Processing 50,000 events per second with Cassandra and Spark

Ben Slater

British Gas Connected Homes: Data Engineering

DataStax Academy

In this talk Josep draws on his experience of building a data platform based on Cassandra and Spark to service the UK's foremost player in the connected homes market. Bringing streams of data online; productionising data science algorithms on spark; and delivering outputs via API's or Kafka messages. Josep will explore the ups and the downs of bringing all this together and share what he's learned from 12 months of Cassandra and Spark development and operations.

Webinar | Building Apps with the Cassandra Python Driver

DataStax Academy

How ReversingLabs Serves File Reputation Service for 10B Files

ScyllaDB

ReversingLabs is on a mission to deliver threat intelligence to their users by providing complete visibility and insight into every destructive object. To deliver on their commitment, they migrated to Scylla to handle thousands of updates per second in their processing engines. In their talk, they will go over their requirements and show how they tuned the system to handle requests from their API frontend.

Our Journey from Cassandra to Scylla

ScyllaDB

Cassandra Community Webinar: Apache Spark Analytics at The Weather Channel - ...

DataStax Academy

The state of analytics has changed dramatically over the last few years. Hadoop is now commonplace, and the ecosystem has evolved to include new tools such as Spark, Shark, and Drill, that live alongside the old MapReduce-based standards. It can be difficult to keep up with the pace of change, and newcomers are left with a dizzying variety of seemingly similar choices. This is compounded by the number of possible deployment permutations, which can cause all but the most determined to simply stick with the tried and true. But there are serious advantages to many of the new tools, and this presentation will give an analysis of the current state–including pros and cons as well as what’s needed to bootstrap and operate the various options. About Robbie Strickland, Software Development Manager at The Weather Channel Robbie works for The Weather Channel’s digital division as part of the team that builds backend services for weather.com and the TWC mobile apps. He has been involved in the Cassandra project since 2010 and has contributed in a variety of ways over the years; this includes work on drivers for Scala and C#, the Hadoop integration, heading up the Atlanta Cassandra Users Group, and answering lots of Stack Overflow questions.

Captial One: Why Stream Data as Part of Data Transformation?

ScyllaDB

Cassandra Community Webinar: MySQL to Cassandra - What I Wish I'd Known

DataStax

A brief intro to how Barracuda Networks uses Cassandra and the ways in which they are replacing their MySQL infrastructure, with Cassandra. This presentation will include the lessons they've learned along the way during this migration. Speaker: Michael Kjellman, Software Engineer at Barracuda Networks Michael Kjellman is a Software Engineer, from San Francisco, working at Barracuda Networks. Michael works across multiple products, technologies, and languages. He primarily works on Barracuda's spam infrastructure and web filter classification data.

Webinar: Dyn + DataStax - helping companies deliver exceptional end-user expe...

DataStax

Dyn delivers exceptional Internet Performance. Enabling high quality services requires data centers around the globe. In order to manage services, customers need timely insight collected from all over the world. Dyn uses DataStax Enterprise (DSE) to deploy complex clusters across multiple datacenters to enable sub 50 ms query responses for hundreds of billions of data points. From granular DNS traffic data, to aggregated counts for a variety of report dimensions, DSE at Dyn has been up since 2013 and has shined through upgrades, data center migrations, DDoS attacks and hardware failures. In this webinar, Principal Engineers Tim Chadwick and Rick Bross cover the requirements which led them to choose DSE as their go-to Big Data solution, the path which led to SPARK, and the lessons that we’ve learned in the process.

Scylla @ GumGum: Contextual Ads

ScyllaDB

DataStax: Testing Cassandra Guarantees Under Diverse Failure Modes With Jepsen

DataStax Academy

The increasing prevalence of large-scale distributed systems necessitates careful testing and understanding of the invariants and guarantees at play. In particular, Kyle Kingsbury's "Call Me Maybe" series has increased awareness of this need for developers and administrators alike. In this talk, Joel will discuss these issues in the context of his efforts as an intern at DataStax to develop extensive testing coverage via Kingsbury's Jepsen library.

AdStage: Monacella: An Relational Object Database using Cassandra as the Data...

DataStax Academy

At AdStage we have a large volume of data about ads and their relationships: campaigns, ad groups, keywords, bids, budgets, targeting info - the list goes on. We started out storing all this data in Postgres, but even before we reached public beta we were already putting too much strain on the largest Postgres instance we could run. We considered sharding, but instead we decided to embark on a project to store our data in Cassandra. Now, after more than a year of development, we present Monacella, a relational object database that uses Cassandra as the datastore. In this talk we'll examine the architecture of Monacella, its features and use cases, and plans for future development.

What's hot

Managing Cassandra Databases with OpenStack Trove

Tesora

C* Capacity Forecasting (Ajay Upadhyay, Jyoti Shandil, Arun Agrawal, Netflix)...

DataStax

Scylla Summit 2018: How Scylla Helps You to be a Better Application Developer

ScyllaDB

Tsinghua University: Two Exemplary Applications in China

DataStax Academy

Azure + DataStax Enterprise Powers Office 365 Per User Store

DataStax Academy

Cassandra Summit 2014: Apache Cassandra Best Practices at Ebay

DataStax Academy

Expedia Group: Our Migration Journey from Apache Cassandra to ScyllaDB

ScyllaDB

Proofpoint: Fraud Detection and Security on Social Media

DataStax Academy

Apache Cassandra in the Cloud

Instaclustr

Building A Diverse Geo-Architecture For Cloud Native Applications In One Day

VMware Tanzu

Processing 50,000 events per second with Cassandra and Spark

Ben Slater

British Gas Connected Homes: Data Engineering

DataStax Academy

Webinar | Building Apps with the Cassandra Python Driver

DataStax Academy

How ReversingLabs Serves File Reputation Service for 10B Files

ScyllaDB

Our Journey from Cassandra to Scylla

ScyllaDB

Cassandra Community Webinar: Apache Spark Analytics at The Weather Channel - ...

DataStax Academy

Captial One: Why Stream Data as Part of Data Transformation?

ScyllaDB

Cassandra Community Webinar: MySQL to Cassandra - What I Wish I'd Known

DataStax

Webinar: Dyn + DataStax - helping companies deliver exceptional end-user expe...

DataStax

Scylla @ GumGum: Contextual Ads

ScyllaDB

What's hot (20)

Managing Cassandra Databases with OpenStack Trove

C* Capacity Forecasting (Ajay Upadhyay, Jyoti Shandil, Arun Agrawal, Netflix)...

Scylla Summit 2018: How Scylla Helps You to be a Better Application Developer

Tsinghua University: Two Exemplary Applications in China

Azure + DataStax Enterprise Powers Office 365 Per User Store

Cassandra Summit 2014: Apache Cassandra Best Practices at Ebay

Expedia Group: Our Migration Journey from Apache Cassandra to ScyllaDB

Proofpoint: Fraud Detection and Security on Social Media

Apache Cassandra in the Cloud

Building A Diverse Geo-Architecture For Cloud Native Applications In One Day

Processing 50,000 events per second with Cassandra and Spark

British Gas Connected Homes: Data Engineering

Webinar | Building Apps with the Cassandra Python Driver

How ReversingLabs Serves File Reputation Service for 10B Files

Our Journey from Cassandra to Scylla

Cassandra Community Webinar: Apache Spark Analytics at The Weather Channel - ...

Captial One: Why Stream Data as Part of Data Transformation?

Cassandra Community Webinar: MySQL to Cassandra - What I Wish I'd Known

Webinar: Dyn + DataStax - helping companies deliver exceptional end-user expe...

Scylla @ GumGum: Contextual Ads

Viewers also liked

DataStax: Testing Cassandra Guarantees Under Diverse Failure Modes With Jepsen

DataStax Academy

AdStage: Monacella: An Relational Object Database using Cassandra as the Data...

DataStax Academy

DataStax: The Cassandra Validation Harness: Achieving More Stable Releases

DataStax Academy

As of 3.0, Apache Cassandra will be moving to a tick-tock release cycle to improve the stability problems that have plagued new major releases. Alongside that, we need to expand beyond our existing test infrastructure, and begin simulating production environments. We'll explore the Cassandra Validation Harness, a new testing infrastructure for improving our detection rate of resource leaks, concurrency problems, and other bugs that only show up at scale.

The Last Pickle: Repeatable, Scalable, Reliable, Observable: Cassandra

DataStax Academy

Apache Cassandra makes it possible to write code on a laptop and deploy to multi-region clusters with a few configuration changes. But what does it take to create repeatable, scalable, reliable, and observable clusters? In this talk Aaron Morton, Co Founder at The Last Pickle and Apache Cassandra Committer, will discuss the tools and techniques they use. From environment planning to implementation for tools such as Chef, Sensu, Graphite, Riemann and LogStash this will be a discussion of the full stack ecosystem for successful projects.

Silicon Valley Data Science: From Oracle to Cassandra with Spark

DataStax Academy

Want your enterprise to scale out 20x faster performance while containing cost? Customer Data Integration is a core Master-Data-Management (MDM) function for most enterprises: being intentional about where to perform your most resource-consuming data management processes, and choosing the right open-source columnar data store, can help you scale in big way. In this talk we will look at the migration of Allant's CDI-keying engine, built using JMS and Oracle, to an architecture centered around distributed data-management technologies with Cassandra as the data store.   Attendees will learn how a legacy Oracle-centric pipeline can be migrated to a Cassandra-based solution by:  1) using Spark to push the bulk of the keying & recognition functions originally executed at the application layer down to the distributed data layer and  2) using Cassandra and Zookeeper to migrate in-process locking mechanisms and to generate unique profile recognition keys to a highly scalable system; and   3) using Spark to create an high-throughput ETL pipeline for loading large datasets into Cassandra.   Allant's original customer recognition engine (employed to integrate customer profiles and their digital handles) was architected and designed to accommodate both batch and real time inputs, with a common API to ensure unique creation of customer profiles. The common API is facilitated via a batch-to-single interface (proprietary, custom software) that drops input records into JMS queues to perform the parsing, hygiene, matching, and keying sub-functions. This pipeline was re-implemented by using Cassandra as a scalable, high-throughput data store, and Spark to achieve highly distributed function execution while maintaining the proven fuzzy matching logic to preserve Allant’s proprietary customer recognition rules.

AddThis: Scaling Cassandra up and down into containers with ZFS

DataStax Academy

ZFS is an advanced file, raid, and volume management system originally developed by Sun Microsystems, 'The Last Word in File Systems' has been unavailable on Linux until recently. AddThis uses ZFS to more effectively scale up dedicated hardware, getting twice the performance at half the cost. ZFS is also fundamental to containerization, allowing nodes from multiple clusters to be co-located with safe persistent storage.

Stratio: Geospatial and bitemporal search in Cassandra with pluggable Lucene ...

DataStax Academy

Stratio presented its open source Lucene-based implementation of Cassandra's secondary indexes at Cassandra Summit London 2014, which provided several search engine features. It used to be distributed as a fork of Apache Cassandra, which was a huge problem both for users and maintainers. Nowadays, due to some changes introduced at Cassandra 2.1.6, we are proud to announce that it has become a plugin that can be attached to the official Apache Cassandra. With the plugin we have been able to provide Cassandra with geospatial capabilities, making it possible to index geographical positions and perform bounding box and radial distance queries. This is achieved through Lucene's geospatial module. Another feature we have provided with our plugin is the possibility of indexing bitemporal data models, which distinguish between system time and business time. This way, it is possible to make queries over Cassandra such as ̢give me what system thought in a certain instant about what happened in another instant. The implementation has been performed combining range prefix trees with the 4R-Tree approach exposed by Bliujute, et al. Both full-text, geospatial and bitemporal queries can be combined with Apache Spark to avoid systematic full-scan, dramatically reducing the amount of data to be processed.

DataStax: A deep look at the CQL WHERE clause

DataStax Academy

GumGum: Multi-Region Cassandra in AWS

DataStax Academy

GumGum relies heavily on Cassandra for storing different kinds of metadata. Currently GumGum reaches 1 billion unique visitors per month using 3 Cassandra datacenters in Amazon Web Services spread across the globe. This presentation will detail how we scaled out from one local Cassandra datacenter to a multi-datacenter Cassandra cluster and all the problems we encountered and choices we made while implementing it. How did we architect multi-region Cassandra in AWS? What were our experiences in implementing multi-datacenter Cassandra? How did we achieve low latency with multi-region Cassandra and the Datastax Driver? What are the different Cassandra use cases at GumGum? How did we integrate our Cassandra with Spark?

SKB Kontur: Digging Cassandra cluster

DataStax Academy

Assume you have a Cassandra cluster with hundreds of tables, and one day the latency of client requests and CPU utilization of the Cassandra process became unacceptable. Our team regularly faced such a problem. A simple look at the metrics did not help us - because of high CPU utilization, we saw bad metrics across almost all of our tables. In this talk I will discuss ways to find out which table (or tables) are the most problematic for the cluster and create problems for other tables. The talk will be fully practical as I will introduce our real steps in this investigation. Some of the steps were successful, others were not. But finally we reduced both latency and CPU utilization in about 10 times without adding additional nodes and hardware resources.

codecentric AG: CQRS and Event Sourcing Applications with Cassandra

DataStax Academy

CQRS (Command Query Responsibility Segregation) is a pattern, which separates the process of querying and updating data. As a query only returns data without any side effects, a command is designed to change data. CQRS is often combined with Event Sourcing. This is an architecture in which all changes to an application state are stored as a sequence of events. Because of its great capability to store time series data Cassandra is the perfect fit for implementing the event store. But there a still a lot of open questions: What about the data modeling? What techniques will be used to process and store data in the Cassandra database? How to access the current state of the application, without replaying every event? And what about failure handling? In this talk, I will give a brief introduction to CQRS and the Event Sourcing pattern and will then answer the questions above using a real life example of a data store for customer data.

Cisco: Cassandra adoption on Cisco UCS & OpenStack

DataStax Academy

Tesora: Managing Cassandra Databases with OpenStack Trove

DataStax Academy

This presentation is aimed at IT managers, DevOps or Developers in an IT organization, as it presents an in depth exploration of the architecture and internals of Cassandra databases with OpenStack Trove. The presentation will start with an overview of the Trove architecture, exploring such concepts as "How does Trove interact with other OpenStack services", "What are the various components of Trove", "What are guest agents", "How are requests to Trove processed", and "How does Trove support multiple database types". It will next explain how Trove supports the Apache Cassandra NoSQL database and how to deploy and manage a Cassandra database with Trove. It will then describe Trove's framework for implementing clustering and how Cassandra clusters are deployed through Trove. It helps the participant understand the internals and architecture of Trove and provides the participant with knowledge that would be useful in assessing, deploying and managing a Cassandra database with Trove.

Restlet: Building a multi-tenant API PaaS with DataStax Enterprise Search

DataStax Academy

Viewers also liked (14)

DataStax: Testing Cassandra Guarantees Under Diverse Failure Modes With Jepsen

AdStage: Monacella: An Relational Object Database using Cassandra as the Data...

DataStax: The Cassandra Validation Harness: Achieving More Stable Releases

The Last Pickle: Repeatable, Scalable, Reliable, Observable: Cassandra

Silicon Valley Data Science: From Oracle to Cassandra with Spark

AddThis: Scaling Cassandra up and down into containers with ZFS

Stratio: Geospatial and bitemporal search in Cassandra with pluggable Lucene ...

DataStax: A deep look at the CQL WHERE clause

GumGum: Multi-Region Cassandra in AWS

SKB Kontur: Digging Cassandra cluster

codecentric AG: CQRS and Event Sourcing Applications with Cassandra

Cisco: Cassandra adoption on Cisco UCS & OpenStack

Tesora: Managing Cassandra Databases with OpenStack Trove

Restlet: Building a multi-tenant API PaaS with DataStax Enterprise Search

Introduction to Data Modeling with Apache Cassandra

DataStax Academy

Cassandra Core Concepts

DataStax Academy

Enabling Search in your Cassandra Application with DataStax Enterprise

DataStax Academy

Bad Habits Die Hard

DataStax Academy

Advanced Data Modeling with Apache Cassandra

DataStax Academy

Advanced Cassandra

DataStax Academy

More from DataStax Academy (20)

Forrester CXNYC 2017 - Delivering great real-time cx is a true craft

Introduction to DataStax Enterprise Graph Database

Introduction to DataStax Enterprise Advanced Replication with Apache Cassandra

Cassandra on Docker @ Walmart Labs

Cassandra 3.0 Data Modeling

Cassandra Adoption on Cisco UCS & Open stack

Data Modeling for Apache Cassandra

Coursera Cassandra Driver

Production Ready Cassandra

Cassandra @ Netflix: Monitoring C* at Scale, Gossip and Tickler & Python

Cassandra @ Sony: The good, the bad, and the ugly part 1

Cassandra @ Sony: The good, the bad, and the ugly part 2

Standing Up Your First Cluster

Real Time Analytics with Dse

Introduction to Data Modeling with Apache Cassandra

Cassandra Core Concepts

Enabling Search in your Cassandra Application with DataStax Enterprise

Bad Habits Die Hard

Advanced Data Modeling with Apache Cassandra

Advanced Cassandra

Recently uploaded

GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...

Sri Ambati

DevOps and Testing slides at DASA Connect

Kari Kakkonen

Neuro-symbolic is not enough, we need neuro-*semantic*

Frank van Harmelen

Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”. All of this illustrated with link prediction over knowledge graphs, but the argument is general.

PHP Frameworks: I want to break free (IPC Berlin 2024)

Ralf Eggert

In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development. This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

FIDO Alliance

From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...

Product School

GraphRAG is All You need? LLM & Knowledge Graph

Guy Korland

Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs. 1. Unifying Large Language Models and Knowledge Graphs: A Roadmap. https://arxiv.org/abs/2306.08302 2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs: https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/

Key Trends Shaping the Future of Infrastructure.pdf

Cheryl Hung

Assuring Contact Center Experiences for Your Customers With ThousandEyes

ThousandEyes

The Future of Platform Engineering

Jemma Hussein Allen

FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf

FIDO Alliance

Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024

Tobias Schneck

As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other? Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

FIDO Alliance

State of ICS and IoT Cyber Threat Landscape Report 2024 preview

Prayukth K V

The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development. The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers: State of global ICS asset and network exposure Sectoral targets and attacks as well as the cost of ransom Global APT activity, AI usage, actor and tactic profiles, and implications Rise in volumes of AI-powered cyberattacks Major cyber events in 2024 Malware and malicious payload trends Cyberattack types and targets Vulnerability exploit attempts on CVEs Attacks on counties – USA Expansion of bot farms – how, where, and why In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East Why are attacks on smart factories rising? Cyber risk predictions Axis of attacks – Europe Systemic attacks in the Middle East Download the full report from here: https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/

When stars align: studies in data quality, knowledge graphs, and machine lear...

Elena Simperl

Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality

Inflectra

In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring. Learn about: • The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks. • Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective. • Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification. • Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process. Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.

Connector Corner: Automate dynamic content and events by pushing a button

DianaGray10

Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to: Create a campaign using Mailchimp with merge tags/fields Send an interactive Slack channel message (using buttons) Have the message received by managers and peers along with a test email for review But there’s more: In a second workflow supporting the same use case, you’ll see: Your campaign sent to target colleagues for approval If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team But—if the “Reject” button is pushed, colleagues will be alerted via Slack message Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors. And... Speakers: Akshay Agnihotri, Product Manager Charlie Greenberg, Host

Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...

Ramesh Iyer

In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.

GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...

James Anderson

Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management. The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM). Speakers: Bob Boule Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle. Gopinath Rebala Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.

De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...

Product School

Recently uploaded (20)

GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...

DevOps and Testing slides at DASA Connect

Neuro-symbolic is not enough, we need neuro-*semantic*

PHP Frameworks: I want to break free (IPC Berlin 2024)

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...

GraphRAG is All You need? LLM & Knowledge Graph

Key Trends Shaping the Future of Infrastructure.pdf

Assuring Contact Center Experiences for Your Customers With ThousandEyes

The Future of Platform Engineering

FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf

Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

State of ICS and IoT Cyber Threat Landscape Report 2024 preview

When stars align: studies in data quality, knowledge graphs, and machine lear...

Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality

Connector Corner: Automate dynamic content and events by pushing a button

Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...

GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...

De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...

MyDrive Solutions: Case Study: Troubleshooting Production Issues as a Developer.

1. Case Study: Troubleshooting Production Issues as a Developer

3. Carlos Alonso - About me 3© 2015. All Rights Reserved. • Spanish Londoner • MSc at Salamanca University, Spain • Software Engineer @ MyDrive Solutions • Enjoying Cassandra since 2014 • Cassandra Certified Developer • DataStax Cassandra MVP 2015 • @calonso / http://mrcalonso.com

4. MyDrive Solutions - About 4© 2015. All Rights Reserved. • World leading driver profiling company. • Using technology and data to understand how to improve driving behaviour. • Enjoying Cassandra since 2012 • Recently acquired by The Generali Group. • @_MyDrive • http://www.mydrivesolutions.com • We are hiring!

6. @calonso @_MyDrive© 2015. All Rights Reserved. 6 trips.csv driver_id start_time end_time start_location end_location 123456 2011-04-18 12:23:36 +0100 2011-04-18 14:11:22 +0100 -11.06238284, -126.71490131 86.02409727, 72.13493018 … import.rb The program

7. @calonso @_MyDrive© 2015. All Rights Reserved. 7 CREATE KEYSPACE drivers WITH replication = { 'class': 'SimpleStrategy', 'replication_factor': '2' }; • 2 environments • 3 nodes clusters • Cassandra 1.2.9 The Setup

8. @calonso @_MyDrive© 2015. All Rights Reserved. 8 CREATE TABLE trips ( driver_id int, end_time timestamp, end_location text, start_location text, start_time timestamp, PRIMARY KEY (driver_id, end_time) ) WITH bloom_filter_fp_chance=0.010000 AND caching='KEYS_ONLY' AND comment='' AND dclocal_read_repair_chance=0.000000 AND gc_grace_seconds=864000 AND read_repair_chance=0.100000 AND replicate_on_write='true' AND populate_io_cache_on_flush='false' AND compaction={'class': 'SizeTieredCompactionStrategy'} AND compression={'sstable_compression': 'SnappyCompressor'};

10.

11. @calonso @_MyDrive Conclusions • Measure everything • Metrics and monitoring let us know we were facing an unexpected performance issue    • Keep calm, read to the end • On first profiling report I could have spotted that the issue was on Cluster#connect but due to my eagerness to find a fix I made a wrong assumption that meant more time (~2 days!). © 2015. All Rights Reserved. 11

12. @calonso @_MyDrive Thank the community • We have great tools such as: • Cassandra: Apache and DataStax • Statsd, Graphite. • RubyProf: All members of RubyProf Organisation  • Special thanks to Patrick McFadin. © 2015. All Rights Reserved. 12

13. Thank you

MyDrive Solutions: Case Study: Troubleshooting Production Issues as a Developer.

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (14)

Similar to MyDrive Solutions: Case Study: Troubleshooting Production Issues as a Developer.

Similar to MyDrive Solutions: Case Study: Troubleshooting Production Issues as a Developer. (20)

More from DataStax Academy

More from DataStax Academy (20)

Recently uploaded

Recently uploaded (20)

MyDrive Solutions: Case Study: Troubleshooting Production Issues as a Developer.