CASSANDRA MEETUP - Choosing the right cloud instances for success

•

0 likes•149 views

This document discusses choosing the appropriate Amazon EC2 instance types for Cassandra workloads. It emphasizes that one size does not fit all, and the instance must be tailored to the specific workload in terms of reads vs writes and CPU vs memory needs. It reviews the storage options of EBS volumes and ephemeral storage, noting the better performance of ephemeral. It then provides examples of suitable instance types for different Cassandra workload volumes, from development and light production to real heavy production use cases.

© DataStax, All Rights Reserved.
Apache Cassandra™
Choosing instances for success
1
Erick Ramirez
DataStax Engineering
@flightc

Welcome
• Your app in focus — reads vs writes, CPU vs RAM

• What IOPS? How much is enough?

• Are ephemeral disks evil?

• False economy — cheaper instances can cost you more

• A time to kill — they’re not your pets

© DataStax, All Rights Reserved.
https://academy.datastax.com
5

© DataStax, All Rights Reserved.
ONE SIZE DOES NOT

FIT ALL
6

Tailor to workload
• intimately understand your app

• reads vs writes

• CPU vs memory

• OLTP vs OLAP

• use case will dictate requirements

© DataStax, All Rights Reserved.
STORAGE OPTIONS
8

© DataStax, All Rights Reserved.
EBS gp2 SSDs
9
• general purpose EBS option

• persistent (durable)

• default volume for EC2 instances

• guaranteed 99% single-digit millisecond latency

• only pay for each GB (IOPS included)

• minimum 10K IOPS for production workloads
3 IOPS/GB (3K IOPS/TB)
Max 10K IOPS/vol
Max 160MB/s throughput/vol
1TB = $122/mo, $1474/yr

© DataStax, All Rights Reserved.
EBS io1 SSDs
10
• fastest available EBS option

• persistent (durable)

• for latency-sensitive OLTP workloads

• guaranteed 99.9%* single-digit millisecond latency

• provisioned IOPS are charged extra

• minimum 10K IOPS for production workloads

* read the fine print
Up to 50 IOPS/GB
Max 20K IOPS/vol
Max 320MB/s throughput/vol
1TB = $141/mo, $1695/yr
1K IOPS = $72/mo, $864/yr

© DataStax, All Rights Reserved.
#spoileralert

EPHEMERAL IS YOUR FRIEND
11

Ephemeral storage
• performance orders of magnitude better than EBS

• already included in instance costs, e.g. m3, c3, i3

• “physically” attached

• not durable across reboots but…

© DataStax, All Rights Reserved.
HELLO, CASSANDRA
13

What is Cassandra
• massively scalable NoSQL database

• fully distributed, no single-point-of-failure

• linear horizontal scaling

© DataStax, All Rights Reserved.
Why Cassandra
15
• all nodes are the same — no SPOF

• real-time, durable writes

• linear scaling on commodity servers

• real-time replication across data centres

• always on — no offline operation

• because you have a scale problem

© DataStax, All Rights Reserved.16
Replication across DCs

© DataStax, All Rights Reserved.
CHEAP INSTANCES

MAY BE COSTING YOU
17

© DataStax, All Rights Reserved.
Real example
18
• deployed on c4.4xlarge

• using EBS io1 with 3K PIOPS

• nodes dropping writes

• high read latencies
16 vCPU, 30GB RAM
Instance $ 5443
EBS io1 1TB $ 1695
PIOPS 3K $ 2592
————————
Annual cost $ 9730

© DataStax, All Rights Reserved.
Recommendation
19
• swap to i3.2xlarge

• 1.9TB NVMe SSDs included

• 3M IOPS, 16GB/s

• 60-70% cheaper than replaced i2.2xlarge
8 vCPU, 61GB RAM
Instance $ 4174
————————
Annual cost $ 4174

© DataStax, All Rights Reserved.
HORSES FOR COURSES
20

© DataStax, All Rights Reserved.
Use case - dev, light prod
21
• m3.large suitable

• entry-level load, testing-the-waters

• minimum 3 C* nodes with RF=3

• use CMS GC with 2GB heap
2 vCPU
7.5GB RAM
1 x 32GB SSD
$ 962/yr

© DataStax, All Rights Reserved.
Use case -

low prod volume
22
• m3.xlarge suitable

• JVM will perform better with the extra RAM

• min 3 C* nodes with RF=3

• use CMS GC with 8GB heap
4 vCPU
15GB RAM
1 x 40GB SSD
$ 1924/yr

© DataStax, All Rights Reserved.
Use case -

moderate prod volume
23
• c3.2xlarge recommended

• more diskspace, extra cores a bonus

• costs 50% more for 2x CPU and 4x diskspace

• min 3 C* nodes with RF=3

• use CMS GC with 8GB heap
8 vCPU
15GB RAM
2 x 80GB SSD
$ 2916/yr

© DataStax, All Rights Reserved.
Use case -

real prod volume
24
• i3.2xlarge recommended

• will handle all kinds of workloads including Analytics,
Graph and Search (Solr)

• min 3 C* nodes with RF=3

• use G1 GC with 24GB heap (32GB for Search nodes)
8 vCPU
61GB RAM
1.9TB NVMe SSD
$ 4174/yr

© DataStax, All Rights Reserved.
https://datastaxacademy.slack.com
25

© DataStax, All Rights Reserved.
Thank you
26

Eventually consistent databases choose to remain available under failure, allowing for conflicting data to be stored in different replicas (later repaired by background processes). Weakening the consistency guarantees improves not only availability, but also performance, as the number of replicas involved in a given operation can be minimized. There are, however, use-cases that require the opposite trade-off. Indeed, Apache Cassandra and Scylla provide Lightweight Transactions (LWT), which allow single-key linearizable updates. The mechanism underlying LWT is asynchronous consensus. In this talk, we'll describe the characteristics and requirements of Scylla's consensus implementation, and how it enables strongly consistent updates. We will also cover how consensus can be applied to other aspects of the system, such as schema changes, node membership, and range movements, in order to improve their reliability and safety. We will thus show that an eventually consistent database can leverage consensus without compromising either availability or performance.

Load testing Cassandra applications

Ben Slater

ScyllaDB @ Apache BigData, may 2016

Tzach Livyatan

Scylla is a new open source NoSQL database that is compatible with Apache Cassandra but provides significantly higher performance through a redesign that takes advantage of modern hardware. Scylla is capable of over 1.8 million operations per second per node with predictable low latencies. It uses an architecture with shard-per-core and reactor programming that avoids locks and threads for near-linear scaling. Scylla also has its own efficient unified cache and I/O scheduler that maximize throughput and allow it to outperform Cassandra on benchmarks by an order of magnitude. Scylla is fully compatible with Cassandra and aims to build an open source community around ongoing core database improvements.

Scylla Summit 2019 Keynote - Dor Laor - Beyond Cassandra

ScyllaDB

ScyllaDB CEO Dor Laor lays out the ten million dollar engineering problem for distributed systems, and how only Scylla is architected to address the issue at the heart of Big Data ROI. He then introduces ScyllaDB's Glauber Costa and Packet's James Malachowski to reveal a new level of performance for a persistent NoSQL datastore. Dor concludes his talk with a bold proposition about how Scylla is uniquely positioned to help companies easily create and scale the software they need to achieve their vision.

How to be Successful with Scylla

ScyllaDB

Should I use more, smaller instances, or fewer, bigger instances? Is 1Gbps enough for my network cards? Should I use batches? Can I have a collection with 3GB in size? Those are just some of the many questions we see users asking themselves on a daily basis over our mailing list, slack, and corporate ticket requests. In this talk, I will explore the answers to these common questions and help you make sure that your deployment is up to the highest standards.

Lessons Learned on Java Tuning for Our Cassandra Clusters (Carlos Monroy, Kne...

DataStax

Customizing JVM settings for the needs of an application can be a tricky business, especially when running externally developed software such as Cassandra. In this talk I will share our experiences and the procedure that we have used to test and validate changes with Java tuning. We'll explore with two recent experiences: changes and monitoring of G1 garbage collection, and moving buffer objects off the heap. For the talk, I'll discuss our tuning process at Knewton. I will share some of the challenges that we faced while identifying what we expected to learn. I'll discuss how we isolated and minimized variables across tests, the importance of the duration of these tests, and how we try to separate correlation from causation. I will demonstrate how to use and interpret the results of the custom scripts that we were driven to develop to gain visibility into our G1GC processes; these scripts will be open sourced. About the Speaker Carlos Monroy Senior Software Engineer, Knewton Carlos Monroy is a senior engineer on the database team at Knewton, an education company that created an adaptive learning platform. Carlos has been developing software professionally since 1998. His experience holding multiple roles on the software lifecycle provides him a wholistic approach. Having used over a half dozen relational database engines, he has recently come over to the NoSQL side, first working with HBase and for the last three years Cassandra.

Diagnosing Problems in Production involves first preparing monitoring tools like OpsCenter, Server monitoring, Application metrics, and Log aggregation. Common issues include incorrect server times causing data inconsistencies, tombstone overhead slowing queries, not using the right snitch, version mismatches breaking functionality, and disk space not being reclaimed properly. Diagnostic tools like htop, iostat, vmstat, dstat, strace, jstack, tcpdump and nodetool can help narrow down issues like performance bottlenecks, garbage collection problems, and compaction issues.

Seattle Cassandra Meetup - HasOffers

btoddb

This document summarizes Apache Cassandra configuration at HasOffers for supporting high write throughput and fault tolerance across multiple datacenters. Keypoints include Cassandra being configured with two column families, multiple secondary indexes, and a replication factor of 1 across 9 nodes in 3 datacenters. HasOffers sees approximately 13.8 million daily inserts replicated across the datacenters, with 3 million daily queries. Daily insert volumes are shown by datacenter. Additional topics covered include AWS instance types used, disk options like EBS RAID vs provisioned IOPS, and client configuration using Pycassa.

Scylla Summit 2018: Rebuilding the Ceph Distributed Storage Solution with Sea...

ScyllaDB

This document summarizes plans to rebuild the Ceph distributed storage system using Seastar, a framework for high-performance event-driven applications. Ceph is an open-source distributed storage platform that provides object, block, and file storage at scale. It uses a thread pool model that has limitations around lock contention and context switching. Seastar uses an asynchronous message passing model without locks that could improve Ceph's performance. The plan is to backfill Ceph components starting with critical I/O paths to prioritize basic functionality, then add supporting features later to fully rebuild Ceph on Seastar.

DynamoDB at HasOffers

Amazon Web Services

10 Devops-Friendly Database Must-Haves - Dor Laor, ScyllaDB - DevOpsDays Tel ...

DevOpsDays Tel Aviv

This document discusses 10 must-have features for a devops-friendly database: 1) being open source, 2) supporting multi-tenancy, 3) allowing isolated test and development environments, 4) integrating with big data technologies, 5) being resilient during failures, 6) being extensible through export/import and scripting, 7) automatically handling tuning and scaling, 8) providing consistent tail latency, 9) easily backing up and restoring data, and 10) always being available across multiple data centers. It promotes ScyllaDB as a database that has these features by providing Cassandra compatibility at Redis speeds.

How to size up an Apache Cassandra cluster (Training)

DataStax Academy

This document discusses how to size a Cassandra cluster based on replication factor, data size, and performance needs. It describes that replication factor, data size, data velocity, and hardware considerations like CPU, memory, and disk type should all be examined to determine the appropriate number of nodes. The goal is to have enough nodes to store data, achieve target throughput levels, and maintain performance and availability even if nodes fail.

How We Made Scylla Maintenance Easier, Safer and Faster

ScyllaDB

Many Scylla maintenance operations require significant data movement between database nodes in a cluster. It is not an easy task to make the management operations efficient while maintaining minimum impact on the workload all the time. In this talk, we will share how we made those maintenance operations easier, safer and faster with the new Scylla features and improvements, e.g., seedless, repair based node operations, smarter off-strategy compaction, io bandwidth limiter for repair and compaction, parallel repair in Scylla Manger and more.

Scylla Summit 2016: Why Kenshoo is about to displace Cassandra with Scylla

ScyllaDB

Scylla Summit 2018: OLAP or OLTP? Why Not Both?

ScyllaDB

OLTP and Analytics are very different. One is characterized by many concurrent small requests, with a high sensitivity to latency, while the other typically processes large streams of data with more emphasis on throughput. The talk will cover: - the different requirements of the two workloads - how ScyllaDB optimizes for both - performance isolation of different workloads within ScyllaDB - how ScyllaDB supports concurrent OLTP and Analytics without sacrificing either latency or throughput - measurements

Scylla’s Journey Towards Being an Elastic Cloud Native Database

ScyllaDB

Scylla Summit 2016: Using ScyllaDB for a Microservice-based Pipeline in Go

ScyllaDB

How do you handle the continuous transformation and refinement of billions of entities with some sort of reliability and performance? In this talk, Henrik will describe how Scylla enabled him and his team to create a pipelined solution using a series of microservices written in Go communicating with each other using Nats. You’ll hear about the mistakes and learnings they had along the way as they built the services that led to the great performance and stability they are experiencing today.

Scylla Summit 2016: Compose on Containing the Database

ScyllaDB

This document discusses how Compose applies containerization best practices to provide database services. It outlines the "Twelve Factors of Stateful Apps" that guide Compose's architecture. These include running databases and data in separate containers, using environment variables for configuration, scaling containers vertically before adding nodes, and collecting logs and metrics within the deployment. By applying these factors, Compose can reliably deploy a range of database technologies like MongoDB, PostgreSQL, and now ScyllaDB across its platform.

Building Scalable, Real Time Applications for Financial Services with DataStax

DataStax

The document discusses building scalable real-time applications for financial services using Apache Cassandra. It provides an overview of Cassandra's key capabilities including linear scalability, high availability, tunable consistency and multi-datacenter support. It also discusses example architectures for resilient application tiers using Cassandra to store session data across multiple data centers.

Date-tiered Compaction Policy for Time-series Data

HBaseCon

Clara Xiong (Flurry/Yahoo!) With petabytes of data on thousands of nodes replicated across multiple data centers, growing at an accelerating rate, we have been running a workload at scale with a bottleneck of IO bandwidth. This talk covers a new compaction policy to improve efficiency for time-range scans of various look-back windows by structuring and maintaining a date-tiered store file layout for time-series data with infrequent updates and deletes.

Scylla Summit 2018: Keeping Your Latency SLAs No Matter What!

ScyllaDB

IMC Summit 2016 Breakout - Andy Pavlo - What Non-Volatile Memory Means for th...

In-Memory Computing Summit

The advent of non-volatile memory (NVM) will fundamentally change the dichotomy between memory and durable storage in database management systems (DBMSs). These new NVM devices are almost as fast as DRAM, but all writes to it are potentially persistent even after power loss. Existing DBMSs are unable to take full advantage of this technology because their internal architectures are predicated on the assumption that memory is volatile. That means when NVM finally arrives, just like when you finally passed that kidney stone after three weeks, everyone will be relieved but the transition will be painful. Many of the components of legacy DBMSs will become unnecessary and will degrade the performance of data intensive applications.

Scylla Summit 2019 Keynote - Avi Kivity

ScyllaDB

Scaling Cassandra for Big Data

DataStax Academy

This document discusses scaling Cassandra for big data applications. It describes how Ooyala uses Cassandra for fast access to data generated by MapReduce, high availability key-value storage from Storm, and playhead tracking for cross-device resume. It outlines Ooyala's experience migrating to newer Cassandra versions as data doubled yearly, including removing expired tombstones, schema changes, and Linux performance tuning.

Performance tuning - A key to successful cassandra migration

Ramkumar Nottath

In last few years, technology has seen a major drift in the dominance of traditional / RDMBS databases across different domains. Expeditious adoption of NoSQL databases especially Cassandra in the industry opens up a lot more discussions on what are the major challenges that are faced during implementation of Cassandra and how to mitigate it. Many a times we conclude that migration or POC (proof of concept) is not successful; however the real flaw might be in the data modeling, identifying the right hardware configurations, database parameters, right consistency level and so on. There's no one good model or configuration which fits all use cases and all applications. Performance tuning an application is truly an art and requires perseverance. This paper delve into different performance tuning considerations and anti-patterns that need to be considered during Cassandra migration / implementation to make sure we are able to reap the benefits of Cassandra, what makes it a ‘Visionary’ in 2014 Gartner’s Magic Quadrant for Operational Database Management Systems.

TechTalk v2.0 - Performance tuning Cassandra + AWS

Pythian

The document discusses various techniques for optimizing input/output (I/O) performance when running Cassandra on Amazon Web Services (AWS). It recommends tuning at the AWS, Cassandra, and file system layers. Specifically, it suggests using AWS i2 instances with SSDs, following Cassandra best practices, mounting the Cassandra data directory on ephemeral storage, and using tools like IOZone to benchmark performance. The document also provides details on testing configurations and measuring performance.

M6d cassandrapresentation

Edward Capriolo

Ceph on All Flash Storage -- Breaking Performance Barriers

Ceph Community

The document discusses a new software-defined all-flash storage system called InfiniFlash that uses Ceph software to break performance barriers. It provides disaggregated storage, compute, and software for better scaling and lower costs compared to traditional monolithic storage systems. InfiniFlash uses flash drive cards in a 3U JBOD configuration and can deliver up to 2 million IOPS, less than 1ms latency, and 15GB/s throughput while consuming only 400W of power. When used with Ceph software, it provides a highly scalable block and object storage solution for private and public cloud environments.

What's hot

Cassandra Day Atlanta 2015: Diagnosing Problems in Production

DataStax Academy

Seattle Cassandra Meetup - HasOffers

btoddb

Scylla Summit 2018: Rebuilding the Ceph Distributed Storage Solution with Sea...

ScyllaDB

DynamoDB at HasOffers

Amazon Web Services

10 Devops-Friendly Database Must-Haves - Dor Laor, ScyllaDB - DevOpsDays Tel ...

DevOpsDays Tel Aviv

How to size up an Apache Cassandra cluster (Training)

DataStax Academy

How We Made Scylla Maintenance Easier, Safer and Faster

ScyllaDB

Scylla Summit 2016: Why Kenshoo is about to displace Cassandra with Scylla

ScyllaDB

Scylla Summit 2018: OLAP or OLTP? Why Not Both?

ScyllaDB

Scylla’s Journey Towards Being an Elastic Cloud Native Database

ScyllaDB

Scylla Summit 2016: Using ScyllaDB for a Microservice-based Pipeline in Go

ScyllaDB

Scylla Summit 2016: Compose on Containing the Database

ScyllaDB

Building Scalable, Real Time Applications for Financial Services with DataStax

DataStax

Date-tiered Compaction Policy for Time-series Data

HBaseCon

Scylla Summit 2018: Keeping Your Latency SLAs No Matter What!

ScyllaDB

IMC Summit 2016 Breakout - Andy Pavlo - What Non-Volatile Memory Means for th...

In-Memory Computing Summit

Scylla Summit 2019 Keynote - Avi Kivity

ScyllaDB

Scaling Cassandra for Big Data

DataStax Academy

Performance tuning - A key to successful cassandra migration

Ramkumar Nottath

TechTalk v2.0 - Performance tuning Cassandra + AWS

Pythian

What's hot (20)

Cassandra Day Atlanta 2015: Diagnosing Problems in Production

Seattle Cassandra Meetup - HasOffers

Scylla Summit 2018: Rebuilding the Ceph Distributed Storage Solution with Sea...

DynamoDB at HasOffers

10 Devops-Friendly Database Must-Haves - Dor Laor, ScyllaDB - DevOpsDays Tel ...

How to size up an Apache Cassandra cluster (Training)

How We Made Scylla Maintenance Easier, Safer and Faster

Scylla Summit 2016: Why Kenshoo is about to displace Cassandra with Scylla

Scylla Summit 2018: OLAP or OLTP? Why Not Both?

Scylla’s Journey Towards Being an Elastic Cloud Native Database

Scylla Summit 2016: Using ScyllaDB for a Microservice-based Pipeline in Go

Scylla Summit 2016: Compose on Containing the Database

Building Scalable, Real Time Applications for Financial Services with DataStax

Date-tiered Compaction Policy for Time-series Data

Scylla Summit 2018: Keeping Your Latency SLAs No Matter What!

IMC Summit 2016 Breakout - Andy Pavlo - What Non-Volatile Memory Means for th...

Scylla Summit 2019 Keynote - Avi Kivity

Scaling Cassandra for Big Data

Performance tuning - A key to successful cassandra migration

TechTalk v2.0 - Performance tuning Cassandra + AWS

Similar to CASSANDRA MEETUP - Choosing the right cloud instances for success

M6d cassandrapresentation

Edward Capriolo

Ceph on All Flash Storage -- Breaking Performance Barriers

Ceph Community

Building Data Pipelines with SMACK: Designing Storage Strategies for Scale an...

DataStax

stackArmor presentation for DevOpsDC ver 4

Gaurav "GP" Pal

The document discusses stackArmor's presentation on using MongoDB and Solr on AWS for a customer's ETL processing needs. The customer receives large amounts of data monthly and needed to process it faster at lower cost. StackArmor designed a solution using MongoDB shards, SolrCloud, and Chef automation to meet the customer's goals of processing 5 million records per hour within budget and compliance requirements. Through tuning the design over time, they were able to improve throughput to over 3 million records per hour sustained.

DevOps for ETL processing at scale with MongoDB, Solr, AWS and Chef

Gaurav "GP" Pal

Large scale data processing for Extract Transform and Loading (ETL) jobs is a very common practice. The stackArmor DevOps team developed a Chef based automation solution to automate the AWS environment provisioning, code deployment and data ingestion processing to ingest and process over 2 TB of Data. This presentation covers the technologies used, the planning phase, AWS instance selection and optimizing the ETL processing for not only performance but also cost. The target was to process 500 million rows within 72 hours with a processing rate of 5 million transactions per hour. The presentation also provides pitfalls and automation optimizations performed to accomplish the targeted processing rates. The presentation was delivered at the DevOpsDC Meetup on May 17, 2016

Sizing MongoDB on AWS with Wired Tiger-Patrick and Vigyan-Final

Vigyan Jain

This document provides guidance on sizing MongoDB deployments on AWS for optimal performance. It discusses key considerations for capacity planning like testing workloads, measuring performance, and adjusting over time. Different AWS services like compute-optimized instances and storage options like EBS are reviewed. Best practices for WiredTiger like sizing cache, effects of compression and encryption, and monitoring tools are covered. The document emphasizes starting simply and scaling based on business needs and workload profiling.

AWS Webcast - Cost and Performance Optimization in Amazon RDS

Amazon Web Services

Amazon RDS makes it easy to set up, operate, and scale relational databases in the cloud. The service offers a variety of options for optimizing the performance level delivered, as well as optimizing your spending. In this webinar, we will show a variety of techniques for implementing the right performance level for your application. Learning Objectives: • Understand the Amazon RDS options that change database performance and cost • Select the appropriate performance and cost level for your specific application Who Should Attend: • Technical Amazon RDS customers and prospective customers

Colvin exadata mistakes_ioug_2014

marvin herrera

This document discusses common mistakes made when implementing Oracle Exadata systems. It describes improperly sized SGAs which can hurt performance on data warehouses. It also discusses issues like not using huge pages, over or under use of indexing, too much parallelization, selecting the wrong disk types, failing to patch systems, and not implementing tools like Automatic Service Request and exachk. The document provides guidance on optimizing these areas to get the best performance from Exadata.

Accelerating hbase with nvme and bucket cache

David Grier

PGConf.ASIA 2019 Bali - Tune Your LInux Box, Not Just PostgreSQL - Ibrar Ahmed

Equnix Business Solutions

This document discusses tuning Linux and PostgreSQL for performance. It recommends: - Tuning Linux kernel parameters like huge pages, swappiness, and overcommit memory. Huge pages can improve TLB performance. - Tuning PostgreSQL parameters like shared_buffers, work_mem, and checkpoint_timeout. Shared_buffers stores the most frequently accessed data. - Other tips include choosing proper hardware, OS, and database based on workload. Tuning queries and applications can also boost performance.

Everyday I'm Scaling... Cassandra (Ben Bromhead, Instaclustr) | C* Summit 2016

DataStax

This presentation will show how create truly elastic Cassandra deployments on AWS allowing you to scale and shrink your large Cassandra deployments multiple times a day. Leveraging a combination of EBS backed disks, JBOD, token pinning and our previous work on bootstrapping from backups you will be able to dramatically reduce costs per cluster by scaling to match your daily workloads. Warning: This presentation will probably contain some references to late 2000's pop group LMFAO About the Speaker Ben Bromhead CTO, Instaclustr Ben Bromhead is the CTO of Instaclustr where he is responsible for working closely with his engineering team and customers to build highly available, scalable applications on top of Cassandra. Instaclustr is the only multi-cloud, self service Cassandra as a Service provider in the world and is dedicated to provider world class support.

Everyday I’m scaling... Cassandra

Instaclustr

Co-Founder and CTO of Instaclustr, Ben Bromhead's presentation at the Cassandra Summit 2016, in San Jose. This presentation will show how create truly elastic Cassandra deployments on AWS allowing you to scale and shrink your large Cassandra deployments multiple times a day. Leveraging a combination of EBS backed disks, JBOD, token pinning and our previous work on bootstrapping from backups you will be able to dramatically reduce costs per cluster by scaling to match your daily workloads.

Running & Scaling Large Elasticsearch Clusters

Fred de Villamil

Aerospike meetup july 2019 | Big Data Demystified

Omid Vahdaty

Building a low latency (sub millisecond), high throughput database that can handle big data AND linearly scale is not easy - but we did it anyway... In this session we will get to know Aerospike, an enterprise distributed primary key database solution. - We will do an introduction to Aerospike - basic terms, how it works and why is it widely used in mission critical systems deployments. - We will understand the 'magic' behind Aerospike ability to handle small, medium and even Petabyte scale data, and still guarantee predictable performance of sub-millisecond latency - We will learn how Aerospike devops is different than other solutions in the market, and see how easy it is to run it on cloud environments as well as on premise. We will also run a demo - showing a live example of the performance and self-healing technologies the database have to offer.

IMC Summit 2016 Breakout - Per Minoborg - Work with Multiple Hot Terabytes in...

In-Memory Computing Summit

Speedment SQL Reflector is a software solution that allows applications to get automatically updated data in real time. The SQL Reflector loads data from your existing SQL database and feeds it into an in-memory data grid e.g. GridGain. When started, the SQL reflector will load your selected existing relational data into your map cluster. Also, any subsequent changes that are made to the relational database (regardless how, via your application, script, SQL commands or even stored procedures) are then continuously fed to your GridGain nodes. Even SQL-transactions are preserved so that your maps will always reflect a valid state of the underlying SQL database.

AWS Summit London 2014 | Uses and Best Practices for Amazon Redshift (200)

Amazon Web Services

Amazon Redshift is a fully managed, petabyte-scale data warehouse service that makes it simple and cost-effective to analyze all your data using standard SQL and existing BI tools. It provides fast query performance by using columnar storage and data compression, automatic parallelization and distribution of data and queries, continuous backups and automated repairs. Customers use Redshift for analytics, business intelligence, and reporting on large datasets for use cases such as reducing costs, improving performance, and making more data available for analysis.

Storage and performance- Batch processing, Whiptail

Internet World

Batch processing allows jobs to run without manual intervention by shifting processing to less busy times. It avoids idling computing resources and allows higher overall utilization. Batch processing provides benefits like prioritizing batch and interactive work. The document then discusses different approaches to batch processing like dedicating all resources to it or sharing resources. It outlines challenges like systems being unavailable during batch processing. The rest of the document summarizes Whiptail's flash storage solutions for accelerating workloads and reducing costs and resources compared to HDDs.

505 kobal exadata

Kam Chan

The Exadata X3 introduces new hardware with dramatically more and faster flash memory, more DRAM memory, faster CPUs, and more connectivity while maintaining the same price as the previous Exadata X2 platform. Key software enhancements include Exadata Smart Flash Write Caching which provides up to 20 times more write I/O performance, and Hybrid Columnar Compression which now supports write-back caching and provides storage savings of up to 15 times. The Exadata X3 provides higher performance, more storage capacity, and lower power usage compared to previous Exadata platforms.

Accelerating HBase with NVMe and Bucket Cache

Nicolas Poggi

on-Volatile-Memory express (NVMe) standard promises and order of magnitude faster storage than regular SSDs, while at the same time being more economical than regular RAM on TB/$. This talk evaluates the use cases and benefits of NVMe drives for its use in Big Data clusters with HBase and Hadoop HDFS. First, we benchmark the different drives using system level tools (FIO) to get maximum expected values for each different device type and set expectations. Second, we explore the different options and use cases of HBase storage and benchmark the different setups. And finally, we evaluate the speedups obtained by the NVMe technology for the different Big Data use cases from the YCSB benchmark. In summary, while the NVMe drives show up to 8x speedup in best case scenarios, testing the cost-efficiency of new device technologies is not straightforward in Big Data, where we need to overcome system level caching to measure the maximum benefits.

Presentation database on flash

xKinAnx

Similar to CASSANDRA MEETUP - Choosing the right cloud instances for success (20)

M6d cassandrapresentation

Ceph on All Flash Storage -- Breaking Performance Barriers

Building Data Pipelines with SMACK: Designing Storage Strategies for Scale an...

stackArmor presentation for DevOpsDC ver 4

DevOps for ETL processing at scale with MongoDB, Solr, AWS and Chef

Sizing MongoDB on AWS with Wired Tiger-Patrick and Vigyan-Final

AWS Webcast - Cost and Performance Optimization in Amazon RDS

Colvin exadata mistakes_ioug_2014

Accelerating hbase with nvme and bucket cache

PGConf.ASIA 2019 Bali - Tune Your LInux Box, Not Just PostgreSQL - Ibrar Ahmed

Everyday I'm Scaling... Cassandra (Ben Bromhead, Instaclustr) | C* Summit 2016

Everyday I’m scaling... Cassandra

Running & Scaling Large Elasticsearch Clusters

Aerospike meetup july 2019 | Big Data Demystified

IMC Summit 2016 Breakout - Per Minoborg - Work with Multiple Hot Terabytes in...

AWS Summit London 2014 | Uses and Best Practices for Amazon Redshift (200)

Storage and performance- Batch processing, Whiptail

505 kobal exadata

Accelerating HBase with NVMe and Bucket Cache

Presentation database on flash

Recently uploaded

Ocean lotus Threat actors project by John Sitima 2024 (1).pptx

SitimaJohn

Ocean Lotus cyber threat actors represent a sophisticated, persistent, and politically motivated group that poses a significant risk to organizations and individuals in the Southeast Asian region. Their continuous evolution and adaptability underscore the need for robust cybersecurity measures and international cooperation to identify and mitigate the threats posed by such advanced persistent threat groups.

UiPath Test Automation using UiPath Test Suite series, part 6

DianaGray10

Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI. UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities. Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes. What will you get from this session? 1. Insights into integrating generative AI. 2. Understanding how this integration enhances test automation within the UiPath platform 3. Practical demonstrations 4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath Topics covered: What is generative AI Test Automation with generative AI and Open AI. UiPath integration with generative AI Speaker: Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP

Digital Marketing Trends in 2024 | Guide for Staying Ahead

Wask

https://www.wask.co/ebooks/digital-marketing-trends-in-2024 Feeling lost in the digital marketing whirlwind of 2024? Technology is changing, consumer habits are evolving, and staying ahead of the curve feels like a never-ending pursuit. This e-book is your compass. Dive into actionable insights to handle the complexities of modern marketing. From hyper-personalization to the power of user-generated content, learn how to build long-term relationships with your audience and unlock the secrets to success in the ever-shifting digital landscape.

Driving Business Innovation: Latest Generative AI Advancements & Success Story

Safe Software

Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency. During the hour, we’ll take you through: Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board. Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes. Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI. We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI. This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!

HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU

panagenda

Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/ DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen! Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell. Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten. Diese Themen werden behandelt - Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten - Wie funktionieren CCB- und CCX-Lizenzen wirklich? - Verstehen des DLAU-Tools und wie man es am besten nutzt - Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw. - Praxisbeispiele und Best Practices zum sofortigen Umsetzen

Serial Arm Control in Real Time Presentation

tolgahangng

How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf

Chart Kalyan

みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...

名前です男

Fueling AI with Great Data with Airbyte Webinar

Zilliz

Artificial Intelligence for XMLDevelopment

Octavian Nadolu

In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject. We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup. Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved. The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring. The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise. By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.

Building Production Ready Search Pipelines with Spark and Milvus

Zilliz

5th LF Energy Power Grid Model Meet-up Slides

DanBrown980551

5th Power Grid Model Meet-up It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology. Power Grid Model The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services. Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability. Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization. What to expect For the upcoming meetup we are organizing, we have an exciting lineup of activities planned: -Insightful presentations covering two practical applications of the Power Grid Model. -An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024. -An interactive brainstorming session to discuss and propose new feature requests. -An opportunity to connect with fellow Power Grid Model enthusiasts and users.

Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf

Malak Abu Hammad

Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers: * What is Vector Search? * Importance and benefits of vector search * Practical use cases across various industries * Step-by-step implementation guide * Live demos with code snippets * Enhancing LLM capabilities with vector search * Best practices and optimization strategies Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications. #MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology

GenAI Pilot Implementation in the organizations

kumardaparthi1024

Choosing The Best AWS Service For Your Website + API.pptx

Brandon Minnick, MBA

Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API? Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose? Which one is cheapest? Which one is fastest? Which one will scale to meet our needs? Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!

20240607 QFM018 Elixir Reading List May 2024

Matthew Sinclair

Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...

Jeffrey Haguewood

Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows. We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases. This video focuses on integration of Salesforce with Bonterra Impact Management. Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.

WeTestAthens: Postman's AI & Automation Techniques

Postman

Taking AI to the Next Level in Manufacturing.pdf

ssuserfac0301

Read Taking AI to the Next Level in Manufacturing to gain insights on AI adoption in the manufacturing industry, such as: 1. How quickly AI is being implemented in manufacturing. 2. Which barriers stand in the way of AI adoption. 3. How data quality and governance form the backbone of AI. 4. Organizational processes and structures that may inhibit effective AI adoption. 6. Ideas and approaches to help build your organization's AI strategy.

Webinar: Designing a schema for a Data Warehouse

Federico Razzoli

Are you new to data warehouses (DWH)? Do you need to check whether your data warehouse follows the best practices for a good design? In both cases, this webinar is for you. A data warehouse is a central relational database that contains all measurements about a business or an organisation. This data comes from a variety of heterogeneous data sources, which includes databases of any type that back the applications used by the company, data files exported by some applications, or APIs provided by internal or external services. But designing a data warehouse correctly is a hard task, which requires gathering information about the business processes that need to be analysed in the first place. These processes must be translated into so-called star schemas, which means, denormalised databases where each table represents a dimension or facts. We will discuss these topics: - How to gather information about a business; - Understanding dictionaries and how to identify business entities; - Dimensions and facts; - Setting a table granularity; - Types of facts; - Types of dimensions; - Snowflakes and how to avoid them; - Expanding existing dimensions and facts.

Recently uploaded (20)

Ocean lotus Threat actors project by John Sitima 2024 (1).pptx

UiPath Test Automation using UiPath Test Suite series, part 6

Digital Marketing Trends in 2024 | Guide for Staying Ahead

Driving Business Innovation: Latest Generative AI Advancements & Success Story

HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU

Serial Arm Control in Real Time Presentation

How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf

Fueling AI with Great Data with Airbyte Webinar

Artificial Intelligence for XMLDevelopment

Building Production Ready Search Pipelines with Spark and Milvus

5th LF Energy Power Grid Model Meet-up Slides

Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf

GenAI Pilot Implementation in the organizations

Choosing The Best AWS Service For Your Website + API.pptx

20240607 QFM018 Elixir Reading List May 2024

Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...

WeTestAthens: Postman's AI & Automation Techniques

Taking AI to the Next Level in Manufacturing.pdf

Webinar: Designing a schema for a Data Warehouse

CASSANDRA MEETUP - Choosing the right cloud instances for success

2. Welcome • Your app in focus — reads vs writes, CPU vs RAM • What IOPS? How much is enough? • Are ephemeral disks evil? • False economy — cheaper instances can cost you more • A time to kill — they’re not your pets

7. Tailor to workload • intimately understand your app • reads vs writes • CPU vs memory • OLTP vs OLAP • use case will dictate requirements

9. © DataStax, All Rights Reserved. EBS gp2 SSDs 9 • general purpose EBS option • persistent (durable) • default volume for EC2 instances • guaranteed 99% single-digit millisecond latency • only pay for each GB (IOPS included) • minimum 10K IOPS for production workloads 3 IOPS/GB (3K IOPS/TB) Max 10K IOPS/vol Max 160MB/s throughput/vol 1TB = $122/mo, $1474/yr

10. © DataStax, All Rights Reserved. EBS io1 SSDs 10 • fastest available EBS option • persistent (durable) • for latency-sensitive OLTP workloads • guaranteed 99.9%* single-digit millisecond latency • provisioned IOPS are charged extra • minimum 10K IOPS for production workloads * read the fine print Up to 50 IOPS/GB Max 20K IOPS/vol Max 320MB/s throughput/vol 1TB = $141/mo, $1695/yr 1K IOPS = $72/mo, $864/yr

12. Ephemeral storage • performance orders of magnitude better than EBS • already included in instance costs, e.g. m3, c3, i3 • “physically” attached • not durable across reboots but…

14. What is Cassandra • massively scalable NoSQL database • fully distributed, no single-point-of-failure • linear horizontal scaling

15. © DataStax, All Rights Reserved. Why Cassandra 15 • all nodes are the same — no SPOF • real-time, durable writes • linear scaling on commodity servers • real-time replication across data centres • always on — no offline operation • because you have a scale problem

18. © DataStax, All Rights Reserved. Real example 18 • deployed on c4.4xlarge • using EBS io1 with 3K PIOPS • nodes dropping writes • high read latencies 16 vCPU, 30GB RAM Instance $ 5443 EBS io1 1TB $ 1695 PIOPS 3K $ 2592 ———————— Annual cost $ 9730

19. © DataStax, All Rights Reserved. Recommendation 19 • swap to i3.2xlarge • 1.9TB NVMe SSDs included • 3M IOPS, 16GB/s • 60-70% cheaper than replaced i2.2xlarge 8 vCPU, 61GB RAM Instance $ 4174 ———————— Annual cost $ 4174

21. © DataStax, All Rights Reserved. Use case - dev, light prod 21 • m3.large suitable • entry-level load, testing-the-waters • minimum 3 C* nodes with RF=3 • use CMS GC with 2GB heap 2 vCPU 7.5GB RAM 1 x 32GB SSD $ 962/yr

22. © DataStax, All Rights Reserved. Use case - low prod volume 22 • m3.xlarge suitable • JVM will perform better with the extra RAM • min 3 C* nodes with RF=3 • use CMS GC with 8GB heap 4 vCPU 15GB RAM 1 x 40GB SSD $ 1924/yr

23. © DataStax, All Rights Reserved. Use case - moderate prod volume 23 • c3.2xlarge recommended • more diskspace, extra cores a bonus • costs 50% more for 2x CPU and 4x diskspace • min 3 C* nodes with RF=3 • use CMS GC with 8GB heap 8 vCPU 15GB RAM 2 x 80GB SSD $ 2916/yr

24. © DataStax, All Rights Reserved. Use case - real prod volume 24 • i3.2xlarge recommended • will handle all kinds of workloads including Analytics, Graph and Search (Solr) • min 3 C* nodes with RF=3 • use G1 GC with 24GB heap (32GB for Search nodes) 8 vCPU 61GB RAM 1.9TB NVMe SSD $ 4174/yr

CASSANDRA MEETUP - Choosing the right cloud instances for success

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to CASSANDRA MEETUP - Choosing the right cloud instances for success

Similar to CASSANDRA MEETUP - Choosing the right cloud instances for success (20)

Recently uploaded

Recently uploaded (20)

CASSANDRA MEETUP - Choosing the right cloud instances for success