SlideShare a Scribd company logo
1 of 40
Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License
CASSANDRA JP 2015
PROBLEM FREE
CASSANDRA UPGRADES
Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License
Aaron Morton
@aaronmorton
Co-Founder &Team Member
AboutThe Last Pickle.
Work with clients to deliver and
improve Apache Cassandra based
solutions.
Based in New Zealand, USA,
Norway.
Why Upgrade?
When to Upgrade
Run Books
Upgrading from 1.2 to 2.0
Why Upgrade?
Bug Fixes
New Features
Performance Improvements
Why Upgrade?
Because it is easy.
Why Upgrade?
Upgrading is a series of short
term node failures.
Why Upgrade?
Cassandra is designed to
maintain availability when
nodes fail.
Why Upgrade?
When to Upgrade
Run Books
Upgrading from 1.2 to 2.0
WhenTo Upgrade
Upgrade before you need to
upgrade.
WhenTo Upgrade
Upgrade during business
hours when people are at
their best.
WhenTo Upgrade
If you are a startup upgrade to
whatever release gives you
features you need.
(Make sure you test.)
WhenTo Upgrade
If you are a mature company
generally avoid the first 6
revisions of a major release.
(Still do testing.)
WhenTo Upgrade
Ideally stay 1 or 2 revisions
behind on the major version.
Why Upgrade?
When to Upgrade
Run Books
Upgrading from 1.2 to 2.0
Run Books
Why are we doing this?
What are we doing?
How will we do it?
Run Books
Phase changes into chunks.
Describe the state of the
system at the end of the Phase
Phases
Tasks that have checks and
rollback.
cut-and-paste actions for
operators.
Run Book Sections
Overview
News
Changes
Tickets
Configuration
Schema
Run Book Sections
Current Situation
Assumptions
Risks
Phases
Why Upgrade?
When to Upgrade
Run Books
Upgrading from 1.2 to 2.0
TLP Run Book
A detailed Run Book for this
upgrade is available at
http://thelastpickle.com/blog/
Overview
Prepare the cluster.
To each node in turn:
Shutdown then node.
Upgrade the installation.
Restart the node.
Upgrade on disk files.
Cleanup.
cassandra.yaml
(There are more changes.)
Setting Change
reduce_cache_sizes_at Setting has been removed.
reduce_cache_capacity_to Setting has been removed.
flush_largest_memtables_at Setting has been removed.
native_transport_min_threads Setting has been removed.
Schema Change
Setting Change
read_repair_chance The default has changed from 0.1 to 0.0.
local_read_repair_chance The default has changed from 0.0 to 0.1.
News
Java 7 Required
Cassandra 1.2.9 Required
Blob Formats
New LCS File Size
Speculative Retry Added
Configurable Metrics
Current Situation
nodetool status
nodetool version
nodetool gossipinfo
&& date
Assumptions
Disk Space
No ERRORs log messages
Gossip Stable
No Dropped Messages
Backups Disabled
Repair Disabled
Assumptions
On Disk File Format
MinimumVersion
Monitoring
Blob Constants
Availability
Risks
Latency
Availability
Incompatibility
Disk Space
Phase 0. Check Assumptions
Ensure the cluster works the
way we think it does.
Phase 1. Prepare the cluster
Snapshot
Update package repositories
Disable Hinted Handoff
Phase 2. Upgrade First Node to 2.0.14
All nodes UP
Graceful shutdown
Clear CommitLog
Upgrade Package
Phase 2. Upgrade First Node to 2.0.14
Update cassandra.yaml
Start Node
Tail system.log
watch nodetool tpstats
Phase 2. Upgrade First Node to 2.0.14
Rollback (if needed)
Backup 2.0 SSTables.
Delete all data.
Restore from snapshot.
Repair.
Phase 3. Upgrade Remaining Nodes to 2.0.14
Ensure all nodes are UP
Rolling upgrade
Phase 4. Cleanup
Enabled Hinted Handoff
Phase 5. Upgrade SSTables
nodetool clearsnapshot -t pre-2.0
nodetool upgradesstables
Phase 6. Restore Support Services
Enabled Backups
Enabled Repair
Thanks.
Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License
Aaron Morton
@aaronmorton
Co-Founder &Team Member
www.thelastpickle.com
Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License

More Related Content

More from datastaxjp

Db tech showcase 2016
Db tech showcase 2016Db tech showcase 2016
Db tech showcase 2016datastaxjp
 
Cassandra Meetup Tokyo, 2016 Spring
Cassandra Meetup Tokyo, 2016 SpringCassandra Meetup Tokyo, 2016 Spring
Cassandra Meetup Tokyo, 2016 Springdatastaxjp
 
Cassandra Meetup Tokyo, 2016 Spring 2
Cassandra Meetup Tokyo, 2016 Spring 2Cassandra Meetup Tokyo, 2016 Spring 2
Cassandra Meetup Tokyo, 2016 Spring 2datastaxjp
 
検索エンジンPatheeがAzureとCassandraをどう利用しているのか
検索エンジンPatheeがAzureとCassandraをどう利用しているのか検索エンジンPatheeがAzureとCassandraをどう利用しているのか
検索エンジンPatheeがAzureとCassandraをどう利用しているのかdatastaxjp
 
(LT)Spark and Cassandra
(LT)Spark and Cassandra(LT)Spark and Cassandra
(LT)Spark and Cassandradatastaxjp
 
SparkとCassandraの美味しい関係
SparkとCassandraの美味しい関係SparkとCassandraの美味しい関係
SparkとCassandraの美味しい関係datastaxjp
 
Cassandra v3.0 at Rakuten meet-up on 12/2/2015
Cassandra v3.0 at Rakuten meet-up on 12/2/2015Cassandra v3.0 at Rakuten meet-up on 12/2/2015
Cassandra v3.0 at Rakuten meet-up on 12/2/2015datastaxjp
 
Investigation of Transactions in Cassandra
Investigation of Transactions in CassandraInvestigation of Transactions in Cassandra
Investigation of Transactions in Cassandradatastaxjp
 
Cassandra summit 2015 レポート
Cassandra summit 2015 レポートCassandra summit 2015 レポート
Cassandra summit 2015 レポートdatastaxjp
 
Cassandra Meetup Tokyo, 2015 Summer
Cassandra Meetup Tokyo, 2015 SummerCassandra Meetup Tokyo, 2015 Summer
Cassandra Meetup Tokyo, 2015 Summerdatastaxjp
 
Cassandra and Spark
Cassandra and Spark Cassandra and Spark
Cassandra and Spark datastaxjp
 
[Cassandra summit Tokyo, 2015] Apache Cassandra日本人コミッターが伝える、"Apache Cassandra...
[Cassandra summit Tokyo, 2015] Apache Cassandra日本人コミッターが伝える、"Apache Cassandra...[Cassandra summit Tokyo, 2015] Apache Cassandra日本人コミッターが伝える、"Apache Cassandra...
[Cassandra summit Tokyo, 2015] Apache Cassandra日本人コミッターが伝える、"Apache Cassandra...datastaxjp
 
[Cassandra summit Tokyo, 2015] Cassandra 2015 最新情報 by ジョナサン・エリス(Jonathan Ellis)
[Cassandra summit Tokyo, 2015] Cassandra 2015 最新情報 by ジョナサン・エリス(Jonathan Ellis)[Cassandra summit Tokyo, 2015] Cassandra 2015 最新情報 by ジョナサン・エリス(Jonathan Ellis)
[Cassandra summit Tokyo, 2015] Cassandra 2015 最新情報 by ジョナサン・エリス(Jonathan Ellis)datastaxjp
 
[db tech showcase Tokyo 2015] E35: Web, IoT, モバイル時代のデータベース、Apache Cassandraを学ぼう
[db tech showcase Tokyo 2015] E35: Web, IoT, モバイル時代のデータベース、Apache Cassandraを学ぼう[db tech showcase Tokyo 2015] E35: Web, IoT, モバイル時代のデータベース、Apache Cassandraを学ぼう
[db tech showcase Tokyo 2015] E35: Web, IoT, モバイル時代のデータベース、Apache Cassandraを学ぼうdatastaxjp
 
[db tech showcase Tokyo 2015] A27: RDBエンジニアの為のNOSQL, 今どうしてNOSQLなのか?
[db tech showcase Tokyo 2015] A27: RDBエンジニアの為のNOSQL, 今どうしてNOSQLなのか?[db tech showcase Tokyo 2015] A27: RDBエンジニアの為のNOSQL, 今どうしてNOSQLなのか?
[db tech showcase Tokyo 2015] A27: RDBエンジニアの為のNOSQL, 今どうしてNOSQLなのか?datastaxjp
 

More from datastaxjp (15)

Db tech showcase 2016
Db tech showcase 2016Db tech showcase 2016
Db tech showcase 2016
 
Cassandra Meetup Tokyo, 2016 Spring
Cassandra Meetup Tokyo, 2016 SpringCassandra Meetup Tokyo, 2016 Spring
Cassandra Meetup Tokyo, 2016 Spring
 
Cassandra Meetup Tokyo, 2016 Spring 2
Cassandra Meetup Tokyo, 2016 Spring 2Cassandra Meetup Tokyo, 2016 Spring 2
Cassandra Meetup Tokyo, 2016 Spring 2
 
検索エンジンPatheeがAzureとCassandraをどう利用しているのか
検索エンジンPatheeがAzureとCassandraをどう利用しているのか検索エンジンPatheeがAzureとCassandraをどう利用しているのか
検索エンジンPatheeがAzureとCassandraをどう利用しているのか
 
(LT)Spark and Cassandra
(LT)Spark and Cassandra(LT)Spark and Cassandra
(LT)Spark and Cassandra
 
SparkとCassandraの美味しい関係
SparkとCassandraの美味しい関係SparkとCassandraの美味しい関係
SparkとCassandraの美味しい関係
 
Cassandra v3.0 at Rakuten meet-up on 12/2/2015
Cassandra v3.0 at Rakuten meet-up on 12/2/2015Cassandra v3.0 at Rakuten meet-up on 12/2/2015
Cassandra v3.0 at Rakuten meet-up on 12/2/2015
 
Investigation of Transactions in Cassandra
Investigation of Transactions in CassandraInvestigation of Transactions in Cassandra
Investigation of Transactions in Cassandra
 
Cassandra summit 2015 レポート
Cassandra summit 2015 レポートCassandra summit 2015 レポート
Cassandra summit 2015 レポート
 
Cassandra Meetup Tokyo, 2015 Summer
Cassandra Meetup Tokyo, 2015 SummerCassandra Meetup Tokyo, 2015 Summer
Cassandra Meetup Tokyo, 2015 Summer
 
Cassandra and Spark
Cassandra and Spark Cassandra and Spark
Cassandra and Spark
 
[Cassandra summit Tokyo, 2015] Apache Cassandra日本人コミッターが伝える、"Apache Cassandra...
[Cassandra summit Tokyo, 2015] Apache Cassandra日本人コミッターが伝える、"Apache Cassandra...[Cassandra summit Tokyo, 2015] Apache Cassandra日本人コミッターが伝える、"Apache Cassandra...
[Cassandra summit Tokyo, 2015] Apache Cassandra日本人コミッターが伝える、"Apache Cassandra...
 
[Cassandra summit Tokyo, 2015] Cassandra 2015 最新情報 by ジョナサン・エリス(Jonathan Ellis)
[Cassandra summit Tokyo, 2015] Cassandra 2015 最新情報 by ジョナサン・エリス(Jonathan Ellis)[Cassandra summit Tokyo, 2015] Cassandra 2015 最新情報 by ジョナサン・エリス(Jonathan Ellis)
[Cassandra summit Tokyo, 2015] Cassandra 2015 最新情報 by ジョナサン・エリス(Jonathan Ellis)
 
[db tech showcase Tokyo 2015] E35: Web, IoT, モバイル時代のデータベース、Apache Cassandraを学ぼう
[db tech showcase Tokyo 2015] E35: Web, IoT, モバイル時代のデータベース、Apache Cassandraを学ぼう[db tech showcase Tokyo 2015] E35: Web, IoT, モバイル時代のデータベース、Apache Cassandraを学ぼう
[db tech showcase Tokyo 2015] E35: Web, IoT, モバイル時代のデータベース、Apache Cassandraを学ぼう
 
[db tech showcase Tokyo 2015] A27: RDBエンジニアの為のNOSQL, 今どうしてNOSQLなのか?
[db tech showcase Tokyo 2015] A27: RDBエンジニアの為のNOSQL, 今どうしてNOSQLなのか?[db tech showcase Tokyo 2015] A27: RDBエンジニアの為のNOSQL, 今どうしてNOSQLなのか?
[db tech showcase Tokyo 2015] A27: RDBエンジニアの為のNOSQL, 今どうしてNOSQLなのか?
 

Recently uploaded

2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdfThe Good Food Institute
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveKeep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveIES VE
 
LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0DanBrown980551
 
20140402 - Smart house demo kit
20140402 - Smart house demo kit20140402 - Smart house demo kit
20140402 - Smart house demo kitJamie (Taka) Wang
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingFrancesco Corti
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarThousandEyes
 
Graphene Quantum Dots-Based Composites for Biomedical Applications
Graphene Quantum Dots-Based Composites for  Biomedical ApplicationsGraphene Quantum Dots-Based Composites for  Biomedical Applications
Graphene Quantum Dots-Based Composites for Biomedical Applicationsnooralam814309
 
Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...DianaGray10
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Libraryshyamraj55
 
My key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIMy key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIVijayananda Mohire
 
Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FESTBillieHyde
 
UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2DianaGray10
 
March Patch Tuesday
March Patch TuesdayMarch Patch Tuesday
March Patch TuesdayIvanti
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
 
Scenario Library et REX Discover industry- and role- based scenarios
Scenario Library et REX Discover industry- and role- based scenariosScenario Library et REX Discover industry- and role- based scenarios
Scenario Library et REX Discover industry- and role- based scenariosErol GIRAUDY
 
Extra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdfExtra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdfInfopole1
 
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox
 
.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptx.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptxHansamali Gamage
 
Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.IPLOOK Networks
 

Recently uploaded (20)

2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveKeep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
 
LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0
 
20140402 - Smart house demo kit
20140402 - Smart house demo kit20140402 - Smart house demo kit
20140402 - Smart house demo kit
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is going
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? Webinar
 
Graphene Quantum Dots-Based Composites for Biomedical Applications
Graphene Quantum Dots-Based Composites for  Biomedical ApplicationsGraphene Quantum Dots-Based Composites for  Biomedical Applications
Graphene Quantum Dots-Based Composites for Biomedical Applications
 
Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Library
 
My key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIMy key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAI
 
Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FEST
 
UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2
 
March Patch Tuesday
March Patch TuesdayMarch Patch Tuesday
March Patch Tuesday
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
Scenario Library et REX Discover industry- and role- based scenarios
Scenario Library et REX Discover industry- and role- based scenariosScenario Library et REX Discover industry- and role- based scenarios
Scenario Library et REX Discover industry- and role- based scenarios
 
Extra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdfExtra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdf
 
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
 
.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptx.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptx
 
Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.
 

[Cassandra summit Tokyo, 2015] Problem free Cassandra upgrades by Aaron Morton