SlideShare a Scribd company logo
Meeting the Challenges of
Archival Storage
Your not so Humble Speaker
• 25+ years of consulting and
writing for trade press
• Columnist/blogger at
NetworkComputing .com
• Chief Scientist DeepStorage, LLC.
• Independent test lab and
analysts
• @DeepStorageNet on Twitter
• Hmarks@DeepStorage.Net
The Data We’re Drowning In is
Archival
• Long-Tail data
– Richer and richer media
• Media and entertainment
• Video surveillance
• Medical imaging
– New paths to value
• Big data analytics
• Instrumentation data
The Storage Market Today
• Performance storage
– Designed for high IOPS/low latency
– High cost/GB
• Storage for backups
– Deduplication driven
– Cost very data dependent
• Object Storage
– Economical at scale
– Requires application support
What Do We Mean By “Archival”
• Many sources
– Video and other media repositories
– Medical imaging
– Instrumentation and log files
– Email or business archives
• Large file storage repositories
– 100MB
– Written and read sequentially or even get/put
• Low access frequency
– But sub-second latency
Archives & Most Storage Don’t
Fit
• Primary storage is too
expensive
– Bandwidth more important that
IOPS
• Dedupeing appliances are
for backups
– Archives already single
instanced
– Media files don’t dedupe
• Object storage needs new
applications
Archive Storage Requirements
• Affordability
– Large quantities * long retention = many TB
– Storage efficiency is key
• Standard interfaces
– Many mixed application workflows
• High reliability
– RAID-6 at a minimum
• Ensure data Integrity
The Data Integrity
• RAID only insures against drive failures
– Drive reliability 1014 to 1015 bits
– 8TB disk=6.4x1013
– Error expected rebuilding 12+2 array
• Undetected corruption on about 3% of
drives*
– Error rate increases over time
* http://research.cs.wisc.edu/wind/Publications/latent-sigmetrics07.pdf
The Solution
• Calculate a hash with each block written
• Store hash separately from data
– Drives sometimes just write the wrong sector
• Verify via hash on reads
– Reconstitute data from parity data on failure
• Scrub and rebuild all data periodically
nTier Verde –
Simply Affordable File Storage
• No previous storage experience required
• Half the cost of traditional file storage
• Never lose your data
• Effortless Setup
– 30 minutes from box
to production!
– Dashboard style
management with
single screen view
– Manage/monitor
Verde remotely with
smart phone interface
No Previous Storage Experience
Required
• Store More, Spend Less
– List price as low as 35¢ per GB
– No software fees
• Compression, Snapshots and Replication* included
• Quick Installation
– PS dollars focus on solution, not install
• PriceLock Support Pricing
– Avoid the support hockey stick
– Will not increase LIST price more than Consumer Price
Index
Half The Cost Of Traditional File
Storage
* Future Release
• Software Eliminates Data Corruption and Loss
– Advanced checksums and multiple parity options
– All writes committed instantaneously
• Rock Solid Hardware
– Highest quality & capacity SAS drives made
• 4TB ~ (6TB)Future
• 7200 RPM
• SAS 6Gb/s
• 24x7 Duty Cycle
• Few Affordable Arrays Offer This Caliber of Drive
Never Lose Your Data
nTier Verde 2U nTier Verde 4U
Supported Hard Drives 4 TB, 7200 RPM, SAS Hard Drives
Master Node Capacity* 6 to 11 Drives
24 TB to 44 TB RAW
10 to 35 Drives
40 TB to 140 TB RAW
Expansion Node Support 1 Expansion Node 9 Expansion Nodes
Max Capacity* 220 TB RAW Max 1.7 PB RAW Max
Parity Options Mirroring, Single Parity, Double Parity, Triple Parity
Software Compression, Thin Provisioning, Snapshots
Hot Spares Unlimited
Supported Protocols NFS, CIFS
3 x 1 GigE Data Ports Standard Standard
2 x 10 GigE Data Ports Optional Standard
nTier Verde
Testing the Verde
• nTier Verde 4u
– 35x 4TB drives
• 10Gbps CIFS and
NFS
• Tested with
– Commvault Simpana
– Veeam B&R
– IOmeter
12UUID
RESETR OM
Virtual Server Hosts
UPERMICRS
UPERMICRS
B
POWER
UID
POWER
UID
A
Veeam/CommVault Server
T540
12UUID
RESETR OM
12UUID
RESETR OM
12UUID
RESETR OM
Supermicro
6016T-NTRF
Brocade 8000
10Gbps Ethernet Switch
Spectra nTier Verde Tintri T540
(VM Storage)
IOmeter Results
0
100
200
300
400
500
600
700
800
900
1000
Backup Writer Backup Reader 8K OLTP 2MB Read 2MB Write
Miror RAID-5 Z2 Z3 Z3 w/compression
Multiple Stream File Transfers
0
10000
20000
30000
40000
50000
60000
70000
80000
1 2 4 8 16
Write Read
Wrapping Up
• Spectra’s nTier Verde meets the archival
test
– High reliability and efficiency via triple parity
RAID
– High capacity nearline disks
– Hashes to ensure data integrity
– Optional compression
– Simple web administration
– Throughput at near line rate
Meeting the Challenges of Archival Storage

More Related Content

What's hot

Ambedded - how to build a true no single point of failure ceph cluster
Ambedded - how to build a true no single point of failure ceph cluster Ambedded - how to build a true no single point of failure ceph cluster
Ambedded - how to build a true no single point of failure ceph cluster
inwin stack
 
test
testtest
test
EMCRyan
 
Ceph Day Seoul - The Anatomy of Ceph I/O
Ceph Day Seoul - The Anatomy of Ceph I/OCeph Day Seoul - The Anatomy of Ceph I/O
Ceph Day Seoul - The Anatomy of Ceph I/O
Ceph Community
 
Arm - ceph on arm update
Arm - ceph on arm updateArm - ceph on arm update
Arm - ceph on arm update
inwin stack
 
Ceph Day KL - Ceph on All-Flash Storage
Ceph Day KL - Ceph on All-Flash Storage Ceph Day KL - Ceph on All-Flash Storage
Ceph Day KL - Ceph on All-Flash Storage
Ceph Community
 
BDAM: Big Data Asset Management
BDAM: Big Data Asset ManagementBDAM: Big Data Asset Management
BDAM: Big Data Asset Management
Perforce
 
Ceph at Spreadshirt (June 2016)
Ceph at Spreadshirt (June 2016)Ceph at Spreadshirt (June 2016)
Ceph at Spreadshirt (June 2016)
Jens Hadlich
 
Build a High Available NFS Cluster Based on CephFS - Shangzhong Zhu
Build a High Available NFS Cluster Based on CephFS - Shangzhong ZhuBuild a High Available NFS Cluster Based on CephFS - Shangzhong Zhu
Build a High Available NFS Cluster Based on CephFS - Shangzhong Zhu
Ceph Community
 
Qnap Product Selection Guide 2016
Qnap Product Selection Guide 2016Qnap Product Selection Guide 2016
Qnap Product Selection Guide 2016
Hernan Lopez
 
Providence net app upgrade plan PPMC
Providence net app upgrade plan PPMCProvidence net app upgrade plan PPMC
Providence net app upgrade plan PPMC
Accenture
 
Ceph's journey at SUSE
Ceph's journey at SUSECeph's journey at SUSE
Ceph's journey at SUSE
Ceph Community
 
inwinSTACK - ceph integrate with kubernetes
inwinSTACK - ceph integrate with kubernetesinwinSTACK - ceph integrate with kubernetes
inwinSTACK - ceph integrate with kubernetes
inwin stack
 
Ceph: Low Fail Go Scale
Ceph: Low Fail Go Scale Ceph: Low Fail Go Scale
Ceph: Low Fail Go Scale
Ceph Community
 
Ceph Object Storage at Spreadshirt
Ceph Object Storage at SpreadshirtCeph Object Storage at Spreadshirt
Ceph Object Storage at Spreadshirt
Jens Hadlich
 
Ceph on rdma
Ceph on rdmaCeph on rdma
Ceph on rdma
Somnath Roy
 
CC-4009, "Optimizing Hadoop Deployments with SeaMicro SM15000" by Satheesh Na...
CC-4009, "Optimizing Hadoop Deployments with SeaMicro SM15000" by Satheesh Na...CC-4009, "Optimizing Hadoop Deployments with SeaMicro SM15000" by Satheesh Na...
CC-4009, "Optimizing Hadoop Deployments with SeaMicro SM15000" by Satheesh Na...
AMD Developer Central
 

What's hot (16)

Ambedded - how to build a true no single point of failure ceph cluster
Ambedded - how to build a true no single point of failure ceph cluster Ambedded - how to build a true no single point of failure ceph cluster
Ambedded - how to build a true no single point of failure ceph cluster
 
test
testtest
test
 
Ceph Day Seoul - The Anatomy of Ceph I/O
Ceph Day Seoul - The Anatomy of Ceph I/OCeph Day Seoul - The Anatomy of Ceph I/O
Ceph Day Seoul - The Anatomy of Ceph I/O
 
Arm - ceph on arm update
Arm - ceph on arm updateArm - ceph on arm update
Arm - ceph on arm update
 
Ceph Day KL - Ceph on All-Flash Storage
Ceph Day KL - Ceph on All-Flash Storage Ceph Day KL - Ceph on All-Flash Storage
Ceph Day KL - Ceph on All-Flash Storage
 
BDAM: Big Data Asset Management
BDAM: Big Data Asset ManagementBDAM: Big Data Asset Management
BDAM: Big Data Asset Management
 
Ceph at Spreadshirt (June 2016)
Ceph at Spreadshirt (June 2016)Ceph at Spreadshirt (June 2016)
Ceph at Spreadshirt (June 2016)
 
Build a High Available NFS Cluster Based on CephFS - Shangzhong Zhu
Build a High Available NFS Cluster Based on CephFS - Shangzhong ZhuBuild a High Available NFS Cluster Based on CephFS - Shangzhong Zhu
Build a High Available NFS Cluster Based on CephFS - Shangzhong Zhu
 
Qnap Product Selection Guide 2016
Qnap Product Selection Guide 2016Qnap Product Selection Guide 2016
Qnap Product Selection Guide 2016
 
Providence net app upgrade plan PPMC
Providence net app upgrade plan PPMCProvidence net app upgrade plan PPMC
Providence net app upgrade plan PPMC
 
Ceph's journey at SUSE
Ceph's journey at SUSECeph's journey at SUSE
Ceph's journey at SUSE
 
inwinSTACK - ceph integrate with kubernetes
inwinSTACK - ceph integrate with kubernetesinwinSTACK - ceph integrate with kubernetes
inwinSTACK - ceph integrate with kubernetes
 
Ceph: Low Fail Go Scale
Ceph: Low Fail Go Scale Ceph: Low Fail Go Scale
Ceph: Low Fail Go Scale
 
Ceph Object Storage at Spreadshirt
Ceph Object Storage at SpreadshirtCeph Object Storage at Spreadshirt
Ceph Object Storage at Spreadshirt
 
Ceph on rdma
Ceph on rdmaCeph on rdma
Ceph on rdma
 
CC-4009, "Optimizing Hadoop Deployments with SeaMicro SM15000" by Satheesh Na...
CC-4009, "Optimizing Hadoop Deployments with SeaMicro SM15000" by Satheesh Na...CC-4009, "Optimizing Hadoop Deployments with SeaMicro SM15000" by Satheesh Na...
CC-4009, "Optimizing Hadoop Deployments with SeaMicro SM15000" by Satheesh Na...
 

Similar to Meeting the Challenges of Archival Storage

Is Disk Now a Viable Solution for Archive - Jon Toigo
Is Disk Now a Viable Solution for Archive - Jon ToigoIs Disk Now a Viable Solution for Archive - Jon Toigo
Is Disk Now a Viable Solution for Archive - Jon Toigo
spectralogic
 
2010 AIRI Petabyte Challenge - View From The Trenches
2010 AIRI Petabyte Challenge - View From The Trenches2010 AIRI Petabyte Challenge - View From The Trenches
2010 AIRI Petabyte Challenge - View From The Trenches
George Ang
 
Pros_and_Cons_of_DW_Apps pdf.pdf
Pros_and_Cons_of_DW_Apps pdf.pdfPros_and_Cons_of_DW_Apps pdf.pdf
Pros_and_Cons_of_DW_Apps pdf.pdf
HernanKlint
 
Backup and Archive Doesn't Have to be Complicated and Expensive
Backup and Archive Doesn't Have to be Complicated and ExpensiveBackup and Archive Doesn't Have to be Complicated and Expensive
Backup and Archive Doesn't Have to be Complicated and Expensive
spectralogic
 
Webinar: Cloud Storage: The 5 Reasons IT Can Do it Better
Webinar: Cloud Storage: The 5 Reasons IT Can Do it BetterWebinar: Cloud Storage: The 5 Reasons IT Can Do it Better
Webinar: Cloud Storage: The 5 Reasons IT Can Do it Better
Storage Switzerland
 
Four Assumptions Killing Backup Storage Webinar
Four Assumptions Killing Backup Storage WebinarFour Assumptions Killing Backup Storage Webinar
Four Assumptions Killing Backup Storage Webinar
Storage Switzerland
 
Managing Security At 1M Events a Second using Elasticsearch
Managing Security At 1M Events a Second using ElasticsearchManaging Security At 1M Events a Second using Elasticsearch
Managing Security At 1M Events a Second using Elasticsearch
Joe Alex
 
S016828 storage-tiering-nola-v1710b
S016828 storage-tiering-nola-v1710bS016828 storage-tiering-nola-v1710b
S016828 storage-tiering-nola-v1710b
Tony Pearson
 
Webinar: Getting Beyond Flash 101 - Flash 102 Selecting the Right Flash Array
Webinar: Getting Beyond Flash 101 - Flash 102 Selecting the Right Flash ArrayWebinar: Getting Beyond Flash 101 - Flash 102 Selecting the Right Flash Array
Webinar: Getting Beyond Flash 101 - Flash 102 Selecting the Right Flash Array
Storage Switzerland
 
Cloud computing UNIT 2.1 presentation in
Cloud computing UNIT 2.1 presentation inCloud computing UNIT 2.1 presentation in
Cloud computing UNIT 2.1 presentation in
RahulBhole12
 
SUSE Storage: Sizing and Performance (Ceph)
SUSE Storage: Sizing and Performance (Ceph)SUSE Storage: Sizing and Performance (Ceph)
SUSE Storage: Sizing and Performance (Ceph)
Lars Marowsky-Brée
 
Group-D ICT presentation updated.pptx
Group-D ICT presentation updated.pptxGroup-D ICT presentation updated.pptx
Group-D ICT presentation updated.pptx
20021519016
 
Meta scale kognitio hadoop webinar
Meta scale kognitio hadoop webinarMeta scale kognitio hadoop webinar
Meta scale kognitio hadoop webinar
Michael Hiskey
 
Where Is Your Data?: An Introduction to Problems and Bottlenecks in Data Systems
Where Is Your Data?: An Introduction to Problems and Bottlenecks in Data SystemsWhere Is Your Data?: An Introduction to Problems and Bottlenecks in Data Systems
Where Is Your Data?: An Introduction to Problems and Bottlenecks in Data Systems
InsightDataScience
 
Webinar: Performance vs. Cost - Solving The HPC Storage Tug-of-War
Webinar: Performance vs. Cost - Solving The HPC Storage Tug-of-WarWebinar: Performance vs. Cost - Solving The HPC Storage Tug-of-War
Webinar: Performance vs. Cost - Solving The HPC Storage Tug-of-War
Storage Switzerland
 
Meta scale kognitio hadoop webinar
Meta scale kognitio hadoop webinarMeta scale kognitio hadoop webinar
Meta scale kognitio hadoop webinar
Kognitio
 
Data management for Quantitative Biology -Basics and challenges in biomedical...
Data management for Quantitative Biology -Basics and challenges in biomedical...Data management for Quantitative Biology -Basics and challenges in biomedical...
Data management for Quantitative Biology -Basics and challenges in biomedical...
QBiC_Tue
 
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)
Lars Marowsky-Brée
 
Webinar: Sizing Up Object Storage for the Enterprise
Webinar: Sizing Up Object Storage for the EnterpriseWebinar: Sizing Up Object Storage for the Enterprise
Webinar: Sizing Up Object Storage for the Enterprise
Storage Switzerland
 
Elastic storage in the cloud session 5224 final v2
Elastic storage in the cloud session 5224 final v2Elastic storage in the cloud session 5224 final v2
Elastic storage in the cloud session 5224 final v2
BradDesAulniers2
 

Similar to Meeting the Challenges of Archival Storage (20)

Is Disk Now a Viable Solution for Archive - Jon Toigo
Is Disk Now a Viable Solution for Archive - Jon ToigoIs Disk Now a Viable Solution for Archive - Jon Toigo
Is Disk Now a Viable Solution for Archive - Jon Toigo
 
2010 AIRI Petabyte Challenge - View From The Trenches
2010 AIRI Petabyte Challenge - View From The Trenches2010 AIRI Petabyte Challenge - View From The Trenches
2010 AIRI Petabyte Challenge - View From The Trenches
 
Pros_and_Cons_of_DW_Apps pdf.pdf
Pros_and_Cons_of_DW_Apps pdf.pdfPros_and_Cons_of_DW_Apps pdf.pdf
Pros_and_Cons_of_DW_Apps pdf.pdf
 
Backup and Archive Doesn't Have to be Complicated and Expensive
Backup and Archive Doesn't Have to be Complicated and ExpensiveBackup and Archive Doesn't Have to be Complicated and Expensive
Backup and Archive Doesn't Have to be Complicated and Expensive
 
Webinar: Cloud Storage: The 5 Reasons IT Can Do it Better
Webinar: Cloud Storage: The 5 Reasons IT Can Do it BetterWebinar: Cloud Storage: The 5 Reasons IT Can Do it Better
Webinar: Cloud Storage: The 5 Reasons IT Can Do it Better
 
Four Assumptions Killing Backup Storage Webinar
Four Assumptions Killing Backup Storage WebinarFour Assumptions Killing Backup Storage Webinar
Four Assumptions Killing Backup Storage Webinar
 
Managing Security At 1M Events a Second using Elasticsearch
Managing Security At 1M Events a Second using ElasticsearchManaging Security At 1M Events a Second using Elasticsearch
Managing Security At 1M Events a Second using Elasticsearch
 
S016828 storage-tiering-nola-v1710b
S016828 storage-tiering-nola-v1710bS016828 storage-tiering-nola-v1710b
S016828 storage-tiering-nola-v1710b
 
Webinar: Getting Beyond Flash 101 - Flash 102 Selecting the Right Flash Array
Webinar: Getting Beyond Flash 101 - Flash 102 Selecting the Right Flash ArrayWebinar: Getting Beyond Flash 101 - Flash 102 Selecting the Right Flash Array
Webinar: Getting Beyond Flash 101 - Flash 102 Selecting the Right Flash Array
 
Cloud computing UNIT 2.1 presentation in
Cloud computing UNIT 2.1 presentation inCloud computing UNIT 2.1 presentation in
Cloud computing UNIT 2.1 presentation in
 
SUSE Storage: Sizing and Performance (Ceph)
SUSE Storage: Sizing and Performance (Ceph)SUSE Storage: Sizing and Performance (Ceph)
SUSE Storage: Sizing and Performance (Ceph)
 
Group-D ICT presentation updated.pptx
Group-D ICT presentation updated.pptxGroup-D ICT presentation updated.pptx
Group-D ICT presentation updated.pptx
 
Meta scale kognitio hadoop webinar
Meta scale kognitio hadoop webinarMeta scale kognitio hadoop webinar
Meta scale kognitio hadoop webinar
 
Where Is Your Data?: An Introduction to Problems and Bottlenecks in Data Systems
Where Is Your Data?: An Introduction to Problems and Bottlenecks in Data SystemsWhere Is Your Data?: An Introduction to Problems and Bottlenecks in Data Systems
Where Is Your Data?: An Introduction to Problems and Bottlenecks in Data Systems
 
Webinar: Performance vs. Cost - Solving The HPC Storage Tug-of-War
Webinar: Performance vs. Cost - Solving The HPC Storage Tug-of-WarWebinar: Performance vs. Cost - Solving The HPC Storage Tug-of-War
Webinar: Performance vs. Cost - Solving The HPC Storage Tug-of-War
 
Meta scale kognitio hadoop webinar
Meta scale kognitio hadoop webinarMeta scale kognitio hadoop webinar
Meta scale kognitio hadoop webinar
 
Data management for Quantitative Biology -Basics and challenges in biomedical...
Data management for Quantitative Biology -Basics and challenges in biomedical...Data management for Quantitative Biology -Basics and challenges in biomedical...
Data management for Quantitative Biology -Basics and challenges in biomedical...
 
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)
Modeling, estimating, and predicting Ceph (Linux Foundation - Vault 2015)
 
Webinar: Sizing Up Object Storage for the Enterprise
Webinar: Sizing Up Object Storage for the EnterpriseWebinar: Sizing Up Object Storage for the Enterprise
Webinar: Sizing Up Object Storage for the Enterprise
 
Elastic storage in the cloud session 5224 final v2
Elastic storage in the cloud session 5224 final v2Elastic storage in the cloud session 5224 final v2
Elastic storage in the cloud session 5224 final v2
 

Recently uploaded

“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
Claudio Di Ciccio
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
Adtran
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Neo4j
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
SOFTTECHHUB
 
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIEnchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Vladimir Iglovikov, Ph.D.
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
DianaGray10
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
Kumud Singh
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Zilliz
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Quotidiano Piemontese
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
mikeeftimakis1
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Neo4j
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
DianaGray10
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Paige Cruz
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Neo4j
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
Ana-Maria Mihalceanu
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
danishmna97
 

Recently uploaded (20)

“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
 
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIEnchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
 

Meeting the Challenges of Archival Storage

  • 1. Meeting the Challenges of Archival Storage
  • 2. Your not so Humble Speaker • 25+ years of consulting and writing for trade press • Columnist/blogger at NetworkComputing .com • Chief Scientist DeepStorage, LLC. • Independent test lab and analysts • @DeepStorageNet on Twitter • Hmarks@DeepStorage.Net
  • 3. The Data We’re Drowning In is Archival • Long-Tail data – Richer and richer media • Media and entertainment • Video surveillance • Medical imaging – New paths to value • Big data analytics • Instrumentation data
  • 4. The Storage Market Today • Performance storage – Designed for high IOPS/low latency – High cost/GB • Storage for backups – Deduplication driven – Cost very data dependent • Object Storage – Economical at scale – Requires application support
  • 5. What Do We Mean By “Archival” • Many sources – Video and other media repositories – Medical imaging – Instrumentation and log files – Email or business archives • Large file storage repositories – 100MB – Written and read sequentially or even get/put • Low access frequency – But sub-second latency
  • 6. Archives & Most Storage Don’t Fit • Primary storage is too expensive – Bandwidth more important that IOPS • Dedupeing appliances are for backups – Archives already single instanced – Media files don’t dedupe • Object storage needs new applications
  • 7. Archive Storage Requirements • Affordability – Large quantities * long retention = many TB – Storage efficiency is key • Standard interfaces – Many mixed application workflows • High reliability – RAID-6 at a minimum • Ensure data Integrity
  • 8. The Data Integrity • RAID only insures against drive failures – Drive reliability 1014 to 1015 bits – 8TB disk=6.4x1013 – Error expected rebuilding 12+2 array • Undetected corruption on about 3% of drives* – Error rate increases over time * http://research.cs.wisc.edu/wind/Publications/latent-sigmetrics07.pdf
  • 9. The Solution • Calculate a hash with each block written • Store hash separately from data – Drives sometimes just write the wrong sector • Verify via hash on reads – Reconstitute data from parity data on failure • Scrub and rebuild all data periodically
  • 10. nTier Verde – Simply Affordable File Storage • No previous storage experience required • Half the cost of traditional file storage • Never lose your data
  • 11. • Effortless Setup – 30 minutes from box to production! – Dashboard style management with single screen view – Manage/monitor Verde remotely with smart phone interface No Previous Storage Experience Required
  • 12. • Store More, Spend Less – List price as low as 35¢ per GB – No software fees • Compression, Snapshots and Replication* included • Quick Installation – PS dollars focus on solution, not install • PriceLock Support Pricing – Avoid the support hockey stick – Will not increase LIST price more than Consumer Price Index Half The Cost Of Traditional File Storage * Future Release
  • 13. • Software Eliminates Data Corruption and Loss – Advanced checksums and multiple parity options – All writes committed instantaneously • Rock Solid Hardware – Highest quality & capacity SAS drives made • 4TB ~ (6TB)Future • 7200 RPM • SAS 6Gb/s • 24x7 Duty Cycle • Few Affordable Arrays Offer This Caliber of Drive Never Lose Your Data
  • 14. nTier Verde 2U nTier Verde 4U Supported Hard Drives 4 TB, 7200 RPM, SAS Hard Drives Master Node Capacity* 6 to 11 Drives 24 TB to 44 TB RAW 10 to 35 Drives 40 TB to 140 TB RAW Expansion Node Support 1 Expansion Node 9 Expansion Nodes Max Capacity* 220 TB RAW Max 1.7 PB RAW Max Parity Options Mirroring, Single Parity, Double Parity, Triple Parity Software Compression, Thin Provisioning, Snapshots Hot Spares Unlimited Supported Protocols NFS, CIFS 3 x 1 GigE Data Ports Standard Standard 2 x 10 GigE Data Ports Optional Standard nTier Verde
  • 15. Testing the Verde • nTier Verde 4u – 35x 4TB drives • 10Gbps CIFS and NFS • Tested with – Commvault Simpana – Veeam B&R – IOmeter 12UUID RESETR OM Virtual Server Hosts UPERMICRS UPERMICRS B POWER UID POWER UID A Veeam/CommVault Server T540 12UUID RESETR OM 12UUID RESETR OM 12UUID RESETR OM Supermicro 6016T-NTRF Brocade 8000 10Gbps Ethernet Switch Spectra nTier Verde Tintri T540 (VM Storage)
  • 16. IOmeter Results 0 100 200 300 400 500 600 700 800 900 1000 Backup Writer Backup Reader 8K OLTP 2MB Read 2MB Write Miror RAID-5 Z2 Z3 Z3 w/compression
  • 17. Multiple Stream File Transfers 0 10000 20000 30000 40000 50000 60000 70000 80000 1 2 4 8 16 Write Read
  • 18. Wrapping Up • Spectra’s nTier Verde meets the archival test – High reliability and efficiency via triple parity RAID – High capacity nearline disks – Hashes to ensure data integrity – Optional compression – Simple web administration – Throughput at near line rate

Editor's Notes

  1. You shouldn’t have to be an expert to configure or manage a box in this space! Boxes in this space are configured and walked away from for months! You’ll never remember how to interface later… you don’t need to with our box
  2. Right scale MB/Min Tops out at 9.59Gbps