SlideShare a Scribd company logo
Quick FAQ
Quick FAQ
What is Erasure Coding and
how does it work?
What is Erasure Coding and how does it work?
What is Erasure Coding and how does it work?
Encoding
Decoding
Simple example for a 3/1 erasure encoding
What is Erasure Coding and how does it work?
Simple example for a 3/1 erasure encoding
• We solve for x:
OR
x+y=12 [+]
x-y= 2
2x =14 [∕2]
X = 7
2x+ y=19 [+]
x- y= 2
3x =21 [∕3]
X = 7
Encoding
Decoding
What is Erasure Coding and how does it work?
Simple example for a 3/1 erasure encoding
• We solve for x:
x+y=12 [+]
x-y= 2
2x =14 [∕2]
X = 7
Encoding
Decoding
What is Erasure Coding and how does it work?
Simple example for a 3/1 erasure encoding
• We solve for x:
OR
• Then we solve for y:
x+y=12 [+]
x-y= 2
2x =14 [∕2]
X = 7
2x+ y=19 [+]
x- y= 2
3x =21 [∕3]
X = 7
7+y=12 [-7]
y=5
Encoding
Decoding
What is Erasure Coding and how does it work?
Simple example for a 3/1 erasure encoding
• We solve for x:
OR
• Then we solve for y:
OR
x+y=12 [+]
x-y= 2
2x =14 [∕2]
X = 7
2x+ y=19 [+]
x- y= 2
3x =21 [∕3]
X = 7
7-y= 2 [-7]
-y=-5 [*-1]
y=5
7+y=12 [-7]
y=5
Encoding
Decoding
What is Erasure Coding and how does it work?
What is Erasure Coding and how does it work?
Erasure Coding is a data protection scheme that breaks data into shards (fragments) that
are encoded with parity (redundant data), and then stored across multiple storage media
and locations.
What is Erasure Coding and how does it work?
Erasure Coding is a data protection scheme that breaks data into shards (fragments) that
are encoded with parity (redundant data), and then stored across multiple storage media
and locations.
Why you should care
What is Erasure Coding and how does it work?
Erasure Coding is a data protection scheme that breaks data into shards (fragments) that
are encoded with parity (redundant data), and then stored across multiple storage media
and locations.
Why you should care
• You only need a subset of the shards to rehydrate data.
What is Erasure Coding and how does it work?
Erasure Coding is a data protection scheme that breaks data into shards (fragments) that
are encoded with parity (redundant data), and then stored across multiple storage media
and locations.
Why you should care
• You only need a subset of the shards to rehydrate data.
• You can replace failed components when convenient, without taking the system offline.
What is Erasure Coding and how does it work?
Erasure Coding is a data protection scheme that breaks data into shards (fragments) that
are encoded with parity (redundant data), and then stored across multiple storage media
and locations.
Why you should care
• You only need a subset of the shards to rehydrate data.
• You can replace failed components when convenient, without taking the system offline.
• You can reduce CAPEX and OPEX compared with mirroring/replication approaches.
Thanks for watching
Clay Ryder
DCS Marketing
itblog.sandisk.com/author/clayryder
© 2017 Western Digital Corporation. All rights reserved. Western Digital and the Western Digital Logo are registered trademarks of Western Digital Corporation
or its affiliates in the U.S. and/or other countries. All other marks are the property of their respective owners.
"Faster Does It" Kevin MacLeod (incompetech.com) — Licensed under Creative Commons: By Attribution 3.0 http://creativecommons.org/licenses/by/3.0/
@WesternDigiDC
SanDisk Data Center Solutions
@BigDataFlash
HGST, a Western Digital brand
What is Erasure Coding and how does it work?
Erasure Coding encodes data and compartmentalizes
it such that only a subset of the pieces are required to
recreate the original information.
What is Erasure Coding and how does it work?
Erasure Coding encodes data and compartmentalizes
it such that only a subset of the pieces are required to
recreate the original information.
For example:
5/2 encoding requires (5-2) of 5 pieces to rehydrate it
10/3 encoding requires (10-3) of 10 pieces
18/5 requires (18-5) of 18
etc.
What is Erasure Coding and how does it work?
Simple example for a 18/5 erasure encoding
• We solve for x:
OR
• Then we solve for y:
OR
x+y=12 [+]
x-y= 2
2x =14 [∕2]
X = 7
2x+ y=19 [+]
x- y= 2
3x =21 [∕3]
X = 7
7-y= 2 [-7]
-y=-5 [*-1]
y=5
7+y=12 [-7]
y=5
Encoding
Decoding
… …1 182
Any 13 of 18
equations to
decode object
Thanks for watching
Clay Ryder
DCS Marketing
itblog.sandisk.com/author/clayryder

More Related Content

What's hot

Raid
RaidRaid
Raid
dinaselim
 
RAID
RAIDRAID
Storage Devices And Backup Media
Storage Devices And Backup MediaStorage Devices And Backup Media
Storage Devices And Backup MediaTyrone Turner
 
Raid levels
Raid levelsRaid levels
Raid levels
sidra naz
 
RAID
RAIDRAID
Raid(Storage Technology)
Raid(Storage Technology)Raid(Storage Technology)
Raid(Storage Technology)
Vanitha shree Rajagopal
 
Presentation On RAID(Redundant Array Of Independent Disks) Basics
Presentation On RAID(Redundant Array Of Independent Disks) BasicsPresentation On RAID(Redundant Array Of Independent Disks) Basics
Presentation On RAID(Redundant Array Of Independent Disks) BasicsKuber Chandra
 
Green Station Overview Final
Green Station Overview   FinalGreen Station Overview   Final
Green Station Overview Finalbeckeys
 
Raid (Redundant Array of Inexpensive Disks) in Computer Architecture
Raid (Redundant Array of Inexpensive Disks) in Computer ArchitectureRaid (Redundant Array of Inexpensive Disks) in Computer Architecture
Raid (Redundant Array of Inexpensive Disks) in Computer Architecture
Aiman Hafeez
 
Raid- Redundant Array of Inexpensive Disks
Raid- Redundant Array of Inexpensive DisksRaid- Redundant Array of Inexpensive Disks
Raid- Redundant Array of Inexpensive Disks
Mudit Mishra
 
SSDs - Improving Performance of Storage Arrays
SSDs - Improving Performance of Storage ArraysSSDs - Improving Performance of Storage Arrays
SSDs - Improving Performance of Storage Arrays
nomathjobs
 
RAID - (Redundant Array of Inexpensive Disks or Drives, or Redundant Array of...
RAID - (Redundant Array of Inexpensive Disks or Drives, or Redundant Array of...RAID - (Redundant Array of Inexpensive Disks or Drives, or Redundant Array of...
RAID - (Redundant Array of Inexpensive Disks or Drives, or Redundant Array of...
Jason Augustine
 
Raid Level
Raid LevelRaid Level
Raid Level
gaurav singh
 
Raid
RaidRaid
Raid
Pari Soni
 
RAID CONCEPT
RAID CONCEPTRAID CONCEPT
RAID CONCEPT
Ramasubbu .P
 
Group04
Group04Group04
Group04
Bmusa057
 
Database 3
Database 3Database 3
Flash card azure disk
Flash card azure diskFlash card azure disk
Flash card azure disk
Yoong Seng Lai
 
Raid and its levels
Raid and its levelsRaid and its levels
Raid and its levels
IGZ Software house
 

What's hot (20)

Raid
RaidRaid
Raid
 
RAID
RAIDRAID
RAID
 
Storage Devices And Backup Media
Storage Devices And Backup MediaStorage Devices And Backup Media
Storage Devices And Backup Media
 
Raid levels
Raid levelsRaid levels
Raid levels
 
RAID
RAIDRAID
RAID
 
Raid(Storage Technology)
Raid(Storage Technology)Raid(Storage Technology)
Raid(Storage Technology)
 
RAID Review
RAID ReviewRAID Review
RAID Review
 
Presentation On RAID(Redundant Array Of Independent Disks) Basics
Presentation On RAID(Redundant Array Of Independent Disks) BasicsPresentation On RAID(Redundant Array Of Independent Disks) Basics
Presentation On RAID(Redundant Array Of Independent Disks) Basics
 
Green Station Overview Final
Green Station Overview   FinalGreen Station Overview   Final
Green Station Overview Final
 
Raid (Redundant Array of Inexpensive Disks) in Computer Architecture
Raid (Redundant Array of Inexpensive Disks) in Computer ArchitectureRaid (Redundant Array of Inexpensive Disks) in Computer Architecture
Raid (Redundant Array of Inexpensive Disks) in Computer Architecture
 
Raid- Redundant Array of Inexpensive Disks
Raid- Redundant Array of Inexpensive DisksRaid- Redundant Array of Inexpensive Disks
Raid- Redundant Array of Inexpensive Disks
 
SSDs - Improving Performance of Storage Arrays
SSDs - Improving Performance of Storage ArraysSSDs - Improving Performance of Storage Arrays
SSDs - Improving Performance of Storage Arrays
 
RAID - (Redundant Array of Inexpensive Disks or Drives, or Redundant Array of...
RAID - (Redundant Array of Inexpensive Disks or Drives, or Redundant Array of...RAID - (Redundant Array of Inexpensive Disks or Drives, or Redundant Array of...
RAID - (Redundant Array of Inexpensive Disks or Drives, or Redundant Array of...
 
Raid Level
Raid LevelRaid Level
Raid Level
 
Raid
RaidRaid
Raid
 
RAID CONCEPT
RAID CONCEPTRAID CONCEPT
RAID CONCEPT
 
Group04
Group04Group04
Group04
 
Database 3
Database 3Database 3
Database 3
 
Flash card azure disk
Flash card azure diskFlash card azure disk
Flash card azure disk
 
Raid and its levels
Raid and its levelsRaid and its levels
Raid and its levels
 

Viewers also liked

CiscoHyperFlex Overview for CIS@Enbridge
CiscoHyperFlex Overview for CIS@EnbridgeCiscoHyperFlex Overview for CIS@Enbridge
CiscoHyperFlex Overview for CIS@EnbridgeAman Syed
 
A Dell and Nutanix solution can boost datacenter efficiency
A Dell and Nutanix solution can boost datacenter efficiencyA Dell and Nutanix solution can boost datacenter efficiency
A Dell and Nutanix solution can boost datacenter efficiency
Principled Technologies
 
Webinar: How NVMe Will Change Flash Storage
Webinar: How NVMe Will Change Flash StorageWebinar: How NVMe Will Change Flash Storage
Webinar: How NVMe Will Change Flash Storage
Storage Switzerland
 
Redis on NVMe SSD - Zvika Guz, Samsung
 Redis on NVMe SSD - Zvika Guz, Samsung Redis on NVMe SSD - Zvika Guz, Samsung
Redis on NVMe SSD - Zvika Guz, Samsung
Redis Labs
 
Creating peer to peer network nadia perandos bisnar
Creating peer to peer network nadia perandos bisnarCreating peer to peer network nadia perandos bisnar
Creating peer to peer network nadia perandos bisnar
Perla Pelicano Corpez
 
2016 Flash Storage-NVMe Brand Leader Mini-Report
2016 Flash Storage-NVMe Brand Leader Mini-Report2016 Flash Storage-NVMe Brand Leader Mini-Report
2016 Flash Storage-NVMe Brand Leader Mini-Report
IT Brand Pulse
 
HDFS Erasure Coding in Action
HDFS Erasure Coding in Action HDFS Erasure Coding in Action
HDFS Erasure Coding in Action
DataWorks Summit/Hadoop Summit
 
COMPUTER ORGANIZATION -Multiplexer,Demultiplexer, Encoder
COMPUTER ORGANIZATION -Multiplexer,Demultiplexer, EncoderCOMPUTER ORGANIZATION -Multiplexer,Demultiplexer, Encoder
COMPUTER ORGANIZATION -Multiplexer,Demultiplexer, EncoderVanitha Chandru
 
Encoding and Decoding
Encoding and DecodingEncoding and Decoding
Encoding and Decoding
mrhaken
 
The Encoding
The EncodingThe Encoding
The EncodingJen W
 
Introduction to Peer-to-Peer Networks
Introduction to Peer-to-Peer Networks Introduction to Peer-to-Peer Networks
Introduction to Peer-to-Peer Networks Venkatesh Iyer
 
Intel µp instruction encoding and decoding
Intel µp instruction encoding and decodingIntel µp instruction encoding and decoding
Intel µp instruction encoding and decoding
yocirem
 
Encoding/Decoding Stuart Hall
Encoding/Decoding Stuart HallEncoding/Decoding Stuart Hall
Encoding/Decoding Stuart HallNic JM
 
Chapter 4 - Digital Transmission
Chapter 4 - Digital TransmissionChapter 4 - Digital Transmission
Chapter 4 - Digital Transmission
Wayne Jones Jnr
 
difference between hub, bridge, switch and router
difference between hub, bridge, switch and routerdifference between hub, bridge, switch and router
difference between hub, bridge, switch and router
Akmal Cikmat
 
Networking devices
Networking devicesNetworking devices
Networking devicesrupinderj
 
Network Hardware And Software
Network Hardware And SoftwareNetwork Hardware And Software
Network Hardware And SoftwareSteven Cahill
 
Coding and encoding
Coding and encodingCoding and encoding
Coding and encoding
Larry Nelson
 
Introduction to computer network
Introduction to computer networkIntroduction to computer network
Introduction to computer network
Ashita Agrawal
 

Viewers also liked (20)

Long term
Long termLong term
Long term
 
CiscoHyperFlex Overview for CIS@Enbridge
CiscoHyperFlex Overview for CIS@EnbridgeCiscoHyperFlex Overview for CIS@Enbridge
CiscoHyperFlex Overview for CIS@Enbridge
 
A Dell and Nutanix solution can boost datacenter efficiency
A Dell and Nutanix solution can boost datacenter efficiencyA Dell and Nutanix solution can boost datacenter efficiency
A Dell and Nutanix solution can boost datacenter efficiency
 
Webinar: How NVMe Will Change Flash Storage
Webinar: How NVMe Will Change Flash StorageWebinar: How NVMe Will Change Flash Storage
Webinar: How NVMe Will Change Flash Storage
 
Redis on NVMe SSD - Zvika Guz, Samsung
 Redis on NVMe SSD - Zvika Guz, Samsung Redis on NVMe SSD - Zvika Guz, Samsung
Redis on NVMe SSD - Zvika Guz, Samsung
 
Creating peer to peer network nadia perandos bisnar
Creating peer to peer network nadia perandos bisnarCreating peer to peer network nadia perandos bisnar
Creating peer to peer network nadia perandos bisnar
 
2016 Flash Storage-NVMe Brand Leader Mini-Report
2016 Flash Storage-NVMe Brand Leader Mini-Report2016 Flash Storage-NVMe Brand Leader Mini-Report
2016 Flash Storage-NVMe Brand Leader Mini-Report
 
HDFS Erasure Coding in Action
HDFS Erasure Coding in Action HDFS Erasure Coding in Action
HDFS Erasure Coding in Action
 
COMPUTER ORGANIZATION -Multiplexer,Demultiplexer, Encoder
COMPUTER ORGANIZATION -Multiplexer,Demultiplexer, EncoderCOMPUTER ORGANIZATION -Multiplexer,Demultiplexer, Encoder
COMPUTER ORGANIZATION -Multiplexer,Demultiplexer, Encoder
 
Encoding and Decoding
Encoding and DecodingEncoding and Decoding
Encoding and Decoding
 
The Encoding
The EncodingThe Encoding
The Encoding
 
Introduction to Peer-to-Peer Networks
Introduction to Peer-to-Peer Networks Introduction to Peer-to-Peer Networks
Introduction to Peer-to-Peer Networks
 
Intel µp instruction encoding and decoding
Intel µp instruction encoding and decodingIntel µp instruction encoding and decoding
Intel µp instruction encoding and decoding
 
Encoding/Decoding Stuart Hall
Encoding/Decoding Stuart HallEncoding/Decoding Stuart Hall
Encoding/Decoding Stuart Hall
 
Chapter 4 - Digital Transmission
Chapter 4 - Digital TransmissionChapter 4 - Digital Transmission
Chapter 4 - Digital Transmission
 
difference between hub, bridge, switch and router
difference between hub, bridge, switch and routerdifference between hub, bridge, switch and router
difference between hub, bridge, switch and router
 
Networking devices
Networking devicesNetworking devices
Networking devices
 
Network Hardware And Software
Network Hardware And SoftwareNetwork Hardware And Software
Network Hardware And Software
 
Coding and encoding
Coding and encodingCoding and encoding
Coding and encoding
 
Introduction to computer network
Introduction to computer networkIntroduction to computer network
Introduction to computer network
 

Similar to Quick Faq - Erasure Coding

Erasure Coding: Revolutionizing Data Durability and Storage Efficiency
Erasure Coding: Revolutionizing Data Durability and Storage EfficiencyErasure Coding: Revolutionizing Data Durability and Storage Efficiency
Erasure Coding: Revolutionizing Data Durability and Storage Efficiency
MaryJWilliams2
 
Data oriented design and c++
Data oriented design and c++Data oriented design and c++
Data oriented design and c++
Mike Acton
 
Если нашлась одна ошибка — есть и другие. Один способ выявить «наследуемые» у...
Если нашлась одна ошибка — есть и другие. Один способ выявить «наследуемые» у...Если нашлась одна ошибка — есть и другие. Один способ выявить «наследуемые» у...
Если нашлась одна ошибка — есть и другие. Один способ выявить «наследуемые» у...
Positive Hack Days
 
Tips And Tricks For Bioinformatics Software Engineering
Tips And Tricks For Bioinformatics Software EngineeringTips And Tricks For Bioinformatics Software Engineering
Tips And Tricks For Bioinformatics Software Engineering
jtdudley
 
Entenda de onde vem toda a potência do Intel® Xeon Phi™
Entenda de onde vem toda a potência do Intel® Xeon Phi™ Entenda de onde vem toda a potência do Intel® Xeon Phi™
Entenda de onde vem toda a potência do Intel® Xeon Phi™
Intel Software Brasil
 
The Ultimate IDS Smackdown
The Ultimate IDS SmackdownThe Ultimate IDS Smackdown
The Ultimate IDS SmackdownMario Heiderich
 
Webinar NETGEAR - Storagecraft e Netgear: soluzioni per il backup e il disast...
Webinar NETGEAR - Storagecraft e Netgear: soluzioni per il backup e il disast...Webinar NETGEAR - Storagecraft e Netgear: soluzioni per il backup e il disast...
Webinar NETGEAR - Storagecraft e Netgear: soluzioni per il backup e il disast...
Netgear Italia
 
Good++
Good++Good++
Oracle SPARC T7 a M7 servery
Oracle SPARC T7 a M7 serveryOracle SPARC T7 a M7 servery
Oracle SPARC T7 a M7 servery
MarketingArrowECS_CZ
 
cleversafe_definitive_guide_white_paper
cleversafe_definitive_guide_white_papercleversafe_definitive_guide_white_paper
cleversafe_definitive_guide_white_paperChris Woeppel
 
Tales from the Field
Tales from the FieldTales from the Field
Tales from the FieldMongoDB
 
136 latest dot net interview questions
136  latest dot net interview questions136  latest dot net interview questions
136 latest dot net interview questionssandi4204
 
Data recovery
Data recoveryData recovery
Data recovery
Ravi Malik
 
Why databases cry at night
Why databases cry at nightWhy databases cry at night
Why databases cry at night
Michael Yarichuk
 
DataEngConf: Uri Laserson (Data Scientist, Cloudera) Scaling up Genomics with...
DataEngConf: Uri Laserson (Data Scientist, Cloudera) Scaling up Genomics with...DataEngConf: Uri Laserson (Data Scientist, Cloudera) Scaling up Genomics with...
DataEngConf: Uri Laserson (Data Scientist, Cloudera) Scaling up Genomics with...
Hakka Labs
 
Predictable Big Data Performance in Real-time
Predictable Big Data Performance in Real-timePredictable Big Data Performance in Real-time
Predictable Big Data Performance in Real-time
Aerospike, Inc.
 
Search - Journey Of Delivery On A Budget (2014)
Search - Journey Of Delivery On A Budget (2014)Search - Journey Of Delivery On A Budget (2014)
Search - Journey Of Delivery On A Budget (2014)
Sam McLeod
 
How to be a Postgres DBA in a Pinch
How to be a Postgres DBA in a Pinch How to be a Postgres DBA in a Pinch
How to be a Postgres DBA in a Pinch
ElizabethGarrettChri
 
Data Privacy with Apache Spark: Defensive and Offensive Approaches
Data Privacy with Apache Spark: Defensive and Offensive ApproachesData Privacy with Apache Spark: Defensive and Offensive Approaches
Data Privacy with Apache Spark: Defensive and Offensive Approaches
Databricks
 
[C12]元気Hadoop! OracleをHadoopで分析しちゃうぜ by Daisuke Hirama
[C12]元気Hadoop! OracleをHadoopで分析しちゃうぜ by Daisuke Hirama[C12]元気Hadoop! OracleをHadoopで分析しちゃうぜ by Daisuke Hirama
[C12]元気Hadoop! OracleをHadoopで分析しちゃうぜ by Daisuke HiramaInsight Technology, Inc.
 

Similar to Quick Faq - Erasure Coding (20)

Erasure Coding: Revolutionizing Data Durability and Storage Efficiency
Erasure Coding: Revolutionizing Data Durability and Storage EfficiencyErasure Coding: Revolutionizing Data Durability and Storage Efficiency
Erasure Coding: Revolutionizing Data Durability and Storage Efficiency
 
Data oriented design and c++
Data oriented design and c++Data oriented design and c++
Data oriented design and c++
 
Если нашлась одна ошибка — есть и другие. Один способ выявить «наследуемые» у...
Если нашлась одна ошибка — есть и другие. Один способ выявить «наследуемые» у...Если нашлась одна ошибка — есть и другие. Один способ выявить «наследуемые» у...
Если нашлась одна ошибка — есть и другие. Один способ выявить «наследуемые» у...
 
Tips And Tricks For Bioinformatics Software Engineering
Tips And Tricks For Bioinformatics Software EngineeringTips And Tricks For Bioinformatics Software Engineering
Tips And Tricks For Bioinformatics Software Engineering
 
Entenda de onde vem toda a potência do Intel® Xeon Phi™
Entenda de onde vem toda a potência do Intel® Xeon Phi™ Entenda de onde vem toda a potência do Intel® Xeon Phi™
Entenda de onde vem toda a potência do Intel® Xeon Phi™
 
The Ultimate IDS Smackdown
The Ultimate IDS SmackdownThe Ultimate IDS Smackdown
The Ultimate IDS Smackdown
 
Webinar NETGEAR - Storagecraft e Netgear: soluzioni per il backup e il disast...
Webinar NETGEAR - Storagecraft e Netgear: soluzioni per il backup e il disast...Webinar NETGEAR - Storagecraft e Netgear: soluzioni per il backup e il disast...
Webinar NETGEAR - Storagecraft e Netgear: soluzioni per il backup e il disast...
 
Good++
Good++Good++
Good++
 
Oracle SPARC T7 a M7 servery
Oracle SPARC T7 a M7 serveryOracle SPARC T7 a M7 servery
Oracle SPARC T7 a M7 servery
 
cleversafe_definitive_guide_white_paper
cleversafe_definitive_guide_white_papercleversafe_definitive_guide_white_paper
cleversafe_definitive_guide_white_paper
 
Tales from the Field
Tales from the FieldTales from the Field
Tales from the Field
 
136 latest dot net interview questions
136  latest dot net interview questions136  latest dot net interview questions
136 latest dot net interview questions
 
Data recovery
Data recoveryData recovery
Data recovery
 
Why databases cry at night
Why databases cry at nightWhy databases cry at night
Why databases cry at night
 
DataEngConf: Uri Laserson (Data Scientist, Cloudera) Scaling up Genomics with...
DataEngConf: Uri Laserson (Data Scientist, Cloudera) Scaling up Genomics with...DataEngConf: Uri Laserson (Data Scientist, Cloudera) Scaling up Genomics with...
DataEngConf: Uri Laserson (Data Scientist, Cloudera) Scaling up Genomics with...
 
Predictable Big Data Performance in Real-time
Predictable Big Data Performance in Real-timePredictable Big Data Performance in Real-time
Predictable Big Data Performance in Real-time
 
Search - Journey Of Delivery On A Budget (2014)
Search - Journey Of Delivery On A Budget (2014)Search - Journey Of Delivery On A Budget (2014)
Search - Journey Of Delivery On A Budget (2014)
 
How to be a Postgres DBA in a Pinch
How to be a Postgres DBA in a Pinch How to be a Postgres DBA in a Pinch
How to be a Postgres DBA in a Pinch
 
Data Privacy with Apache Spark: Defensive and Offensive Approaches
Data Privacy with Apache Spark: Defensive and Offensive ApproachesData Privacy with Apache Spark: Defensive and Offensive Approaches
Data Privacy with Apache Spark: Defensive and Offensive Approaches
 
[C12]元気Hadoop! OracleをHadoopで分析しちゃうぜ by Daisuke Hirama
[C12]元気Hadoop! OracleをHadoopで分析しちゃうぜ by Daisuke Hirama[C12]元気Hadoop! OracleをHadoopで分析しちゃうぜ by Daisuke Hirama
[C12]元気Hadoop! OracleをHadoopで分析しちゃうぜ by Daisuke Hirama
 

More from Western Digital

Unstructured & Structured Data Growth Is Exploding
Unstructured & Structured Data Growth Is ExplodingUnstructured & Structured Data Growth Is Exploding
Unstructured & Structured Data Growth Is Exploding
Western Digital
 
The Future of Data infographic
The Future of Data infographicThe Future of Data infographic
The Future of Data infographic
Western Digital
 
Storage Class Memory: Learning from 3D NAND
Storage Class Memory: Learning from 3D NANDStorage Class Memory: Learning from 3D NAND
Storage Class Memory: Learning from 3D NAND
Western Digital
 
5 Things You Need to Know About Enterprise Fl
 5 Things You Need to Know About Enterprise Fl 5 Things You Need to Know About Enterprise Fl
5 Things You Need to Know About Enterprise Fl
Western Digital
 
All-Flash Versus Hybrid VMware Virtual SAN™: Performance vs. Price
All-Flash Versus Hybrid VMware Virtual SAN™: Performance vs. Price All-Flash Versus Hybrid VMware Virtual SAN™: Performance vs. Price
All-Flash Versus Hybrid VMware Virtual SAN™: Performance vs. Price
Western Digital
 
The All-Flash SAP HANA Solution: Performance, Economics, and Reliability
The All-Flash SAP HANA Solution: Performance, Economics, and Reliability The All-Flash SAP HANA Solution: Performance, Economics, and Reliability
The All-Flash SAP HANA Solution: Performance, Economics, and Reliability
Western Digital
 
IDC Infographic - How Flash Fits into Your Cloud
IDC Infographic - How Flash Fits into Your CloudIDC Infographic - How Flash Fits into Your Cloud
IDC Infographic - How Flash Fits into Your Cloud
Western Digital
 
Consolidation on Flash- Hardware for Nothing, Get Your Flash for Free (I want...
Consolidation on Flash- Hardware for Nothing, Get Your Flash for Free (I want...Consolidation on Flash- Hardware for Nothing, Get Your Flash for Free (I want...
Consolidation on Flash- Hardware for Nothing, Get Your Flash for Free (I want...
Western Digital
 
5 Tips to Building a Successful Big Data Strategy
5 Tips to Building a Successful Big Data Strategy5 Tips to Building a Successful Big Data Strategy
5 Tips to Building a Successful Big Data Strategy
Western Digital
 
Flash Stories: How Customers Make Smarter Decisions Faster
Flash Stories: How Customers Make Smarter Decisions FasterFlash Stories: How Customers Make Smarter Decisions Faster
Flash Stories: How Customers Make Smarter Decisions Faster
Western Digital
 
Flash & Open Source: Creating New Possibilites in the Data Center
Flash & Open Source: Creating New Possibilites in the Data CenterFlash & Open Source: Creating New Possibilites in the Data Center
Flash & Open Source: Creating New Possibilites in the Data Center
Western Digital
 
Deploying All-Flash Cloud Infrastructure without Breaking the Bank
Deploying All-Flash Cloud Infrastructure without Breaking the BankDeploying All-Flash Cloud Infrastructure without Breaking the Bank
Deploying All-Flash Cloud Infrastructure without Breaking the Bank
Western Digital
 
5 Tips for a More Efficient Data Center
5 Tips for a More Efficient Data Center5 Tips for a More Efficient Data Center
5 Tips for a More Efficient Data Center
Western Digital
 
The Disruptive Power of Flash in the Data Center
The Disruptive Power of Flash in the Data CenterThe Disruptive Power of Flash in the Data Center
The Disruptive Power of Flash in the Data Center
Western Digital
 

More from Western Digital (14)

Unstructured & Structured Data Growth Is Exploding
Unstructured & Structured Data Growth Is ExplodingUnstructured & Structured Data Growth Is Exploding
Unstructured & Structured Data Growth Is Exploding
 
The Future of Data infographic
The Future of Data infographicThe Future of Data infographic
The Future of Data infographic
 
Storage Class Memory: Learning from 3D NAND
Storage Class Memory: Learning from 3D NANDStorage Class Memory: Learning from 3D NAND
Storage Class Memory: Learning from 3D NAND
 
5 Things You Need to Know About Enterprise Fl
 5 Things You Need to Know About Enterprise Fl 5 Things You Need to Know About Enterprise Fl
5 Things You Need to Know About Enterprise Fl
 
All-Flash Versus Hybrid VMware Virtual SAN™: Performance vs. Price
All-Flash Versus Hybrid VMware Virtual SAN™: Performance vs. Price All-Flash Versus Hybrid VMware Virtual SAN™: Performance vs. Price
All-Flash Versus Hybrid VMware Virtual SAN™: Performance vs. Price
 
The All-Flash SAP HANA Solution: Performance, Economics, and Reliability
The All-Flash SAP HANA Solution: Performance, Economics, and Reliability The All-Flash SAP HANA Solution: Performance, Economics, and Reliability
The All-Flash SAP HANA Solution: Performance, Economics, and Reliability
 
IDC Infographic - How Flash Fits into Your Cloud
IDC Infographic - How Flash Fits into Your CloudIDC Infographic - How Flash Fits into Your Cloud
IDC Infographic - How Flash Fits into Your Cloud
 
Consolidation on Flash- Hardware for Nothing, Get Your Flash for Free (I want...
Consolidation on Flash- Hardware for Nothing, Get Your Flash for Free (I want...Consolidation on Flash- Hardware for Nothing, Get Your Flash for Free (I want...
Consolidation on Flash- Hardware for Nothing, Get Your Flash for Free (I want...
 
5 Tips to Building a Successful Big Data Strategy
5 Tips to Building a Successful Big Data Strategy5 Tips to Building a Successful Big Data Strategy
5 Tips to Building a Successful Big Data Strategy
 
Flash Stories: How Customers Make Smarter Decisions Faster
Flash Stories: How Customers Make Smarter Decisions FasterFlash Stories: How Customers Make Smarter Decisions Faster
Flash Stories: How Customers Make Smarter Decisions Faster
 
Flash & Open Source: Creating New Possibilites in the Data Center
Flash & Open Source: Creating New Possibilites in the Data CenterFlash & Open Source: Creating New Possibilites in the Data Center
Flash & Open Source: Creating New Possibilites in the Data Center
 
Deploying All-Flash Cloud Infrastructure without Breaking the Bank
Deploying All-Flash Cloud Infrastructure without Breaking the BankDeploying All-Flash Cloud Infrastructure without Breaking the Bank
Deploying All-Flash Cloud Infrastructure without Breaking the Bank
 
5 Tips for a More Efficient Data Center
5 Tips for a More Efficient Data Center5 Tips for a More Efficient Data Center
5 Tips for a More Efficient Data Center
 
The Disruptive Power of Flash in the Data Center
The Disruptive Power of Flash in the Data CenterThe Disruptive Power of Flash in the Data Center
The Disruptive Power of Flash in the Data Center
 

Recently uploaded

一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
v3tuleee
 
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdf
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdfEnhanced Enterprise Intelligence with your personal AI Data Copilot.pdf
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdf
GetInData
 
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
slg6lamcq
 
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
AbhimanyuSinha9
 
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...
Subhajit Sahu
 
Malana- Gimlet Market Analysis (Portfolio 2)
Malana- Gimlet Market Analysis (Portfolio 2)Malana- Gimlet Market Analysis (Portfolio 2)
Malana- Gimlet Market Analysis (Portfolio 2)
TravisMalana
 
做(mqu毕业证书)麦考瑞大学毕业证硕士文凭证书学费发票原版一模一样
做(mqu毕业证书)麦考瑞大学毕业证硕士文凭证书学费发票原版一模一样做(mqu毕业证书)麦考瑞大学毕业证硕士文凭证书学费发票原版一模一样
做(mqu毕业证书)麦考瑞大学毕业证硕士文凭证书学费发票原版一模一样
axoqas
 
The affect of service quality and online reviews on customer loyalty in the E...
The affect of service quality and online reviews on customer loyalty in the E...The affect of service quality and online reviews on customer loyalty in the E...
The affect of service quality and online reviews on customer loyalty in the E...
jerlynmaetalle
 
Criminal IP - Threat Hunting Webinar.pdf
Criminal IP - Threat Hunting Webinar.pdfCriminal IP - Threat Hunting Webinar.pdf
Criminal IP - Threat Hunting Webinar.pdf
Criminal IP
 
Unleashing the Power of Data_ Choosing a Trusted Analytics Platform.pdf
Unleashing the Power of Data_ Choosing a Trusted Analytics Platform.pdfUnleashing the Power of Data_ Choosing a Trusted Analytics Platform.pdf
Unleashing the Power of Data_ Choosing a Trusted Analytics Platform.pdf
Enterprise Wired
 
Analysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performanceAnalysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performance
roli9797
 
Learn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queriesLearn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queries
manishkhaire30
 
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
dwreak4tg
 
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
g4dpvqap0
 
Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...
Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...
Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...
Subhajit Sahu
 
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
u86oixdj
 
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
mbawufebxi
 
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
axoqas
 
My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.
rwarrenll
 
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
oz8q3jxlp
 

Recently uploaded (20)

一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
 
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdf
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdfEnhanced Enterprise Intelligence with your personal AI Data Copilot.pdf
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdf
 
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
 
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
 
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...
 
Malana- Gimlet Market Analysis (Portfolio 2)
Malana- Gimlet Market Analysis (Portfolio 2)Malana- Gimlet Market Analysis (Portfolio 2)
Malana- Gimlet Market Analysis (Portfolio 2)
 
做(mqu毕业证书)麦考瑞大学毕业证硕士文凭证书学费发票原版一模一样
做(mqu毕业证书)麦考瑞大学毕业证硕士文凭证书学费发票原版一模一样做(mqu毕业证书)麦考瑞大学毕业证硕士文凭证书学费发票原版一模一样
做(mqu毕业证书)麦考瑞大学毕业证硕士文凭证书学费发票原版一模一样
 
The affect of service quality and online reviews on customer loyalty in the E...
The affect of service quality and online reviews on customer loyalty in the E...The affect of service quality and online reviews on customer loyalty in the E...
The affect of service quality and online reviews on customer loyalty in the E...
 
Criminal IP - Threat Hunting Webinar.pdf
Criminal IP - Threat Hunting Webinar.pdfCriminal IP - Threat Hunting Webinar.pdf
Criminal IP - Threat Hunting Webinar.pdf
 
Unleashing the Power of Data_ Choosing a Trusted Analytics Platform.pdf
Unleashing the Power of Data_ Choosing a Trusted Analytics Platform.pdfUnleashing the Power of Data_ Choosing a Trusted Analytics Platform.pdf
Unleashing the Power of Data_ Choosing a Trusted Analytics Platform.pdf
 
Analysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performanceAnalysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performance
 
Learn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queriesLearn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queries
 
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
 
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
 
Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...
Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...
Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...
 
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
 
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
 
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
 
My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.
 
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
 

Quick Faq - Erasure Coding

  • 2. Quick FAQ What is Erasure Coding and how does it work?
  • 3. What is Erasure Coding and how does it work?
  • 4. What is Erasure Coding and how does it work? Encoding Decoding Simple example for a 3/1 erasure encoding
  • 5. What is Erasure Coding and how does it work? Simple example for a 3/1 erasure encoding • We solve for x: OR x+y=12 [+] x-y= 2 2x =14 [∕2] X = 7 2x+ y=19 [+] x- y= 2 3x =21 [∕3] X = 7 Encoding Decoding
  • 6. What is Erasure Coding and how does it work? Simple example for a 3/1 erasure encoding • We solve for x: x+y=12 [+] x-y= 2 2x =14 [∕2] X = 7 Encoding Decoding
  • 7. What is Erasure Coding and how does it work? Simple example for a 3/1 erasure encoding • We solve for x: OR • Then we solve for y: x+y=12 [+] x-y= 2 2x =14 [∕2] X = 7 2x+ y=19 [+] x- y= 2 3x =21 [∕3] X = 7 7+y=12 [-7] y=5 Encoding Decoding
  • 8. What is Erasure Coding and how does it work? Simple example for a 3/1 erasure encoding • We solve for x: OR • Then we solve for y: OR x+y=12 [+] x-y= 2 2x =14 [∕2] X = 7 2x+ y=19 [+] x- y= 2 3x =21 [∕3] X = 7 7-y= 2 [-7] -y=-5 [*-1] y=5 7+y=12 [-7] y=5 Encoding Decoding
  • 9. What is Erasure Coding and how does it work?
  • 10. What is Erasure Coding and how does it work? Erasure Coding is a data protection scheme that breaks data into shards (fragments) that are encoded with parity (redundant data), and then stored across multiple storage media and locations.
  • 11. What is Erasure Coding and how does it work? Erasure Coding is a data protection scheme that breaks data into shards (fragments) that are encoded with parity (redundant data), and then stored across multiple storage media and locations. Why you should care
  • 12. What is Erasure Coding and how does it work? Erasure Coding is a data protection scheme that breaks data into shards (fragments) that are encoded with parity (redundant data), and then stored across multiple storage media and locations. Why you should care • You only need a subset of the shards to rehydrate data.
  • 13. What is Erasure Coding and how does it work? Erasure Coding is a data protection scheme that breaks data into shards (fragments) that are encoded with parity (redundant data), and then stored across multiple storage media and locations. Why you should care • You only need a subset of the shards to rehydrate data. • You can replace failed components when convenient, without taking the system offline.
  • 14. What is Erasure Coding and how does it work? Erasure Coding is a data protection scheme that breaks data into shards (fragments) that are encoded with parity (redundant data), and then stored across multiple storage media and locations. Why you should care • You only need a subset of the shards to rehydrate data. • You can replace failed components when convenient, without taking the system offline. • You can reduce CAPEX and OPEX compared with mirroring/replication approaches.
  • 15. Thanks for watching Clay Ryder DCS Marketing itblog.sandisk.com/author/clayryder
  • 16. © 2017 Western Digital Corporation. All rights reserved. Western Digital and the Western Digital Logo are registered trademarks of Western Digital Corporation or its affiliates in the U.S. and/or other countries. All other marks are the property of their respective owners. "Faster Does It" Kevin MacLeod (incompetech.com) — Licensed under Creative Commons: By Attribution 3.0 http://creativecommons.org/licenses/by/3.0/ @WesternDigiDC SanDisk Data Center Solutions @BigDataFlash HGST, a Western Digital brand
  • 17. What is Erasure Coding and how does it work? Erasure Coding encodes data and compartmentalizes it such that only a subset of the pieces are required to recreate the original information.
  • 18. What is Erasure Coding and how does it work? Erasure Coding encodes data and compartmentalizes it such that only a subset of the pieces are required to recreate the original information. For example: 5/2 encoding requires (5-2) of 5 pieces to rehydrate it 10/3 encoding requires (10-3) of 10 pieces 18/5 requires (18-5) of 18 etc.
  • 19. What is Erasure Coding and how does it work? Simple example for a 18/5 erasure encoding • We solve for x: OR • Then we solve for y: OR x+y=12 [+] x-y= 2 2x =14 [∕2] X = 7 2x+ y=19 [+] x- y= 2 3x =21 [∕3] X = 7 7-y= 2 [-7] -y=-5 [*-1] y=5 7+y=12 [-7] y=5 Encoding Decoding … …1 182 Any 13 of 18 equations to decode object
  • 20. Thanks for watching Clay Ryder DCS Marketing itblog.sandisk.com/author/clayryder