SlideShare a Scribd company logo
1 of 11
Download to read offline
Big Data and the Future of Storage
Dr. Evangelos Eleftheriou
IBM Fellow
© 2014 International Business Machines Corporation 2
Big Data: More than just volume
Volume
Terabytes to
exabytes of existing
data to store and
process
Velocity
Streaming data,
milliseconds to
seconds to respond
Variety
Structured,
unstructured, text &
multimedia
Veracity
Uncertainty from
inconsistency,
ambiguities, etc.
© 2014 International Business Machines Corporation 3
Memory / Storage Stack
Significant advancements in
non-volatile memories blur
the boundaries between
storage and memory by being
low-cost, fast, and cost-
effective.
101
1
102
103
104
105
106
107
108
109
1010
CPU Operation (1ns)
Fetch data from the L2 Cache (10ns)
Fetch data from DRAM (60ns)
Read / Write SCM (100 – 2000 ns)
Read from a Flash device (20 μs)
Write to a Flash device (1 ms)
Read from / Write to disk (5 ms)
Read from / Write to tape (40 s)
Memory
SCM
Storage
Time(ns)
second
minute
hour
day
week
month
year
decade
century
SLOW
FAST
© 2014 International Business Machines Corporation 4
IBM FlashSystem: The Tipping Point
• The first time Flash storage outperformed
hard disks in all aspects, including
capacity and performance density, cost
per I/O Operations Per Second (IOPS)
and energy efficiency!
"
• With IBM FlashSystem 820 we achieved
more than 6 million IOPS running an IBM
DB2 workload on IBM Power servers

• 19 kilowatts vs. 4.5 megawatts with high
capacity hard disks, 236x better

• Installed in less than 48 Hours, HDD
would require two years !
© 2014 International Business Machines Corporation 5
IBM FlashSystem: The Tipping Point
© 2014 International Business Machines Corporation 6
Storage infrastructure in the new era
High IOPS
Ultra-low Latency
Memory-speed
Data Processing
Infrastructure
Elasticity
Reduced time
for deployment
Bottomless
Capacity
Optimized TCO
Integrated global
infrastructure
Simplicity
Security
Reliability
Pay-as-you-go
Extreme Scalability
Block Interface Novel Interfaces
MicroLatency Storage Global File System
POSIX NFS CIFS Object
GPFS Active!
File Management
POSIX CIFS ObjectNFS
ObjectNFSPOSIX CIFS
GPFS
GPFS
© 2014 International Business Machines Corporation 8
Tape Storage: Big Data Needs Tape
• Faster than Hard Disks at streaming!
!
• Reliability: read after three decades,
against five years for disks!
!
• Zero Power Consumption (when idle)!
!
• Security: 50 Petabytes on an HDD can
be deleted in minutes, tape would take
years!
!
• Cost: 1 GB of disk storage costs 10
cents, versus 4 cents for tape !
!
• The Next Target: the 100 TB cartridge
The Economist, Magnetic tape to the rescue. 30 Nov 2013
© 2014 International Business Machines Corporation 9
Multi-cloud Storage Tool Kit (live demo in CeBIT booth)
• Can connect to one or
more clouds including: 

IBM Softlayer, AWS, Azure,
Rackspace 

(public, private, or hybrid) !
• Enterprise features,
including: encryption,
integrity, and resiliency,
it’s transparent to GPFS
and manages keys/
metadata!
• Drag-and-drop usability
© 2014 International Business Machines Corporation© 2014 International Business Machines Corporation !9
© 2014 International Business Machines Corporation 10
A universal, non-volatile
memory technology superior to
Flash. PCM is very durable and
can endure at least 10 million
write cycles, compared to
current enterprise-class flash at
30,000 cycles or consumer-
class flash at 3,000 cycles.
Phase Change Memory (PCM)
DRAM Flash PCM
Multi-Level
PCM
Timeframe Invented in 1966 Invented in 1980s
Used in smartphones
– wide adoption
expected by 2016
2016
Speed
Density
Endurance
Retention
Scaling
Best in class
Good / Adequate
Average / Inadequate
Bad / Worst in class
➢ The qualification is relative and depends on the application!
➢ Racetrack memory is not included in this time horizon
© 2014 International Business Machines Corporation 11
Theseus Project: PCM-based 

PCI-e Prototype Card

Hybrid Storage/Caching subsystem based on PCM
• Design and implementation of a high-
performance PCI-e card using PCM!
• Consistent ultra-low latency, high IOPS
even for very small operations!
• Integration of PCM and Flash for hybrid
use cases: !
• Caching!
• Tiering!
• Persistent Key/Value Store

More Related Content

What's hot

IBM Tape Update Dezember18 - TS1160
IBM Tape Update Dezember18 - TS1160IBM Tape Update Dezember18 - TS1160
IBM Tape Update Dezember18 - TS1160Josef Weingand
 
IBM Tape the future of tape
IBM Tape the future of tapeIBM Tape the future of tape
IBM Tape the future of tapeJosef Weingand
 
OSBConf 2015 | Contemporary and cost efficient backups to to tape by josef we...
OSBConf 2015 | Contemporary and cost efficient backups to to tape by josef we...OSBConf 2015 | Contemporary and cost efficient backups to to tape by josef we...
OSBConf 2015 | Contemporary and cost efficient backups to to tape by josef we...NETWAYS
 
Quantum Enterprise Expansion Product Suite
Quantum Enterprise Expansion Product SuiteQuantum Enterprise Expansion Product Suite
Quantum Enterprise Expansion Product SuiteQuantum
 
[xSIG 2018]InvisibleWriteRule: Extended write protocol for 1VCC
[xSIG 2018]InvisibleWriteRule: Extended write protocol for 1VCC[xSIG 2018]InvisibleWriteRule: Extended write protocol for 1VCC
[xSIG 2018]InvisibleWriteRule: Extended write protocol for 1VCCSho Nakazono
 

What's hot (7)

IBM Tape Update Dezember18 - TS1160
IBM Tape Update Dezember18 - TS1160IBM Tape Update Dezember18 - TS1160
IBM Tape Update Dezember18 - TS1160
 
The future of tape
The future of tapeThe future of tape
The future of tape
 
IBM Tape the future of tape
IBM Tape the future of tapeIBM Tape the future of tape
IBM Tape the future of tape
 
OSBConf 2015 | Contemporary and cost efficient backups to to tape by josef we...
OSBConf 2015 | Contemporary and cost efficient backups to to tape by josef we...OSBConf 2015 | Contemporary and cost efficient backups to to tape by josef we...
OSBConf 2015 | Contemporary and cost efficient backups to to tape by josef we...
 
Quantum Enterprise Expansion Product Suite
Quantum Enterprise Expansion Product SuiteQuantum Enterprise Expansion Product Suite
Quantum Enterprise Expansion Product Suite
 
[xSIG 2018]InvisibleWriteRule: Extended write protocol for 1VCC
[xSIG 2018]InvisibleWriteRule: Extended write protocol for 1VCC[xSIG 2018]InvisibleWriteRule: Extended write protocol for 1VCC
[xSIG 2018]InvisibleWriteRule: Extended write protocol for 1VCC
 
Storage class memory
Storage class memoryStorage class memory
Storage class memory
 

Viewers also liked

Blend Data with Ease: Event Technology for the API Economy
Blend Data with Ease:  Event Technology for the API EconomyBlend Data with Ease:  Event Technology for the API Economy
Blend Data with Ease: Event Technology for the API EconomyHubb
 
Design - Start Your API Journey Today
Design - Start Your API Journey TodayDesign - Start Your API Journey Today
Design - Start Your API Journey TodayLaurenWendler
 
The Future of Customer Centricity
The Future of Customer Centricity The Future of Customer Centricity
The Future of Customer Centricity Software AG
 
Ame 2280 mq for z os latest features
Ame 2280 mq for z os latest featuresAme 2280 mq for z os latest features
Ame 2280 mq for z os latest featuresPete Siddall
 
2012 IBM Tech Trends Report: Fast track to the future
2012 IBM Tech Trends Report: Fast track to the future2012 IBM Tech Trends Report: Fast track to the future
2012 IBM Tech Trends Report: Fast track to the futureSusanne Hupfer, Ph.D.
 
The State of Cybersecurity and Digital Trust 2016
The State of Cybersecurity and Digital Trust 2016The State of Cybersecurity and Digital Trust 2016
The State of Cybersecurity and Digital Trust 2016HfS Research
 
Impact of BIG Data on MDM
Impact of BIG Data on MDMImpact of BIG Data on MDM
Impact of BIG Data on MDMSubhendu Dey
 
WIFUTURE Conference - Artificial Intelligence and ChatBots - Milan
WIFUTURE Conference - Artificial Intelligence and ChatBots - MilanWIFUTURE Conference - Artificial Intelligence and ChatBots - Milan
WIFUTURE Conference - Artificial Intelligence and ChatBots - MilanPietro Leo
 
It delivery 2016 v5
It delivery 2016 v5It delivery 2016 v5
It delivery 2016 v5Pini Cohen
 
Growing up hybrid: Accelerating digital transformation
Growing up hybrid: Accelerating digital transformationGrowing up hybrid: Accelerating digital transformation
Growing up hybrid: Accelerating digital transformationSusanne Hupfer, Ph.D.
 
API Management: Unlock the Value of Your Unique Assets with a Robust API
API Management: Unlock the Value of Your Unique Assets with a Robust APIAPI Management: Unlock the Value of Your Unique Assets with a Robust API
API Management: Unlock the Value of Your Unique Assets with a Robust APISoftware AG
 
IBM App Connect - Let Your Apps Work For You
IBM App Connect - Let Your Apps Work For YouIBM App Connect - Let Your Apps Work For You
IBM App Connect - Let Your Apps Work For YouIBM Integration
 
Social-Psycho Analyses
Social-Psycho AnalysesSocial-Psycho Analyses
Social-Psycho AnalysesJerry Smith
 
The cognitive advantage: Insights from early adopters on driving business value
The cognitive advantage: Insights from early adopters on driving business valueThe cognitive advantage: Insights from early adopters on driving business value
The cognitive advantage: Insights from early adopters on driving business valueSusanne Hupfer, Ph.D.
 
The Dilemmas of Innovation Management
The Dilemmas of Innovation ManagementThe Dilemmas of Innovation Management
The Dilemmas of Innovation ManagementIBM Research
 
The Intelligent Enterprise: How Companies are Using Cognitive Computing to Dr...
The Intelligent Enterprise: How Companies are Using Cognitive Computing to Dr...The Intelligent Enterprise: How Companies are Using Cognitive Computing to Dr...
The Intelligent Enterprise: How Companies are Using Cognitive Computing to Dr...Susanne Hupfer, Ph.D.
 
DevOps 101 - Moving Fast with Confidence
DevOps 101 - Moving Fast with ConfidenceDevOps 101 - Moving Fast with Confidence
DevOps 101 - Moving Fast with ConfidenceNew Relic
 

Viewers also liked (20)

Blend Data with Ease: Event Technology for the API Economy
Blend Data with Ease:  Event Technology for the API EconomyBlend Data with Ease:  Event Technology for the API Economy
Blend Data with Ease: Event Technology for the API Economy
 
Design - Start Your API Journey Today
Design - Start Your API Journey TodayDesign - Start Your API Journey Today
Design - Start Your API Journey Today
 
The Future of Customer Centricity
The Future of Customer Centricity The Future of Customer Centricity
The Future of Customer Centricity
 
Ame 2280 mq for z os latest features
Ame 2280 mq for z os latest featuresAme 2280 mq for z os latest features
Ame 2280 mq for z os latest features
 
2012 IBM Tech Trends Report: Fast track to the future
2012 IBM Tech Trends Report: Fast track to the future2012 IBM Tech Trends Report: Fast track to the future
2012 IBM Tech Trends Report: Fast track to the future
 
Industrial Strategy 10 Pillars
Industrial Strategy 10 PillarsIndustrial Strategy 10 Pillars
Industrial Strategy 10 Pillars
 
The State of Cybersecurity and Digital Trust 2016
The State of Cybersecurity and Digital Trust 2016The State of Cybersecurity and Digital Trust 2016
The State of Cybersecurity and Digital Trust 2016
 
Impact of BIG Data on MDM
Impact of BIG Data on MDMImpact of BIG Data on MDM
Impact of BIG Data on MDM
 
WIFUTURE Conference - Artificial Intelligence and ChatBots - Milan
WIFUTURE Conference - Artificial Intelligence and ChatBots - MilanWIFUTURE Conference - Artificial Intelligence and ChatBots - Milan
WIFUTURE Conference - Artificial Intelligence and ChatBots - Milan
 
It delivery 2016 v5
It delivery 2016 v5It delivery 2016 v5
It delivery 2016 v5
 
Growing up hybrid: Accelerating digital transformation
Growing up hybrid: Accelerating digital transformationGrowing up hybrid: Accelerating digital transformation
Growing up hybrid: Accelerating digital transformation
 
API Management: Unlock the Value of Your Unique Assets with a Robust API
API Management: Unlock the Value of Your Unique Assets with a Robust APIAPI Management: Unlock the Value of Your Unique Assets with a Robust API
API Management: Unlock the Value of Your Unique Assets with a Robust API
 
IBM App Connect - Let Your Apps Work For You
IBM App Connect - Let Your Apps Work For YouIBM App Connect - Let Your Apps Work For You
IBM App Connect - Let Your Apps Work For You
 
Social-Psycho Analyses
Social-Psycho AnalysesSocial-Psycho Analyses
Social-Psycho Analyses
 
The cognitive advantage: Insights from early adopters on driving business value
The cognitive advantage: Insights from early adopters on driving business valueThe cognitive advantage: Insights from early adopters on driving business value
The cognitive advantage: Insights from early adopters on driving business value
 
The Dilemmas of Innovation Management
The Dilemmas of Innovation ManagementThe Dilemmas of Innovation Management
The Dilemmas of Innovation Management
 
ARIS World
ARIS World ARIS World
ARIS World
 
The Intelligent Enterprise: How Companies are Using Cognitive Computing to Dr...
The Intelligent Enterprise: How Companies are Using Cognitive Computing to Dr...The Intelligent Enterprise: How Companies are Using Cognitive Computing to Dr...
The Intelligent Enterprise: How Companies are Using Cognitive Computing to Dr...
 
DevOps 101 - Moving Fast with Confidence
DevOps 101 - Moving Fast with ConfidenceDevOps 101 - Moving Fast with Confidence
DevOps 101 - Moving Fast with Confidence
 
API Economy Journey Map
API Economy Journey MapAPI Economy Journey Map
API Economy Journey Map
 

Similar to Big Data and the Future of Storage

Flash Ahead: IBM Flash System Selling Point
Flash Ahead: IBM Flash System Selling PointFlash Ahead: IBM Flash System Selling Point
Flash Ahead: IBM Flash System Selling PointCTI Group
 
NVMe over Fibre Channel Introduction
NVMe over Fibre Channel IntroductionNVMe over Fibre Channel Introduction
NVMe over Fibre Channel IntroductionCalvin Zito
 
IBM Upgrades SVC with Solid State Drives — Achieves Better Storage Utilization
IBM Upgrades SVC with Solid State Drives — Achieves Better Storage UtilizationIBM Upgrades SVC with Solid State Drives — Achieves Better Storage Utilization
IBM Upgrades SVC with Solid State Drives — Achieves Better Storage UtilizationIBM India Smarter Computing
 
IBM Upgrades SVC with Solid State Drives — Achieves Better Storage Utilization
IBM Upgrades SVC with Solid State Drives — Achieves Better Storage UtilizationIBM Upgrades SVC with Solid State Drives — Achieves Better Storage Utilization
IBM Upgrades SVC with Solid State Drives — Achieves Better Storage UtilizationIBM India Smarter Computing
 
IMCSummit 2015 - Day 2 IT Business Track - Drive IMC Efficiency with Flash E...
IMCSummit 2015 - Day 2  IT Business Track - Drive IMC Efficiency with Flash E...IMCSummit 2015 - Day 2  IT Business Track - Drive IMC Efficiency with Flash E...
IMCSummit 2015 - Day 2 IT Business Track - Drive IMC Efficiency with Flash E...In-Memory Computing Summit
 
In-Memory Computing: Myths and Facts
In-Memory Computing: Myths and FactsIn-Memory Computing: Myths and Facts
In-Memory Computing: Myths and FactsDATAVERSITY
 
Z109889 z4 r-storage-dfsms-jburg-v1909d
Z109889 z4 r-storage-dfsms-jburg-v1909dZ109889 z4 r-storage-dfsms-jburg-v1909d
Z109889 z4 r-storage-dfsms-jburg-v1909dTony Pearson
 
Z4R: Intro to Storage and DFSMS for z/OS
Z4R: Intro to Storage and DFSMS for z/OSZ4R: Intro to Storage and DFSMS for z/OS
Z4R: Intro to Storage and DFSMS for z/OSTony Pearson
 
Z109889 z4 r-storage-dfsms-vegas-v1910b
Z109889 z4 r-storage-dfsms-vegas-v1910bZ109889 z4 r-storage-dfsms-vegas-v1910b
Z109889 z4 r-storage-dfsms-vegas-v1910bTony Pearson
 
Dell whitepaper busting solid state storage myths
Dell whitepaper busting solid state storage mythsDell whitepaper busting solid state storage myths
Dell whitepaper busting solid state storage mythsNatalie Cerullo
 
S de0882 new-generation-tiering-edge2015-v3
S de0882 new-generation-tiering-edge2015-v3S de0882 new-generation-tiering-edge2015-v3
S de0882 new-generation-tiering-edge2015-v3Tony Pearson
 
Helathcare modernize-tebc105-v1704a
Helathcare modernize-tebc105-v1704aHelathcare modernize-tebc105-v1704a
Helathcare modernize-tebc105-v1704aTony Pearson
 
SOUG_SDM_OracleDB_V3
SOUG_SDM_OracleDB_V3SOUG_SDM_OracleDB_V3
SOUG_SDM_OracleDB_V3UniFabric
 
S016828 storage-tiering-nola-v1710b
S016828 storage-tiering-nola-v1710bS016828 storage-tiering-nola-v1710b
S016828 storage-tiering-nola-v1710bTony Pearson
 
Tape Storage Future Directions and the Data Explosion
Tape Storage Future Directions and the Data ExplosionTape Storage Future Directions and the Data Explosion
Tape Storage Future Directions and the Data ExplosionIBM India Smarter Computing
 
Ceph Day Chicago - Brining Ceph Storage to the Enterprise
Ceph Day Chicago - Brining Ceph Storage to the Enterprise Ceph Day Chicago - Brining Ceph Storage to the Enterprise
Ceph Day Chicago - Brining Ceph Storage to the Enterprise Ceph Community
 
Flash Storage Trends
Flash Storage TrendsFlash Storage Trends
Flash Storage TrendsOsys AG
 
T12.Fujitsu World Tour India 2016-Your Datacenter‘s backbone
T12.Fujitsu World Tour India 2016-Your Datacenter‘s backboneT12.Fujitsu World Tour India 2016-Your Datacenter‘s backbone
T12.Fujitsu World Tour India 2016-Your Datacenter‘s backboneFujitsu India
 
Webinar: Where is the All-Flash Data Center?
Webinar: Where is the All-Flash Data Center?Webinar: Where is the All-Flash Data Center?
Webinar: Where is the All-Flash Data Center?Storage Switzerland
 
Fujitsu - 27mai2011
Fujitsu - 27mai2011Fujitsu - 27mai2011
Fujitsu - 27mai2011Agora Group
 

Similar to Big Data and the Future of Storage (20)

Flash Ahead: IBM Flash System Selling Point
Flash Ahead: IBM Flash System Selling PointFlash Ahead: IBM Flash System Selling Point
Flash Ahead: IBM Flash System Selling Point
 
NVMe over Fibre Channel Introduction
NVMe over Fibre Channel IntroductionNVMe over Fibre Channel Introduction
NVMe over Fibre Channel Introduction
 
IBM Upgrades SVC with Solid State Drives — Achieves Better Storage Utilization
IBM Upgrades SVC with Solid State Drives — Achieves Better Storage UtilizationIBM Upgrades SVC with Solid State Drives — Achieves Better Storage Utilization
IBM Upgrades SVC with Solid State Drives — Achieves Better Storage Utilization
 
IBM Upgrades SVC with Solid State Drives — Achieves Better Storage Utilization
IBM Upgrades SVC with Solid State Drives — Achieves Better Storage UtilizationIBM Upgrades SVC with Solid State Drives — Achieves Better Storage Utilization
IBM Upgrades SVC with Solid State Drives — Achieves Better Storage Utilization
 
IMCSummit 2015 - Day 2 IT Business Track - Drive IMC Efficiency with Flash E...
IMCSummit 2015 - Day 2  IT Business Track - Drive IMC Efficiency with Flash E...IMCSummit 2015 - Day 2  IT Business Track - Drive IMC Efficiency with Flash E...
IMCSummit 2015 - Day 2 IT Business Track - Drive IMC Efficiency with Flash E...
 
In-Memory Computing: Myths and Facts
In-Memory Computing: Myths and FactsIn-Memory Computing: Myths and Facts
In-Memory Computing: Myths and Facts
 
Z109889 z4 r-storage-dfsms-jburg-v1909d
Z109889 z4 r-storage-dfsms-jburg-v1909dZ109889 z4 r-storage-dfsms-jburg-v1909d
Z109889 z4 r-storage-dfsms-jburg-v1909d
 
Z4R: Intro to Storage and DFSMS for z/OS
Z4R: Intro to Storage and DFSMS for z/OSZ4R: Intro to Storage and DFSMS for z/OS
Z4R: Intro to Storage and DFSMS for z/OS
 
Z109889 z4 r-storage-dfsms-vegas-v1910b
Z109889 z4 r-storage-dfsms-vegas-v1910bZ109889 z4 r-storage-dfsms-vegas-v1910b
Z109889 z4 r-storage-dfsms-vegas-v1910b
 
Dell whitepaper busting solid state storage myths
Dell whitepaper busting solid state storage mythsDell whitepaper busting solid state storage myths
Dell whitepaper busting solid state storage myths
 
S de0882 new-generation-tiering-edge2015-v3
S de0882 new-generation-tiering-edge2015-v3S de0882 new-generation-tiering-edge2015-v3
S de0882 new-generation-tiering-edge2015-v3
 
Helathcare modernize-tebc105-v1704a
Helathcare modernize-tebc105-v1704aHelathcare modernize-tebc105-v1704a
Helathcare modernize-tebc105-v1704a
 
SOUG_SDM_OracleDB_V3
SOUG_SDM_OracleDB_V3SOUG_SDM_OracleDB_V3
SOUG_SDM_OracleDB_V3
 
S016828 storage-tiering-nola-v1710b
S016828 storage-tiering-nola-v1710bS016828 storage-tiering-nola-v1710b
S016828 storage-tiering-nola-v1710b
 
Tape Storage Future Directions and the Data Explosion
Tape Storage Future Directions and the Data ExplosionTape Storage Future Directions and the Data Explosion
Tape Storage Future Directions and the Data Explosion
 
Ceph Day Chicago - Brining Ceph Storage to the Enterprise
Ceph Day Chicago - Brining Ceph Storage to the Enterprise Ceph Day Chicago - Brining Ceph Storage to the Enterprise
Ceph Day Chicago - Brining Ceph Storage to the Enterprise
 
Flash Storage Trends
Flash Storage TrendsFlash Storage Trends
Flash Storage Trends
 
T12.Fujitsu World Tour India 2016-Your Datacenter‘s backbone
T12.Fujitsu World Tour India 2016-Your Datacenter‘s backboneT12.Fujitsu World Tour India 2016-Your Datacenter‘s backbone
T12.Fujitsu World Tour India 2016-Your Datacenter‘s backbone
 
Webinar: Where is the All-Flash Data Center?
Webinar: Where is the All-Flash Data Center?Webinar: Where is the All-Flash Data Center?
Webinar: Where is the All-Flash Data Center?
 
Fujitsu - 27mai2011
Fujitsu - 27mai2011Fujitsu - 27mai2011
Fujitsu - 27mai2011
 

More from IBM Research

IBM Research - Zurich Celebrates 60 Years of Science and Innovation
IBM Research - Zurich Celebrates 60 Years of Science and InnovationIBM Research - Zurich Celebrates 60 Years of Science and Innovation
IBM Research - Zurich Celebrates 60 Years of Science and InnovationIBM Research
 
IBM/ASTRON DOME 64-bit Hot Water Cooled Microserver
IBM/ASTRON DOME  64-bit Hot Water Cooled MicroserverIBM/ASTRON DOME  64-bit Hot Water Cooled Microserver
IBM/ASTRON DOME 64-bit Hot Water Cooled MicroserverIBM Research
 
IBM and ASTRON 64-Bit Microserver Prototype Prepares for Big Bang's Big Data,...
IBM and ASTRON 64-Bit Microserver Prototype Prepares for Big Bang's Big Data,...IBM and ASTRON 64-Bit Microserver Prototype Prepares for Big Bang's Big Data,...
IBM and ASTRON 64-Bit Microserver Prototype Prepares for Big Bang's Big Data,...IBM Research
 
IBM and ASTRON 64bit μServer for DOME
IBM and ASTRON 64bit μServer for DOMEIBM and ASTRON 64bit μServer for DOME
IBM and ASTRON 64bit μServer for DOMEIBM Research
 
A Prototype Storage Subsystem based on Phase Change Memory
A Prototype Storage Subsystem based on Phase Change MemoryA Prototype Storage Subsystem based on Phase Change Memory
A Prototype Storage Subsystem based on Phase Change MemoryIBM Research
 
The New Era of Cognitive Computing
The New Era of Cognitive ComputingThe New Era of Cognitive Computing
The New Era of Cognitive ComputingIBM Research
 
Das IBM Forschungslabor als Arbeitgeber
Das IBM Forschungslabor als ArbeitgeberDas IBM Forschungslabor als Arbeitgeber
Das IBM Forschungslabor als ArbeitgeberIBM Research
 
Nano, SuperMUC and Photovoltaics:A Day in the Life of IBM Research - Zurich
Nano, SuperMUC and Photovoltaics:A Day in the Life of IBM Research - ZurichNano, SuperMUC and Photovoltaics:A Day in the Life of IBM Research - Zurich
Nano, SuperMUC and Photovoltaics:A Day in the Life of IBM Research - ZurichIBM Research
 
Dechema Conference: Istanbul
Dechema Conference: IstanbulDechema Conference: Istanbul
Dechema Conference: IstanbulIBM Research
 
IBM Research: IBM 2010 Investor Briefing
IBM Research: IBM 2010 Investor BriefingIBM Research: IBM 2010 Investor Briefing
IBM Research: IBM 2010 Investor BriefingIBM Research
 

More from IBM Research (11)

IBM Research - Zurich Celebrates 60 Years of Science and Innovation
IBM Research - Zurich Celebrates 60 Years of Science and InnovationIBM Research - Zurich Celebrates 60 Years of Science and Innovation
IBM Research - Zurich Celebrates 60 Years of Science and Innovation
 
IBM/ASTRON DOME 64-bit Hot Water Cooled Microserver
IBM/ASTRON DOME  64-bit Hot Water Cooled MicroserverIBM/ASTRON DOME  64-bit Hot Water Cooled Microserver
IBM/ASTRON DOME 64-bit Hot Water Cooled Microserver
 
IBM and ASTRON 64-Bit Microserver Prototype Prepares for Big Bang's Big Data,...
IBM and ASTRON 64-Bit Microserver Prototype Prepares for Big Bang's Big Data,...IBM and ASTRON 64-Bit Microserver Prototype Prepares for Big Bang's Big Data,...
IBM and ASTRON 64-Bit Microserver Prototype Prepares for Big Bang's Big Data,...
 
IBM and ASTRON 64bit μServer for DOME
IBM and ASTRON 64bit μServer for DOMEIBM and ASTRON 64bit μServer for DOME
IBM and ASTRON 64bit μServer for DOME
 
A Prototype Storage Subsystem based on Phase Change Memory
A Prototype Storage Subsystem based on Phase Change MemoryA Prototype Storage Subsystem based on Phase Change Memory
A Prototype Storage Subsystem based on Phase Change Memory
 
The New Era of Cognitive Computing
The New Era of Cognitive ComputingThe New Era of Cognitive Computing
The New Era of Cognitive Computing
 
Das IBM Forschungslabor als Arbeitgeber
Das IBM Forschungslabor als ArbeitgeberDas IBM Forschungslabor als Arbeitgeber
Das IBM Forschungslabor als Arbeitgeber
 
Nano, SuperMUC and Photovoltaics:A Day in the Life of IBM Research - Zurich
Nano, SuperMUC and Photovoltaics:A Day in the Life of IBM Research - ZurichNano, SuperMUC and Photovoltaics:A Day in the Life of IBM Research - Zurich
Nano, SuperMUC and Photovoltaics:A Day in the Life of IBM Research - Zurich
 
Meet IBM Research
Meet IBM ResearchMeet IBM Research
Meet IBM Research
 
Dechema Conference: Istanbul
Dechema Conference: IstanbulDechema Conference: Istanbul
Dechema Conference: Istanbul
 
IBM Research: IBM 2010 Investor Briefing
IBM Research: IBM 2010 Investor BriefingIBM Research: IBM 2010 Investor Briefing
IBM Research: IBM 2010 Investor Briefing
 

Recently uploaded

New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfRankYa
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 

Recently uploaded (20)

New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 

Big Data and the Future of Storage

  • 1. Big Data and the Future of Storage Dr. Evangelos Eleftheriou IBM Fellow
  • 2. © 2014 International Business Machines Corporation 2 Big Data: More than just volume Volume Terabytes to exabytes of existing data to store and process Velocity Streaming data, milliseconds to seconds to respond Variety Structured, unstructured, text & multimedia Veracity Uncertainty from inconsistency, ambiguities, etc.
  • 3. © 2014 International Business Machines Corporation 3 Memory / Storage Stack Significant advancements in non-volatile memories blur the boundaries between storage and memory by being low-cost, fast, and cost- effective. 101 1 102 103 104 105 106 107 108 109 1010 CPU Operation (1ns) Fetch data from the L2 Cache (10ns) Fetch data from DRAM (60ns) Read / Write SCM (100 – 2000 ns) Read from a Flash device (20 μs) Write to a Flash device (1 ms) Read from / Write to disk (5 ms) Read from / Write to tape (40 s) Memory SCM Storage Time(ns) second minute hour day week month year decade century SLOW FAST
  • 4. © 2014 International Business Machines Corporation 4 IBM FlashSystem: The Tipping Point • The first time Flash storage outperformed hard disks in all aspects, including capacity and performance density, cost per I/O Operations Per Second (IOPS) and energy efficiency! " • With IBM FlashSystem 820 we achieved more than 6 million IOPS running an IBM DB2 workload on IBM Power servers
 • 19 kilowatts vs. 4.5 megawatts with high capacity hard disks, 236x better
 • Installed in less than 48 Hours, HDD would require two years !
  • 5. © 2014 International Business Machines Corporation 5 IBM FlashSystem: The Tipping Point
  • 6. © 2014 International Business Machines Corporation 6 Storage infrastructure in the new era High IOPS Ultra-low Latency Memory-speed Data Processing Infrastructure Elasticity Reduced time for deployment Bottomless Capacity Optimized TCO Integrated global infrastructure Simplicity Security Reliability Pay-as-you-go Extreme Scalability Block Interface Novel Interfaces MicroLatency Storage Global File System POSIX NFS CIFS Object GPFS Active! File Management POSIX CIFS ObjectNFS ObjectNFSPOSIX CIFS GPFS GPFS
  • 7. © 2014 International Business Machines Corporation 8 Tape Storage: Big Data Needs Tape • Faster than Hard Disks at streaming! ! • Reliability: read after three decades, against five years for disks! ! • Zero Power Consumption (when idle)! ! • Security: 50 Petabytes on an HDD can be deleted in minutes, tape would take years! ! • Cost: 1 GB of disk storage costs 10 cents, versus 4 cents for tape ! ! • The Next Target: the 100 TB cartridge The Economist, Magnetic tape to the rescue. 30 Nov 2013
  • 8. © 2014 International Business Machines Corporation 9 Multi-cloud Storage Tool Kit (live demo in CeBIT booth) • Can connect to one or more clouds including: 
 IBM Softlayer, AWS, Azure, Rackspace 
 (public, private, or hybrid) ! • Enterprise features, including: encryption, integrity, and resiliency, it’s transparent to GPFS and manages keys/ metadata! • Drag-and-drop usability
  • 9. © 2014 International Business Machines Corporation© 2014 International Business Machines Corporation !9
  • 10. © 2014 International Business Machines Corporation 10 A universal, non-volatile memory technology superior to Flash. PCM is very durable and can endure at least 10 million write cycles, compared to current enterprise-class flash at 30,000 cycles or consumer- class flash at 3,000 cycles. Phase Change Memory (PCM) DRAM Flash PCM Multi-Level PCM Timeframe Invented in 1966 Invented in 1980s Used in smartphones – wide adoption expected by 2016 2016 Speed Density Endurance Retention Scaling Best in class Good / Adequate Average / Inadequate Bad / Worst in class ➢ The qualification is relative and depends on the application! ➢ Racetrack memory is not included in this time horizon
  • 11. © 2014 International Business Machines Corporation 11 Theseus Project: PCM-based 
 PCI-e Prototype Card
 Hybrid Storage/Caching subsystem based on PCM • Design and implementation of a high- performance PCI-e card using PCM! • Consistent ultra-low latency, high IOPS even for very small operations! • Integration of PCM and Flash for hybrid use cases: ! • Caching! • Tiering! • Persistent Key/Value Store