1 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Dr. Axel Koester – Storage Chief Technologi...
2 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013BIG DATA ≠≠≠≠ MUCH DATA
3 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013MUCH DATAExample
4 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013www.extremetech.compress release August 2011
5 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Details of the 120 PB ClusterNo "Replicatio...
6 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013RAID technology wont work in a 200.000 disk...
7 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013There is always something broken, somewhere...
8 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013120 PB storage grid : One of many p775 stor...
9 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Power 775 rack with 3 "Dense Disk Enclosure...
11 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013IBM GSS*: "120 PB predictive technology" f...
12 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Cost per 1MB sequenced genomic datawww.cro...
13 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Sometimes…MUCH DATA » BIG DATA
14 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013BIG DATA without MUCH DATA
15 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Watson answers complex "trivia" questions ...
16 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Jeopardy! all-time champions Ken Jennings ...
17 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Watsons "knowledge" is collected, not code...
18 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Watson holds all its knowledge in RAM90 × ...
19 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Watsons inventors surprised by amazing cor...
20 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013BIG DATA and ENERGY
21 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 201320 Watt20 WattIBM Watson: 200.000 Watt
22 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Human brains are incredibly good at "low p...
23 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Root causeCoronary Syndrome 60%Pneumonia 2...
24 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Projects SyNAPSE and BlueBrain: Simulating...
25 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013The brain transistorno more science-fictio...
26 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013March 2013 : IBM publishes a liquid-based ...
27 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013by IBM Fellow Stuart Parkin, inventor of "...
28 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013IT energy problems? Start learning from na...
30 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013axel.koester@de.ibm.com
Upcoming SlideShare
Loading in...5
×

Technology Outlook - The new Era of computing

1,910

Published on

From the lab to the data center - an outlook on future storage technologies and current storage research projects with focus on scalability.

Published in: Technology, Business
1 Comment
1 Like
Statistics
Notes
  • This talk was held at the 7th meeting on May 13 at IBM Zurich by Dr. Axel Koester, Chief Storage Technologist, IBM European Storage Competence Center
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
1,910
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

Technology Outlook - The new Era of computing

  1. 1. 1 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Dr. Axel Koester – Storage Chief Technologist, European Storage Competence CenterFuture perspectives: the new Era of ComputingBIG DATA from a storage point of view
  2. 2. 2 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013BIG DATA ≠≠≠≠ MUCH DATA
  3. 3. 3 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013MUCH DATAExample
  4. 4. 4 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013www.extremetech.compress release August 2011
  5. 5. 5 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Details of the 120 PB ClusterNo "Replication Cluster" like Amazon EC2 or Google Cloud– no 3 copies of each data blockWhy?– Would require 550.000 instead of 200.000 disk drives– and produce 30 instead of 12 daily failures (at identical net capacity)
  6. 6. 6 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013RAID technology wont work in a 200.000 disk cluster1TB disk drives × 200.0008+3 software RAID clusterJBODJBOD JBODJBOD JBODJBOD JBODJBODJBODJBOD JBODJBOD JBODJBOD JBODJBODJBODJBOD JBODJBOD JBODJBODhttp://www.almaden.ibm.com/storagesystems/projects/perseus/No spareNo spare
  7. 7. 7 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013There is always something broken, somewhere8+3 Reed Solomon encoding = tolerates up to 3 faultsJBODJBOD JBODJBOD JBODJBOD JBODJBODJBODJBOD JBODJBOD JBODJBOD JBODJBODJBODJBOD JBODJBOD JBODJBODhttp://www.almaden.ibm.com/storagesystems/projects/perseus/One fault: low rebuild thread priorityTwo faults: prioritize rebuildThree faults: rebuild asap< 4 min 20 sec in this stateNo spareNo spare
  8. 8. 8 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013120 PB storage grid : One of many p775 storage enclosuresDense Disk Enclosure – 384 disks per unit (192 front & back)2% Flash SSD for metadata
  9. 9. 9 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Power 775 rack with 3 "Dense Disk Enclosures" (1152 drives)
  10. 10. 11 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013IBM GSS*: "120 PB predictive technology" for everyone2x IBM x3650 M42x IBM x3650 M4GPFS1PB*16GB/s1PB*16GB/s2U2U4U4U4U4U240 SAS Disks @ 4TB(currently 720 TB @3TB disks)(*) GSS = GPFS Storage Server, RAID-less (General Parallel File System)4 M odules – 240 disks1 PB4 M odules – 240 disks1 PB
  11. 11. 12 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Cost per 1MB sequenced genomic datawww.crops.orgFor whom?predictedactual» Statistical Medicine
  12. 12. 13 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Sometimes…MUCH DATA » BIG DATA
  13. 13. 14 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013BIG DATA without MUCH DATA
  14. 14. 15 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Watson answers complex "trivia" questions from any subject area.Real money is at stake.
  15. 15. 16 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Jeopardy! all-time champions Ken Jennings & Brad RutterYou dont have this hereditarylack of pigment. You just needa little more sun!
  16. 16. 17 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Watsons "knowledge" is collected, not codedKnowledge base = English WWWMultiple millions of analyses per second200 million memorized book pages,with Wikipedia alone totaling 2,25 Mio.10km10km~2000 years to read
  17. 17. 18 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Watson holds all its knowledge in RAM90 × 32 core IBM Power®750 / 16 TB RAM, 1 TB data, 500 GB/secout of 4 TB GPFS disk space, with 16TB = 15 TiB RAM"information aggregator""information aggregator"*Parallel access to 100% of the data,no Internet access during games
  18. 18. 19 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Watsons inventors surprised by amazing correct answersDr. Jennifer Chu-Carrol,Watson algorithmsWhy? Because Watsons knowledge is collected, not coded.Only rules are coded, but subject to machine learning.
  19. 19. 20 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013BIG DATA and ENERGY
  20. 20. 21 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 201320 Watt20 WattIBM Watson: 200.000 Watt
  21. 21. 22 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Human brains are incredibly good at "low power"http://www.ibm.com/smarterplanet/us/en/business_analytics/article/cognitive_computing.html20 Watt20 WattRecognize a face in a crowd – efficientlyDistinguish own from outside soundCombine unrelated factsFilter & distill informationHow?Brains are not 100% accurate. Bit errors dont bother.
  22. 22. 23 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Root causeCoronary Syndrome 60%Pneumonia 25%Pulmonary Embolism 9%Congestion predictionRing A99 in 2 hrs 95%Feeder A8 in 1 hr 90%Energy productionLine load estimationProduction mix %:"Lower power" from abandoning the 100% bit-accurate ITPatient symptomsRoad traffic sensorsWind & sun forecast:UnreliableinputdataApproximationApproximations based on unreliable datashould not require bit-accurate processing !
  23. 23. 24 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013Projects SyNAPSE and BlueBrain: Simulating the brainhttp://www.ibm.com/smarterplanet/us/en/business_analytics/article/cognitive_computing.htmlUnderstandlearning downto synapsesUnderstandlearning downto synapsesExplore large-scale brainsimulationsExplore large-scale brainsimulationsDesign a chipthat "learns" atmolecular levelDesign a chipthat "learns" atmolecular level
  24. 24. 25 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013The brain transistorno more science-fiction, since March
  25. 25. 26 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013March 2013 : IBM publishes a liquid-based transistorthat process data like the human brain
  26. 26. 27 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013by IBM Fellow Stuart Parkin, inventor of "Racetrack Memory"Droplet that can be turned into"liquid metal" and switch currentsStuart ParkinStuart Parkin“We turn this material into ametal and maintain it withoutany need to supply power.”metalized ions liquidThe programmable liquid can be theinformation conveyor, not just a bit cell
  27. 27. 28 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013IT energy problems? Start learning from nature!copied nature « » optimized engineering
  28. 28. 30 © 2013 IBM Corporationaxel.koester@de.ibm.comIBM Big Data Usergroup, Mai 2013axel.koester@de.ibm.com

×