Application Driven Datacenter Computing
Upcoming SlideShare
Loading in...5
×
 

Application Driven Datacenter Computing

on

  • 531 views

 

Statistics

Views

Total Views
531
Views on SlideShare
531
Embed Views
0

Actions

Likes
1
Downloads
7
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Application Driven Datacenter Computing Application Driven Datacenter Computing Presentation Transcript

    • Applicaon-­‐Driven    Datacenter  Compung   Shiding Lin EDCS-HPCA, Shenzhen 2013/2/24
    • Let’s Start from the Search Engine… Web Web   Pages   Index   Central  Repository   Building Inverted   of   Index   Web  Pages   Data   Mining
    • To Build a High-Throughput Storage System In-Memory Records Update Query {<key, data>} Dump MemoryLog-­‐based  Structure   Log Block-N Disk   …        Block  I/O   Log Block-1 Commit New stream        Batch  Commit   Stream Block 0        Stream  R/W   Block0 Block 1 … Block 1 Block Y … Block X
    • To Build a High-Throughput Storage System Block Block <key, data> …Maximize  Parallelism   Memory dump   Disk        NO  RAID,  Raw  Disk   Block Block Block        Direct  I/O   @disk0 @disk1 … @diskN        Independent  of  FS   A  Big  Virtual  File  by  Blocks  
    • 3-Layer Architecture of a Typical Storage System Table   Base  Stream   Mod  Stream   Index  Stream   Patch  Stream   Block   Block   Block   …   Block   Block   Block  
    • To Make It Large-Scale Which  Layer  to  ParNNon,  and  the  ReplicaNon  Granularity?            Complexity          Data  Exchange  Traffic          Reliability  
    • Replication Scheme 1Table   Table   Table   Base   Mod   Base   Mod   Base   Mod   Stream   Stream   Stream   Stream   Stream   Stream   Index   Patch   Index   Patch   Index   Patch   Stream   Stream   Stream   Stream   Stream   Stream  Block   …   …   Block   Block   …   …   Block   Block   …   …   Block   Replica  1   Replica  2   Replica  3   3x  Commit  Cost   Local  I/O  Only  
    • Replication Scheme 2 Base  Stream   Mod  Stream  Block   …   Block   …   Block   …   Block   …   Block   …   Block   …   Replica  1   Replica  2   Replica  3   Replica  1   Replica  2   Replica  3   Index  Stream   Patch  Stream  Block   …   Block   …   Block   …   Block   …   Block   …   Block   …   Replica  1   Replica  2   Replica  3   Replica  1   Replica  2   Replica  3   1x  Commit  Cost   Network  &  Disk  I/O  
    • Map to Physical ArchitectureLogical  Layer   Physical  Boundary   Physical  Layer          Table          Datacenter          Memory          Stream          Cluster          Flash          Block          Rack          Disk          Node  
    • What Are Changed? Single-­‐User  MulN-­‐Task  à  MulN-­‐User  Single-­‐Task     Scale  &  Cost     Speed  of  Delivery  
    • Software Architecture Principles in Datacenter Layered  à  VerNcal       Out-­‐of-­‐the-­‐Box          Datacenter  as  a  Computer          To  Tolerate  Component  Failure  
    • Hardware Architecture Principles in Datacenter Dummy          Control  Logic  Goes  SoXware          ReplicaNon/Checksum/Buffer  Goes  Global     Programmable          Expose  All  Interfaces          Collect  All  Data  
    • Hardware Architecture Principles in Datacenter Modularized  and  Configurable     Reduce  All  the  Unnecessary     Share  All  the  Possible  
    • Practice 1: Baidu SSD Raw  Channels   No  Shadow  Buffer   No  Wear  Leveling  
    • Practice 2: Smart Disk Replacement Failure  and  Repair   Predict  Failure   Logs   Reduce  False-­‐Alarm   Failure  Model  
    • Practice 3: ARM Server2U,  6  Nodes,  12  HDD/U  Internal  Network  Switch