Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Sn wf12 amd fabric server (satheesh nanniyur) oct 12


Published on

Big Data has influenced the data center architecture in ways unimagined before. This presentation explores the Fabric Compute and Storage architectures to enable extreme scale-out, low power, high density Big Data deployments

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Sn wf12 amd fabric server (satheesh nanniyur) oct 12

  1. 1. Fabric ArchitectureA Big Idea for the Big Data infrastructureSatheesh NanniyurSenior Product Line ManagerAMD Data Center Server Solutions (formerly, SeaMicro)
  2. 2. Agenda• Defining Big Data from an Infrastructure perspective• Fabric Architecture for Big Data• An overview of the Fabric Server and Fabric Storage• Illustrating Fabric Architecture Benefits for Hadoop• Conclusion
  3. 3. Have you come across Big Data? Apple’s virtual smartphone assistant, Siri, uses complex machine learning techniques Target’s “pregnancy prediction score” – NY Times: “How companies learn your secrets” – Feb 2012
  4. 4. So, what really is Big Data?Business• “Key basis of competition and growth…”Observational• “Too big, moves too fast, or doesn’t fit the structures of your database”Mathematical• “Every day, we create 2.5 “million trillion” (quintillion) bytes of data"Systems• “Exceeds the processing capacity of conventional database”
  5. 5. The Infrastructural definition of Big Data Massive • Store “all” data not knowing its Storage use in advance Massive • Ask a query, and when you do, Compute get the answer fast
  6. 6. Big Data infrastructure is not business asusual Massive • Petabyte scale high density storage Storage • Flexible storage to compute ratio to meet evolving business needs Massive • High density scale-out compute Compute • Power and space efficient infrastructure The IT architectural approach used in clustered environments such as a large Hadoop grid is radically different from the converged and virtualized IT environmentsIDC White Paper, “Big Data: What It Is and Why You Should Care”
  7. 7. Fabric Architecture for Big DataThe holy grail of Big Data InfrastructureImagine a world where you could simply stack up servers,with each server: Flexible Fraction of a Share over 5 10GE network provisioning of rack unit PB of storage with no cabling storage
  8. 8. A deeper look at the traditional rack-mount architecture Aggregation ToR Cabling and Management Nodes • Compromise between Compute and Storage density • Rigid compute to storage ratio • Oversubscribed network suitable for north-south traffic, not heavy east-west required for Big Data • Too many adapters (NIC, Storage Ctlr) and cabling that can fail
  9. 9. Fabric with 3-D Torus for Big DataInfrastructure Big Data is a big shift from North-South traffic to East-West High Speed and Low Latency Interconnection Switchless Linear Scalability that avoids bottlenecks Highly available network minimizing node loss and data reconstruction High density scale-out architecture with low power and space
  10. 10. An overview of the Fabric Server Y+ X-Z+X+ PCIe Z- • 512 x86 cores with 4TB Y- DRAM in 10RU x86 Server • Up to 5 petabytes of SeaMicro Fabric Node with storage IOVT • Flexible Storage to Compute ratio • 10GE network per server 160GE of uplink bandwidth
  11. 11. Fabric Storage ... for Big Data? Isn’t Big Data always deployed with DAS? “.. the rate of change was killing us, where the data volumes were practically doubling every month. Trying to keep up with that growth was an extreme challenge to say the least.. “ Customer quote from IDC white paper - “Big Data – What It Is and Why You Should Care” Underutilized Compute • Add storage capacity & Network Rigid Storage to Compute independent of compute Ratio (Traditional Rackmount) to increase cluster efficiencyCompute Flexible Fabric Storage to • Flexibly provision storage Compute Ratio capacity to meet evolving customer needs Storage
  12. 12. Massive capacity scale-out Fabric Storage• Massive scale-out capacity with commodity drives• Decoupled from Compute and Network to grow storage independently Captive DAS with Rigid Flexible scale-out Fabric Storage Storage to Compute Ratio up to 5PB Intel /AMD x86 Freedom Fabric servers Traditional Rackmount
  13. 13. Hadoop and the SMAQ stack Built to scale linearly with massive scale-out storage (HDFS) and compute (MapReduce) Query Pig, Hive MapReduce Data Processing Framework Data Storage HDFS
  14. 14. Hadoop data processing phasesFabric Architecture cost efficiently meets the Hadoopinfrastructure needs Storage Compute Network Compute Storage Intensive Intensive Intensive Intensive Intensive Map Reduce Map Reduce Map HDFS Map and Shuffle Reduce HDFS Input Intermediate Output Data Write512 x86 cores 5 Petabytes of 10 Gpbs Inter- 160 Gbps sharedwith 4TB DRAM storage capacity Node Bandwidth uplink for Inter-per Fabric Server with independent per server Rack trafficin 10RU scale-out
  15. 15. Hadoop resource usage pattern Based on Terasort run on SeaMicro SM15000 MapCompute Shuffle Reduce Map ShuffleStorage Reduce ShuffleNetwork
  16. 16. Deployment Challenges of Hadoop• Plan for peak utilization – Hadoop infrastructure utilization is bursty• Compute, Storage, and Network mix dependent on application workload – Flexible ratios optimize deployment• Power and Space Efficiency key to large scale deployment• Administrative cost can increase as rapidly as your data – Simplified deployment and reduced hardware components decrease TCO
  17. 17. Fabric Server for Hadoop DeploymentFabric Server offers 60% more compute and storage in the samepower and space envelope Traditional SeaMicro Fabric Rackmount Server Intel Xeon Cores 320 512 AMD Opteron Cores* 320 1024 Storage 720 TB 1136 TB Storage Scalability None Up to 4PB Network B/W per Up to 2Gbps Up to 8Gbps server Network Downlinks 40 0 ToR Switch 2 0 (Built-in) Aggregation (End of 1 1 Row) switch/router Based on SeaMicro SM15000 and HP DL380 Gen8 2U dual socket octal core servers in a 42U rack
  18. 18. Summary Traditional architectures cannot scale to meet the needs of Big DataEfficient Big Data deployments need flexible storage to compute ratio Conventional wisdom of reduced hardware components still holds Fabric Servers provide unprecedented density, bandwidth, and scalability for Big Data deployments
  19. 19. more information, visit or