Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos

1,677 views
1,597 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,677
On SlideShare
0
From Embeds
0
Number of Embeds
21
Actions
Shares
0
Downloads
24
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos

  1. 1. ANALYST SUMMIT 2007 Redshift: The Explosion of Massive Scale Systems Dr. Greg Papadopoulos Executive Vice President, R&D Chief Technology Officer
  2. 2. These slides contain forward-looking statements regarding the future results and performance of Sun Microsystems, Inc., including statements regarding future industry trends and growth, demand for HPC and future trends in computing. These forward- looking statements involve risks and uncertainties, and actual results could differ materially from those contained in these forward-looking statements. Factors that could cause actual results to differ materially from those contained in these forward-looking statements include: risks associated with developing, designing, manufacturing and distributing new products; lack of success in technological advancements; pricing pressures; lack of customer acceptance of new products; the possibility of errors or defects in new products; competition; adverse business conditions; failure to retain key employees; the cancellation or delay of projects; our reliance on single-source suppliers; risks associated with our ability to purchase a sufficient amount of components to meet demand; inventory risks; risks associated with our international customers and operations; delays in product development or customer acceptance and implementation of new products and technologies; our dependence on significant customers and specific industries; and our dependence on channel partners. Please also refer to Sun's periodic reports that are filed with the Securities and Exchange Commission, including Sun's annual report on Form 10- K for the fiscal year ended June 30, 2006 and its quarterly report on Form 10-Q for the fiscal quarter ended October 1, 2006. Sun assumes no obligation to, and currently does not intend to, update these forward-looking statements.
  3. 3. 25 Yrs: 100,000 to 1M Times More MIPS/$ 1010 105 10,000MIPS/$1K MIPS/1000 USD 100 0.01 MIPS/$1K 10-5 10-10 Hermann Brunner, Max-Planck-Institut fuer Extraterrestriche Physik, Germany -15 10 1880 1900 1920 1940 1960 1980 2000 2020 Year
  4. 4. Where is the Demand? 1M 10K Processor Performance 1K T1000 Sun 1 10,000 MIPS 100 1 MIPS $5K (2005$) $40K (2005$) 10 1 $17T World GDP $36T .1 1985 1990 1995 2000 2005 2010 2015
  5. 5. Core per Enterprise Demand 1M 1000x 100K 10K 1K Moore’s Law 100 10 1 Core Enterprise Demand .1 1995 2000 2005 2010
  6. 6. “Last Mile” Bandwidth 10G 1G Fiber 100M 50–100m 22m 10M DSL/Cable 4m 1M 1.55m 56k 100K 384k POTS 28.8k 10K 14.4k 9.6kbps 1K 1985 1990 1995 2000 2005 2010 2015
  7. 7. ΣBW – Filling the Pipe ΣBW Ning In = Out
  8. 8. ΣBW – Filling the Pipe ΣBW Ning In = Out # Servers * BW/Server = # Devices * BW/Device
  9. 9. ΣBW – Filling the Pipe 1M 1M ΣBW 100K 10K 1K 1,000 1,000 Moore’s Law 100 50 Core 10 1 .1 1995 2000 2005 2010
  10. 10. HPC – Insatiable Demand
  11. 11. HPC – Insatiable Demand 1M ΣBW 100K HPC 10K 1K Moore’s Law 100 Core 10 1 .1 1995 2000 2005 2010
  12. 12. *-Prise Workday Network.com Internet
  13. 13. *-Prise 1M ΣBW 100K HPC *Prise 10K 1K Moore’s Law 100 CRM Core 10 1 .1 1995 2000 2005 2010
  14. 14. Redshift – a Move to Massive Scale 1M ΣBW 100K HPC *Prise 10K 1K Moore’s Law 100 Core 10 1 .1 1995 2000 2005 2010
  15. 15. Diverging Concerns 1M ΣBW 100K HPC *Prise 10K 1K Moore’s Law 100 Core 10 Consolidation Cost 1 Services .1 1995 2000 2005 2010
  16. 16. Diverging Concerns 1M ΣBW Scale HPC 100K Efficiency Products *Prise 10K 1K Moore’s Law 100 Core 10 Consolidation Cost 1 Services .1 1995 2000 2005 2010
  17. 17. “The Google Question” If all you cared about was massive scale, what would you build and how would you operate it?
  18. 18. An Answer With “Brutal Efficiency” • Hardened metrics (utilization, power, security) • Predictability of service level • Idea-to-deploy time and productivity
  19. 19. An Answer With “Brutal Efficiency” • Hardened metrics (utilization, power, security) • Predictability of service level • Idea-to-deploy time and productivity Deep, At-scale Engineering of Systems
  20. 20. How Deep? Efficiency and predictability at massive scale are as mission-critical to Redshift as RAS has been to the core enterprise
  21. 21. Don’t Confuse Commoditization of Computers
  22. 22. Don’t Confuse With Commoditization of Commoditization of Computers Computing
  23. 23. System Includes... Network Service Core Services and Platforms O/S Instances Base HW Plant (Servers, Storage and Switches)
  24. 24. System Includes... Network Service Core Services and Platforms Massively Parallel O/S Instances System Base HW Plant (Servers, Storage and Switches)
  25. 25. SMPs Are Back (With a Vengeance!) 64 Threads in 1997 E10K Full Rack Size 9620 Watts Power (Systems at peak utilization) 1,800 lbs. Weight ~150k tpm Performance
  26. 26. SMPs Are Back (With a Vengeance!) z 64 Threads in 1997 64 Threads in 2007 E10K Niagara 2 Full Rack Size 9620 Watts Power (Systems at peak utilization) 1,800 lbs. Weight ~150k tpm Performance
  27. 27. SMPs Are Back (With a Vengeance!) z 64 Threads in 1997 64 Threads in 2007 E10K Niagara 2 Full Rack Size 1U 30x 9620 Watts Power 24x 410 Watts (Systems at peak utilization) 1,800 lbs. Weight 45 lbs. 40x ~150k tpm Performance 2x ~300k tpm
  28. 28. SMPs Are Back (With a Vengeance!) z 64 Threads in 1997 64 Threads in 2007 E10K Niagara 2
  29. 29. Neptune and Crossbow • Solaris virtual networking stacks for high speed networks, based on Crossbow protocol, service, or container • Does the “careful” unleashing of 10G • A shared, virtualized, non-blocking, multi-homed 10 Gigabit Ethernet Neptune PCI-e network interface • Gets the most out of your system and your I/O bus > Processors get faster faster than I/O buses get faster – PCI-e is (or soon will be) your bottleneck
  30. 30. Neptune and Crossbow Solaris Zones 1000’s Logical Processes Connections Virtualized Network Streams Solaris PCIe Crossbow Neptune 4x10GE and PEF MultiCore Microprocessor
  31. 31. Java RTS + Solaris = Open Real Time Operating System Application Java RTS 2.0 Solaris Hardware
  32. 32. Java RTS + Solaris: 1000 Times More Predictable Maximum Latency Jitter 20 10 microseconds microseconds RTS 10 5 milliseconds milliseconds 12 7 milliseconds milliseconds
  33. 33. Real-Time Application Server Results Comparison of Conventional Java SE to Java RTS Distribution of Request/Reply Round-trip Times 10,000 Number of Requests 1,000 Java RTS Results: ALL priority transactions complete in 11 milliseconds or less Standard Java SE Results: Priority 100 transactions range from 11 ms to 3.5 seconds 10 1 0 200,000 400,000 600,000 800,000 Round-trip Time (US)
  34. 34. Real-Time Java on Solaris: Inverted Pendulum Greg Bollella Distinguished Engineer, Sun Microsystems
  35. 35. NetBeans IDE Support Download the Java RTS Plugin, and • Cross-develop on the host • Deploy over the network • Execute on the target • ...from the NetBeans IDE
  36. 36. Co-Design Computer + Storage + Network + Power + Cooling + Software = BLACKBOX
  37. 37. But Wait, There’s More • The next wave will include leading-edge enterprises • Critical to this wave will be the bridge between core IT and redshifting services: Identity and security Procedural languages and scripting SOA and Web 2.0 New clients
  38. 38. The Bridge 1M ΣBW 100K HPC *Prise 10K Identity RSWS 1K New 100 Clients 10 1 .1 1995 2000 2010
  39. 39. Enterprise Bridge Workday Network.com Internet Core Enterprise
  40. 40. Putting it all Together: The Binary Distribution Network Service Core Services and Platforms “The Distro” O/S Instances Base HW Plant (Servers, Storage and Switches)
  41. 41. Source vs. Binary Platform Developers Platform (Open Source) open Java open ...
  42. 42. Source vs. Binary Platform Developers Platform (Open Source) Distribution open Java open ... 1.1 1.2 1.2.1 Binary Distribution ...
  43. 43. Source vs. Binary Platform Developers Application Developers Platform Applications (Open Source) Distribution open Java Application Binaries open ... 1.1 1.2 1.2.1 Binary Distribution ...
  44. 44. A Likely Scenario Big Computers 100Ks – 1Ms Nodes Globally Distributed GCC Live.com Devices
  45. 45. A Likely Scenario Big Computers 100Ks – 1Ms Nodes Globally Distributed GCC Live.com Distribution Networks Devices
  46. 46. A Likely Scenario Big Computers 100Ks – 1Ms Nodes Startups Globally Distributed GCC Live.com Distribution Networks Devices
  47. 47. ANALYST SUMMIT 2007 Thank the Computer. The Network is You. TM Dr. Greg Papadopoulos Executive Vice President, R&D Chief Technology Officer

×