Sun sparc enterprise t5120 and t5220 servers technical presentation
Upcoming SlideShare
Loading in...5
×
 

Sun sparc enterprise t5120 and t5220 servers technical presentation

on

  • 700 views

 

Statistics

Views

Total Views
700
Slideshare-icon Views on SlideShare
700
Embed Views
0

Actions

Likes
0
Downloads
20
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Sun sparc enterprise t5120 and t5220 servers technical presentation Sun sparc enterprise t5120 and t5220 servers technical presentation Presentation Transcript

    • Sun SPARC EnterpriseT5120 and T5220Rack Server TTTPeter A. WilsonTechnical MarketingSun Microsystems, Inc. 1
    • Agenda• Introduction• Overview of UltraSPARC T2• Sun SPARC Enterprise T5x20 servers > Comparison to Sun Fire T1000/T2000 > System Architecture and Tours > Memory, Networking and IO Expansion > Disks, Fans and Power Supplies > Solaris for T5x20• ILOM 2.0• LDOMs for T5x20• CRUs/FRUs and Configuration Options• Tuning and Performance• Tools, Information and References• Q & A ... Sun Internal and Approved Partners Only 2
    • CMT Processor Strategy First True System on a Chip• Scaling performance with frequency is a dead-end• Multi-core/Multi-thread processors can exploit application parallelism• Able to Aggressively attack memory latency• Provide rich availability feature set• Integrate networking, I/O and security capability• Virtualize the architecture Sun Internal and Approved Partners Only 3
    • Agenda• Introduction• Overview of UltraSPARC T2• Sun SPARC Enterprise T5x20 servers > Comparison to Sun Fire T1000/T2000 > System Architecture and Tours > Memory, Networking and IO Expansion > Disks, Fans and Power Supplies > Solaris for T5x20• ILOM 2.0• LDOMs for T5x20• CRUs/FRUs and Configuration Options• Tuning and Performance• Tools, Information and References• Q & A ... Sun Internal and Approved Partners Only 4
    • UltraSPARC T2: • Up to 8 cores @1.2GHz or 1.4GHz • Up to 64 threads per CPUTrueSystem On a Chip • Up to 16 FB-DIMMs, 4 memory controllers > Up to 64GB memory (4GB DIMMs) FB DIMM FB DIMM FB DIMM FB DIMM > 2.5x memory BW of UltraSPARC T1 FB DIMM FB DIMM FB DIMM FB DIMM • 8 x fully pipelined Floating Point units / core, 1 per core MCU MCU MCU MCU • Dual 10Gbit Ethernet and PCI-E L2$ L2$ L2$ L2$ L2$ L2$ L2$ L2$ integrated onto chip Full Cross Bar • 4MB L2 (8 banks) 16 way C0 C1 C2 C3 C4 C5 C6 C7 associative FPU FPU FPU FPU FPU FPU FPU FPU • Enhanced MAU/Security co- MAU MAU MAU MAU MAU MAU MAU MAU DMA Sub-System processor per core > DES, 3DES, AES, RC4, SHA1, NIU Sys I/F SHA256, MD5, RSA to 2048 key, PCI-E (E-net+) Buffer Switch Core ECC,CRC32 Power ~95W • Advanced Power saving features 2x 10GE Ethernet <1.5W/thread x8 @2.5GHz 4GBytes/s bi-directional • 65nm process technology Sun Internal and Approved Partners Only 5
    • UltraSPARC T2 Processor L2 Data L2 Data Bank 0 Bank 4 L2B0 SPARC SPARC SPARC SPARC L2B4 L2 Data Core 0 Core 1 Core 5 Core 4 L2 Data Bank 1 Bank 5 L2B1 L2B5 L2 L2 L2 L2 MCU0 MCU2 NCU TAG0 TAG1 TAG5 TAG4 FSR EFU MCU1 MCU3 SIO SII L2B3 CCX L2B7 FSR CCU L2 Data L2 Data Bank 3 Bank 7 L2 L2 L2 L2 L2B2 L2B6 TAG2 TAG3 TAG7 TAG6 L2 Data L2 Data Bank 2 Bank 6 SPARC SPARC SPARC SPARC DMU Core 2 Core 3 Core 7 Core 6 RDP TDS PEU RTX PSR ESR FSR MACFor a video tour with Rick Hetherington, go to http://www.youtube.com/watch?v=B8DNexYcsa4 Sun Internal and Approved Partners Only 6
    • UltraSPARC T2 Core Block Diagram • IFU – Instruction Fetch Unit > 16 KB I$, 32B lines, 8-way SA TLU IFU > 64-entry fully-associative ITLB • EXU0/1 – Integer Execution Units > 4 threads share each unit > Executes one integer EXU0 EXU1 instruction/cycle • LSU – Load/Store Unit MMU/ > 8KB D$, 16B lines, 4-way SA HWTW > 128-entry fully-associative DTLB • FGU – Floating/Graphics Unit SPU FGU LSU • SPU – Stream Processing Unit > Cryptographic acceleration • TLU – Trap Logic Unit > Updates machine state, handles exceptions and interrupts Gasket • MMU – Memory Management Unit > Hardware tablewalk (HWTW) xbar/L2 > 8KB, 64KB, 4MB, 256MB pages Sun Internal and Approved Partners Only 7
    • Floating Point Unit• Each Core has its own Floating Point Unit. This removes completely the 2% FP restriction of T1.• Only a 6 cycle penalty to the FPU• Fully-pipelined (except divide/sqrt) > Divide/sqrt in parallel with add or multiply operations of Data other threads• Full VIS 2.0 implementation• FPU performs integer multiply, divide, population count• Multiplier enhancements for modular arithmetic operations in the Crypto Unit• Performance is impressive...testing and tuning still happening Sun Internal and Approved Partners Only 8
    • Niagara 2 Crypto Performance • Supports ten most common ciphers and Comment secure hashing functionsCycles/ Iterations/ Block Block (bytes) Block GB/S 2 rounds/cycle > RC4, DES, 3DES, AES-128, AES0192, AES-256, MDS, SHA-1, 2 rounds/cycle SHA-256 16 rounds 8 8/9+ 12.0 3 overlapped cycles 48 rounds 8 25/24 4.0 Data • On-chip crypto accelerators and built-in 10GbE vs. 4 words into 44 words 1 1 37683 12.0 accelerator cards 6 words into 52 words - - 32 - with UltraSPARC T2 8 words into 60 words - - 37 - New Ciphers >2 column over two cycles <36% crypto performance 41 - - -2 column over two cycles 10 rounds 16 30/30 6.36 >2 column over two cycles >23% CPU overhead 12 rounds 16 36/36 5.28 2 cycles per step • Leading cryptography performance 2 cycles per step 14 rounds 80 steps 16 64 42/42 160/160 4.56 4.80 2 cycles per step > Over 30x greater RSA1024 performance than 2-socket 64 steps 64 128/128 6.00 IBM p510 32 bits per cycle 64 steps 64 128/128 6.00 • Best SSL Performance 1 4 1 48.00 > 2.2x better performace/watt than 4-socket HP DL585G2 > 1.74x better performance/watt than 2-socket HP DL380G5 > 15.6x better AES128 performance than crypto acclerator card Sun Internal and Approved Partners Only 9
    • FB-DIMM Memory Interface Feature UltraSPARC T1 UltraSPARC T2 Require Pins 1000 440 Available Bandwidth 23GBytes/second 50Gbytes/second Data Up to 64x DIMM slots Memory Capacity up to 16x DIMM slots T5x20 uses 16 DIMM slots Memory Controllers 4 4 DIMMS/Channel/Controller 02/02/04 08/02/04 Memory Latency 96ns 126ns DIMM speeds 400MHz 667MHz DIMM Power 7watts/DIMM 12.5Watts/DIMM Sun Internal and Approved Partners Only 10
    • 10GbE Network Interface Unit• Fully Integrated Network Interface Unit > Eliminates PCI-E chipset latencies > 30% better performance vs PCI-E NIC – Mainly through reduced latency• Two 10 GbE ports (XAUI)• 16 Rx and 16 Tx DMA channels for virtualization Data• Packet classification > Incoming flows are classified at layer 2 ,3 and 4 and put into the Rx DMA buffer according to the classification rules that matched the flow• nxge device driver, source base shared with Atlas• Performance estimates, testing/tuning ongoing – 1 port TX/RX = about 9.4Gbps in each direction with about only 40-50% CPU utilisation – 2 port TX/RX = 12GbpsTX, 18GbpsRX with 70-80% CPU utilisation (still 20 threads available for application) Sun Internal and Approved Partners Only 11
    • Agenda• Introduction• Overview of UltraSPARC T2• Sun SPARC Enterprise T5x20 servers > Comparison to SunFire T1000/T2000 > System Architecture and Tours > Memory, Networking and IO Expansion > Disks, Fans and Power Supplies > Solaris for T5x20• ILOM 2.0• LDOMs for T5x20• CRUs/FRUs and Configuration Options• Tuning and Performance• Tools, Information and References• Q & A ... Sun Internal and Approved Partners Only 12
    • Sun SPARC Enterprise T5x20 Extreme Rackmount Density UltraSPARC T2 Platform • 1/2U chassis, 28” depth • Up to 8 cores @ 1.2 - 1.4GHz > Up to 64 threads • 16 FB-DIMM memory slots > 64GB of memory, (4GB DIMMs) High Reliability • 4 to 16 hot plug SAS 2.5” disk• Optimized for best price / drive (RAID 0,1) performance & performance / watt • N+N Redundant, hot-swappable PSUs and Fans. DC PSU option• Scale-Out of web & network for 1U systems only infrastructure Expandable• Extension to T1000/T2000 • 3/6 PCI-E expansion slots (including 2 x 10 GBE I/0 slots) product line • 4 x 1GbE Ethernet as standard • 4 USB ports Available Now Sun Internal Only 13
    • Sun Fire T1000 / T5120 Differences Feature Sun Fire T1000 SPARC Enterprise T5120 Form Factor 1U, 18¾” deep 1U, 28” deep UltraSPARC T1 UltraSPARC T2 CPU 1.0 GHz, 32 threads 1.2/1.4 GHz 64 threads DDR2 32GB Max FB-DIMM 64GB MAX Memory 8x slots 16x slots Network 4x GbE 4x GbE + 2x 10GbE 1x 3.5” SATA, non-hot-swap Internal Storage Up to 8 x 2.5” SAS, hot-swap OR 2x 2.5” SAS Removable Media None 1x DVD-ROM Serial 1x RS-232, No USB 1x RS-232, 4x USB PCI Express slots 1x (low profile) 3x(low profile) 1x 300W AC 2x 650W AC & DC option Power Supply Non-Redundant Redundant/Hot-Swap Fans Non-Hot-Swap Redundant/Hot-Swap Sun Internal and Approved Partners Only 14
    • Sun Fire T2000 / T5220 Differences Feature Sun Fire T2000 SPARC Enterprise T5220 Form Factor 2U, 24” deep 2U, 28” deep UltraSPARC T1 UltraSPARC T2 CPU 1.0 - 1.4GHz 32 threads 1.2/1.4 GHz 64 threads DDR2, 64GB MAX FB-DIMM, 64GB MAX Memory 16x Slots 16x Slots Network(10/100/1000) 4x GbE 4x GbE + 2x 10GbE Internal Storage 4x 2.5” SAS, hot-swap Up to 16 x 2.5” SAS, hot-swap Removable Media 1x DVD-ROM 1x DVD-ROM Serial 1x RS-232, 4x USB 1x RS-232, 4x USB PCI Express slots 3x(low profile) 6x(low profile) PCI-X slots 2 x (low profile) None 2x 450W AC 2x 750W AC Power Supply Redundant/Hot-Swap Redundant/Hot-Swap Sun Internal and Approved Partners Only 15
    • Front Panel Dual USB 2.0 portsLocator LED /Button DVD-ROM Disk Drive MapFault LED Top Fault LEDStatus LED DISK 0 DISK 1 DISK 2 DISK 3 PSU Fault LEDPower Button Temp Fault LEDSerial Number label DISK 1 DISK 3 DISK 5 DVD-ROM DISK 0 DISK 2 DISK 4 DISK 6 DISK 7Locator LED /ButtonFault LED Top Fault LEDStatus LED PSU Fault LEDPower Button Temp Fault LED DISK 1 DISK 3 DISK 5 DVD-ROM Disk Drive MapSerial Number DISK 0 DISK 2 DISK 4 DISK 6 DISK 7 Dual USB 2.0 ports DISK 3 DISK 7 DISK 12 DISK 15 DISK 2 DISK 6 DISK 11 DISK 14 DISK 1 DISK 5 DISK 9 DVD-ROM DISK 0 DISK 4 DISK 8 DISK 10 DISK 13 Sun Internal and Approved Partners Only 16
    • Rear Panel PCI-E Slot 0/XAUI 0 PCI-E Slot 1/XAUI 1 PCI-E Slot 2 x8 Slot/x4 Electrical x8 Slot/x4 Electrical x8 Slot/x8 Electrical PCI/XAUI 0 PCI/XAUI 1 PCI 2PSU PSU 0 PSU 1StatusLEDs Chassis Status LEDs Quad Gigabit Dual POSIX DB-9 and Locator Button Ethernet Ports USB 2.0 Serial Port SP SP Ports Serial Network Port Port PSU 1 PCI 3 PCI 4 PCI 5 PCI/XAUI 0 PCI/XAUI 1 PCI 2 PSU 0 Chassis PSU Status LEDs Quad Gigabit POSIX DB-9 Status Dual and Locator Button Ethernet Ports USB 2.0 Serial Port LEDs SP SP Ports Serial Network Port Port Sun Internal and Approved Partners Only 17
    • Architecture Block Diagram - T5120 16 x FB-DIMM 4 Memory Controllers, 2 Channels, 2 DIMMS (DDR2-667) Disk Chassis 1RU SSI x8 FPGA 10GbE PCI-E 10GbE x4 LSI x4 Switch x8 MPC885 SAS links x4 1068E ILOM PLX 8533 Service DVD x4 x4 Processor PCI-E Switch PCI-E USB 2.0 PCI-E x1 Switch PLX 8533 to IDE x8 2.0 to 10GbE USB PLX 8517 x4 SerDes BCM8704 USB 2.0 x4 x4 Hub 10GbE USB Intel Intel Cu PHY 2.0 Dual Dual BCMxxxx GbE GbE XFPFront Panel 10GbE 0 1 2 3 Fibre Plugin USB Quad GbE PCI-E PCI-E PCI-E Serial Network POSIX Rear Panel Connectors x16 x8 x8 Mgt Mgt Serial DB-9 Sun Internal and Approved Partners Only 18
    • Architecture Block Diagram - T5220 16 x FB-DIMM 4 Memory Controllers, 2 Channels, 2 DIMMS (DDR2-667) Disk Chassis 1RU 2RU/8 SSI x8 FPGA 10GbE x4 PCI-E 10GbE LSI x4 Switch x8 MPC885 SAS links x4 1068E ILOM PLX 8533 Service DVD x4 x4 Processor PCI-E x8 Switch PCI-E USB 2.0 PCI-E x1 Switch PLX 8533 to IDE x8 2.0 to 10GbE USB PLX 8517 x4 x4 x4 SerDes BCM8704 USB 2.0 x4 x4 Hub 2RU Only 10GbE USB Intel Intel Cu PHY 2.0 Dual Dual BCMxxxx GbE GbE XFPFront Panel 10GbE 0 1 2 3 Fibre Plugin USB Quad GbE PCI-E PCI-E PCI-E PCI-E PCI-E PCI-E Serial Network POSIX Rear Panel Connectors x16 x8 x8 x8 x8 x8 Mgt Mgt Serial DB-9 Sun Internal and Approved Partners Only 19
    • Internal View (T5120) Fault Remind System Configuration Button Card (SCC)EEPROM Service Processor FB1/FM0 FB-DIMMs PCI-E Slot 2 x16(m)/x8(e) FB-DIMMs FB0/FM1 FB1/FM1 PCI-E Slot 1 UltraSPARC x8(m)/x4(e) or T2 XAUI 1 Disk Chassis FB-DIMMs FB1/FM2 PCI-E Slot 0 x8(m)/x4(e) FB-DIMMs or XAUI 0 PSU1 Power Power Board Supplies PSU0SAS x4 cable Power BUS Bars Chassis Intrusion Reed to 4 Fan Assemblies are required Switch X4 SAS Connector 0 (cable attached)Disk Chassis RED = +12Volts (magnet on underside of lid) X4 SAS Connector 1 (not used in 1RU) (chassis can take up to 8) BLUE = Ground Sun Internal and Approved Partners Only 20
    • Internal View (T5220) Fault Remind System Configuration Button EEPROM Service Processor PCI-E Slot 2 FB0/FM0 FB-DIMMs x8(m)/x8(e) x16(m)/x8(e) FB-DIMMs PCI-E Slot 1 UltraSPARC x8(m)/x4(e) T2 x8(m)/x4(e) FB0/FM1 or XAUI 1 Disk Chassis FB-DIMMs PCI-E Slot 0 x8(m)/x4(e) FB-DIMMs x8(m)/x4(e) or XAUI 0 FB0/FM2 Power PSUs Board Power Vertically Supplies Stacked Two Power BUS Bars Chassis Intrusion ReedSAS x4 cables 3 Fan Assemblies are required Switch X4 SAS Connector 0 (cable attached) to RED = +12Volts (magnet on underside of lid) X4 SAS Connector 1 (cable attached) (chassis can take up to 6) BLUE = Ground Disk Chassis Sun Internal and Approved Partners Only 21
    • Memory Physical LayoutChassis ChassisFront Rear Channel1/DIMM 1/Branch 1 3 Channel1/DIMM 0/Branch 1 2 Channel0/DIMM 1/Branch 1 3 Memory Population Rules Channel0/DIMM 0/Branch 1 1 1. All Branches must have at least one DIMM populated Channel1/DIMM 1/Branch 0 3 2 - min config is 4 x 1GB DIMMS Channel1/DIMM 0/Branch 0 Channel0/DIMM 1/Branch 0 3 2. Populate Branches starting with the nearest slot to the CPU first Channel0/DIMM 0/Branch 0 1 - See “1” in the diagram C0D0B0, C0D0B1, C0D0B2, C0D0B3 UltraSPARC 3. All DIMMS must be the same size T2 4. In Branch all DIMMS must be indentical (SUN Part No.) - Upgrading from 4 to 8 DIMM configs means moving DIMMS to Channel0/DIMM 0/Branch 2 1 ensure sameness Channel0/DIMM 1/Branch 2 3 5. 12 DIMM configurations are NOT Channel1/DIMM 0/Branch 2 2 supported. Channel1/DIMM 1/Branch 2 3 6. Only 1GB, 2GB and 4GB DIMMS are 1 supported Channel0/DIMM 0/Branch 3 Channel0/DIMM 1/Branch 3 3 7. Memory is FB-DIMM DDR2-667, there is no clocking down of memory in any Channel1/DIMM 0/Branch 3 2 config. Channel1/DIMM 1/Branch 3 3 Sun Internal and Approved Partners Only 22
    • Memory Configuration Rules• Populate at least 4 DIMMS at a time• First > Fill DIMM0 in Channel 0 in each Branch• Second > Fill DIMM0 in Channel 1 in each Branch• Third > Fill remaining DIMM slots.• All DIMMS must be the same size(i.e all 1GB, 2GB or 4GB, no mixing)• In a Branch all DIMMS must have the same SUN Part No.• 12 DIMM configurations are NOT supported.• No clocking down of memory in any config. Sun Internal and Approved Partners Only 23
    • Memory Population Examples 4 DIMM configuration 8 DIMM Channel1/DIMM 1/Branch 1 3 configuration Channel1/DIMM 0/Branch 1 2 16 DIMM Channel0/DIMM 1/Branch 1 3 Channel1/DIMM 1/Branch 1 3 configuration Channel0/DIMM 0/Branch 1 1 Channel1/DIMM 0/Branch 1 2Channel1/DIMM 1/Branch 0 3 Channel0/DIMM 1/Branch 1 3 Channel1/DIMM 1/Branch 1 3Channel1/DIMM 0/Branch 0 2 Channel0/DIMM 0/Branch 1 1 Channel1/DIMM 0/Branch 1 2Channel0/DIMM 1/Branch 0 3 Channel1/DIMM 1/Branch 0 3 Channel0/DIMM 1/Branch 1 3Channel0/DIMM 0/Branch 0 1 Channel1/DIMM 0/Branch 0 2 Channel0/DIMM 0/Branch 1 1 Channel0/DIMM 1/Branch 0 3 Channel1/DIMM 1/Branch 0 3 Channel0/DIMM 0/Branch 0 1 Channel1/DIMM 0/Branch 0 2 Channel0/DIMM 1/Branch 0 3 UltraSPARC Channel0/DIMM 0/Branch 0 1 T2 UltraSPARC T2Channel0/DIMM 0/Branch 2 1 UltraSPARCChannel0/DIMM 1/Branch 2 3 T2Channel1/DIMM 0/Branch 2 2 Channel0/DIMM 0/Branch 2 1Channel1/DIMM 1/Branch 2 3 Channel0/DIMM 1/Branch 2 3Channel0/DIMM 0/Branch 3 1 Channel1/DIMM 0/Branch 2 2 Channel0/DIMM 0/Branch 2 1Channel0/DIMM 1/Branch 3 3 Channel1/DIMM 1/Branch 2 3 Channel0/DIMM 1/Branch 2 3Channel1/DIMM 0/Branch 3 2 Channel0/DIMM 0/Branch 3 1 Channel1/DIMM 0/Branch 2 2Channel1/DIMM 1/Branch 3 3 Channel0/DIMM 1/Branch 3 3 Channel1/DIMM 1/Branch 2 3 Channel1/DIMM 0/Branch 3 2 Channel0/DIMM 0/Branch 3 1 Channel1/DIMM 1/Branch 3 3 Channel0/DIMM 1/Branch 3 3 Channel1/DIMM 0/Branch 3 2 Channel1/DIMM 1/Branch 3 3 Sun Internal and Approved Partners Only 24
    • Network Ports and Options XAUI 0 XAUI 1 Device NET 0 NET 1 NET 2 NET 3 XAUI 0 XAUI 1 Driver e1000g0 e1000g1 e1000g2 e1000g3 nxge0 nxge1 PCI Device PCI 0/0/1/0/2 PCI 0/0/1/0/3 NIU@80/0 NUI@80/1 NOTES NET 0 and NET 1 share a common Intel Ophir chip, as do NET 2 and NET 3. XAUI Adapter cards are required to take advantage of the built-in 10GbE networking There are two variations, Fibre and Copper Fibre XAUI cards take one of three current 10GbE transceivers Short Reach(SR), Long Reach(LR) and Long Reach Multimode(LRM) Copper versions of the XAUI card are expected Post-RR XAUI Cards consume the same physical space in the chassis as a PCI card, so are mutually exclusive with PCI cards in the same location Sun Internal and Approved Partners Only 25
    • 10GbE XAUI Option Cards• Interface to the outside world for the Onboard 2 x 10GbE interfaces from the UltraSPARC T2• Cards each consume a PCI-E slot if installed• Attach to dedicated XAUI sockets on special risers• 10Gbe XAUI Card – SESY7XA1Z > Requires Fibre Tranceivers – 10GbE Short Reach(SR) – SESY7XT1Z – 10GbE Long Reach (LR) – SESY7XT2Z – 10GbE Long Reach multimode (LRM) – part No. TBD• Copper connector XAUI versions to follow post-RR Sun Internal and Approved Partners Only 26
    • PCI Riser Cards• Various Riser Cards are used• Risers should not be moved between slots• All slots accommodate low Profile PCI-Express cards• T5120 – 25 Watts per card, 75 Watts Max (3 Slots)• T5220 – 25 Watts per card, 102 Watts Max (6 Slots) > Riser Card Mechanical and Electrical widths for T5120 and T5220 T5120 Slot 0 Slot 1 Slot 2 Only Slot x8/XAUI(m) x4(e) x8/XAUI(m) x4(e) x16(m) x8(e) T5220 Slot 1 Slot 1 Slot 2 Top Slot x8(m) x4(e) x8(m) x4(e) x8(m) x8(e) Sun Internal and Approved Partners Only 27
    • Disk Cage• Disk chassis is a FRU – 4 screws and disconnect cables• Contains DVD-ROM/Dual USB module as a sub-FRU/CRU – Standard in all default configs – remove disk 3/7 to access release latch, pre-RR• Non-DVD module available as X-Option Remove Disk and locate – Still provides Dual USB ports latch beneath DVD Press up and pull DVD out• T5120 will be available initially with only a 4 disk chassis, 8 disk in qualification > Only uses one or two, of the x4 SAS links• T5220 will be available initially with only an 8 disk chassis, 16 disk in qualification > Uses both of the x4 SAS link cables Sun Internal and Approved Partners Only 28
    • Disk Cage - T5120• Chassis accommodates up to 4 or 8 SAS or SATA 2.5” drives – No mixing in a chassis – No chassis changes needed• Disk Chassis requires only a single cable Fan Paddle x4 SAS cable• Connects to Fan Paddle board with press fit connector Board press- fit connector and connector• Mounting ensures fan to disk vibration isolation Chassis Screws Chassis Screws Locator Tabs T head locator bolts Sun Internal and Approved Partners Only 29
    • Disk Cage - T5220• Accommodates up to 8 or 16 SAS or SATA 2.5” drives• Requires both SAS cables be connected• Two x4 SAS connector on chassis• Even 16 disks provides lots of airflow Paddle Board press-fit• Cage and Fan install ensures vibration connector x4 SAS cables isolation and connectors Chassis Screws Chassis Screws Locator Tabs T head locator bolts Sun Internal and Approved Partners Only 30
    • Disk Drives• Disk drives are all 2.5” form factor• Chassis support both SAS and SATA (post-RR)• SAS Drives – 73GB @ 10K RPM – 146GB @ 10K RPM Improved – 73GB @ 15K RPM (post-RR) Airflow above and below disks – 146GB @ 15K RPM (post-RR)• SATA Drives – 200GB @ 5400 RPM (soon after RR) – 300GB @ 5400 RPM (H1CY08) Disk LEDs Ready to Remove – 500GB @ 7200 RPM (end CY08) Fault Status• New mounting bracket “Marlin” > Allows better cooling in denser chassis Sun Internal and Approved Partners Only 31
    • Disk Controller Options• T5x20 has embedded LSI1068E on the motherboard – JBOD up to 8 disks – Controls internal disks only – RAID 0 (striping) and RAID 1(mirroring) – Hot-plug supported – Hot-swap in non-RAID configs is done with cfgadm• Optional Extended RAID functionality cards (available Q4CY07) > Cougar - SGXPCIESAS-R-INT-Z – 8 port SAS RAID 0,1,1E,5,5EE,6,10,50,60 (depending on No. disks available) – Adaptec Card and Firmware with Intel SunRiseLake chipset – Battery Backed Write Cache (BBWC) of 256MB – BBWC battery at least 72 hours (much longer when new/fully charged) – Only for internal disks, replaces cables to route to standard disk chassis > Prometheus – SGXPCIESAS-R-EXT-Z – External port version of Cougar, for external disk arrays only (no internal ports) Sun Internal and Approved Partners Only 32
    • Fans and Cooling• T5120 > Requires 4 Fan modules FBD1 FBD1 FBD0 FBD0 FM3 PSUs FM3 FM3 FM3 > Two cooling groups – Group 1 – N2 cooling Disk Chassis Disk Chassis FBD1 FBD0 FBD1 FBD0 FM2 FM2 FM2 FM2 – FANBD0/FM1 – FANBD1/FM1 FBD1 FBD0 FBD1 FBD0 FM1 Motherboard FM1 FM1 FM1 – Group 2 – FB-DIMM cooling FBD1 FBD0 FBD1 FBD0 – FANBD1/FM0 FM0 FM0 FM0 FM0 – FANBD1/FM2 – Blank Module in FANBD1/FM3 Back Front• T5220 PSUs FBD1 FBD0 FBD1 FBD0 FM2 FM2 > Requires 3 Fan Modules Disk Chassis Disk Chassis > Single group for N2 and FB-DIMM FBD1 FBD0 FBD1 FBD0 FM1 FM1 – FANBD0/FM0 Motherboard – FANBD0/FM1 – FANBD0/FM2 FBD1 FBD0 FBD1 FBD0 FM0 FM0 FM0 FM0• Higher disk densities may require Internal andfans Partners Only Sun more Approved 33
    • Fans modules and boards• Fans are speed controlled by the Service Processor• Each fan has an Amber FAULT LED – if lit the fan module is faulty• 1RU modules Fan Fault – 2 Fans at 15K RPM Max. Amber LED Lit if fan• 2RU modules faulty – 2 Fans at 12K RPM Max• Monitored through ILOM/IPMI/Web Vibration Absorbing Silicon Dampers Sun Internal and Approved Partners Only 34
    • Power Supplies• PSUs are high efficiency• Output 12V (and 3.3v standby) PSU LEDs Ready to• Most DC-DC conversion done in MBD Fault Remove• Shared with Doradi/o, Tucani/a AC Present• Integrated Fan• T5120 > Side-by-Side > AC 650Watts > DC 660 Watts option coming post-RR > DC or AC, no sharing• T5220 > Vertically Stacked > 750Watts (may require the 1050Watt Psu for the 16 disk chassis, pending quals) > No DC option > BEWARE... Same form factor as SunFire X4200 PSUs ... DO NOT MIX Sun Internal and Approved Partners Only 35
    • Rack Mounting Options• Two Rack mounting options are available..• Standard Rack Mount (4 post)SESY9RK1Z > Same as Galaxy servers, T2000, etc.. > Standard configs contain this kit – ATO : SESY9RK1Z, X-ATO : SESY9RK1Z• Tool-less Rack Kit (4 post) > Optional kit, purchased seperately > Snaps into most 3rd party Racks > Requires no tools to fit or remove – ATO : SESY9RK2Z, X-ATO : SESX9RK2Z Sun Internal and Approved Partners Only 36
    • Solaris for Sun SPARC Enterprise T5x20• T5x20 servers will require Solaris10 8/07 (update 4)• This incorporates several UltraSPARC T1/T2 optimisations for CMT servers in general• Also includes some specific T5x20 optimisations > UltraSPARC T2 Crypto optimisations > nxge driver for 10GbE devices > UltraSPARC T2 performance counters > Processor grouping, page coloring and default sizing and bcopy/memcpy Sun Internal and Approved Partners Only 37
    • Solaris FMA for T5x20• Now aware of CPU Core oflining• PCIe Fabric diagnostics improved• Crypto Unit offlining• Network interface Units (10GbE)• POST memory CE tolerance improved• FRU identity info improved in fault reports (part/ser #)• Diagnostic Engines (DE) – Existing DE have been extended for T5x20 – New Eversholt DE used for HostBridge, PCIe and NUI diagnostics• Limitations... – Cant isolate Uncorrectable Errors to a single DIMM – PCIe riser cards not in topology – XAUI card and XFP not in the topology – SouthBridge chip is notand Approved Partners Only this time (DVD, USB,...) Sun Internal diagnosed at 38
    • Agenda• Introduction• Overview of UltraSPARC T2• Sun SPARC Enterprise T5x20 servers > Comparison to SunFire T1000/T2000 > System Architecture and Tours > Memory, Networking and IO Expansion > Disks, Fans and Power Supplies > Solaris for T5x20• ILOM 2.0• LDOMs for T5x20• CRUs/FRUs and Configuration Options• Tuning and Performance• Tools, Information and References• Q & A ... Sun Internal and Approved Partners Only 39
    • ILOM Features at a Glance• Hardware Control, Inventory, Monitoring• Host Configuration and Administration• SP Configuration and Administration• Logging, Auditing and Alerting• Authentication : – LDAP, RADIUS, Active Directory, Local Users• Services : – SSH, HTTP(S), NTP, Syslog, SMTP• Out-of-Band – Via Serial port, or dedicated Ethernet port• In-Band – Via Host OS, firmware upload, Explorer, Syslog Sun Internal and Approved Partners Only 40
    • ILOM – Industry Standard Manageability • ILOM enables Sun systems to be easily integrated into third party management infrastructures • Key functionality > IPMI 2.0 > SNMP V1, V2c, V3 > SSH 2.0 > LDAP, RADIUS & MS Active Directory authentication > Browser UI, SMASH DMTF CLP + ALOM style CLI • NOTE : No RKVMS functionality like Galaxy ILOM > No Java plugin required • Browsers tested include – Mozilla 1.7 on Solaris 10, IE 6.0 on W2K Server, Firefox 1.0 and 1.5 on W2K server Sun Internal and Approved Partners Only 41
    • ILOM – Industry Standard Manageability • Default user is root with password changeme > Use for ILOM WEB GUI access and SSH access and IPMI • The ALOM admin user is not defined by default, create this user with an ALOM CLI • Lost root password and ILOM config is reset using Motherboard Jumper J6801 • Note... sunservice user shares the password of the root user ONLY FOR SERVICE ACCESS* • Passwords must be 8 to 16 chars (used to be max 8) • Max of 10 users allowed (used to be 16) Sun Internal and Approved Partners Only 42
    • ILOM Common Tasks• Setting up an ALOM admin user from the ILOM CLI > SUNSP00144F4641BD login: root Password: changeme Waiting for daemons to initialize... Daemons ready Sun(TM) Integrated Lights Out Manager Version 2.0.4.0 Copyright 2007 Sun Microsystems, Inc. All rights reserved. Use is subject to license terms. Warning: password is set to factory default. ->create /SP/users/admin role=Administrator cli_mode=alom Creating user... Enter new password: ******** Enter new password again: ******** Created /SP/users/admin2 -> exit SUNSP00144F4641BD login: admin Password: ******** Waiting for daemons to initialize... Daemons ready Sun(TM) Integrated Lights Out Manager Version 2.0.4.0 Copyright 2007 Sun Microsystems, Inc. All rights reserved. Use is subject to license terms. sc> Sun Internal and Approved Partners Only 43
    • ILOM Common Tasks• Setting up ILOM IP configuration from the ILOM CLI > SUNSP00144F4641BD login: root Password: changeme Waiting for daemons to initialize... Daemons ready Sun(TM) Integrated Lights Out Manager Version 2.0.4.0 Copyright 2007 Sun Microsystems, Inc. All rights reserved. Use is subject to license terms. Warning: password is set to factory default. -> set /SP/network state=enabled Set ’state’ to ’enabled’ -> set /SP/network pendingipaddress=10.8.31.163 Set ’pendingipaddress’ to ’10.8.31.163’ -> set /SP/network pendingipdiscovery=static Set ’pendingipdiscovery’ to ’static’ -> set /SP/network pendingipnetmask=255.255.252.0 Static IP Set ’pendingipnetmask’ to ’255.255.252.0’ configuration -> set /SP/network pendingipgateway=10.8.31.248 Set ’pendingipgateway’ to ’10.8.31.248’ -> set /SP/network commitpending=true Set ’commitpending’ to ’true’ -> -> set/SP/pendingipdiscovery=dhcp Set ’pendingipdiscovery’ to ’dhcp’ -> set /SP/network commitpending=true Set ’commitpending’ to ’true’ Dynamic IP -> (DHCP) configuration Sun Internal and Approved Partners Only 44
    • ILOM IPMI functionality• IPMI is a widely used management protocol for x86 servers• ... But its new for SPARC boxes...• Download the required tools from > The Solaris Supplemental CD (an older binary package) > Latest is available at http://ipmitool.sourceforge.net – Binary packages (usually older versions) are available for Solaris x86 and SPARC, as well as Linux, Windows, Mac...etc > There is a Blueprint available from Sun also (mainlyx64 centric, but applies to SPARC now too) – http://www.sun.com/blueprints/0107/820-1011.pdf -bash-ventus2$ /opt/ipmitool/bin/ipmitool -U root -P changeme -H 10.10.10.10 sdr /MB/V_+1V0_VDD | 1.01 Volts | ok /MB/V_+3V3_STBY | 3.36 Volts | ok Get the /MB/V_+3V3_MAIN | 3.36 Volts | ok Sensor Data /MB/V_+5V0_VCC | 4.97 Volts | ok Repository /MB/V_+12V0_MAIN | 12.16 Volts | ok .. .. -bash-ventus2$ /opt/ipmitool/bin/ipmitool -U root -P changeme -H 10.10.10.10 sunoem sbled set /SYS/LOCATE on Set the System /SYS/LOCATE | ON Locator LED -bash-ventus2$ flashing Sun Internal and Approved Partners Only 45
    • ILOM Common Tasks• Logging into the ILOM Web GUI • Point Web Browser at the IP address of the ILOM > Will automatically redirect to a secure HTTPS port > Best to disable any Web Proxies for best performance > Make sure port 80, 443 are open on any firewall • Log in as any defined user, default Administration user is root Sun Internal and Approved Partners Only 46
    • ILOM/ALOM Common Tasks• A Tour of the ILOM Web GUI • GUI provides ways to set up IP address configurations for ILOM • Configure additional ILOM/ALOM users • Access Sensor Readings • Control System configurations • Carry out firmware updates. • etc.... Sun Internal and Approved Partners Only 47
    • ALOM 2.0 vs. 1.4 CLI Differences• Default CLI appearance is ILOM – User is root – Must create an admin Administrator user with the ALOM CLI• Flashupdate in new ALOM expects a TFTP server not an FTP server• ALOM IP address changes now active only after running ... – setsc netsc_commit true – Previously changes were active only after SC reset – DHCP is enabled by default• View firmware versions with sc> showhost version Sun System Firmware 7.0.0_build_13 2007/08/10 23:44 – showhost version Host flash versions: Hypervisor 1.5.0.build_13**PROTOTYPE** 2007/08/10 22:54 OBP 4.27.0.build_04***PROTOTYPE BUILD*** 2007/08/10 13:25• View FPGA version with Sun Fire[TM] Huron POST 4.27.0.build_04***PROTOTYPE BUILD*** 2007/08/10 13:51 – fpga version sc> fpga version FPGA Version: 1.5.0.0 Platform ID Major revision : 1 : 5 Minor revision : 0• Only Admin and Operator roles Lab Debug Version: 0 sc> are defined Sun Internal and Approved Partners Only 48
    • ALOM backward Compatibilty CLI• Uses ALOM-CMT 1.2 for a base feature set• Includes ALOM-CMT 1.4 features... > break -d > Host Watchdog Timer > Bootmode LDOMs awareness > Forward SP events to Hot/Solaris syslog via inband channel Sun Internal and Approved Partners Only 49
    • Agenda• Introduction• Overview of UltraSPARC T2• Sun SPARC Enterprise T5x20 servers > Comparison to SunFire T1000/T2000 > System Architecture and Tours > Memory, Networking and IO Expansion > Disks, Fans and Power Supplies > Solaris for T5x20• ILOM 2.0• LDOMs for T5x20• CRUs/FRUs and Configuration Options• Tuning and Performance• Tools, Information and References• Q & A ... Sun Internal and Approved Partners Only 50
    • Logical Domains 1.0.1 FeaturesReleased, aligned with UltraSPARC T2/T5x20• UltraSPARC T2 Platform Support > 32 domains at RR, 64 domains following• I/O domain can reboot while the other domains keep running > Dependant domains would wait for IO transactions to complete• LDoms SNMP MIB > Modeled after industry standard DMTF CIM (Common Information Model) > Running on control domain to enable remote SNMP monitoring• Domain minimization support > Minimal/reduced Solaris installation > Base minimal packages plus optional ones for guest, control, I/O or service domains Sun Internal and Approved Partners Only 51
    • Logical Domains 1.0.1 Requirements • Solaris 10 08/07 (update 4) • Systems firmware version – T5x20/Glendale Firmware 7.0.x = SP FW 2.0.4.0+ – T2000/T1000/T6300/Pelton Firmware 6.5.x • SUNWldm.v LDOMs Manager Package – Version 1.0.1 • T5x20 System FPGA – Version 1.5.0+ (Not a concern for Post-RR systems) Sun Internal and Approved Partners Only 52
    • Agenda• Introduction• Overview of UltraSPARC T2• Sun SPARC Enterprise T5x20 servers > Comparison to SunFire T1000/T2000 > System Architecture and Tours > Memory, Networking and IO Expansion > Disks, Fans and Power Supplies > Solaris for T5x20• ILOM 2.0• LDOMs for T5x20• CRUs/FRUs and Configuration Options• Tuning and Performance• Tools, Information and References• Q & A ... Sun Internal and Approved Partners Only 53
    • CRU/FRU List - T5120• Lots shared with other Jake and Elwood systems Item Description FRU/CRU Disk Backplane 4 drive Disk Backplane to support 4 Drives only FRU Fan Module 1U Dual Fan module, 4 populated in system CRU Fan Board 1U Fan connector board to accommodate Fan Modules, 2 per system FRU DVD Assembly DVD Assembly including front USB board FRU Paddle board 1U Conection between PDB, fan boards and Disk Backplane FRU PDB 1U Power distribution board, contains master chassis serial number FRU PSU 1U Power supply (650 Watts) CRU X8 &XAUI, 1U PCIe Riser PCI Riser assembly for left and centre motherboard slots CRU x16 1U PCIe Riser PCI Riser for right motherboard slot CRU Rack mount kit (requires tools) Rack mount kit CRU Rack Mount Kit (tooless) Rack mount kit, no tools required for installation CRU CMA Cable management arm CRU Bus Bars Power Bus Bars between PDB and Motherboard FRU PDB to MB Ribbon Cable Ribbon cable for control connections between PDB abd Motherboard FRU Light pipe kit Light pipe assemblies for left and right of disk cage FRU Motherboard assembly Motherboard assembly, part no varies depending on Core count FRU DIMMs FBDIMMs (three different part numbers for 1GB, 2GB, 4GB) CRU XAUI board optical/copper XAUI Adapter card (two different parts for copper/optical) CRU XFP module XFP GBIC module for use in XAUI cards (three different parts SR, HR, HRMM) CRU Battery Battery backup for NVRAM of Service Processor CRU HDD 73GB/146GB SAS Hard Disk drive, different part numbers for capacities CRU System Configurations PROM System Configuration PROM FRU Sun Internal and Approved Partners Only 54
    • CRU/FRU List - T5220• Lots shared with other Jake and Elwood systems Item Description FRU/CRU Disk Backplane 8 drive Disk Backplane to support 8 Drives only FRU Fan Module 2U Dual Fan module, 3 populated in system CRU Fan Board 2U Fan connector board to accommodate Fan Modules, 2 per system FRU DVD Assembly DVD Assembly including front USB board FRU Paddle board 2U Connection between PDB, fan boards and Disk Backplane FRU PDB 2U Power distribution board, contains master chassis serial number FRU PSU 2U Power supply (750 Watts) CRU X8,X8, XAUI, 2U PCIe Riser PCI Riser assembly for left and centre motherboard slots CRU X8, x16 2U PCIe Riser PCI Riser for right motherboard slot CRU Rack mount kit (requires tools) Rack mount kit CRU Rack Mount Kit (tooless) Rack mount kit, no tools required for installation CRU CMA Cable management arm CRU Bus Bars Power Bus Bars between PDB and Motherboard FRU PDB to MB Ribbon Cable Ribbon cable for control connections between PDB abd Motherboard FRU Light pipe kit Light pipe assemblies for left and right of disk cage FRU Motherboard assembly Motherboard assembly, part no varies depending on Core count FRU DIMMs FBDIMMs (three different part numbers for 1GB, 2GB, 4GB) CRU XAUI board optical/copper XAUI Adapter card (two different parts for copper/optical) CRU XFP module XFP GBIC module for use in XAUI cards (three different parts SR, HR, HRMM) CRU Battery Battery backup for NVRAM of Service Processor CRU HDD 73GB/146GB SAS Hard Disk drive, different part numbers for capacities CRU System Configurations PROM System Configuration PROM FRU Sun Internal and Approved Partners Only 55
    • Option Parts List Part Number Option Description Avialable SECY9PS11Z AC PSU 1RU (650 Watts) RR SEDY9PS31Z AC PSU 2RU (750 Watts) RR SECY9PS21Z DC PSU 1RU (660Watts) post-RR SESY2A1Z 2GB Memory (2x1GB) RR SESY2B1Z 4GB memory (2x2GB) RR SESY2C1Z 8GB Memory (2x4GB) RR SESY3A11Z 73GB, 10KRpm 2.5” SAS Drive RR SESY3C11Z 146GB, 10KRpm 2.5” SAS Drive RR - SATA Drives (200, 300,500GB) post-RR SESY9DV1Z DVD, 8X RW, Slot Load RR SESY9RK1Z Slide Rail Kit, requires screwdriver (1RU and 2RU) RR SESY9RK2Z Slide Rail Kit, Tool-less (1RU and 2RU) RR Sun Internal and Approved Partners Only 56
    • Option Parts List (cont..) Part Number Option Description Avialable SG-PCIE1FC-QF4 Qlogic, 4Gb, PCI-E Single Port (up to 3/6) RR SG-PCIE2FC-QF4 Qlogic, 4Gb, PCI-E Dual Port (up to 3/6) RR SG-PCIE1FC-EM4 Emulex, 4Gb, PCI-E Single Port (up to 3/6) RR SG-PCIE2FC-EM4 Emulex, 4Gb, PCI-E Dual Port (up to 3/6) RR SG-PCIE2SCSI-U320Z U320 SCSI, PCI-E Dual Port (up to 3/6) RR SG-PCIE8SAS-E-Z SAS, 8 Port, PCI-E (up to 1) RR SGXPCIESAS-R-INT-Z SAS, 8 Port, RAID, PCI-E Internal Drive Controller (up to 1) RR+3 SGCPCIESAS-R-EXT-Z SAS, 8 Port, RAID, PCI-E, External Drive Controller (up to 1) RR+3 7280A-2 Northstar Dual port GigE MMF Low profile PCI-E (up to 3/6) RR 7281A-2 Northstar Dual port GigE UTP Low profile PCI-E (up to 3/6) RR 1236A-Z Infiniband HBA, Dual 4x ports, PCI-E (up to 3/6) RR 4447A-X Atlas Quad Port, GigE Copper, PCI-E (up to 3/6) RR X4240A XVR-300, PCI-E x16/x8 Graphics cards (up to 1) TBD Sun Internal and Approved Partners Only 57
    • Agenda• Introduction• Overview of UltraSPARC T2• Sun SPARC Enterprise T5x20 servers > Comparison to SunFire T1000/T2000 > System Architecture and Tours > Memory, Networking and IO Expansion > Disks, Fans and Power Supplies > Solaris for T5x20• ILOM 2.0• LDOMs for T5x20• CRUs/FRUs and Configuration Options• Tuning and Performance• Tools, Information and References• Q & A ... Sun Internal and Approved Partners Only 58
    • Tuning Out of the Box• Using Solaris 10 08/07... > Default pages sizes for – Initdata Segments is 64kB pages – Private Anon, Heap, Process Stack, Text, Shared Anon segments are 4MB pages – ISM and DISM use 256MB pages – New variables introduced to control these sizes if changes are needed – max_uheap_lpsize,max_ustack_lpsize, max_privmap_lpsize, max_shm_lpsize, max_uidata_lpsize, max_utext_lpsize, max_shm_lpsize > L2 Cache hashing implemented – helps randomise accesses to L2 caches and ease thrashing > Stack biasing implemented – D-Cache is only 4 Way associative... 8 threads – Running multiple copies of a binary can lead process stacks aligning in the cache and to hot lines in the cache – Stack Bias slews the stack variableOnly processes. Sun Internal and Approved Partners for 59
    • Tuning Out of the Box (continued..) > Processor Groups introduced – Helps associate CPUs with some shared resources (int/FP pipelines, Crypto units etc) – Enables improved multi-level CMT load balancing and affinity policies > bcopy/uiomove Optimizations – Key kernel operations – new implementation about 2x faster than previous• Some optimisations did not make it into S10 8/07 > These will be post RR patches as they become available > Most fixes are already committed in update 5 > Check patch levels for customers with performance issues > Also check Firmware versions as this can improve performance too... Sun Internal and Approved Partners Only 60
    • Tuning Recommendations• Less involved than with T2000• S10 8/07 is more automatic – OOB optimised page size selection, thread scheduling, block copy, IP instances for zones, page coloring etc...• If tuning is necessary remember... > Improved pipeline results in Single threaded performance around 20% better > Floating Point is no longer an issue > Start with T2000 tuning options > Use updated trapstat, cpustat,• LDOMs may be an easier way to scale poorly threaded applications• Solaris Crypto framework is still being tuned > Great for big packets, working on improving small packets• A Blueprint is being prepared (Denis Sheahan) Sun Internal and Approved Partners Only 61
    • Tuning Recommendations (cont..)• Refer to UltraSPARC T1 applications tuning best practices... > http://www.sun.com/servers/coolthreads/tnb/applications.jsp• CoolTools are being updated for UltraSPARC T2 > http://www.sun.com/servers/coolthreads/overview/cooltools.jsp• Sun Studio 12 is UltraSPARC T1 and T2 aware > Recompiling without code changes should benefit T1/T2 apps• Processes are being worked on for LDOMS migrations 1.0 to 1.0.1 > LDOMs configs can benefit from a transition to T5x20 platform > Make sure appropriate LDOMs tuning and sizing is carried out• Wider Crypto capability, so ensure code is using PKCS#11 framework > Ensure that the ncp (SSL, RSA, DSA etc) and n2cp (bulk ciphers), n2rng(random number generator) drivers are used appropriately• Sun Internal and Approved Partners Only 62
    • Agenda• Introduction• Overview of UltraSPARC T2• Sun SPARC Enterprise T5x20 servers > Comparison to SunFire T1000/T2000 > System Architecture and Tours > Memory, Networking and IO Expansion > Disks, Fans and Power Supplies > Solaris for T5x20• ILOM 2.0• LDOMs for T5x20• CRUs/FRUs and Configuration Options• Tuning and Performance• Tools, Information and References• Q & A ... Sun Internal and Approved Partners Only 63
    • Tools and Information• Cool Tools – http://www.sun.com/servers/coolthreads/overview/cooltools.jsp• Architecture WhitePaper – http://www.sun.com/• SUNWIN Tokens > Sun Intro : 514416 > Architecture Whitepaper : 512750 > This Technical Presentation: 512749 Sun Internal and Approved Partners Only 64
    • Reference Information• T5120 and T5220 (Huron) Engineering TOI – http://ittdev.east.sun.com/TechTalk2/EngineeringTOIs/Huron_Apr07/• ILOM/ALOM User Guides – http://www.sun.com/products-n-solutions/hardware/docs/pdf/819-1160-13.pdf – http://www.sun.com/products-n-solutions/hardware/docs/pdf/819-7981-10.pdf• Solaris FMA – http://fma.eng – http://lightside.sfbay/n2-fma/docs.php• Sun Internal and Approved Partners Only 65
    • Peter A. WilsonSenior Technical Marketing EngineerSun Microsystems 66
    • Huron PCI Topology Sun Internal and Approved Partners Only 67
    • T5x20 PCI Device Tree “/pci@0" 0 "px" UltraSPARC T2 PCI bus “/pci@0/pci@0" 0 "pxb_plx" To First Level PCI-E Switch Chip “/pci@0/pci@0/pci@1" 1 "pxb_plx" To Second level PCI-E Switch Chip “/pci@0/pci@0/pci@1/pci@0" 3 "pxb_plx" “/pci@0/pci@0/pci@1/pci@0/pci@1" 4 "pxb_plx" “/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0" 0 "px_pci" TO SouthBridge USB Devices “/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0" 0 "ohci" “/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,1" 1 "ohci" Standard USB Device manager “/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2" 0 "ehci" Enhanced USB Device manger “/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4" 0 "hubd" USB Dual Port HUB “/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/keyboard@1" 2 "hid" Rear Port 0 “/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/hub@4/keyboard@2" 0 "hid" Rear Port 1 “/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@1" 0 "scsa2usb" USB to IDE Converter “/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@1/disk@0,0" 0 "sd" DVD ROM Drive “/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@1/disk@0,1" 3 "sd" “/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@2" 1 "scsa2usb" “/pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@2/disk@0,0" 2 "sd" Front USB Port 0 “pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@3" 2 "scsa2usb" “”pci@0/pci@0/pci@1/pci@0/pci@1/pci@0/usb@0,2/storage@3/disk@0,0" 4 "sd" Front USB Port 1 “/pci@0/pci@0/pci@1/pci@0/pci@2" 5 "pxb_plx" To Dual GbE Controller 0 “/pci@0/pci@0/pci@1/pci@0/pci@2/network@0" 0 "e1000g" NIC 0 "/pci@0/pci@0/pci@1/pci@0/pci@2/network@0,1" 1 "e1000g" NIC 1 "/pci@0/pci@0/pci@1/pci@0/pci@3" 6 "pxb_plx" To Dual GbE Controller 1 "/pci@0/pci@0/pci@1/pci@0/pci@3/network@0" 2 "e1000g" NIC 2 "/pci@0/pci@0/pci@1/pci@0/pci@3/network@0,1" 3 "e1000g" NIC 3 "/pci@0/pci@0/pci@2" 2 "pxb_plx" To SAS Controller "/pci@0/pci@0/pci@2/scsi@0" 0 "mpt" SAS Controller "/pci@0/pci@0/pci@2/scsi@0/sd@0,0" 1 "sd" SAS Disk Device "/pci@0/pci@0/pci@8" 7 "pxb_plx" To Second Level PCI-E Switch Chip "/pci@0/pci@0/pci@8/pci@0" 9 "pxb_plx" "/pci@0/pci@0/pci@8/pci@0/pci@1" 10 "pxb_plx" PCI-E Slot 1/2RU "/pci@0/pci@0/pci@8/pci@0/pci@2" 11 "pxb_plx" PCI-E Slot 0/2RU "/pci@0/pci@0/pci@8/pci@0/pci@8" 12 "pxb_plx" PCI-E Slot 2/2RU "/pci@0/pci@0/pci@8/pci@0/pci@9" 13 "pxb_plx" PCI-E Slot 1/1RU "/pci@0/pci@0/pci@8/pci@0/pci@a" 14 "pxb_plx" PCI-E Slot 0/1RU "/pci@0/pci@0/pci@9" 8 "pxb_plx" PCI-E Slot 2 "/niu@80" 0 "niumx" Network Interface Unit "/niu@80/network@0" 0 "nxge" 10Gb_NIC 0 "/niu@80/network@1" 1 "nxge" Sun Internal and Approved Partners Only 10Gb_NIC 1 68