Presentation video: https://youtu.be/H-guf07meWY Company: Lenovo Presenter: Karl Hansen Location: Future Cloud Summit Tallinn 2018 website: https://www.futurecloudsummit2018.pilw.io

1. Transforming the Data Center
Hyperscale –
Power/Cooling –
Open
KarlHansen | SeniorTechnicalManager,LenovoDCG Nordic
Tallinn,October,2018
2017 Lenovo. All rights reserved.

2. 2017 Lenovo Internal. All rights reserved. 2H Y P E R S C A L E
Lenovo: A Global Technology Leader
Worldwide facilities
36 manufacturing facilities,
7 research centers and
71 offices globally
Globally diverse leadership
team
7 different nationalities in
leadership executive council
(LEC)
Five nationalities represented on
Board of Directors.
A $43 billion,
Fortune 500 company
~52,000 employees serving
customers in 160+ countries
100% publicly traded on the
Hong Kong Stock Exchange
$1.2B
AI AND IOT INVESTMENT
#226 Fortune 500
ONE OF THE GREAT
BUSINESS STORIES IN THE
LAST 10YRS
Magazine

3. 2017 Lenovo Internal. All rights reserved. 3H Y P E R S C A L E
Data Center Group: An Experienced, Innovative, Global Team
2,000
DEVELOPERS ACROSS 7LABS
GLOBALLY
20MSERVERS SHIPPED; SURPASSING
MILESTONE THIS YEAR
Headquarters
Lenovo Manufacturing
Other Manufacturing
25 Years
IN THE DATA CENTER
10,000SERVICE PROFESSIONALS
NUMBER OF
FASTEST SYSTEMS
ON TOP500AND
FASTEST GROWING
HPC COMPANY
#1100
OWNED /
CONTROLLED
MFG SITES
6
100 SERVERS PER HR

4. 2017 Lenovo. All rights reserved. H Y P E R S C A L E
FAST
4
On-time
Delivery
95%
To Ship New
Products
1st
Current world
record performance
benchmarks
87
Gartner Top Supply
Chains
#24
Response
Times
2hour

5. 2017 Lenovo. All rights reserved. H Y P E R S C A L E
FLEXIBLE
5
HIGH-
PERFORMANCE
COMPUTING /
ARTIFICIAL
INTELLIGENCE
DATA CENTER
INFRASTRUCTURE
SOFTWARE-
DEFINED
DATA CENTER
HYPERSCALE
ENGINEERED SOLUTIONS
SERVICES

6. 62018 Lenovo Internal. All rights reserved.
Lenovo SDI Portfolio: Choose your Path to Cloud
Building Blocks Turnkey
Cloud
Model
Simplicity
Agility
HCI-ready HCI Cloud
Modular
Scaling
ThinkAgile
CP
Series
Composable
Resources
Platform
Services
ThinkAgile
SXM
On-Demand
Self-Service
End-to-End
Automation
Integrated
Infrastructure
Integrated
Software ThinkAgile
VX
Appliance
ThinkAgile
HX
ApplianceCertified
Hardware
ThinkAgile VX
Certified Node
ThinkAgile HX
Certified Node
ThinkAgile MX
Certified Node
coupled compute+storage
prescribed
resource
offerings
build your
own cloud
using
VMware
vRealize
Suite
(Operations,
Automation,
Insight)
build your own
Integration & Networking
build your
own cloud
using
VMware
vRealize
Suite
(Operations,
Automation,
Insight)
build your
own cloud
using
Nutanix
Prism
Central
&
Calm
orchestration
build your
own cloud
using
Nutanix
Prism
Central
&
Calm
build your own
cloud
using
custom
tools
&
automation
VMware
vSphere
+ vSAN
Nutanix
Acropolis
Microsoft
Windows Server
+ WSSD
100% turn-key

8. 8
WORKING WITH 7 OF THE 10
LARGEST DATA CENTER
COMPANIES IN
THE WORLD
HYPERSCALE

9. 2017 Lenovo Internal. All rights reserved. 9H Y P E R S C A L E
HYPERSCALE
Leadership
Paul Ju
Vice President,
WW Hyperscale
Segment GM
FASTER & more
FLEXIBLE than OEMs
Larger Commodity
SCALE than ODMs
More GLOBAL &
TECHNICAL than SIs
H Y P E R S C A L E C U S T O M E R S A R E L O O K I N G F O R
C O M PA N I E S T H AT A R E …
• Dedicated organization provides end to end services
• Industry leading engineering that can customize high
quality products with fast response
• Flexible supply chain model with leverage of entire
Lenovo scale
• The only company that can provide self-owned
manufacturing & global service capability
O U R VA L U E P R O P O S I T I O N
Mission
• Build a WW customer-
centric business model
that gives the client a
strategic partner that they
can trust to deliver value;
• Offering faster innovation,
lower cost products,
unique time-to-bid cycles,
and flexible service;

10. 2017 Lenovo Internal. All rights reserved. 10H Y P E R S C A L E
We Own Our End To End Capabilities
TECHNICAL
ARCHITECT
MANUFACTURING
PCBA  L12
CUSTOMER
ENGAGEMENT
ENGINEERING
DESIGN
GLOBAL DEPLOYMENT
& SERVICES

11. 2017 Lenovo Internal. All rights reserved. 11H Y P E R S C A L E
Hyperscale portfolio
Industry Trend | Evolving Technology
RackScaleStandardRack
Computing / Warm Storage Server
HIGH Volume | Commoditized
Acceleration & Storage Rich Server
Specific Application Optimized
Infrastructure Nodes / ToR
+ =
• Hyperscale data center
• Disaggregated , High-density
and Power optimize
infrastructures
• Software Defined X
 Computing
 Storage
 Networking
• Modulize design to fulfill diversify Hyperscale workloads
• Optimize Performance/Price ratio with excellent engineering
• Customize BMC/BIOS for Tier 1 customer
• Innovation design to support PCIe/ Nvlink GPU cards,
particularly suited to deep learning and resource-intensive
applications
• Optimize performance for object based storage requirements
1U 2U Multi-Node
Server
GPU Server Storage rich server
Compute
Storage
Networking
GPU Node
ARM/AMD Node
Global..
China… ODCC Scorpio
OCS
OCP ORV2
Olympus
FPGA

12. 2017 Lenovo Internal. All rights reserved. 12H Y P E R S C A L E
ChinaChina
HungaryHungary
BrazilBrazil
MexicoMexico
USUS
Lenovo Owned Mfg
Build Locations
Lenovo Dedicated Mfg
Partner Owned
Global Manufacturing, Delivery & Service
Monterrey, MX – 2009
• Primary Support Region: Americas
• L10 Capacity 1400x/day
• Current L11/12 Capacity 21 Cells
Whitsett, US – 2015
• Primary Support Region: USA
• L10 Capacity 200x/day
• Current L11/12 Capability 2 racks/day
Shenzhen, China – 2009
• Primary Support Region: Worldwide
• L10 Capacity 4800x/day
• Current L11/L12 Capacity 25 cells
Indaiatuba, Brazil - 2014
• Primary Support Region: Brazil
• L10 Support for in-country shipments – 200x/day
• L11/L12 Capability for key engagements
Sarvar, Hungary – 2011
• Primary Support Region: EMEA
• L10 Capacity 1000x/day
• L11/L12 Capability 12 Cells

13. 2017 Lenovo Internal. All rights reserved. 13H Y P E R S C A L E
Overall Design Capabilities
Data Center Level
Rack Scale Level
Component Level
Product Design Validation
• L11 S/V capacity
• HDD system performance optimization
Infrastructure
• Open-RMC
• 100G Networking
• NVMe over fabric
Commodity
• Lenovo NVMe SSD
Customization
FW / SW
• BMC / BIOS
• Big data predictive
failure analytics
Power / Thermal Solutions
• Liquid cooling technology
• Natural air cooling technology
• Battery and DC power
Software Defined
• Pooled resources
• SDN / SDS
From Component level to Rack Scale level to Data Center level…
The design is done by Lenovo engineers, independent of ODMs

14. 2017 Lenovo Internal. All rights reserved. 14H Y P E R S C A L E
WE DESIGN IT | WE BUILD IT | WE DEPLOY IT
HYPERSCALE

16. Why Energy Is A Big And Rapidly Growing Problem For Data Centers
Data centres of the world will consume 1/5 of Earth's power by 2025 ...
Energy Hogs: Can World's Huge Data Centers Be Made More Efficient ...
Data Center Design and its Energy Consumption - Stanford University

17. 17
0
200
400
600
800
1000
1200
1400
1600
1800
2000
60
80
100
120
140
160
180
200
220
240
6/1/2006
11/1/2006
4/1/2007
9/1/2007
2/1/2008
7/1/2008
12/1/2008
5/1/2009
10/1/2009
3/1/2010
8/1/2010
1/1/2011
6/1/2011
11/1/2011
4/1/2012
9/1/2012
2/1/2013
7/1/2013
12/1/2013
5/1/2014
10/1/2014
3/1/2015
8/1/2015
1/1/2016
6/1/2016
11/1/2016
4/1/2017
Intel Xeon Processor & Spec_fp Rate
TDP CFP2006 Rate2018 Lenovo - All rights reserved.
Performance-Power relation
500400320300 35024020585 12075
NVIDIA / AMD GPU
XEON
PHIAMD
NERVANA/CREST
NVIDIA SXM
• Maintaining Moore’s Law with increased competition
is resulting in higher component power
• Increased memory count, NVMe adoption, and I/O
requirements are driving packaging and feature
tradeoffs (superset of features doesn’t fit in 1U)
• Shared cooling fan power savings no longer exist
for dense 2S nodes architectures due to non-
spreadcore CPU layout high airflow requirements
 For highest performance systems will have to
reduce density or move to optimized cooling.
ARM SOC
Haswell
Sandy Bridge / IvyBridge

18. 18
What Will Tomorrow’s IT Look Like?
18
Data Center
limitations
Increasing
Electricity Costs
Thermals Capping
Performance
Waste Heat
Reuse
Higher Power
Processors
<205W >205W-240W >240W
1U Half wide Air/Hybrid Water Water
1U Full wide Air/Hybrid Air/Hybrid Water
2U Half wide Air Air Water
2U Full Wide Air Air Air
CPU TDP Wattage
ServerForm
Factor
18
<205W >205W-240W >240W
1U Half wide Air/Hybrid Water Water
1U Full wide Air/Hybrid Air/Hybrid Water
2U Half wide Air Air Water
2U Full Wide Air Air Air
CPU TDP Wattage
CHOICES
Go Higher
Go Wider
Go Water
2018 Lenovo. All rights reserved.

19. 19
Liquid Cooling Goes Mainstream
2012: A Watershed Moment
2018 Lenovo. All rights reserved.
2012
2014
2018
Leadership
Hardware
Smart Control
Software
Driving
Innovation
30+
patents
Integrated
Infrastructure

20. 20
Lenovo ThinkSystem SD650
Direct Water Cooling Delivers Maximum Performance
 Proven
Over 30,000 Direct Water Cooled Nodes installed
 Dense
Up to 24 205W+ CPUs per 6U Enclosure
 Performance
Turbo always on + cooler CPUs run faster
 Efficient
90% heat to water
Runs at 50C reducing data center energy by 40%
PUE 1.08 capable
 Flexible
Wide variety of fabrics, DIMM, drives and software

21. 2017 Lenovo Internal. All rights reserved. 21H Y P E R S C A L E2018 Lenovo - All rights reserved.
Power efficiency in the datacenter
• Power Usage Effectiveness (PUE)
• How much of total datacenter power is not used for IT?
• Cooling infrastructure, UPS, power , …
• Ideal value is 1.0
• Does not say anything about IT power efficiency
• IT Usage Effectiveness (ITUE)
• How much IT power is not used for computing?
• PSUs, fans, voltage regulators, …
• Ideal value is 1.0
• Does not say anything about datacenter power efficiency
• Total-Power Usage Effectiveness (TUE)
• How much datacenter power is not used for computing?
• TUE = PUE * ITUE
• Ideal value is 1.0
• Energy Reuse Effectiveness (ERE)
• How much power is lost to the universe?
• Ideal value is 0.0.
• Equal to PUE if no reuse

22. 222018 Lenovo - All rights reserved.
Waste Energy reuse - ERE
Energy Waste Direct Reuse Indirect Reuse
Why are high water outlet temperatures important?
Pictures: Leibniz Supercomputing Centre

23. 23
• Standard Air flow with
internal fans cooled with
the room climatization
• Broadest choice of
configurable options
supported
• Relatively inefficient cooling
• Air cooled but heat
removed with RDHX
through chilled water
• Retains high flexibility
• Enables extremely tight
rack placement
• Potentially room neutral
Air Cooled Air Cooled
w/ Rear Door Heat Exch.
2018 Lenovo - All rights reserved.
Lenovo Cooling Technologies
Choose for broadest choice
of customizable options
Choose for increased energy
efficiency with broad choice
PUE ~2.0 – 1.5
ERE ~2.0 – 1.5
PUE ~1.4 – 1.2
ERE ~1.4 – 1.2
• Most heat removed by
onboard-waterloop with
up to 50°C temperature
• Supports highest TDP CPU
at densest footprint
• Higher performance
• Free cooling
Direct Water Cooled
Choose for max performance
and high energy efficiency
PUE <=1.1
ERE <=1.1
• Waste heat reused to
generate coldness to cool
non-DWC components
• Retains highest TDP,
footprint and performance
• Potentially all system heat
covered through DWC
Direct Water Cooled
w/ Adsorption Chilling
Choose for max performance
and max energy efficiency
PUE <=1.1
ERE <1

24. 242018 Lenovo - All rights reserved.
Why use water to move the heat away?
• Energy (Joules)
1 Joules is the energy required to
– Let the human heart work for 1 second
– Lift a chocolade bar (100g) by 1 meter (g=9.81 m/s²)
– Let electrical power of 1 W flow for 1 second
Same mass of water stores 4x energy
compared to air at a given
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
Air Aluminium Copper Steel Water
Isobaric mass heat capacity• Mass heat capacity ( )
– ability of a given mass of a substance to store
internal energy while undergoing a given
temperature change

25. 252018 Lenovo - All rights reserved.
Why use copper to transfer heat?
• Heat power transfer (W)
is dependent on surface, thickness, temperature
difference and thermal conductivity of material
sourcesink
Heat transfer
0
50
100
150
200
250
300
350
400
450
Water Steel, stainless Aluminium Copper (pure)
Thermal conductivity
Copper is a very efficient material
to transfer energy from source to sink
• Thermal conductivity ( )
is the property of a material to conduct heat
per thickness and per temperature differential

28. 28
Open – Our Software Strategy...
and some Lenovo Contribution to the Open Community
• Rewriting our entire management stack
• Creating a new NOS – Lenovo CNOS
• Main contributor to XCAT – rewriting tools in Python
• Confluent Project
• https://github.com/openstack/pyghmi
• https://github.com/lenovo
• https://wiki.openstack.org/wiki/Neutron/ML2/LenovoML2Mechanism
• https://www.youtube.com/watch?v=1kMrY4QqzVI