A presentation from Networkshop46.

1. Developing an internal
business case, and the
associated benefits and
savings, to support use of
the Jisc shared datacentre
Mike Cope, UCL

2. Mike Cope, CIO at UCL (mike.cope@ucl.ac.uk)
Making the internal case for the Shared Datacentre
27th March 2018

3. Agenda
 Historical context at UCL
 Datacentre strategy at UCL
 Making the case for the Shared Datacentre
 Advantages of the Shared Datacentre contract
 Current situation
29-Mar-18 © UCL 2018 3

4. What is PUE
 PUE (Power Usage Effectiveness) is a measure of datacentre efficiency
 PUE = Total power into the DC / Power into actual computing kit
 The difference between power into the DC and power into computing kit
is mainly energy to run the cooling but there can be other losses as well
 Highly efficient modern DCs can achieve a PUE around 1.2
– DC needs to be almost fully utilised to achieve low PUEs
– Waste energy recover from heat generated by IT kit can reduce PUE further
29-Mar-18 © UCL 2018 4

5. Historical context at UCL
 Three significant centrally managed on-campus datacentres
 Plus 50+ server rooms in academic departments
– Many small and poorly maintained, about 200 racks in total
– About 840m2 of space used in academic buildings for server rooms
– Server rooms in odd places, one in the top of a lift shaft and one required
stepladders to access
 UCL’s central London location means space is highly constrained
 Poor PUEs:
– Often around 2.2 for small server rooms
– Central datacentres around 2.0
– Modern facilities should achieve 1.25 or less
 All due to history and a lack of strategy
29-Mar-18 © UCL 2016 5

6. The good, the bad and the ugly
29-Mar-18 © UCL 2018 6
One of the better Server Room examples
and how it looks in the Shared datacentre

7. Have a strategy and for UCL this was…
 Have more than one but less than three datacentres
– Centrally managed, large and flexible
– Roughly equal sized, at least 20 miles apart
– Independent active-active with globally load balancing and no L2 between them
 One needs to be on-site due:
– Nature of some computing related research
– Consolidating dispersed academic server rooms into a modern shared facility was
a big prize. Seeking to do this and also move offsite was too much for one step
 HPC benefits from modern, efficient and scalable DC facilities
– Focus central HPC in offsite DC
– Local departmental HPC based in on-site DC
 …but this will be the last on-site DC we ever build…
29-Mar-18 © UCL 2018 7

8. Making the case to use the Shared Datacentre
 Financials:
– Space savings
– Power savings
– … these offset the rack monthly costs
 Flexibility and agility:
– One shared space is much more flexible than many smaller spaces
– Shared DC better supports collaboration and consortium research grants
 Maintenance and reliability:
– Dispersed academic server rooms not maintained to a good standard due low
investment and general Estates maintenance approach meant poor reliability
 The world has moved on:
– DC space now a utility like power and water
– Allows the organisation to focus on more strategic objectives
29-Mar-18 © UCL 2018 8

9. Making the case – power savings
 Assessed the power taken by the existing computing kit
 Assessed the overall power consumption of the existing DCs and Server
Rooms containing this computing kit (high PUE)
 Calculated the power that would be taken at the Shared DC for the
same power taken by the existing computing kit (at a PUE of 1.25)
 Power consumption differences drive savings since each MW reduction
saves about £1m pa
 Don’t forget sustainability and reduced CO2 emissions from reduced
power consumption
29-Mar-18 © UCL 2018 9

10. Shared DC contract tailored to HEI needs
 KISS principle (Keep It Simple Stupid)
– Only co-location facilities and smart hands
– Other higher level support services can be layered on top
– Institutions able to trade between each other
 Shaped to meet HE needs including research intensive
– Racks up to 30kW each, not typical of the commercial world
 Strong SLAs and competitive prices
– Set prices and indexation
– Contractually capped PUE of 1.25
– Collective discount with more volume
 Easy contracting
– Each university contracts under the JISC framework
– No long procurement, turn around in weeks
 Long term stability
– 15 year contract (3 x 5 years)
29-Mar-18 © UCL 2018 10

11. Current situation
 Southern Shared DC
– Institutions have taken 294 racks and 2.96MW of usable power
– Average power per rack of 10kW. Largest 19kW and smallest 4kW
– Largest installation for one institution - 71 racks
– 21 institutions in total
 UCL
– Great success
– Most HPC now located in Shared DC
– Academic departments keener than expected to use the new facilities
– UCL first order in late 14, now have 46 racks
29-Mar-18 © UCL 2018 11

12. Summary
 Have a clear strategy
 Making the case:
– Power savings
– Space savings
– Agility, scalability and professionalism
– Reliability
– Allowing the organisation to focus on more strategic objectives
 There are alternatives but the Shared DC contract is shaped to meet
HEI needs
– 21 institutions using the Southern Shared DC with 294 racks
– New Northern Shared DC established in Leeds
29-Mar-18 © UCL 2018 12