Migrating national services to the Cloud

Mike Jones
Mike JonesRetired Head of Infrastructure at home
Jisc told Mimas to use the cloud; what happened next was amazing...
2015-04-02 Migrating national services to the Cloud
Mike Jones
Jisc
August 2013
“Dear Mimas,
How long would it take to migrate into the
cloud if you started tomorrow?
Love Jisc”
erm...
10 Production Services and 10-20 projects
across ~80 Nodes ~120TB
Sun VMware Stand Alone Servers
14VMs
(+14)
30VMs
75 x 300 GB Disk Clariion
HP P2000 SAN 40TB
75 x 300 GB Disk Clariion
Dell Powervault 8TB
+Native server storage
Storage
Ja.net (via University)
~128 Public IPs
~750 DNS Records
~30 Zones
Connectivity
» + other service components already in the cloud
IaaS
With perhaps a smattering of PaaS
With perhaps a smattering of PaaS
Or not
1 – 2 years
For a numbed of reasons, mostly procurement
and spending public money
3 months of head scratching ensued...
Options
»Use the “Jisc” cloud
»Use the Janet Framework
»G-cloud
»EGI federated cloud
»Commodity Clouds
Options Appraisal
»Use the “Jisc” cloud – it didn't exist
»Use the Janet Framework – Rent some rack space in a
yet to be built machine room, buy kit and build our own
cloud.
»G-cloud – tl;dr
»EGI – a bit too bleeding edge
»Commodity Clouds
Decision
»Opted for the Commodity Clouds
»How to choose:
● Cost
● Location
● Connectivity
● Familiarity
AWS with perhaps a smattering of Azure
How do we buy it?
After lots of Indirection and Misdirection
»AWS via Arcus through https://janet.arcusglobal.com
»Azure via reseller directly-ish
Ploughing into AWS
First up: HistoricalTexts
»Jisc Historic Books (later also Jisc Journal Archives)
● Elastic Search cluster of 4 x i2.xlarge
● Seadragon cluster 2 x m2.2xlarge
● Shibboleth or IP authenticated web front end
● MySQL
● ~40TB S3 storage
– mounted vis fuse s3fs*
● OCR
Next up: Jorum
»
● Elastic Search cluster of 2 (m2.2xlarge)
● Dspace & tomcat (m1.medium)
– Modest EBS* – asset store
– Shibboleth
● Web front end (micro)
Next up: Jusp and Mimas “Small Servers”
»
● Bespoke mashup of perl, php, Mysql, Apache
and Shibboleth
● We also host a number of small legacy
VitrualHosts
● 1 x m3.xlarge
● EBS for each additional web site
Next up: Census
»
● Reverse Proxy to various content
● Shibboleth Authentication
● initially micro instance -> small
Next up: Archives Hub
»
● Lots of service components
– Cheshire
– PostgreSQL
– VariousTripple stores.
– Tomcat
– Blogs andWikis
● CreateVM and let admins loose.
Large image slide headline style
Sub-headline if required
And some use of Azure
Copac*
»
● 2TB Oracle DB
– Enterprise Oracle
● Blog
● Website
● Shibboleth
Zetoc*
»
● Oracle DB
● Website
● Shibboleth
● Python
● Email
– Lots of emails
Miscellaneous Detail
AWS services used
»
● EC2, EIP, EBS, Snapshots
● Route53
● VPC
● IAM
– Shibboleth: http://bit.ly/ShibAWS
● S3
● SES
● SNS
Savings
»
● Mimas would spend at least ~£600k on kit per 3-5 years refresh
● Mimas would spend ~£250k p.a. on local support: 3 dedicated
external FTEs
● Mimas would spend ~£85k p.a. on internal support: 1 internal
FTE.
● Mimas would also pay salary overheads on all staff ~£15k p.c. p.a.
● a proportion of which would be donated the the ongoing support
of University IT
●
● ~£400k p.a. to support Mimas services
Savings
»
● Fewer dedicated IT staff (simpler systems)
● Mirrored Services -> Snapshots
Leaving remarks
»
● Don't want to get unduly tied into a cloud
provider
● Lift and shift is the quickest way to move
services into cloud
● There's money to be saved if you look carefully
1 of 31

More Related Content

What's hot(20)

The promise of multi cloudThe promise of multi cloud
The promise of multi cloud
Rishidot Research162 views
Cloud Computing - An IntroductionCloud Computing - An Introduction
Cloud Computing - An Introduction
Ravindra Dastikop3.6K views
Microsoft Cloud Computing E-BookMicrosoft Cloud Computing E-Book
Microsoft Cloud Computing E-Book
ZAG Technical Services951 views
Cloud management (IBM)Cloud management (IBM)
Cloud management (IBM)
Vivek Trivedi1.4K views
Microsoft Azure cloud computingMicrosoft Azure cloud computing
Microsoft Azure cloud computing
Roman Antonov99 views
Cloud computing IntroductionCloud computing Introduction
Cloud computing Introduction
Venkatesh Jambulingam188 views
Cloud ComputingCloud Computing
Cloud Computing
Subhajeet Paria523 views
Cloud Computing by AGDMOUN KhalidCloud Computing by AGDMOUN Khalid
Cloud Computing by AGDMOUN Khalid
خالد اكدمون40.7K views
cloud computing Multi cloudcloud computing Multi cloud
cloud computing Multi cloud
Dr.Neeraj Kumar Pandey870 views
Cloud 101 - What is the Cloud?Cloud 101 - What is the Cloud?
Cloud 101 - What is the Cloud?
RapidScale1.8K views
Enterprise Journey to the CloudEnterprise Journey to the Cloud
Enterprise Journey to the Cloud
Amazon Web Services10K views
Cloud presentationCloud presentation
Cloud presentation
nich2533545 views
Cloud coumputingCloud coumputing
Cloud coumputing
Jai Sadana55 views

Similar to Migrating national services to the Cloud

cloud.pptcloud.ppt
cloud.pptsourabhsinghbhopal
1 view31 slides
Cloud ComputingCloud Computing
Cloud ComputingBiswajit Pratihari
2.3K views31 slides
Cloud computingCloud computing
Cloud computingMOHIT PANDEY
225 views31 slides

Similar to Migrating national services to the Cloud(20)

Cloud introduction2.pptCloud introduction2.ppt
Cloud introduction2.ppt
Bala Anand3 views
cloud.pptcloud.ppt
cloud.ppt
sourabhsinghbhopal1 view
JOSA TechTalks - Downgrade your CostsJOSA TechTalks - Downgrade your Costs
JOSA TechTalks - Downgrade your Costs
Jordan Open Source Association103 views
Cloud ComputingCloud Computing
Cloud Computing
Biswajit Pratihari2.3K views
Cloud computingCloud computing
Cloud computing
MOHIT PANDEY225 views
OSOM Operations in the CloudOSOM Operations in the Cloud
OSOM Operations in the Cloud
mstuparu402 views
OSOM - Operations in the CloudOSOM - Operations in the Cloud
OSOM - Operations in the Cloud
Marcela Oniga294 views
MySQL Ecosystem in 2020MySQL Ecosystem in 2020
MySQL Ecosystem in 2020
Alkin Tezuysal511 views
L2-3.FA19.pptL2-3.FA19.ppt
L2-3.FA19.ppt
mohaaalsa4 views
L2-3.FA19.pptL2-3.FA19.ppt
L2-3.FA19.ppt
VishalKumar5858484 views
Cloud ComputingCloud Computing
Cloud Computing
JAGUARGAMING118 views
L2-3.FA19.pptL2-3.FA19.ppt
L2-3.FA19.ppt
kesrinath4 views
L2 3.fa19L2 3.fa19
L2 3.fa19
Kv Sagar40 views
L2-3.FA19.pptL2-3.FA19.ppt
L2-3.FA19.ppt
VarchasvaTiwari24 views
L2-3.FA19.pptL2-3.FA19.ppt
L2-3.FA19.ppt
EcoSmith7 views
L2-3.FA19.pptL2-3.FA19.ppt
L2-3.FA19.ppt
SoundariyaSathish2 views
L2-3.FA19.pptL2-3.FA19.ppt
L2-3.FA19.ppt
UbaidURRahman782 views
L2-3.FA19.pptL2-3.FA19.ppt
L2-3.FA19.ppt
ssuserd237111 view
Hacking apache cloud stackHacking apache cloud stack
Hacking apache cloud stack
Nitin Mehta5.7K views

Migrating national services to the Cloud

  • 1. Jisc told Mimas to use the cloud; what happened next was amazing... 2015-04-02 Migrating national services to the Cloud
  • 3. August 2013 “Dear Mimas, How long would it take to migrate into the cloud if you started tomorrow? Love Jisc”
  • 5. 10 Production Services and 10-20 projects across ~80 Nodes ~120TB Sun VMware Stand Alone Servers 14VMs (+14) 30VMs 75 x 300 GB Disk Clariion HP P2000 SAN 40TB 75 x 300 GB Disk Clariion Dell Powervault 8TB +Native server storage Storage Ja.net (via University) ~128 Public IPs ~750 DNS Records ~30 Zones Connectivity » + other service components already in the cloud
  • 7. With perhaps a smattering of PaaS
  • 8. With perhaps a smattering of PaaS Or not
  • 9. 1 – 2 years
  • 10. For a numbed of reasons, mostly procurement and spending public money 3 months of head scratching ensued...
  • 11. Options »Use the “Jisc” cloud »Use the Janet Framework »G-cloud »EGI federated cloud »Commodity Clouds
  • 12. Options Appraisal »Use the “Jisc” cloud – it didn't exist »Use the Janet Framework – Rent some rack space in a yet to be built machine room, buy kit and build our own cloud. »G-cloud – tl;dr »EGI – a bit too bleeding edge »Commodity Clouds
  • 13. Decision »Opted for the Commodity Clouds »How to choose: ● Cost ● Location ● Connectivity ● Familiarity
  • 14. AWS with perhaps a smattering of Azure
  • 15. How do we buy it?
  • 16. After lots of Indirection and Misdirection »AWS via Arcus through https://janet.arcusglobal.com »Azure via reseller directly-ish
  • 18. First up: HistoricalTexts »Jisc Historic Books (later also Jisc Journal Archives) ● Elastic Search cluster of 4 x i2.xlarge ● Seadragon cluster 2 x m2.2xlarge ● Shibboleth or IP authenticated web front end ● MySQL ● ~40TB S3 storage – mounted vis fuse s3fs* ● OCR
  • 19. Next up: Jorum » ● Elastic Search cluster of 2 (m2.2xlarge) ● Dspace & tomcat (m1.medium) – Modest EBS* – asset store – Shibboleth ● Web front end (micro)
  • 20. Next up: Jusp and Mimas “Small Servers” » ● Bespoke mashup of perl, php, Mysql, Apache and Shibboleth ● We also host a number of small legacy VitrualHosts ● 1 x m3.xlarge ● EBS for each additional web site
  • 21. Next up: Census » ● Reverse Proxy to various content ● Shibboleth Authentication ● initially micro instance -> small
  • 22. Next up: Archives Hub » ● Lots of service components – Cheshire – PostgreSQL – VariousTripple stores. – Tomcat – Blogs andWikis ● CreateVM and let admins loose.
  • 23. Large image slide headline style Sub-headline if required
  • 24. And some use of Azure
  • 25. Copac* » ● 2TB Oracle DB – Enterprise Oracle ● Blog ● Website ● Shibboleth
  • 26. Zetoc* » ● Oracle DB ● Website ● Shibboleth ● Python ● Email – Lots of emails
  • 28. AWS services used » ● EC2, EIP, EBS, Snapshots ● Route53 ● VPC ● IAM – Shibboleth: http://bit.ly/ShibAWS ● S3 ● SES ● SNS
  • 29. Savings » ● Mimas would spend at least ~£600k on kit per 3-5 years refresh ● Mimas would spend ~£250k p.a. on local support: 3 dedicated external FTEs ● Mimas would spend ~£85k p.a. on internal support: 1 internal FTE. ● Mimas would also pay salary overheads on all staff ~£15k p.c. p.a. ● a proportion of which would be donated the the ongoing support of University IT ● ● ~£400k p.a. to support Mimas services
  • 30. Savings » ● Fewer dedicated IT staff (simpler systems) ● Mirrored Services -> Snapshots
  • 31. Leaving remarks » ● Don't want to get unduly tied into a cloud provider ● Lift and shift is the quickest way to move services into cloud ● There's money to be saved if you look carefully