2. Agenda: A Cloud Architect Project
Introduction to Plan 9
Abstract
Requirements
Current System
Research Work
Proposed System
Real World Testing
Timeline of the project
Final costs
Summary
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
3. Introduction to Plan 9
Plan 9 Labs is an automation company that provides data robots to
companies needing data transfer between data sources or CRMs.
Removing Noise from existing data and provisioning robots that can sync
data across multiple services is the key aim of Plan9 to secure proper
data tagging, routing, and mapping between data sources.
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
4. PLAN 9 AUTOMATION
Web App
Servers
Blogs and Websites
Adobe Experience Cloud
CRMs Milestones Database
Dashboard and
Reporting
Plan 9 Bot
Plan 9 Bot
Plan 9 Bot
Segment
Destination
Data lake
Destination
Web App
source
Server
source
Multiple other
sources
6. Requirements
SERVER
Single Server with Backups
and a multi hour MTTR
(manual restore).
OS
Support for FreeBSD
Must run Ubuntu 22.04 LTS
No desktop GUI/ CLI only
SPECS
8-16 Core CPU
32-64 GB RAM
500-2 TB STORAGE (SSD)
MONITORING & BACKUP
Live monitoring
SMS alerting
Backup with AWS S3 buckets
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
7. Current System
Linode Server with 8 Cores, 16GB RAM
and 300 GB RAM. Support for Ubuntu
22.04 LTS , but no support for FreeBSD
Backup done through AWS S3 buckets.
Whole images of data backed up daily
and can be restored in 4-6 Hours
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
AWS S3
Linode VPS Cloud Server
8. Research Work
Determining the
Type of Server
Virtual Private Server,
Virtual Data Center or
Hosted Dedicated Server
Backup and
Failover Solutions
Having a Failover of less
than 2 hours if the server is
down is important
Best Server
Location Strategies
To provide for the least
latency, the server should
be located on the US east
coast.
Monitoring and
Alerts via SMS and
Email
For instant response to
restore servers manually
(automatic restore not
required).
1 2 3 4
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
9. Research Work
Determining the
Type of Server
Virtual Private Server,
Virtual Data Center or
Hosted Dedicated Server
1
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
Virtual Private Server- It is a server virtualized on physical server
hardware, providing scalability and flexibility of configurations with its
secondary storage virtualized using LUNs with stacks of HDD or SSD
units. This is advantageous for scalability and flexibility only, but
read/write speeds are significantly slower
Virtual Data Center- It is a collection of VPSes connected over
virtualized networks and having databases, monitoring servers, and
other servers all on a single physical virtualized infrastructure. Only
provided by Ionos
Hosted Dedicated Server: It is a physical rack-based server that is
configured with physical components and is hosted over a network.
High performance but loses on Virtualization features.
Finally, the best type of Server boils down to the pricing and what fits in
Plan9’s budget.
10. Research Work
Plan9 had a budget of 600 USD(per month) to spare for their whole infrastructure.
We had to configure everything including servers, backup, monitoring and failover for that price
Research work was started to find the best servers for the money, according to the specifications
provided. This table is a small snippet of the prices we found. (in USD per Month)
We chose InterServer and Ionos as our finalists because of the price, and a hosted dedicated
server because of performance. We could thus use our budget to configure better backup infra.
Specifications AWS
Digital
Ocean
Google
Cloud
InterServer Ionos
32GB RAM Server 720 252 908 98 185
64GB RAM Server - 504 - 110 210
128GB RAM Server - 1008 - 209 256
Type of Server VPS VPS VPS DS & VPS DS VPS & VDC
11. Research Work
Backup and
Failover Solutions
Having a Failover of less
than 2 hours if the server
is down is important
2
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
Plan 9’s current system consists of a single server with images being
pushed to AWS S3 bucket server. There is also a redundant backup
which is pushed to dropbox.
However, with the new system, we propose a 2 server system, where
the second server takes over immediately in case of the first server
failing. PostgreSQL data is pushed to both servers at the same time,
properly syncing data between the two servers. Bots still push a
snapshot of data daily to Amazon S3 (more snapshots daily to be
implemented soon).
Amazon S3 data can be restored onto a new server configured within
minutes and the bots redeployed in less than an hour just like the
previous configuration.
Dropbox is still used in case Amazon S3 fails
12. Research Work
Best Server
Location Strategies
To provide for the least
latency, the server should
be located on the US east
coast
3
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
Plan 9’s main servers need to be located on the US east coast to
prevent any latency and network issues that may arise.
Having the server in the same regional location is also beneficial for
application and data processing performance. The Linode server that
Plan 9 uses is based in New York, and Plan 9 needed something similar
for their server upgrade.
Both our finalists Ionos and InterServer are located in New Jersey,
which is a city a few miles from New York, and on the east coast.
For the second server, we needed a server from a different region and
location to prevent data center failures, or environmental hazards
which can cause a certain server in a region to fail (ex- storms,
hurricanes, tornados or earthquakes)
Therefore, we configured the second server in central US- Las Vegas
13. Research Work
Monitoring and
Alerts via SMS and
Email
For instant response to
restore servers manually
(automatic restore not
required)
4
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION 13
Plan 9 has always used a monitoring system to alert them when the
server was down. Linode offered good services for their VPS products
with SMS alerting and stats monitoring.
The main open-source monitoring solutions that we discovered were
Zabbix and Nagios. These are cli configurable monitoring that requires
a server to run and lots of configuration. This was however very
complex to install and run and provided features that wouldn’t be
useful to us.
Therefore, we dropped that and used a domain monitoring solution
called UptimeRobot which could monitor the status of the domain
(which we configured using Namecheap, for our server)
This was also within budget and costed only 30 USD per month.
14. Proposed System
Main Server is located in NJ and Backup
Server in LA
All the bots are run as Cron jobs (scheduled
tasks)
We use an AWS S3 server as an archive for
the data being created or run through the
robots on the main server.
PostgreSQL database is run on both servers
and synced together through the AWS S3
server.
ML Training is done on the main server or
Plan 9’s development laptop.
AWS S3
Main Server Backup Server
15. Proposed System
AWS S3
Main Server Backup Server
UptimeRobot
Monitoring
Web Server
All the data is also pushed through to
Dropbox, in case the S3 instance fails. It
takes longer to restore from Dropbox, but it is
a proper archiving solution.
Uptime Robot monitors both servers.
The web app is run on the backup server, and
the website is on a web server (Plan 9 could
switch to a hosting provider in the future)
Dropbox
16. Real World Testing
For every product or IT asset, Testing is important to ensure that the product is functional and performs well.
Such was the case with our finalists- Ionos Intel Xeon 6126 and InterServer’s AMD EPYC 7402P. We tested it
using a high-resource-demanding bot created from Plan 9
The resultant scores were disappointing against the Plan 9 development laptop. This made us switch to a
desktop-class processor which would perform better than the Plan 9 laptop – AMD Ryzen 9 5950X. This
config had less RAM, but this processor was much faster than the other
Specifications Bot Write Copy
Linode VPS- 16GB RAM 8 Core Intel 27.9 13.7 1.6
Ionos Intel Xeon 6126 25.6 16.8 1.4
InterServer AMD EPYC 7402P 18.7 10.9 1.3
Plan 9 Development Laptop 20.2 8.3 0.8
InterServer AMD RYZEN 5950X 14.6 8.1 1.2
server processors, which was more beneficial. We also chose a lower
spec of the Ryzen (5900X) for the backup server.
17. Project Timeline
Project
Requirement
Allocation
Jan 2
Budget and
Metrics Calculation
Jan 4
Application
Analysis
Jan 5
Research Work
to find the best
providers
Jan 10
Plan9
chooses the
best finalists,
starts work
on testing
Jan 16
Servers
Provisioned
Jan 18
Performance
Testing
Jan 20
Provisioning
AMD 5950X
Jan 25
Installing
Software
– python,
bots etc
Feb 2
Research
work for
monitoring
solutions
Feb 5
UptimeRobot
Configured
Feb 10
Backup
Server
configured
Feb 15
Server
Migration
Feb 20
AWS S3
Feb 21
Fixing
Bugs and
Issues
Feb 23
Dropbox
config
Feb 27
Project
Audit
Mar 1
Project
Complete
Mar 10
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
18. Final Costs
Budget- USD 600 P.M.
When we started with this
budget, it was challenging
to build a solution to fit this
limited budget.
However, we now have a
fully-fledged solution,
which is high performing
and functional.
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
Component Product Cost
Main Server
InterServer AMD Ryzen 9 5950X
128GB RAM, 2TB NVMe
209 USD
Backup/ Webapp
Server
InterServer AMD Ryzen 9 5900X
128GB RAM, 2TB NVMe
125 USD
Domain NameCheap 20 USD
Storage Backup Amazon S3 2TB 10 USD
Archive Backup Dropbox Premium 2TB 20 USD
Monitoring UptimeRobot 30 USD
Total Costs Per Month 414 USD
19. Summary
As an infrastructure solutions architect, the aim of this project was to
make Plan 9’s infrastructure have the best specifications to suit their
company’s application while sticking to the budget- thus creating the
most value.
I was able to save Plan 9 around 176 USD per month while building a
solution that will benefit Plan 9 for years to come.
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION