SlideShare a Scribd company logo
1 of 20
Download to read offline
Building
Infrastructure for
an IT Organization
Daniel Jude Gonsalves
Plan9 Labs
Agenda: A Cloud Architect Project
Introduction to Plan 9
Abstract
Requirements
Current System
Research Work
Proposed System
Real World Testing
Timeline of the project
Final costs
Summary
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
Introduction to Plan 9
Plan 9 Labs is an automation company that provides data robots to
companies needing data transfer between data sources or CRMs.
Removing Noise from existing data and provisioning robots that can sync
data across multiple services is the key aim of Plan9 to secure proper
data tagging, routing, and mapping between data sources.
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
PLAN 9 AUTOMATION
Web App
Servers
Blogs and Websites
Adobe Experience Cloud
CRMs Milestones Database
Dashboard and
Reporting
Plan 9 Bot
Plan 9 Bot
Plan 9 Bot
Segment
Destination
Data lake
Destination
Web App
source
Server
source
Multiple other
sources
Requirements
To upgrade Plan 9’s Infrastructure
Requirements
SERVER
Single Server with Backups
and a multi hour MTTR
(manual restore).
OS
Support for FreeBSD
Must run Ubuntu 22.04 LTS
No desktop GUI/ CLI only
SPECS
8-16 Core CPU
32-64 GB RAM
500-2 TB STORAGE (SSD)
MONITORING & BACKUP
Live monitoring
SMS alerting
Backup with AWS S3 buckets
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
Current System
Linode Server with 8 Cores, 16GB RAM
and 300 GB RAM. Support for Ubuntu
22.04 LTS , but no support for FreeBSD
Backup done through AWS S3 buckets.
Whole images of data backed up daily
and can be restored in 4-6 Hours
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
AWS S3
Linode VPS Cloud Server
Research Work
Determining the
Type of Server
Virtual Private Server,
Virtual Data Center or
Hosted Dedicated Server
Backup and
Failover Solutions
Having a Failover of less
than 2 hours if the server is
down is important
Best Server
Location Strategies
To provide for the least
latency, the server should
be located on the US east
coast.
Monitoring and
Alerts via SMS and
Email
For instant response to
restore servers manually
(automatic restore not
required).
1 2 3 4
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
Research Work
Determining the
Type of Server
Virtual Private Server,
Virtual Data Center or
Hosted Dedicated Server
1
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
Virtual Private Server- It is a server virtualized on physical server
hardware, providing scalability and flexibility of configurations with its
secondary storage virtualized using LUNs with stacks of HDD or SSD
units. This is advantageous for scalability and flexibility only, but
read/write speeds are significantly slower
Virtual Data Center- It is a collection of VPSes connected over
virtualized networks and having databases, monitoring servers, and
other servers all on a single physical virtualized infrastructure. Only
provided by Ionos
Hosted Dedicated Server: It is a physical rack-based server that is
configured with physical components and is hosted over a network.
High performance but loses on Virtualization features.
Finally, the best type of Server boils down to the pricing and what fits in
Plan9’s budget.
Research Work
Plan9 had a budget of 600 USD(per month) to spare for their whole infrastructure.
We had to configure everything including servers, backup, monitoring and failover for that price
Research work was started to find the best servers for the money, according to the specifications
provided. This table is a small snippet of the prices we found. (in USD per Month)
We chose InterServer and Ionos as our finalists because of the price, and a hosted dedicated
server because of performance. We could thus use our budget to configure better backup infra.
Specifications AWS
Digital
Ocean
Google
Cloud
InterServer Ionos
32GB RAM Server 720 252 908 98 185
64GB RAM Server - 504 - 110 210
128GB RAM Server - 1008 - 209 256
Type of Server VPS VPS VPS DS & VPS DS VPS & VDC
Research Work
Backup and
Failover Solutions
Having a Failover of less
than 2 hours if the server
is down is important
2
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
Plan 9’s current system consists of a single server with images being
pushed to AWS S3 bucket server. There is also a redundant backup
which is pushed to dropbox.
However, with the new system, we propose a 2 server system, where
the second server takes over immediately in case of the first server
failing. PostgreSQL data is pushed to both servers at the same time,
properly syncing data between the two servers. Bots still push a
snapshot of data daily to Amazon S3 (more snapshots daily to be
implemented soon).
Amazon S3 data can be restored onto a new server configured within
minutes and the bots redeployed in less than an hour just like the
previous configuration.
Dropbox is still used in case Amazon S3 fails
Research Work
Best Server
Location Strategies
To provide for the least
latency, the server should
be located on the US east
coast
3
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
Plan 9’s main servers need to be located on the US east coast to
prevent any latency and network issues that may arise.
Having the server in the same regional location is also beneficial for
application and data processing performance. The Linode server that
Plan 9 uses is based in New York, and Plan 9 needed something similar
for their server upgrade.
Both our finalists Ionos and InterServer are located in New Jersey,
which is a city a few miles from New York, and on the east coast.
For the second server, we needed a server from a different region and
location to prevent data center failures, or environmental hazards
which can cause a certain server in a region to fail (ex- storms,
hurricanes, tornados or earthquakes)
Therefore, we configured the second server in central US- Las Vegas
Research Work
Monitoring and
Alerts via SMS and
Email
For instant response to
restore servers manually
(automatic restore not
required)
4
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION 13
Plan 9 has always used a monitoring system to alert them when the
server was down. Linode offered good services for their VPS products
with SMS alerting and stats monitoring.
The main open-source monitoring solutions that we discovered were
Zabbix and Nagios. These are cli configurable monitoring that requires
a server to run and lots of configuration. This was however very
complex to install and run and provided features that wouldn’t be
useful to us.
Therefore, we dropped that and used a domain monitoring solution
called UptimeRobot which could monitor the status of the domain
(which we configured using Namecheap, for our server)
This was also within budget and costed only 30 USD per month.
Proposed System
Main Server is located in NJ and Backup
Server in LA
All the bots are run as Cron jobs (scheduled
tasks)
We use an AWS S3 server as an archive for
the data being created or run through the
robots on the main server.
PostgreSQL database is run on both servers
and synced together through the AWS S3
server.
ML Training is done on the main server or
Plan 9’s development laptop.
AWS S3
Main Server Backup Server
Proposed System
AWS S3
Main Server Backup Server
UptimeRobot
Monitoring
Web Server
All the data is also pushed through to
Dropbox, in case the S3 instance fails. It
takes longer to restore from Dropbox, but it is
a proper archiving solution.
Uptime Robot monitors both servers.
The web app is run on the backup server, and
the website is on a web server (Plan 9 could
switch to a hosting provider in the future)
Dropbox
Real World Testing
For every product or IT asset, Testing is important to ensure that the product is functional and performs well.
Such was the case with our finalists- Ionos Intel Xeon 6126 and InterServer’s AMD EPYC 7402P. We tested it
using a high-resource-demanding bot created from Plan 9
The resultant scores were disappointing against the Plan 9 development laptop. This made us switch to a
desktop-class processor which would perform better than the Plan 9 laptop – AMD Ryzen 9 5950X. This
config had less RAM, but this processor was much faster than the other
Specifications Bot Write Copy
Linode VPS- 16GB RAM 8 Core Intel 27.9 13.7 1.6
Ionos Intel Xeon 6126 25.6 16.8 1.4
InterServer AMD EPYC 7402P 18.7 10.9 1.3
Plan 9 Development Laptop 20.2 8.3 0.8
InterServer AMD RYZEN 5950X 14.6 8.1 1.2
server processors, which was more beneficial. We also chose a lower
spec of the Ryzen (5900X) for the backup server.
Project Timeline
Project
Requirement
Allocation
Jan 2
Budget and
Metrics Calculation
Jan 4
Application
Analysis
Jan 5
Research Work
to find the best
providers
Jan 10
Plan9
chooses the
best finalists,
starts work
on testing
Jan 16
Servers
Provisioned
Jan 18
Performance
Testing
Jan 20
Provisioning
AMD 5950X
Jan 25
Installing
Software
– python,
bots etc
Feb 2
Research
work for
monitoring
solutions
Feb 5
UptimeRobot
Configured
Feb 10
Backup
Server
configured
Feb 15
Server
Migration
Feb 20
AWS S3
Feb 21
Fixing
Bugs and
Issues
Feb 23
Dropbox
config
Feb 27
Project
Audit
Mar 1
Project
Complete
Mar 10
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
Final Costs
Budget- USD 600 P.M.
When we started with this
budget, it was challenging
to build a solution to fit this
limited budget.
However, we now have a
fully-fledged solution,
which is high performing
and functional.
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
Component Product Cost
Main Server
InterServer AMD Ryzen 9 5950X
128GB RAM, 2TB NVMe
209 USD
Backup/ Webapp
Server
InterServer AMD Ryzen 9 5900X
128GB RAM, 2TB NVMe
125 USD
Domain NameCheap 20 USD
Storage Backup Amazon S3 2TB 10 USD
Archive Backup Dropbox Premium 2TB 20 USD
Monitoring UptimeRobot 30 USD
Total Costs Per Month 414 USD
Summary
As an infrastructure solutions architect, the aim of this project was to
make Plan 9’s infrastructure have the best specifications to suit their
company’s application while sticking to the budget- thus creating the
most value.
I was able to save Plan 9 around 176 USD per month while building a
solution that will benefit Plan 9 for years to come.
BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
Thank you
Daniel Jude Gonsalves
111720036021
In collaboration with
Plan 9 Labs

More Related Content

Similar to Building Infrastructure for an IT Organization

AAI-4847 Full Disclosure on the Performance Characteristics of WebSphere Appl...
AAI-4847 Full Disclosure on the Performance Characteristics of WebSphere Appl...AAI-4847 Full Disclosure on the Performance Characteristics of WebSphere Appl...
AAI-4847 Full Disclosure on the Performance Characteristics of WebSphere Appl...WASdev Community
 
IBM World of Watson 2016 - DB2 Analytics Accelerator on Cloud
IBM World of Watson 2016 - DB2 Analytics Accelerator on CloudIBM World of Watson 2016 - DB2 Analytics Accelerator on Cloud
IBM World of Watson 2016 - DB2 Analytics Accelerator on CloudDaniel Martin
 
RDS Postgres and Aurora Postgres | AWS Public Sector Summit 2017
RDS Postgres and Aurora Postgres | AWS Public Sector Summit 2017RDS Postgres and Aurora Postgres | AWS Public Sector Summit 2017
RDS Postgres and Aurora Postgres | AWS Public Sector Summit 2017Amazon Web Services
 
Pivotal Cloud Foundry 2.6: A First Look
Pivotal Cloud Foundry 2.6: A First LookPivotal Cloud Foundry 2.6: A First Look
Pivotal Cloud Foundry 2.6: A First LookVMware Tanzu
 
Level up your SQL and Azure, by using Rubrik
Level up your SQL and Azure, by using RubrikLevel up your SQL and Azure, by using Rubrik
Level up your SQL and Azure, by using RubrikJaap Brasser
 
SUSE Container as a Service Platform
SUSE Container as a Service PlatformSUSE Container as a Service Platform
SUSE Container as a Service PlatformSUSE
 
Introduction to Azure IaaS
Introduction to Azure IaaSIntroduction to Azure IaaS
Introduction to Azure IaaSRobert Crane
 
[よくわかるAmazon Redshift]Amazon Redshift最新情報と導入事例のご紹介
[よくわかるAmazon Redshift]Amazon Redshift最新情報と導入事例のご紹介[よくわかるAmazon Redshift]Amazon Redshift最新情報と導入事例のご紹介
[よくわかるAmazon Redshift]Amazon Redshift最新情報と導入事例のご紹介Amazon Web Services Japan
 
Deploying couchbaseserverazure cihanbiyikoglu_microsoft
Deploying couchbaseserverazure cihanbiyikoglu_microsoftDeploying couchbaseserverazure cihanbiyikoglu_microsoft
Deploying couchbaseserverazure cihanbiyikoglu_microsoftCihan Biyikoglu
 
Dimension data cloud for the enterprise architect
Dimension data cloud for the enterprise architectDimension data cloud for the enterprise architect
Dimension data cloud for the enterprise architectDavid Sawatzke
 
Strategies for Seamless Backup and Disaster Recovery with AWS
Strategies for Seamless Backup and Disaster Recovery with AWSStrategies for Seamless Backup and Disaster Recovery with AWS
Strategies for Seamless Backup and Disaster Recovery with AWSAmazon Web Services
 
Data Core Riverved Dr 22 Sep08
Data Core Riverved Dr 22 Sep08Data Core Riverved Dr 22 Sep08
Data Core Riverved Dr 22 Sep08michaelking
 
Virtual Design Master Challenge 1 - Akmal
Virtual Design Master Challenge 1  - AkmalVirtual Design Master Challenge 1  - Akmal
Virtual Design Master Challenge 1 - Akmaltovmug
 
Pivotal Cloud Foundry 2.5: A First Look
Pivotal Cloud Foundry 2.5: A First LookPivotal Cloud Foundry 2.5: A First Look
Pivotal Cloud Foundry 2.5: A First LookVMware Tanzu
 
Azure en Nutanix: your journey to the hybrid cloud
Azure en Nutanix: your journey to the hybrid cloudAzure en Nutanix: your journey to the hybrid cloud
Azure en Nutanix: your journey to the hybrid cloudICT-Partners
 
RedisConf17 - Building Large High Performance Redis Databases with Redis Ente...
RedisConf17 - Building Large High Performance Redis Databases with Redis Ente...RedisConf17 - Building Large High Performance Redis Databases with Redis Ente...
RedisConf17 - Building Large High Performance Redis Databases with Redis Ente...Redis Labs
 

Similar to Building Infrastructure for an IT Organization (20)

AAI-4847 Full Disclosure on the Performance Characteristics of WebSphere Appl...
AAI-4847 Full Disclosure on the Performance Characteristics of WebSphere Appl...AAI-4847 Full Disclosure on the Performance Characteristics of WebSphere Appl...
AAI-4847 Full Disclosure on the Performance Characteristics of WebSphere Appl...
 
Databases - State of the Union
Databases - State of the UnionDatabases - State of the Union
Databases - State of the Union
 
IBM World of Watson 2016 - DB2 Analytics Accelerator on Cloud
IBM World of Watson 2016 - DB2 Analytics Accelerator on CloudIBM World of Watson 2016 - DB2 Analytics Accelerator on Cloud
IBM World of Watson 2016 - DB2 Analytics Accelerator on Cloud
 
RDS Postgres and Aurora Postgres | AWS Public Sector Summit 2017
RDS Postgres and Aurora Postgres | AWS Public Sector Summit 2017RDS Postgres and Aurora Postgres | AWS Public Sector Summit 2017
RDS Postgres and Aurora Postgres | AWS Public Sector Summit 2017
 
Pivotal Cloud Foundry 2.6: A First Look
Pivotal Cloud Foundry 2.6: A First LookPivotal Cloud Foundry 2.6: A First Look
Pivotal Cloud Foundry 2.6: A First Look
 
Level up your SQL and Azure, by using Rubrik
Level up your SQL and Azure, by using RubrikLevel up your SQL and Azure, by using Rubrik
Level up your SQL and Azure, by using Rubrik
 
SUSE Container as a Service Platform
SUSE Container as a Service PlatformSUSE Container as a Service Platform
SUSE Container as a Service Platform
 
Introduction to Azure IaaS
Introduction to Azure IaaSIntroduction to Azure IaaS
Introduction to Azure IaaS
 
Technolo Report
Technolo ReportTechnolo Report
Technolo Report
 
[よくわかるAmazon Redshift]Amazon Redshift最新情報と導入事例のご紹介
[よくわかるAmazon Redshift]Amazon Redshift最新情報と導入事例のご紹介[よくわかるAmazon Redshift]Amazon Redshift最新情報と導入事例のご紹介
[よくわかるAmazon Redshift]Amazon Redshift最新情報と導入事例のご紹介
 
Resume
ResumeResume
Resume
 
Deploying couchbaseserverazure cihanbiyikoglu_microsoft
Deploying couchbaseserverazure cihanbiyikoglu_microsoftDeploying couchbaseserverazure cihanbiyikoglu_microsoft
Deploying couchbaseserverazure cihanbiyikoglu_microsoft
 
Mdb dn 2016_11_ops_mgr
Mdb dn 2016_11_ops_mgrMdb dn 2016_11_ops_mgr
Mdb dn 2016_11_ops_mgr
 
Dimension data cloud for the enterprise architect
Dimension data cloud for the enterprise architectDimension data cloud for the enterprise architect
Dimension data cloud for the enterprise architect
 
Strategies for Seamless Backup and Disaster Recovery with AWS
Strategies for Seamless Backup and Disaster Recovery with AWSStrategies for Seamless Backup and Disaster Recovery with AWS
Strategies for Seamless Backup and Disaster Recovery with AWS
 
Data Core Riverved Dr 22 Sep08
Data Core Riverved Dr 22 Sep08Data Core Riverved Dr 22 Sep08
Data Core Riverved Dr 22 Sep08
 
Virtual Design Master Challenge 1 - Akmal
Virtual Design Master Challenge 1  - AkmalVirtual Design Master Challenge 1  - Akmal
Virtual Design Master Challenge 1 - Akmal
 
Pivotal Cloud Foundry 2.5: A First Look
Pivotal Cloud Foundry 2.5: A First LookPivotal Cloud Foundry 2.5: A First Look
Pivotal Cloud Foundry 2.5: A First Look
 
Azure en Nutanix: your journey to the hybrid cloud
Azure en Nutanix: your journey to the hybrid cloudAzure en Nutanix: your journey to the hybrid cloud
Azure en Nutanix: your journey to the hybrid cloud
 
RedisConf17 - Building Large High Performance Redis Databases with Redis Ente...
RedisConf17 - Building Large High Performance Redis Databases with Redis Ente...RedisConf17 - Building Large High Performance Redis Databases with Redis Ente...
RedisConf17 - Building Large High Performance Redis Databases with Redis Ente...
 

Recently uploaded

Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Alan Dix
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxMaximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxOnBoard
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Neo4j
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxnull - The Open Security Community
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr LapshynFwdays
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonetsnaman860154
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays
 
APIForce Zurich 5 April Automation LPDG
APIForce Zurich 5 April  Automation LPDGAPIForce Zurich 5 April  Automation LPDG
APIForce Zurich 5 April Automation LPDGMarianaLemus7
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksSoftradix Technologies
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 

Recently uploaded (20)

Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxMaximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptx
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
 
APIForce Zurich 5 April Automation LPDG
APIForce Zurich 5 April  Automation LPDGAPIForce Zurich 5 April  Automation LPDG
APIForce Zurich 5 April Automation LPDG
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other Frameworks
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptxVulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
 

Building Infrastructure for an IT Organization

  • 1. Building Infrastructure for an IT Organization Daniel Jude Gonsalves Plan9 Labs
  • 2. Agenda: A Cloud Architect Project Introduction to Plan 9 Abstract Requirements Current System Research Work Proposed System Real World Testing Timeline of the project Final costs Summary BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
  • 3. Introduction to Plan 9 Plan 9 Labs is an automation company that provides data robots to companies needing data transfer between data sources or CRMs. Removing Noise from existing data and provisioning robots that can sync data across multiple services is the key aim of Plan9 to secure proper data tagging, routing, and mapping between data sources. BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
  • 4. PLAN 9 AUTOMATION Web App Servers Blogs and Websites Adobe Experience Cloud CRMs Milestones Database Dashboard and Reporting Plan 9 Bot Plan 9 Bot Plan 9 Bot Segment Destination Data lake Destination Web App source Server source Multiple other sources
  • 5. Requirements To upgrade Plan 9’s Infrastructure
  • 6. Requirements SERVER Single Server with Backups and a multi hour MTTR (manual restore). OS Support for FreeBSD Must run Ubuntu 22.04 LTS No desktop GUI/ CLI only SPECS 8-16 Core CPU 32-64 GB RAM 500-2 TB STORAGE (SSD) MONITORING & BACKUP Live monitoring SMS alerting Backup with AWS S3 buckets BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
  • 7. Current System Linode Server with 8 Cores, 16GB RAM and 300 GB RAM. Support for Ubuntu 22.04 LTS , but no support for FreeBSD Backup done through AWS S3 buckets. Whole images of data backed up daily and can be restored in 4-6 Hours BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION AWS S3 Linode VPS Cloud Server
  • 8. Research Work Determining the Type of Server Virtual Private Server, Virtual Data Center or Hosted Dedicated Server Backup and Failover Solutions Having a Failover of less than 2 hours if the server is down is important Best Server Location Strategies To provide for the least latency, the server should be located on the US east coast. Monitoring and Alerts via SMS and Email For instant response to restore servers manually (automatic restore not required). 1 2 3 4 BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
  • 9. Research Work Determining the Type of Server Virtual Private Server, Virtual Data Center or Hosted Dedicated Server 1 BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION Virtual Private Server- It is a server virtualized on physical server hardware, providing scalability and flexibility of configurations with its secondary storage virtualized using LUNs with stacks of HDD or SSD units. This is advantageous for scalability and flexibility only, but read/write speeds are significantly slower Virtual Data Center- It is a collection of VPSes connected over virtualized networks and having databases, monitoring servers, and other servers all on a single physical virtualized infrastructure. Only provided by Ionos Hosted Dedicated Server: It is a physical rack-based server that is configured with physical components and is hosted over a network. High performance but loses on Virtualization features. Finally, the best type of Server boils down to the pricing and what fits in Plan9’s budget.
  • 10. Research Work Plan9 had a budget of 600 USD(per month) to spare for their whole infrastructure. We had to configure everything including servers, backup, monitoring and failover for that price Research work was started to find the best servers for the money, according to the specifications provided. This table is a small snippet of the prices we found. (in USD per Month) We chose InterServer and Ionos as our finalists because of the price, and a hosted dedicated server because of performance. We could thus use our budget to configure better backup infra. Specifications AWS Digital Ocean Google Cloud InterServer Ionos 32GB RAM Server 720 252 908 98 185 64GB RAM Server - 504 - 110 210 128GB RAM Server - 1008 - 209 256 Type of Server VPS VPS VPS DS & VPS DS VPS & VDC
  • 11. Research Work Backup and Failover Solutions Having a Failover of less than 2 hours if the server is down is important 2 BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION Plan 9’s current system consists of a single server with images being pushed to AWS S3 bucket server. There is also a redundant backup which is pushed to dropbox. However, with the new system, we propose a 2 server system, where the second server takes over immediately in case of the first server failing. PostgreSQL data is pushed to both servers at the same time, properly syncing data between the two servers. Bots still push a snapshot of data daily to Amazon S3 (more snapshots daily to be implemented soon). Amazon S3 data can be restored onto a new server configured within minutes and the bots redeployed in less than an hour just like the previous configuration. Dropbox is still used in case Amazon S3 fails
  • 12. Research Work Best Server Location Strategies To provide for the least latency, the server should be located on the US east coast 3 BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION Plan 9’s main servers need to be located on the US east coast to prevent any latency and network issues that may arise. Having the server in the same regional location is also beneficial for application and data processing performance. The Linode server that Plan 9 uses is based in New York, and Plan 9 needed something similar for their server upgrade. Both our finalists Ionos and InterServer are located in New Jersey, which is a city a few miles from New York, and on the east coast. For the second server, we needed a server from a different region and location to prevent data center failures, or environmental hazards which can cause a certain server in a region to fail (ex- storms, hurricanes, tornados or earthquakes) Therefore, we configured the second server in central US- Las Vegas
  • 13. Research Work Monitoring and Alerts via SMS and Email For instant response to restore servers manually (automatic restore not required) 4 BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION 13 Plan 9 has always used a monitoring system to alert them when the server was down. Linode offered good services for their VPS products with SMS alerting and stats monitoring. The main open-source monitoring solutions that we discovered were Zabbix and Nagios. These are cli configurable monitoring that requires a server to run and lots of configuration. This was however very complex to install and run and provided features that wouldn’t be useful to us. Therefore, we dropped that and used a domain monitoring solution called UptimeRobot which could monitor the status of the domain (which we configured using Namecheap, for our server) This was also within budget and costed only 30 USD per month.
  • 14. Proposed System Main Server is located in NJ and Backup Server in LA All the bots are run as Cron jobs (scheduled tasks) We use an AWS S3 server as an archive for the data being created or run through the robots on the main server. PostgreSQL database is run on both servers and synced together through the AWS S3 server. ML Training is done on the main server or Plan 9’s development laptop. AWS S3 Main Server Backup Server
  • 15. Proposed System AWS S3 Main Server Backup Server UptimeRobot Monitoring Web Server All the data is also pushed through to Dropbox, in case the S3 instance fails. It takes longer to restore from Dropbox, but it is a proper archiving solution. Uptime Robot monitors both servers. The web app is run on the backup server, and the website is on a web server (Plan 9 could switch to a hosting provider in the future) Dropbox
  • 16. Real World Testing For every product or IT asset, Testing is important to ensure that the product is functional and performs well. Such was the case with our finalists- Ionos Intel Xeon 6126 and InterServer’s AMD EPYC 7402P. We tested it using a high-resource-demanding bot created from Plan 9 The resultant scores were disappointing against the Plan 9 development laptop. This made us switch to a desktop-class processor which would perform better than the Plan 9 laptop – AMD Ryzen 9 5950X. This config had less RAM, but this processor was much faster than the other Specifications Bot Write Copy Linode VPS- 16GB RAM 8 Core Intel 27.9 13.7 1.6 Ionos Intel Xeon 6126 25.6 16.8 1.4 InterServer AMD EPYC 7402P 18.7 10.9 1.3 Plan 9 Development Laptop 20.2 8.3 0.8 InterServer AMD RYZEN 5950X 14.6 8.1 1.2 server processors, which was more beneficial. We also chose a lower spec of the Ryzen (5900X) for the backup server.
  • 17. Project Timeline Project Requirement Allocation Jan 2 Budget and Metrics Calculation Jan 4 Application Analysis Jan 5 Research Work to find the best providers Jan 10 Plan9 chooses the best finalists, starts work on testing Jan 16 Servers Provisioned Jan 18 Performance Testing Jan 20 Provisioning AMD 5950X Jan 25 Installing Software – python, bots etc Feb 2 Research work for monitoring solutions Feb 5 UptimeRobot Configured Feb 10 Backup Server configured Feb 15 Server Migration Feb 20 AWS S3 Feb 21 Fixing Bugs and Issues Feb 23 Dropbox config Feb 27 Project Audit Mar 1 Project Complete Mar 10 BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
  • 18. Final Costs Budget- USD 600 P.M. When we started with this budget, it was challenging to build a solution to fit this limited budget. However, we now have a fully-fledged solution, which is high performing and functional. BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION Component Product Cost Main Server InterServer AMD Ryzen 9 5950X 128GB RAM, 2TB NVMe 209 USD Backup/ Webapp Server InterServer AMD Ryzen 9 5900X 128GB RAM, 2TB NVMe 125 USD Domain NameCheap 20 USD Storage Backup Amazon S3 2TB 10 USD Archive Backup Dropbox Premium 2TB 20 USD Monitoring UptimeRobot 30 USD Total Costs Per Month 414 USD
  • 19. Summary As an infrastructure solutions architect, the aim of this project was to make Plan 9’s infrastructure have the best specifications to suit their company’s application while sticking to the budget- thus creating the most value. I was able to save Plan 9 around 176 USD per month while building a solution that will benefit Plan 9 for years to come. BUILDING INFRASTRUCTURE FOR AN IT ORGANIZATION
  • 20. Thank you Daniel Jude Gonsalves 111720036021 In collaboration with Plan 9 Labs