A presentation summarizing my Cloud Based Payroll project. The presentation highlights how Google Cloud components and custom servers were combined to develop an auto scalable cloud based payroll system that could scale/adapt based on CPU demand.
2. Introduction
RESTful Auto-Scaling Payroll API
Several Custom Components (Application, Translator, Accruals, Delivery)
Google Components (Load Balancer, Auto Scaling, Pub/Sub, Cloud Spanner)
Users send/receive data as JSON objects through HTTP Requests
4. Load Balancer
Distributes our users to various applications that are in service
Distributes to a Managed Instance Group
Managed Instance Groups are just groups of identical instances built from an
Instance Template
Instance Template is an image of our Application
If changes are made to the Instance Template, these changes are rolled out to all
instances in the Managed Instance Group
5. Auto Scaling
Monitors the CPU level of our Application Instances
If it gets too high, new Application instances are created to help reduce the load
When the traffic dies down, these instances are shut off and deleted
automatically
6. Pub/Sub
Google’s Messaging Service that allows you send and receive messages
between independent applications.
Allowed for easy communication between all of our components
Topics:
● insert-new-company
● insert-new-employee
● insert-timesheets
● calculate-accruals
● paystub-delivery
7. Application
Flask Server
Routes Publishes To Returns
createCompany insert-new-company Response
addEmployee insert-new-employee Response
submit insert-timesheets Response
calculateAccruals calculate-accruals Response
deliveryRequest delivery-request Response
paystub - Your paystub
8. Translator
This component subscribes to 3 topics:
● insert-new-company
● insert-new-employee
● insert-timesheets
Parses the JSON message, generates a database insert statement, then inserts
the data
10. Delivery
This instance has one job, to deliver pay stubs to the front-end application
Subscribes to delivery-requests
Queries the database for an employee’s pay stub
Publishes your pay stub to paystub-delivery
13. Design Challenges
● Designing our components. What should they do? How do they interact with
the rest of our components?
● Constructing the JSON objects our users would be required to send
● Designing the database to work with the JSON
● We tried Google SQL first, but it was not horizontally scalable
● How to scale the Application server and add a Load Balancer?
● Issues with various Kafka libraries led to Google’s Pub/Sub
14. Conclusion
Things that would be nice:
● Auto calculate accruals (chron job, every 2 weeks or so)
● Full horizontal scalability (Translator, Delivery, Accruals)
● Find alternative for database cluster component instead of Cloud Spanner
● Costs $9000 per node/year ($0.90/hour for just 1 node)
● Can’t be shut down