SlideShare a Scribd company logo
1 of 15
A RESTful, Auto-Scaling Payroll API
Mike, Joe, Edson
Introduction
RESTful Auto-Scaling Payroll API
Several Custom Components (Application, Translator, Accruals, Delivery)
Google Components (Load Balancer, Auto Scaling, Pub/Sub, Cloud Spanner)
Users send/receive data as JSON objects through HTTP Requests
Component Model
Load Balancer
Distributes our users to various applications that are in service
Distributes to a Managed Instance Group
Managed Instance Groups are just groups of identical instances built from an
Instance Template
Instance Template is an image of our Application
If changes are made to the Instance Template, these changes are rolled out to all
instances in the Managed Instance Group
Auto Scaling
Monitors the CPU level of our Application Instances
If it gets too high, new Application instances are created to help reduce the load
When the traffic dies down, these instances are shut off and deleted
automatically
Pub/Sub
Google’s Messaging Service that allows you send and receive messages
between independent applications.
Allowed for easy communication between all of our components
Topics:
● insert-new-company
● insert-new-employee
● insert-timesheets
● calculate-accruals
● paystub-delivery
Application
Flask Server
Routes Publishes To Returns
createCompany insert-new-company Response
addEmployee insert-new-employee Response
submit insert-timesheets Response
calculateAccruals calculate-accruals Response
deliveryRequest delivery-request Response
paystub - Your paystub
Translator
This component subscribes to 3 topics:
● insert-new-company
● insert-new-employee
● insert-timesheets
Parses the JSON message, generates a database insert statement, then inserts
the data
Accruals
Subscribes to calculate-accruals
Calculates your pay stub
Inserts your pay stub into the database
It only calculates your pay when you request it
Delivery
This instance has one job, to deliver pay stubs to the front-end application
Subscribes to delivery-requests
Queries the database for an employee’s pay stub
Publishes your pay stub to paystub-delivery
Cloud Spanner Database
Relational
Strong Consistency
High Availability
Automatic Replication
Horizontally Scalable
SQL-like queries
Component Model
Design Challenges
● Designing our components. What should they do? How do they interact with
the rest of our components?
● Constructing the JSON objects our users would be required to send
● Designing the database to work with the JSON
● We tried Google SQL first, but it was not horizontally scalable
● How to scale the Application server and add a Load Balancer?
● Issues with various Kafka libraries led to Google’s Pub/Sub
Conclusion
Things that would be nice:
● Auto calculate accruals (chron job, every 2 weeks or so)
● Full horizontal scalability (Translator, Delivery, Accruals)
● Find alternative for database cluster component instead of Cloud Spanner
● Costs $9000 per node/year ($0.90/hour for just 1 node)
● Can’t be shut down
Demo!

More Related Content

What's hot

3 pm3 t_we%20-%20setting%20up%20the%20control%20account%20structure
3 pm3 t_we%20-%20setting%20up%20the%20control%20account%20structure3 pm3 t_we%20-%20setting%20up%20the%20control%20account%20structure
3 pm3 t_we%20-%20setting%20up%20the%20control%20account%20structure
harwoodr
 

What's hot (19)

Top open source tools to consider for web service performance testing
Top open source tools to consider for web service performance testingTop open source tools to consider for web service performance testing
Top open source tools to consider for web service performance testing
 
Integration with dropbox using mule esb
Integration with dropbox using mule esbIntegration with dropbox using mule esb
Integration with dropbox using mule esb
 
Adopting serverless
Adopting serverlessAdopting serverless
Adopting serverless
 
Kube London May 2018
Kube London May 2018Kube London May 2018
Kube London May 2018
 
Cloud ftp a case study of migrating traditional applications to the cloud
Cloud ftp a case study of migrating traditional applications to the cloudCloud ftp a case study of migrating traditional applications to the cloud
Cloud ftp a case study of migrating traditional applications to the cloud
 
Testing soapui
Testing soapuiTesting soapui
Testing soapui
 
АРТЕМ КОБРІН «Achieve Networking at Scale with a Self-Service Network Solutio...
АРТЕМ КОБРІН «Achieve Networking at Scale with a Self-Service Network Solutio...АРТЕМ КОБРІН «Achieve Networking at Scale with a Self-Service Network Solutio...
АРТЕМ КОБРІН «Achieve Networking at Scale with a Self-Service Network Solutio...
 
Google apps engine
Google apps engineGoogle apps engine
Google apps engine
 
AWS Config Rules - Advanced AWS Meetup
AWS Config Rules - Advanced AWS MeetupAWS Config Rules - Advanced AWS Meetup
AWS Config Rules - Advanced AWS Meetup
 
AWS Config - Advanced AWS Meetup SF
AWS Config - Advanced AWS Meetup SFAWS Config - Advanced AWS Meetup SF
AWS Config - Advanced AWS Meetup SF
 
Cloud ftp a case study of migrating traditional applications to the cloud
Cloud ftp a case study of migrating traditional applications to the cloudCloud ftp a case study of migrating traditional applications to the cloud
Cloud ftp a case study of migrating traditional applications to the cloud
 
Azuresatpn19 - An Introduction To Azure Data Factory
Azuresatpn19 - An Introduction To Azure Data FactoryAzuresatpn19 - An Introduction To Azure Data Factory
Azuresatpn19 - An Introduction To Azure Data Factory
 
SoapUi using WebServices
SoapUi using WebServicesSoapUi using WebServices
SoapUi using WebServices
 
WebServices using Soap
WebServices using SoapWebServices using Soap
WebServices using Soap
 
3 pm3 t_we%20-%20setting%20up%20the%20control%20account%20structure
3 pm3 t_we%20-%20setting%20up%20the%20control%20account%20structure3 pm3 t_we%20-%20setting%20up%20the%20control%20account%20structure
3 pm3 t_we%20-%20setting%20up%20the%20control%20account%20structure
 
Integrating with Aws s3
Integrating with Aws s3Integrating with Aws s3
Integrating with Aws s3
 
Riviera Jug - 20/03/2018 - KSQL
Riviera Jug - 20/03/2018 - KSQLRiviera Jug - 20/03/2018 - KSQL
Riviera Jug - 20/03/2018 - KSQL
 
Serverless
ServerlessServerless
Serverless
 
How to Optimize GIS Workflows
How to Optimize GIS WorkflowsHow to Optimize GIS Workflows
How to Optimize GIS Workflows
 

Similar to Scalable Google Cloud Payroll Project- Presentation

An Empirical Performance Study of AppEngine and AppScale
An Empirical Performance Study of AppEngine and AppScaleAn Empirical Performance Study of AppEngine and AppScale
An Empirical Performance Study of AppEngine and AppScale
Fei Dong
 
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptxTransform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx
kmani5
 
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).ppt
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).pptTransform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).ppt
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).ppt
HusseinWassof
 
Final Report To Executive ManagersXXXXXCCA 625Un
Final Report To Executive ManagersXXXXXCCA 625UnFinal Report To Executive ManagersXXXXXCCA 625Un
Final Report To Executive ManagersXXXXXCCA 625Un
ChereCheek752
 

Similar to Scalable Google Cloud Payroll Project- Presentation (20)

Scalable Google Cloud Payroll Project - Paper
Scalable Google Cloud Payroll Project - PaperScalable Google Cloud Payroll Project - Paper
Scalable Google Cloud Payroll Project - Paper
 
Unit 5.pptx
Unit 5.pptxUnit 5.pptx
Unit 5.pptx
 
How to – wrap soap web service around a database
How to – wrap soap web service around a databaseHow to – wrap soap web service around a database
How to – wrap soap web service around a database
 
AWS Step Functions을 활용한 서버리스 앱 오케스트레이션
AWS Step Functions을 활용한 서버리스 앱 오케스트레이션AWS Step Functions을 활용한 서버리스 앱 오케스트레이션
AWS Step Functions을 활용한 서버리스 앱 오케스트레이션
 
An Empirical Performance Study of AppEngine and AppScale
An Empirical Performance Study of AppEngine and AppScaleAn Empirical Performance Study of AppEngine and AppScale
An Empirical Performance Study of AppEngine and AppScale
 
Automate Your Big Data Workflows (SVC201) | AWS re:Invent 2013
Automate Your Big Data Workflows (SVC201) | AWS re:Invent 2013Automate Your Big Data Workflows (SVC201) | AWS re:Invent 2013
Automate Your Big Data Workflows (SVC201) | AWS re:Invent 2013
 
WebServices using Soapui
WebServices using SoapuiWebServices using Soapui
WebServices using Soapui
 
Testing soapui
Testing soapuiTesting soapui
Testing soapui
 
Managing Large Flask Applications On Google App Engine (GAE)
Managing Large Flask Applications On Google App Engine (GAE)Managing Large Flask Applications On Google App Engine (GAE)
Managing Large Flask Applications On Google App Engine (GAE)
 
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptxTransform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx
 
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).ppt
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).pptTransform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).ppt
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).ppt
 
February 2016 Webinar Series Migrate Your Apps from Parse to AWS
February 2016 Webinar Series   Migrate Your Apps from Parse to AWSFebruary 2016 Webinar Series   Migrate Your Apps from Parse to AWS
February 2016 Webinar Series Migrate Your Apps from Parse to AWS
 
Final Report To Executive ManagersXXXXXCCA 625Un
Final Report To Executive ManagersXXXXXCCA 625UnFinal Report To Executive ManagersXXXXXCCA 625Un
Final Report To Executive ManagersXXXXXCCA 625Un
 
Shattering The Monolith(s) (Martin Kess, Namely) Kafka Summit SF 2019
Shattering The Monolith(s) (Martin Kess, Namely) Kafka Summit SF 2019 Shattering The Monolith(s) (Martin Kess, Namely) Kafka Summit SF 2019
Shattering The Monolith(s) (Martin Kess, Namely) Kafka Summit SF 2019
 
Azure Functions.pptx
Azure Functions.pptxAzure Functions.pptx
Azure Functions.pptx
 
Project Report
Project ReportProject Report
Project Report
 
Real-World Pulsar Architectural Patterns
Real-World Pulsar Architectural PatternsReal-World Pulsar Architectural Patterns
Real-World Pulsar Architectural Patterns
 
Evolution of netflix conductor
Evolution of netflix conductorEvolution of netflix conductor
Evolution of netflix conductor
 
Appscale at CLOUDCOMP '09
Appscale at CLOUDCOMP '09Appscale at CLOUDCOMP '09
Appscale at CLOUDCOMP '09
 
AWS Batch: Simplifying Batch Computing in the Cloud
AWS Batch: Simplifying Batch Computing in the CloudAWS Batch: Simplifying Batch Computing in the Cloud
AWS Batch: Simplifying Batch Computing in the Cloud
 

Recently uploaded

Final DBMS Manual (2).pdf final lab manual
Final DBMS Manual (2).pdf final lab manualFinal DBMS Manual (2).pdf final lab manual
Final DBMS Manual (2).pdf final lab manual
BalamuruganV28
 
electrical installation and maintenance.
electrical installation and maintenance.electrical installation and maintenance.
electrical installation and maintenance.
benjamincojr
 
Maher Othman Interior Design Portfolio..
Maher Othman Interior Design Portfolio..Maher Othman Interior Design Portfolio..
Maher Othman Interior Design Portfolio..
MaherOthman7
 

Recently uploaded (20)

handbook on reinforce concrete and detailing
handbook on reinforce concrete and detailinghandbook on reinforce concrete and detailing
handbook on reinforce concrete and detailing
 
Lab Manual Arduino UNO Microcontrollar.docx
Lab Manual Arduino UNO Microcontrollar.docxLab Manual Arduino UNO Microcontrollar.docx
Lab Manual Arduino UNO Microcontrollar.docx
 
8th International Conference on Soft Computing, Mathematics and Control (SMC ...
8th International Conference on Soft Computing, Mathematics and Control (SMC ...8th International Conference on Soft Computing, Mathematics and Control (SMC ...
8th International Conference on Soft Computing, Mathematics and Control (SMC ...
 
Final DBMS Manual (2).pdf final lab manual
Final DBMS Manual (2).pdf final lab manualFinal DBMS Manual (2).pdf final lab manual
Final DBMS Manual (2).pdf final lab manual
 
Piping and instrumentation diagram p.pdf
Piping and instrumentation diagram p.pdfPiping and instrumentation diagram p.pdf
Piping and instrumentation diagram p.pdf
 
litvinenko_Henry_Intrusion_Hong-Kong_2024.pdf
litvinenko_Henry_Intrusion_Hong-Kong_2024.pdflitvinenko_Henry_Intrusion_Hong-Kong_2024.pdf
litvinenko_Henry_Intrusion_Hong-Kong_2024.pdf
 
5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...
 
Research Methodolgy & Intellectual Property Rights Series 1
Research Methodolgy & Intellectual Property Rights Series 1Research Methodolgy & Intellectual Property Rights Series 1
Research Methodolgy & Intellectual Property Rights Series 1
 
"United Nations Park" Site Visit Report.
"United Nations Park" Site  Visit Report."United Nations Park" Site  Visit Report.
"United Nations Park" Site Visit Report.
 
What is Coordinate Measuring Machine? CMM Types, Features, Functions
What is Coordinate Measuring Machine? CMM Types, Features, FunctionsWhat is Coordinate Measuring Machine? CMM Types, Features, Functions
What is Coordinate Measuring Machine? CMM Types, Features, Functions
 
Artificial Intelligence in due diligence
Artificial Intelligence in due diligenceArtificial Intelligence in due diligence
Artificial Intelligence in due diligence
 
NO1 Best Powerful Vashikaran Specialist Baba Vashikaran Specialist For Love V...
NO1 Best Powerful Vashikaran Specialist Baba Vashikaran Specialist For Love V...NO1 Best Powerful Vashikaran Specialist Baba Vashikaran Specialist For Love V...
NO1 Best Powerful Vashikaran Specialist Baba Vashikaran Specialist For Love V...
 
21scheme vtu syllabus of visveraya technological university
21scheme vtu syllabus of visveraya technological university21scheme vtu syllabus of visveraya technological university
21scheme vtu syllabus of visveraya technological university
 
Research Methodolgy & Intellectual Property Rights Series 2
Research Methodolgy & Intellectual Property Rights Series 2Research Methodolgy & Intellectual Property Rights Series 2
Research Methodolgy & Intellectual Property Rights Series 2
 
Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)
 
Introduction to Artificial Intelligence and History of AI
Introduction to Artificial Intelligence and History of AIIntroduction to Artificial Intelligence and History of AI
Introduction to Artificial Intelligence and History of AI
 
Low Altitude Air Defense (LAAD) Gunner’s Handbook
Low Altitude Air Defense (LAAD) Gunner’s HandbookLow Altitude Air Defense (LAAD) Gunner’s Handbook
Low Altitude Air Defense (LAAD) Gunner’s Handbook
 
electrical installation and maintenance.
electrical installation and maintenance.electrical installation and maintenance.
electrical installation and maintenance.
 
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdfInvolute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
 
Maher Othman Interior Design Portfolio..
Maher Othman Interior Design Portfolio..Maher Othman Interior Design Portfolio..
Maher Othman Interior Design Portfolio..
 

Scalable Google Cloud Payroll Project- Presentation

  • 1. A RESTful, Auto-Scaling Payroll API Mike, Joe, Edson
  • 2. Introduction RESTful Auto-Scaling Payroll API Several Custom Components (Application, Translator, Accruals, Delivery) Google Components (Load Balancer, Auto Scaling, Pub/Sub, Cloud Spanner) Users send/receive data as JSON objects through HTTP Requests
  • 4. Load Balancer Distributes our users to various applications that are in service Distributes to a Managed Instance Group Managed Instance Groups are just groups of identical instances built from an Instance Template Instance Template is an image of our Application If changes are made to the Instance Template, these changes are rolled out to all instances in the Managed Instance Group
  • 5. Auto Scaling Monitors the CPU level of our Application Instances If it gets too high, new Application instances are created to help reduce the load When the traffic dies down, these instances are shut off and deleted automatically
  • 6. Pub/Sub Google’s Messaging Service that allows you send and receive messages between independent applications. Allowed for easy communication between all of our components Topics: ● insert-new-company ● insert-new-employee ● insert-timesheets ● calculate-accruals ● paystub-delivery
  • 7. Application Flask Server Routes Publishes To Returns createCompany insert-new-company Response addEmployee insert-new-employee Response submit insert-timesheets Response calculateAccruals calculate-accruals Response deliveryRequest delivery-request Response paystub - Your paystub
  • 8. Translator This component subscribes to 3 topics: ● insert-new-company ● insert-new-employee ● insert-timesheets Parses the JSON message, generates a database insert statement, then inserts the data
  • 9. Accruals Subscribes to calculate-accruals Calculates your pay stub Inserts your pay stub into the database It only calculates your pay when you request it
  • 10. Delivery This instance has one job, to deliver pay stubs to the front-end application Subscribes to delivery-requests Queries the database for an employee’s pay stub Publishes your pay stub to paystub-delivery
  • 11. Cloud Spanner Database Relational Strong Consistency High Availability Automatic Replication Horizontally Scalable SQL-like queries
  • 13. Design Challenges ● Designing our components. What should they do? How do they interact with the rest of our components? ● Constructing the JSON objects our users would be required to send ● Designing the database to work with the JSON ● We tried Google SQL first, but it was not horizontally scalable ● How to scale the Application server and add a Load Balancer? ● Issues with various Kafka libraries led to Google’s Pub/Sub
  • 14. Conclusion Things that would be nice: ● Auto calculate accruals (chron job, every 2 weeks or so) ● Full horizontal scalability (Translator, Delivery, Accruals) ● Find alternative for database cluster component instead of Cloud Spanner ● Costs $9000 per node/year ($0.90/hour for just 1 node) ● Can’t be shut down
  • 15. Demo!