SlideShare a Scribd company logo
#UXDXAPAC Handling Sudden Growth
Handling Sudden Growth
Kristine Joy Paas (@kjcpaas)
Engineering Manager, Quipper Ltd.
UXDX APAC
#UXDXAPAC Handling Sudden Growth
Kristine Joy Paas (Joy)
- Engineering Manager at Quipper Ltd.
- Ruby, Javascript (mostly React js),
Unix, Kubernetes
- Hobbies
- Pre-COVID: travelling, swimming
- COVID: reading, watching series
and movies
#UXDXAPAC Handling Sudden Growth
#UXDXAPAC Handling Sudden Growth
Agenda | 01
02
03
04
05
06
Early 2020
Our Challenges
Our Response
Results
The Future
Summary
#UXDXAPAC Handling Sudden Growth
Early 2020
01
#UXDXAPAC Handling Sudden Growth
Early 2020
Restructuring: Splitting the Japanese business to the rest of the global business
WHY?
1. Difference in Academic calendars
2. Ensures Future business sustainability
#UXDXAPAC Handling Sudden Growth
If you want to place an image with full screen,
remove the top blue bar and the footer
But then on March 2020…
#UXDXAPAC Handling Sudden Growth
Changes in user behavior
Supplemental
service
Synchronous
education
#UXDXAPAC Handling Sudden Growth
Changes in user behavior
Only using
Quipper
contents
Now creating
their own
content
#UXDXAPAC Handling Sudden Growth
Teacher’s usage pattern (Oct 2019 vs Oct 2020)
#UXDXAPAC Handling Sudden Growth
Student’s usage pattern (Oct 2019 vs Oct 2020)
#UXDXAPAC Handling Sudden Growth
Number of Access (Teacher vs Students)
pre-covid covid
#UXDXAPAC Handling Sudden Growth
Our Challenges
02
#UXDXAPAC Handling Sudden Growth
Effects on our product
- More demand for features to facilitate online education
- Higher traffic due to more users with higher activity per user
- More bugs discovered due to increased usage
- Higher urgency in addressing and fixing bug reports
#UXDXAPAC Handling Sudden Growth
August 2020:
- Start of some private schools
October 2020:
- Start of public schools
Incidents Happened... Incidents
#UXDXAPAC Handling Sudden Growth
Incident Investigation Results
Main Symptoms:
- Slow access on some pages, leading to errors
- Late delivery of activities to students
Causes:
- Slow performance on database need further optimizations
- No CDN on some platforms set up CDN
- Autoscaling setup didn’t work improve scaling based on peak times
#UXDXAPAC Handling Sudden Growth
Our Response
03
#UXDXAPAC Handling Sudden Growth
We were not ready for this!
We didn’t expect to grow this fast!
😱
#UXDXAPAC Handling Sudden Growth
Platform Optimization Efforts
WHY?
- By focusing on developing features, we had less attention on scalability.
- We accumulated technical debt and now it’s the time to pay back.
- Developing new features is useless if users cannot use our core features.
Which lead to…
a freeze on developing new features
#UXDXAPAC Handling Sudden Growth
Platform Optimization Efforts
I was chosen to lead the efforts and we broke down our efforts into 3 main
focus:
Technical
Communications
Operations
#UXDXAPAC Handling Sudden Growth
Platform Optimization Efforts
Technical
Allows us to prevent these incidents from happening again
- Clean up old code
- Remove unnecessary requests
- Improve performance via caching, N+1 fixes, improving algorithms, etc
- Setup auto-scaling and scheduled scaling
- Ensure apps have enough resources so they don’t crash
- Improve database for higher capacity
#UXDXAPAC Handling Sudden Growth
Platform Optimization Efforts
Communications
Keeps everyone on the same page
- Agree on timeline with the stakeholders
- Communicate with users about the efforts being done
- Weekly checkup with Platform leads on the optimization efforts, to also
maintain cross-platform communication
- Compile common issues encountered so that support officers can
respond to users faster
- Create an incident management flow so that everyone is up-to-date on
status of incidents that may happen
#UXDXAPAC Handling Sudden Growth
Platform Optimization Efforts
Operations
Allows us to recover from incidents as quickly as possible
- Have developers on stand by (on a rotation basis) from 6:30 am for
possible incidents.
#UXDXAPAC Handling Sudden Growth
Results
04
#UXDXAPAC Handling Sudden Growth
Platform Optimization Results
- Less crashes
#UXDXAPAC Handling Sudden Growth
Platform Optimization Results
- Improved overall response time
#UXDXAPAC Handling Sudden Growth
Platform Optimization Results
- No major incidents, despite increasing usage
- Our scaling system worked perfectly
- Established cross-platform communications
- Raised awareness on how developments in different departments affect
each other
#UXDXAPAC Handling Sudden Growth
But the war is not over...
We managed the traffic at peak
times!!! 🎉
#UXDXAPAC Handling Sudden Growth
The Future
05
#UXDXAPAC Handling Sudden Growth
- Degradations at some points of
the day
- We found that we hit our limit
of concurrent usage.
Battle is won but the war is not yet over... Incident?
#UXDXAPAC Handling Sudden Growth
More Future Improvements
- Establish workflows and processes
- Improve our incident response process
- Switch from reactive to proactive
- Changes on an organizational level from the lessons learned
#UXDXAPAC Handling Sudden Growth
Summary
06
#UXDXAPAC Handling Sudden Growth
Takeaways
- Always analyze the data to make sound decisions
- Don’t procrastinate in addressing technical issues. Technical debt will
come back to bite you when you are less prepared.
- We can prepare all we want but we will always encounter situations we
are not ready for. Learn to adapt!
- Be decisive, for the users. Make a call and own it. Taking a lot of time to
cascade discussions and decisions can be costly.
#UXDXAPAC Handling Sudden Growth
Visit https://career.quipper.com/
Be a distributor of wisdom and join us!
We’re looking for:
● Software Engineers (Web & Mobile)
● Product Designers (UI & UX)
● SRE
󾓥 🇮🇩
🇵🇭
#UXDXAPAC Handling Sudden Growth
Thank you!

More Related Content

More from UXDXConf

A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
UXDXConf
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
UXDXConf
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at Comcast
UXDXConf
 
Improving UX Research Quality with Cross-Department Collaboration
Improving UX Research Quality with Cross-Department CollaborationImproving UX Research Quality with Cross-Department Collaboration
Improving UX Research Quality with Cross-Department Collaboration
UXDXConf
 
The UX of Automation by AJ King, Senior UX Researcher, Ocado
The UX of Automation by AJ King, Senior UX Researcher, OcadoThe UX of Automation by AJ King, Senior UX Researcher, Ocado
The UX of Automation by AJ King, Senior UX Researcher, Ocado
UXDXConf
 
We're Agile. So why haven't our outcomes improved?
We're Agile. So why haven't our outcomes improved?We're Agile. So why haven't our outcomes improved?
We're Agile. So why haven't our outcomes improved?
UXDXConf
 
Breaking Silos_The Shift from a Software Delivery to a Product Development Mi...
Breaking Silos_The Shift from a Software Delivery to a Product Development Mi...Breaking Silos_The Shift from a Software Delivery to a Product Development Mi...
Breaking Silos_The Shift from a Software Delivery to a Product Development Mi...
UXDXConf
 
How Intercom built ‘Fin’, a GPT-4 powered chatbot_Fergal Reid_UXDX_EMEA_2023
How Intercom built ‘Fin’, a GPT-4 powered chatbot_Fergal Reid_UXDX_EMEA_2023How Intercom built ‘Fin’, a GPT-4 powered chatbot_Fergal Reid_UXDX_EMEA_2023
How Intercom built ‘Fin’, a GPT-4 powered chatbot_Fergal Reid_UXDX_EMEA_2023
UXDXConf
 
Leveling Up Design Maturity in a Large-Scale Organisation_ Daniel Heaslip_ U...
Leveling Up  Design Maturity in a Large-Scale Organisation_ Daniel Heaslip_ U...Leveling Up  Design Maturity in a Large-Scale Organisation_ Daniel Heaslip_ U...
Leveling Up Design Maturity in a Large-Scale Organisation_ Daniel Heaslip_ U...
UXDXConf
 
Continuous-Research_Mike Brown_UXDX_ EMEA_2023
Continuous-Research_Mike Brown_UXDX_ EMEA_2023Continuous-Research_Mike Brown_UXDX_ EMEA_2023
Continuous-Research_Mike Brown_UXDX_ EMEA_2023
UXDXConf
 
Crafting Digital Products for Connected Appliances and Other Stories_ Alexis ...
Crafting Digital Products for Connected Appliances and Other Stories_ Alexis ...Crafting Digital Products for Connected Appliances and Other Stories_ Alexis ...
Crafting Digital Products for Connected Appliances and Other Stories_ Alexis ...
UXDXConf
 
Integrating AI _King's journey of Technology Transformation_Steven Collins_ U...
Integrating AI _King's journey of Technology Transformation_Steven Collins_ U...Integrating AI _King's journey of Technology Transformation_Steven Collins_ U...
Integrating AI _King's journey of Technology Transformation_Steven Collins_ U...
UXDXConf
 
Seamless UX: Invisible Transactions_Sudev Balakrishan_UXDX_EMEA_2023
Seamless UX: Invisible Transactions_Sudev Balakrishan_UXDX_EMEA_2023Seamless UX: Invisible Transactions_Sudev Balakrishan_UXDX_EMEA_2023
Seamless UX: Invisible Transactions_Sudev Balakrishan_UXDX_EMEA_2023
UXDXConf
 
Roadmaps Unveiled_ Lessons from successes, failures and the crucial role of s...
Roadmaps Unveiled_ Lessons from successes, failures and the crucial role of s...Roadmaps Unveiled_ Lessons from successes, failures and the crucial role of s...
Roadmaps Unveiled_ Lessons from successes, failures and the crucial role of s...
UXDXConf
 
Operational Evolution_ Harnessing Design for Internal Business Transformation...
Operational Evolution_ Harnessing Design for Internal Business Transformation...Operational Evolution_ Harnessing Design for Internal Business Transformation...
Operational Evolution_ Harnessing Design for Internal Business Transformation...
UXDXConf
 
Design Delivery Principles, Accelerating Product Delivery Outcomes in a Compl...
Design Delivery Principles, Accelerating Product Delivery Outcomes in a Compl...Design Delivery Principles, Accelerating Product Delivery Outcomes in a Compl...
Design Delivery Principles, Accelerating Product Delivery Outcomes in a Compl...
UXDXConf
 
Innovation Without Adoption Is Meaningless_ Rhiannon White_ UXDX_ EMEA_2023
Innovation Without Adoption Is Meaningless_ Rhiannon White_ UXDX_ EMEA_2023Innovation Without Adoption Is Meaningless_ Rhiannon White_ UXDX_ EMEA_2023
Innovation Without Adoption Is Meaningless_ Rhiannon White_ UXDX_ EMEA_2023
UXDXConf
 
Learning Balanced Leadership with Horses_Martina Hodges-Schell_ UXDX_ EMEA_2023
Learning Balanced Leadership with Horses_Martina Hodges-Schell_ UXDX_ EMEA_2023Learning Balanced Leadership with Horses_Martina Hodges-Schell_ UXDX_ EMEA_2023
Learning Balanced Leadership with Horses_Martina Hodges-Schell_ UXDX_ EMEA_2023
UXDXConf
 
An Unexpected Approach to Design: Uncovering Hidden Parallels With Writing St...
An Unexpected Approach to Design: Uncovering Hidden Parallels With Writing St...An Unexpected Approach to Design: Uncovering Hidden Parallels With Writing St...
An Unexpected Approach to Design: Uncovering Hidden Parallels With Writing St...
UXDXConf
 
Embracing Uncertainty in Product Delivery_ Martin Reilly_ UXDX_ EMEA_2023
Embracing Uncertainty in Product Delivery_ Martin Reilly_ UXDX_ EMEA_2023Embracing Uncertainty in Product Delivery_ Martin Reilly_ UXDX_ EMEA_2023
Embracing Uncertainty in Product Delivery_ Martin Reilly_ UXDX_ EMEA_2023
UXDXConf
 

More from UXDXConf (20)

A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at Comcast
 
Improving UX Research Quality with Cross-Department Collaboration
Improving UX Research Quality with Cross-Department CollaborationImproving UX Research Quality with Cross-Department Collaboration
Improving UX Research Quality with Cross-Department Collaboration
 
The UX of Automation by AJ King, Senior UX Researcher, Ocado
The UX of Automation by AJ King, Senior UX Researcher, OcadoThe UX of Automation by AJ King, Senior UX Researcher, Ocado
The UX of Automation by AJ King, Senior UX Researcher, Ocado
 
We're Agile. So why haven't our outcomes improved?
We're Agile. So why haven't our outcomes improved?We're Agile. So why haven't our outcomes improved?
We're Agile. So why haven't our outcomes improved?
 
Breaking Silos_The Shift from a Software Delivery to a Product Development Mi...
Breaking Silos_The Shift from a Software Delivery to a Product Development Mi...Breaking Silos_The Shift from a Software Delivery to a Product Development Mi...
Breaking Silos_The Shift from a Software Delivery to a Product Development Mi...
 
How Intercom built ‘Fin’, a GPT-4 powered chatbot_Fergal Reid_UXDX_EMEA_2023
How Intercom built ‘Fin’, a GPT-4 powered chatbot_Fergal Reid_UXDX_EMEA_2023How Intercom built ‘Fin’, a GPT-4 powered chatbot_Fergal Reid_UXDX_EMEA_2023
How Intercom built ‘Fin’, a GPT-4 powered chatbot_Fergal Reid_UXDX_EMEA_2023
 
Leveling Up Design Maturity in a Large-Scale Organisation_ Daniel Heaslip_ U...
Leveling Up  Design Maturity in a Large-Scale Organisation_ Daniel Heaslip_ U...Leveling Up  Design Maturity in a Large-Scale Organisation_ Daniel Heaslip_ U...
Leveling Up Design Maturity in a Large-Scale Organisation_ Daniel Heaslip_ U...
 
Continuous-Research_Mike Brown_UXDX_ EMEA_2023
Continuous-Research_Mike Brown_UXDX_ EMEA_2023Continuous-Research_Mike Brown_UXDX_ EMEA_2023
Continuous-Research_Mike Brown_UXDX_ EMEA_2023
 
Crafting Digital Products for Connected Appliances and Other Stories_ Alexis ...
Crafting Digital Products for Connected Appliances and Other Stories_ Alexis ...Crafting Digital Products for Connected Appliances and Other Stories_ Alexis ...
Crafting Digital Products for Connected Appliances and Other Stories_ Alexis ...
 
Integrating AI _King's journey of Technology Transformation_Steven Collins_ U...
Integrating AI _King's journey of Technology Transformation_Steven Collins_ U...Integrating AI _King's journey of Technology Transformation_Steven Collins_ U...
Integrating AI _King's journey of Technology Transformation_Steven Collins_ U...
 
Seamless UX: Invisible Transactions_Sudev Balakrishan_UXDX_EMEA_2023
Seamless UX: Invisible Transactions_Sudev Balakrishan_UXDX_EMEA_2023Seamless UX: Invisible Transactions_Sudev Balakrishan_UXDX_EMEA_2023
Seamless UX: Invisible Transactions_Sudev Balakrishan_UXDX_EMEA_2023
 
Roadmaps Unveiled_ Lessons from successes, failures and the crucial role of s...
Roadmaps Unveiled_ Lessons from successes, failures and the crucial role of s...Roadmaps Unveiled_ Lessons from successes, failures and the crucial role of s...
Roadmaps Unveiled_ Lessons from successes, failures and the crucial role of s...
 
Operational Evolution_ Harnessing Design for Internal Business Transformation...
Operational Evolution_ Harnessing Design for Internal Business Transformation...Operational Evolution_ Harnessing Design for Internal Business Transformation...
Operational Evolution_ Harnessing Design for Internal Business Transformation...
 
Design Delivery Principles, Accelerating Product Delivery Outcomes in a Compl...
Design Delivery Principles, Accelerating Product Delivery Outcomes in a Compl...Design Delivery Principles, Accelerating Product Delivery Outcomes in a Compl...
Design Delivery Principles, Accelerating Product Delivery Outcomes in a Compl...
 
Innovation Without Adoption Is Meaningless_ Rhiannon White_ UXDX_ EMEA_2023
Innovation Without Adoption Is Meaningless_ Rhiannon White_ UXDX_ EMEA_2023Innovation Without Adoption Is Meaningless_ Rhiannon White_ UXDX_ EMEA_2023
Innovation Without Adoption Is Meaningless_ Rhiannon White_ UXDX_ EMEA_2023
 
Learning Balanced Leadership with Horses_Martina Hodges-Schell_ UXDX_ EMEA_2023
Learning Balanced Leadership with Horses_Martina Hodges-Schell_ UXDX_ EMEA_2023Learning Balanced Leadership with Horses_Martina Hodges-Schell_ UXDX_ EMEA_2023
Learning Balanced Leadership with Horses_Martina Hodges-Schell_ UXDX_ EMEA_2023
 
An Unexpected Approach to Design: Uncovering Hidden Parallels With Writing St...
An Unexpected Approach to Design: Uncovering Hidden Parallels With Writing St...An Unexpected Approach to Design: Uncovering Hidden Parallels With Writing St...
An Unexpected Approach to Design: Uncovering Hidden Parallels With Writing St...
 
Embracing Uncertainty in Product Delivery_ Martin Reilly_ UXDX_ EMEA_2023
Embracing Uncertainty in Product Delivery_ Martin Reilly_ UXDX_ EMEA_2023Embracing Uncertainty in Product Delivery_ Martin Reilly_ UXDX_ EMEA_2023
Embracing Uncertainty in Product Delivery_ Martin Reilly_ UXDX_ EMEA_2023
 

Recently uploaded

みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial ApplicationsLarge Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications
Rohit Gautam
 
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Zilliz
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Albert Hoitingh
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
SOFTTECHHUB
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Speck&Tech
 
20 Comprehensive Checklist of Designing and Developing a Website
20 Comprehensive Checklist of Designing and Developing a Website20 Comprehensive Checklist of Designing and Developing a Website
20 Comprehensive Checklist of Designing and Developing a Website
Pixlogix Infotech
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
Claudio Di Ciccio
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
DianaGray10
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
KatiaHIMEUR1
 
Data structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdfData structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdf
TIPNGVN2
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
KAMESHS29
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Neo4j
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
DianaGray10
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
Kumud Singh
 

Recently uploaded (20)

みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial ApplicationsLarge Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications
 
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
 
20 Comprehensive Checklist of Designing and Developing a Website
20 Comprehensive Checklist of Designing and Developing a Website20 Comprehensive Checklist of Designing and Developing a Website
20 Comprehensive Checklist of Designing and Developing a Website
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
 
Data structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdfData structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdf
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
 

Handling Sudden Growth

  • 1. #UXDXAPAC Handling Sudden Growth Handling Sudden Growth Kristine Joy Paas (@kjcpaas) Engineering Manager, Quipper Ltd. UXDX APAC
  • 2. #UXDXAPAC Handling Sudden Growth Kristine Joy Paas (Joy) - Engineering Manager at Quipper Ltd. - Ruby, Javascript (mostly React js), Unix, Kubernetes - Hobbies - Pre-COVID: travelling, swimming - COVID: reading, watching series and movies
  • 4. #UXDXAPAC Handling Sudden Growth Agenda | 01 02 03 04 05 06 Early 2020 Our Challenges Our Response Results The Future Summary
  • 5. #UXDXAPAC Handling Sudden Growth Early 2020 01
  • 6. #UXDXAPAC Handling Sudden Growth Early 2020 Restructuring: Splitting the Japanese business to the rest of the global business WHY? 1. Difference in Academic calendars 2. Ensures Future business sustainability
  • 7. #UXDXAPAC Handling Sudden Growth If you want to place an image with full screen, remove the top blue bar and the footer But then on March 2020…
  • 8. #UXDXAPAC Handling Sudden Growth Changes in user behavior Supplemental service Synchronous education
  • 9. #UXDXAPAC Handling Sudden Growth Changes in user behavior Only using Quipper contents Now creating their own content
  • 10. #UXDXAPAC Handling Sudden Growth Teacher’s usage pattern (Oct 2019 vs Oct 2020)
  • 11. #UXDXAPAC Handling Sudden Growth Student’s usage pattern (Oct 2019 vs Oct 2020)
  • 12. #UXDXAPAC Handling Sudden Growth Number of Access (Teacher vs Students) pre-covid covid
  • 13. #UXDXAPAC Handling Sudden Growth Our Challenges 02
  • 14. #UXDXAPAC Handling Sudden Growth Effects on our product - More demand for features to facilitate online education - Higher traffic due to more users with higher activity per user - More bugs discovered due to increased usage - Higher urgency in addressing and fixing bug reports
  • 15. #UXDXAPAC Handling Sudden Growth August 2020: - Start of some private schools October 2020: - Start of public schools Incidents Happened... Incidents
  • 16. #UXDXAPAC Handling Sudden Growth Incident Investigation Results Main Symptoms: - Slow access on some pages, leading to errors - Late delivery of activities to students Causes: - Slow performance on database need further optimizations - No CDN on some platforms set up CDN - Autoscaling setup didn’t work improve scaling based on peak times
  • 17. #UXDXAPAC Handling Sudden Growth Our Response 03
  • 18. #UXDXAPAC Handling Sudden Growth We were not ready for this! We didn’t expect to grow this fast! 😱
  • 19. #UXDXAPAC Handling Sudden Growth Platform Optimization Efforts WHY? - By focusing on developing features, we had less attention on scalability. - We accumulated technical debt and now it’s the time to pay back. - Developing new features is useless if users cannot use our core features. Which lead to… a freeze on developing new features
  • 20. #UXDXAPAC Handling Sudden Growth Platform Optimization Efforts I was chosen to lead the efforts and we broke down our efforts into 3 main focus: Technical Communications Operations
  • 21. #UXDXAPAC Handling Sudden Growth Platform Optimization Efforts Technical Allows us to prevent these incidents from happening again - Clean up old code - Remove unnecessary requests - Improve performance via caching, N+1 fixes, improving algorithms, etc - Setup auto-scaling and scheduled scaling - Ensure apps have enough resources so they don’t crash - Improve database for higher capacity
  • 22. #UXDXAPAC Handling Sudden Growth Platform Optimization Efforts Communications Keeps everyone on the same page - Agree on timeline with the stakeholders - Communicate with users about the efforts being done - Weekly checkup with Platform leads on the optimization efforts, to also maintain cross-platform communication - Compile common issues encountered so that support officers can respond to users faster - Create an incident management flow so that everyone is up-to-date on status of incidents that may happen
  • 23. #UXDXAPAC Handling Sudden Growth Platform Optimization Efforts Operations Allows us to recover from incidents as quickly as possible - Have developers on stand by (on a rotation basis) from 6:30 am for possible incidents.
  • 24. #UXDXAPAC Handling Sudden Growth Results 04
  • 25. #UXDXAPAC Handling Sudden Growth Platform Optimization Results - Less crashes
  • 26. #UXDXAPAC Handling Sudden Growth Platform Optimization Results - Improved overall response time
  • 27. #UXDXAPAC Handling Sudden Growth Platform Optimization Results - No major incidents, despite increasing usage - Our scaling system worked perfectly - Established cross-platform communications - Raised awareness on how developments in different departments affect each other
  • 28. #UXDXAPAC Handling Sudden Growth But the war is not over... We managed the traffic at peak times!!! 🎉
  • 29. #UXDXAPAC Handling Sudden Growth The Future 05
  • 30. #UXDXAPAC Handling Sudden Growth - Degradations at some points of the day - We found that we hit our limit of concurrent usage. Battle is won but the war is not yet over... Incident?
  • 31. #UXDXAPAC Handling Sudden Growth More Future Improvements - Establish workflows and processes - Improve our incident response process - Switch from reactive to proactive - Changes on an organizational level from the lessons learned
  • 32. #UXDXAPAC Handling Sudden Growth Summary 06
  • 33. #UXDXAPAC Handling Sudden Growth Takeaways - Always analyze the data to make sound decisions - Don’t procrastinate in addressing technical issues. Technical debt will come back to bite you when you are less prepared. - We can prepare all we want but we will always encounter situations we are not ready for. Learn to adapt! - Be decisive, for the users. Make a call and own it. Taking a lot of time to cascade discussions and decisions can be costly.
  • 34. #UXDXAPAC Handling Sudden Growth Visit https://career.quipper.com/ Be a distributor of wisdom and join us! We’re looking for: ● Software Engineers (Web & Mobile) ● Product Designers (UI & UX) ● SRE 󾓥 🇮🇩 🇵🇭
  • 35. #UXDXAPAC Handling Sudden Growth Thank you!