Karl Patrick, a senior backend software engineer at Twitch, discusses scaling challenges for Twitch extensions during his talk at TwitchCon 2018. He emphasizes the importance of handling traffic spikes effectively through strategies such as using pubsub for event updates, randomizing wait times, and managing write traffic to prevent spam. Key tactics include being lazy in requests, spreading out traffic, and using fake early responses to enhance user experience during high-demand situations.

1. Extensions Developer Services Team
@MovingViolation
Karl Patrick
Senior Backend
Software Engineer

2. Extension Design Patterns for Twitch Scale

3. TwitchCon 2018 San Jose, California
@XPCagey
15 years of experience prior to Twitch
Single person hobby projects to AAA titles
Build/Tools/Engine/Server/Gameplay
Karl Patrick
Game Developer

4. TwitchCon 2018 San Jose, California
Extensions
Usually Fail to Scale
-Popular technologies don’t work for
high-volume channels
-False starts with high-profile
Extensions inspired this talk
-High concurrent users is only part of
the story
TWITCH SCALE

5. TwitchCon 2018 San Jose, California
- When a streamer activates
Extensions during a stream
- When a channel is “raided” by
another audience
- When viewers refresh due to a
technical issue
Traffic Spikes
10,000x traffic in
One Second
WHAT’S THE BIG DEAL?
~20 viewers / 10 seconds
40,000 viewers / 10 seconds

6. TwitchCon 2018 San Jose, California
How can Extensions
Scale for Traffic
Bursts?
TwitchCon 2018 San Jose, California
Server

7. TwitchCon 2018 San Jose, CaliforniaTwitchCon 2018 San Jose, California
Server Twitch
PubSub
Call Twitch PubSub from
your service to push events
instead of polling for
updates.
If you need to whisper
secrets, shout the list of
people who need to ask for
them instead of having
every user ask.
Use a
Megaphone
SCALING READS

8. TwitchCon 2018 San Jose, California
Be Lazy:
Wait Before You
Ask for Data
- Use a splash screen when you first
load your Extension
- Schedule your requests after a pause
- Cancel the request if it is answered
while you’re waiting
SCALING READS
Document
loads
Schedule
request
PubSub sent
before timer
expires?
Cancel
request
Send
request
Yes
No

9. TwitchCon 2018 San Jose, California
Roll the Dice:
Randomize Your
Wait Time
SCALING READS
- Make each client pick a number
instead of using one delay
- “Jitter” will spread out traffic if
clients load at the same time
Spread out work with a
random delay
All requests at one time

10. TwitchCon 2018 San Jose, California
- Tell clients how long they can wait to
respond
- Roll the dice on each client and cancel
if the response comes while waiting
- On each client request, add more wait
time to the next response
- Larger audiences automatically get
longer waits and lower traffic per
viewer
Listen to Yourself:
Send Wait Times
From the Server
SCALING READS
Server
Send time
remaining
Use time to decide
when to send
Add time on
every request
PubSub

11. TwitchCon 2018 San Jose, California
Green
Purple
Orange
Blue
What’s your favorite color?
What About
Interactive
Extensions?

12. TwitchCon 2018 San Jose, California
- Authorize requests and use cool
downs to prevent spam
- Popular Extensions have seen
attempts to spam within 20 minutes
of release
- Chrome developer tools make it
simple to repeat service requests
Stop Cheaters
SCALING WRITES
!

13. TwitchCon 2018 San Jose, California
- Be lazy and roll the dice to spread
write traffic; listen to yourself to
make sure you answer in time
- Predict and apply fake responses
while you’re waiting
- Once you have a real response,
replace the fake one
Start Cheating:
Fake Early Responses
SCALING WRITES
User
Clicks
Schedule & Send
Request
Service Calculates
Result
PubSub Message
Arrives
Show
Result

14. TwitchCon 2018 San Jose, California
Take Command:
Batch Frequent
Requests
SCALING WRITES

15. How can Extensions Scale for Traffic Bursts?
- Use a Megaphone
- Listen to Yourself
- Be Lazy
- Roll the Dice
- Stop Cheaters
- Start Cheating
- Take Command

16. Questions?