Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Realtime Webpack Pushing on-demand bundling to the limits March 2017 @oliverwoodings
We help our clients deliver highly persuasive personalization at scale, to their best customers, at the right time.
What does that mean? Collect data Surface insights (machine learning etc) Define segments, design personalisations Serve pe...
How do we deliver web assets Build Pipeline Qubit Platform CDN Browser End UserQubit Customer Prod Bundle Personalisation ...
How do we deliver web assets Build Pipeline Qubit Platform CDN Browser End UserQubit Customer Prod Bundle Personalisation ...
Filesystem - index.js - app/ - actions/ - components/ - views/ - reducers/ - App.js - node_modules/ Yarn Webpack Dev Serve...
Why doesn't this work for us?
Each client is different! Why doesn't this work for us?
Filesystem - index.js - app/ - actions/ - components/ - views/ - reducers/ - App.js - node_modules/ Yarn Webpack Dev Serve...
1. Get rid of the fs Webpack Dev Server Browser MemoryFS - entry.js - .cache - config.js - experiences-2499/ - activation....
2. Replace webpack dev server Redis package cache npm registry Dynamic compiler service Browser Redis bundle cache My SQL ...
Redis package cache 3. Scale Worker PoolMySQL Dynamic compiler service MemoryFS Loadbalancer MemoryFS MemoryFS Dynamic com...
Redis package cache Worker PoolMySQL Dynamic compiler service MemoryFS Loadbalancer MemoryFS MemoryFS Dynamic compiler ser...
The result 11k builds/day on average 2.6 seconds average build duration 4 preview builds/second with 6 preview workers
Performance under load
Performance under load • 1000 webpack builds
Performance under load • 1000 webpack builds • 100 workers
Performance under load • 1000 webpack builds • 100 workers • Completed in 2 minutes
Performance under load • 1000 webpack builds • 100 workers • Completed in 2 minutes • Peak throughput of 9 builds per seco...
So what?
So what? • Make it snappy - HappyPack
So what? • Make it snappy - HappyPack • Keep it simple - Preact and Buble
So what? • Make it snappy - HappyPack • Keep it simple - Preact and Buble • Tidy up after yourself - clean webpack config
So what? • Make it snappy - HappyPack • Keep it simple - Preact and Buble • Tidy up after yourself - clean webpack config •...
So what? • Make it snappy - HappyPack • Keep it simple - Preact and Buble • Tidy up after yourself - clean webpack config •...
@oliverwoodings oli@qubit.com Thanks!
Upcoming SlideShare
Loading in …5
×

Realtime Webpack - Pushing on-demand bundling to the limits by Oliver Woodings

617 views

Published on

Bundling JavaScript nowadays is trivial. But what if you need to do it 10,000 times a day to thousands of users, with sourcemaps, served directly to the browser in under 2 seconds? Oliver will show us how they did this at Qubit

Published in: Technology
no profile picture user

  • Be the first to comment

Realtime Webpack - Pushing on-demand bundling to the limits by Oliver Woodings

  1. 1. Realtime Webpack Pushing on-demand bundling to the limits March 2017 @oliverwoodings
  2. 2. We help our clients deliver highly persuasive personalization at scale, to their best customers, at the right time.
  3. 3. What does that mean? Collect data Surface insights (machine learning etc) Define segments, design personalisations Serve personalisations /segments back to end users
  4. 4. How do we deliver web assets Build Pipeline Qubit Platform CDN Browser End UserQubit Customer Prod Bundle Personalisation & Segments Preview Bundle Event stream
  5. 5. How do we deliver web assets Build Pipeline Qubit Platform CDN Browser End UserQubit Customer Prod Bundle Personalisation & Segments Preview Bundle Event stream
  6. 6. Filesystem - index.js - app/ - actions/ - components/ - views/ - reducers/ - App.js - node_modules/ Yarn Webpack Dev Server Browser A typical webpack setup
  7. 7. Why doesn't this work for us?
  8. 8. Each client is different! Why doesn't this work for us?
  9. 9. Filesystem - index.js - app/ - actions/ - components/ - views/ - reducers/ - App.js - node_modules/ Yarn Webpack Dev Server Browser Let’s fix it up…
  10. 10. 1. Get rid of the fs Webpack Dev Server Browser MemoryFS - entry.js - .cache - config.js - experiences-2499/ - activation.js - variation-1.js - variation-1.css MySQL Redis package cache npm registry
  11. 11. 2. Replace webpack dev server Redis package cache npm registry Dynamic compiler service Browser Redis bundle cache My SQL MemoryFS - entry.js - .cache - config.js - experiences-2499/ - activation.js - variation-1.js - variation-1.css
  12. 12. Redis package cache 3. Scale Worker PoolMySQL Dynamic compiler service MemoryFS Loadbalancer MemoryFS MemoryFS Dynamic compiler service Dynamic compiler service Redis bundle cache npm registry Browser
  13. 13. Redis package cache Worker PoolMySQL Dynamic compiler service MemoryFS Loadbalancer MemoryFS MemoryFS Dynamic compiler service Dynamic compiler service Redis bundle cache npm registry Browser 4. Queue and autoscale Autoscaler Job Queue CDN
  14. 14. The result 11k builds/day on average 2.6 seconds average build duration 4 preview builds/second with 6 preview workers
  15. 15. Performance under load
  16. 16. Performance under load • 1000 webpack builds
  17. 17. Performance under load • 1000 webpack builds • 100 workers
  18. 18. Performance under load • 1000 webpack builds • 100 workers • Completed in 2 minutes
  19. 19. Performance under load • 1000 webpack builds • 100 workers • Completed in 2 minutes • Peak throughput of 9 builds per second
  20. 20. So what?
  21. 21. So what? • Make it snappy - HappyPack
  22. 22. So what? • Make it snappy - HappyPack • Keep it simple - Preact and Buble
  23. 23. So what? • Make it snappy - HappyPack • Keep it simple - Preact and Buble • Tidy up after yourself - clean webpack config
  24. 24. So what? • Make it snappy - HappyPack • Keep it simple - Preact and Buble • Tidy up after yourself - clean webpack config • Be realistic - tree shaking
  25. 25. So what? • Make it snappy - HappyPack • Keep it simple - Preact and Buble • Tidy up after yourself - clean webpack config • Be realistic - tree shaking • Try it flat - Rollup
  26. 26. @oliverwoodings oli@qubit.com Thanks!

×