This document discusses the technical choices and rationale for building high-traffic, dynamic websites at Ask. It describes using a configuration-driven approach with reusable components and libraries to rapidly launch new domains and sites within a 1 sprint turnaround. The key aspects are leveraging modular view components, the Archie Node.js library, and JSON configuration files to declaratively configure domains, pages, and site features in a scalable way.

1. Building High-Traffic
Dynamic Websites at Ask
Tech choices and rationale

2. Background
● 35 Base domains (e.g. Ask.com, Consumersearch.com, Shop411.com,
Informationvine.com, etc.)
● 10 base domains have 5 to 14 localized versions (e.g. de.ask.com).
● 13 front-end apps serve all of these sites
● 800 QPS peak on busiest apps, > 100MM visitors/month
● New domains getting launched all the time—turnaround ~1 sprint, depending
on site
● Site categories: search, content/articles, shopping
○ Some sites have social slideshow/listicle experiences in addition to the above

3. Examples

4. Leveraging Modules and Libraries
● Page template
● Data providers
● View components (HTML, JS, CSS, Assets)
● Archie—our node-based library

5. View components
● Reusable UI building block
● Custom skins
● Independent of data source
● Configurable

6. Related Search

7. Archie
● Node + express.js
● Provides building blocks leveraged by all frontends
● Configuration-driven development
○ Multi-domain support
○ Page configuration
○ UI components / Data providers

8. Cranking out new domains
● Set-up time is ~ 4 days
○ No time for writing custom javascript, CSS for each new domain
○ Can’t “start from scratch” on everything
● Answer: Configuration-driven development
○ Make (almost) everything declarative
○ Hostname, pages, page partials, data sources—all config-driven
○ Templated pages and reusable components are key

9. Site Config
● Domains
● Pages / Routes
● Site level config

10. Page Config
● Routes
● Page skin
● Data providers
● Partials

11. View component
Data Source
Configuration
No CSS!

12. Building and Deploying a Site
● Building the Application
● Running the integration tests
● Deploying to Staging
● Deploying to Production

13. CI/CD Pipeline

14. Building Websites @ Scale is Hard!
● Relevant Results
● Fast Landing Page
● Consistent UX to User & “Good” Bots
● Modern LaF
● Mobile-friendly/first
● Optimal Ad-Content Ratio
● X-Browser Support
● I18N

15. Node/Express
● Easy to get started
● Middleware simple to understand
● Fully embrace the async nature of the Web
● npm makes code sharing easy
● Terminal-friendly
● FE Devs can contribute to BE as well
● Makes hiring easier

16. What’s Missing in Node/Express
● Ability to override overall experience dynamically
● Separation of Concerns
○ Rendering engine
○ Logic-less templates
○ Reusable UI components
○ Asset management
● Integration with our BE services
● Page Introspection
● Simpler API to deal with microservices

17. Enter Archie
● Adds sophisticated ways to override the page
○ URL, Site/Page config, HTTP-based, Inline, Command-line…
● Dust.js as the rendering/templating engine (Streaming support)
○ Not building SPAs with complex state, so didn’t need React/Angular
● Partials abstract a UI component (JS/HTML/Less/Assets)
● Providers abstract data source while providing a synchronous interface
● OOTB integration with our in-house User Analytics, Feature Flag, A/B
Testing, Geo-IP, Query Classification and Search services
● Includes IT F/W
● Common HTTP Client
● Embeddable - Available as an npm module

18. Typical Page configuration (JSON)