Condé Nast uses machine learning across various applications like recommendations, personalization, advertising, and content understanding. They developed an MLOps platform called Spire to manage the machine learning lifecycle including datasets, modeling, inference, and tracking. Spire includes components like Aleph for feature engineering, Kalos for model training and deployment, and utilizes technologies like Astronomer, Databricks, and APIs to execute workflows and integrate with other systems. Condé Nast aims to improve their platform over time by enhancing models and features as well as consolidating libraries.

1. Machine Learning Infrastructure at Condé Nast
James Evers & Max Cantor

5. 110M+
Monthly Unique Visitors
800M+
Monthly Page Views
100K+
Monthly New Or Revised
Pieces of Content
Digital Scale at Condé Nast

6. Landscape of MLOps
- Datasets

7. Landscape of MLOps
- Datasets
- Modeling

8. Landscape of MLOps
- Datasets
- Modeling
- Inference

9. Landscape of MLOpsLandscape of MLOps
- Datasets
- Modeling
- Inference
- Tracking

10. MLOps Across the Industry
- Michelangelo & Ludwig (Uber)
- FBLearner (Facebook)
- Noether (Spotify)
- Bighead (Airbnb)
- Metaflow (Netflix)

11. MLOps at
Condé Nast

12. Motivations
- Recommendations &
Personalization
- Advertisement & User
Segmentation
- Content Understanding
- Audience Forecasting
- Search Optimization

13. Challenges
● Facts of the media industry
○ low on-site times (engagement and bounce rate)
○ one and done’s (cold start)
○ Dependence on social media
● Business vs Engineering interests
● Legacy infrastructure requirements
● Staffing & overall technical resources

14. Features
Labeled Data
Dataprep
Spire Workflow Stages
Model
Train
Scores
Predict
Thresholds

15. Aleph
- User-level behavioral feature space
- Content-derived feature generation
- Vetted feature pipelines for models
Features
Labeled Data

16. Aleph

17. - Model interface standardization
- Model serialization
- Model versioning & tracking
- Hyperparameter tuning
Model
Kalos

18. Kalos

19. ML as a
Software
Product

20. Execution Environments
- Astronomer / Airflow
- Databricks
- Local Development

21. Spire Command-Line Interface
- Development
- User Queries
- Models
- Execution

22. API Wrappers
- Bridges Airflow Operators and Spark Jobs
- Close integration with Databricks APIs
- Development iteration, speed, and stability in processes

23. - Spire versioning
- Spire releases
- GitHub/repository
- CI/CD
Release
Management

24. Future
- Model reduction
- Model architecture enhancement
- Extend abstraction flexibility to feature sets
- In-house library consolidation

25. Thank you!