Advertisement
Check these out next
![[AI] ML Operationalization with Microsoft Azure](https://cdn.slidesharecdn.com/ss_thumbnails/wds-mlops-trainer-kyle-akepanidtaworn-v03-190924094657-thumbnail.jpg?width=600&height=600&fit=bounds)
An Architecture for Agile Machine Learning in Real-Time Applications
Aug. 11, 2015•0 likes•3,770 views
7 likes
Be the first to like this
Show More
views
Total views
0
On Slideshare
0
From embeds
0
Number of embeds
0
Download to read offline
Report
Technology
Presented at KDD, August 11, 2015. Abstract of the paper: Machine learning techniques have proved effective in recommender systems and other applications, yet teams working to deploy them lack many of the advantages that those in more established software disciplines today take for granted. The well-known Agile methodology advances projects in a chain of rapid development cycles, with subsequent steps often informed by production experiments. Support for such workflow in machine learning applications remains primitive. The platform developed at if(we) embodies a specific machine learning approach and a rigorous data architecture constraint, so allowing teams to work in rapid iterative cycles. We require models to consume data from a time-ordered event history, and we focus on facilitating creative feature engineering. We make it practical for data scientists to use the same model code in development and in production deployment, and make it practical for them to collaborate on complex models. We deliver real-time recommendations at scale, returning top results from among 10,000,000 candidates with sub-second response times and incorporating new updates in just a few seconds. Using the approach and architecture described here, our team can routinely go from ideas for new models to production-validated results within two weeks.
Johann Schleier-SmithFollow
Board Member at if(we)Advertisement
Advertisement
Advertisement
Recommended
ReStream: Accelerating Backtesting and Stream Replay with Serial-Equivalent P...Johann Schleier-Smith
Real time machine learningVinoth Kannan
Agile Machine Learning for Real-time Recommender SystemsJohann Schleier-Smith
Architecting for Data ScienceJohann Schleier-Smith
Blind spots in big data erez koren @ forterIdo Shilon
Machine Learning with Apache SparkIBM Cloud Data Services
An Architecture for Agile Machine Learning in Real-Time Applications
- An Architecture for Agile Machine Learning in Real-Time Applications johann@ifwe.co@jssmith github.com/ifwe Johann Schleier-Smith if(we) Inc. August 11, 2015 KDD, Sydney Australia
- • Profitable startup actively pursuing big opportunities in social apps • Millions of users on existing products • Thousands of social contacts per second
- Overview • Agile machine learning can be difficult— but brings big benefits • Key challenges in deployment and feature engineering • Solution in single path to data
- production development serve personalized recommendations data collection model updates
- production development serve personalized recommendations data collection model updates study & understand train & backtest design new models & features
- production development serve personalized recommendations data collection study & understand design new models & features model updates train & backtest
- model updates train & backtest write spec e-mail model to engineers request engineering why did we want this? QA bug fixes meetingswait export to Excel check parameters Java development new database schema
- model updates train & backtest • Shared path to data • Shared feature definition code
- production development serve personalized recommendations data collection model updates study & understand train & backtest design new models & features
- • >10 million candidates • >1000 updates/sec • Must be responsive to current activity • Users expect instant query results Recommendation Engine for Dating Product
- Model
- Model
- Model • Decompose likelihood of match between vote outcomes and vote occurrence • Logistic regression • Real-time personalization through feature vector evolution • Model parameters trained offline by data scientists • Consider 1000s of features, select 50-100
- Application APIs & Business Logic RDBMS
- Application APIs & Business Logic RDBMS Data Warehouse / Hadoop
- Application APIs & Business Logic RDBMS Data Warehouse / Hadoop Streaming Logs
- Application APIs & Business Logic RDBMS Data Warehouse / Hadoop Streaming Logs
- Application APIs & Business Logic RDBMS production development Exploratory Analysis Training & Backtesting Data Warehouse / Hadoop Streaming Logs
- Application APIs & Business Logic RDBMS production development Exploratory Analysis Training & Backtesting Batch Predictions Data Warehouse / Hadoop Streaming Logs
- Application APIs & Business Logic RDBMS production development Exploratory Analysis Training & Backtesting Batch Predictions Predictive Services / Ranking Data Warehouse / Hadoop Streaming Logs
- Application APIs & Business Logic RDBMS production development Exploratory Analysis Training & Backtesting Batch Predictions Predictive Services / Ranking Data Warehouse / Hadoop Streaming Logs
- Events Time
- Aggregation first( ) last( ) count( ) sum( ) max( ) count( ) avg( ) min( ) Events Time
- Machine learning inputAggregation first( ) last( ) count( ) sum( ) max( ) count( ) avg( ) min( ) Events Time
- Event History API trait EventHistory { def publishEvent(e: Event) def getEvents( startTime: Date, endTime: Date, eventFilter: EventFilter, eventHandler: EventHandler ) }
- Event History API trait EventHistory { def publishEvent(e: Event) def getEvents( startTime: Date, endTime: Date, eventFilter: EventFilter, eventHandler: EventHandler ) }
- Event History API trait EventHistory { def publishEvent(e: Event) def getEvents( startTime: Date, endTime: Date, eventFilter: EventFilter, eventHandler: EventHandler ) }
- Event History API trait EventHistory { def publishEvent(e: Event) def getEvents( startTime: Date, endTime: Date, eventFilter: EventFilter, eventHandler: EventHandler ) } +∞ for real-time streaming
- Events Alice updates profile Bob opens app Bob sees Alice in recommendations Bob swipes yes on Alice Alice receives push notification Alice sees Bob in recommendations Alice sends message to Bob Time
- Online feature stateEvents Alice updates profile Bob opens app Bob sees Alice in recommendations Bob swipes yes on Alice Alice receives push notification Alice sees Bob in recommendations Alice sends message to Bob Time
- Machine learning inputOnline feature stateEvents Alice updates profile Bob opens app Bob sees Alice in recommendations Bob swipes yes on Alice Alice receives push notification Alice sees Bob in recommendations Alice sends message to Bob Time
- RDBMS Application APIs & Business Logic Event History Repository Ranking Real-Time State Updates
- RDBMS Application APIs & Business Logic Event History Repository Ranking Real-Time State Updates
- RDBMS Application APIs & Business Logic Event History Repository Ranking Real-Time State Updates
- RDBMS Application APIs & Business Logic Event History Repository Ranking Real-Time State Updates State Updates Exploratory Analysis Training & Backtesting production development
- RDBMS Application APIs & Business Logic Event History Repository Ranking Real-Time State Updates State Updates Exploratory Analysis Training & Backtesting production development
- RDBMS Application APIs & Business Logic Event History Repository Ranking Real-Time State Updates State Updates Exploratory Analysis Training & Backtesting production development
- RDBMS Application APIs & Business Logic Event History Repository Ranking Real-Time State Updates State Updates Exploratory Analysis Training & Backtesting production development
- RDBMS Application APIs & Business Logic Event History Repository Ranking Real-Time State Updates State Updates Exploratory Analysis Training & Backtesting production development
- RDBMS Application APIs & Business Logic Event History Repository Ranking Real-Time State Updates State Updates Exploratory Analysis Training & Backtesting production development
- Monitoring RDBMS Application APIs & Business Logic Event History Repository Ranking Real-Time State Updates State Updates Exploratory Analysis Training & Backtesting production development
- • Single path to data for real-time streaming and history • Shared feature engineering code for development and production • Team shares access to code and data • Fine-grained alignment of feature state and prediction outcomes • Temporally accurate modeling ensured (no looking ahead) Event History API
- 15 new models released and tested within 6 months >30% cumulative improvement in usage shown in A/B testing 0 500,000 1,000,000 1,500,000 2,000,000 Apr 2013 Jul 2013 Oct 2013 Jan 2014 Apr 2014 DailyUniqueUsers Matchers Voters New model released A/Btestupdated
- • Open source implementation derived from if(we)’s proprietary platform • Provides Scala DSL for building online features from event history • Examples include dating recommendations and product search with learning to rank • Not yet ready for scale or production • Seeking collaborators
- Production Serving Data Science Ranking R MatlabPython Feature Engineering Event History API Kafka Streaming data Storm Historical data S3 NFSHDFS Antelope Open Source Vision
- Agile Machine Learning with Event History • Solving deployment yields quick product cycles • All data saved and retrieved as time-ordered events • Single path to data for both historical and real-time access • Same feature engineering code used in development and production • Agile success • Team shares access to code and data • Production product iterations measured in days rather than months github.com/ifwe/antelope@jssmith
Advertisement