Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Rushin Shah, Engineering Manager, Facebook at MLconf SF 2017


Published on

Natural Language Understanding @ Facebook Scale:
At Facebook, text understanding is the key to surfacing content that’s relevant and personalized, plus enabling new experiences like social recommendations and Marketplace suggestions. In this talk, I will introduce you to DeepText, Facebook’s platform for text understanding, and discuss the various models it supports.

Bio: Rushin Shah is an engineering leader at Facebook, currently working on natural language understanding and dialog. Previously, he was at Siri at Apple for 5 years, where he built and headed the natural language understanding group, which included teams dedicated to modeling, engineering and data science. He also worked at the query understanding group at Yahoo. He has worked on a broad range of problems in the NLP area including parsing, information extraction, dialog and question answering. He holds degrees in language technologies and computer science from Carnegie Mellon and IIT Kharagpur.

Published in: Technology

Rushin Shah, Engineering Manager, Facebook at MLconf SF 2017

  1. 1. Natural Language Understanding @ Facebook Scale ENGINEERING MANAGER, FB Rushin Shah
  2. 2. Our Goal: Understand textual content with near human accuracy at Facebook scale
  3. 3. Friends ML Conferences ¬ Hockey The Daily Puck The Canadiens ran through a high-intensity practice on Wednesday ahead of Thursday's game. 1 hr • Ron Timpany, Dan Fell and 80 others Like Comment Share Montreal Hockey Insider Canadiens on verge of clinching playoff spot, Lindgren called up. 1 hr • Bill Russell, Joe Tony and 116 others Like Comment Share Benoit Dumoulin was watching Vancouver Canucks vs. Montreal Canadiens. Yes! In OT! 1 hr • Bill Russell, Joe Tony and 234 others Like Comment Share VAN 3 FINAL OT 4 MTL
  4. 4. NLU Tasks • Text classification • Word classification • Content similarity • Entity Resolution
  5. 5. Text Classification Entity: Delicious Food Jole Simmons 1 hr • Sarah Russell and 23 others 4 Comments Like Comment Share Topic: Cooking I'm trying out this new recipe for a coconut curry tonight. It looks DELICIOUS!!! Entity: Kaepernick Jole Simmons 1 hr • Sarah Russell and 23 others 4 Comments Like Comment Share Topic: Sports or Cooking ? Last night's game was absolutely incredible. Once Curry gets cooking, there's no stopping that guy!
  6. 6. Word Classification
  7. 7. Content similarity SIMILARITY 0.75
  8. 8. MUCKBUCKET SUNSHINE MUSIC BAND Parsing out Entities "DID YOU KNOW THAT MUCK BUCKET SUNSHINE IS PERFORMING LIVE AT THE BOOM BOOM ROOM?" TOPIC LIVE EVENT The Boom Boom Room 1 hr • Tonight before you Dance Dance Dance, join us for a special evening. Boom Boom Room Presents: Muckbucket Sunshine! Doors 8pm. Sarah Russell and 23 others 4 Comments Like Comment Share
  9. 9. • Deep Learning
 For NLU • Continuous
 Representation • Can Solve Hard 
 NLP Problems Natural Language Processing (Almost) From Scratch
  10. 10. DeepText
  11. 11. DeepText Features Multiple Tasks Multiple Languages Multiple Architectures
  12. 12. Model Structure Learning Algorithm Data Loader Tokenizer Feature Extraction Model Training DeploymentFeat Prep DeepText Platform architecture
  13. 13. DeepText Tasks Sequence LabelingClassification
  14. 14. DeepText Document classification
  15. 15. DeepText Document classification - LSTM Hidden MLP Messi todayscored HiddenHidden EmbeddingEmbeddingEmbedding
  16. 16. DeepText Document classification - CNN Messi todayscored EmbeddingEmbeddingEmbedding MLP CNN
  17. 17. DeepText Classification: FastText Messi todayscored Unigram Bigram Unigram Bigram Unigram Output Hidden
  18. 18. DeepText Word classification
  19. 19. MLP CNN/LSTM Messi Embedding today Embedding Classification(Messi) MLP Classification(today) scored a hat-trick DeepText Classification: Word classification
  20. 20. DeepText Content Similarity Messi todayscored EmbeddingEmbeddingEmbedding CNN / LSTM Messi todayscored EmbeddingEmbeddingEmbedding CNN / LSTM MLP Ranking loss
  21. 21. FC Barcelona DeepText Entity recognition and linking Real Madrid C.F.
  22. 22. Document Candidate Selection Mention Detection Disambiguation Entity Annotations Entity recognition and linking Architecture
  23. 23. Exploring Use Cases In Facebook
  24. 24. For Sale Post for SaleNot Selling Create a Sales post to sell your items faster. Only post as a discussion if you’re not selling something. Are You Selling Something?
  25. 25. Social Recs
  26. 26. Scale ML Experts Reuse
 Models Optimize
  27. 27. CLUE Label Efficiency Active Learning Democratize Single entry point for NLU Flexibility and Scale Deep Text Scale ML Experts Reuse
 Models Optimize
  28. 28. Active Learning Self Service With CLUE Train Classifier Label Data Collect Data (Search) Review Threshold Precision POOLING ... W1 W2 W3 W4 WN CONVOLUTIO I want some subway or burger king I want some subway or burger king Slots [ { "name": "cu:restaurant" "value": "subway" "start": 12 "end": 19 "contextldx": 0 } { "name": "cu:restaurant" "value": "burger king" "start": 22 "end": 32 "contextldx": 0
  29. 29. MODELS 200+