Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Applicable bert

30 views

Published on

PPT presented DPS(https://digitalproductschool.io/) week 8, at engineering weekly.
Explaining how BERT model can be used in industry.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Applicable bert

  1. 1. Applicable BERT Jiwon Kim Engineering Weekly 8
  2. 2. How can we apply SOTA NLP model to the specific domain?
  3. 3. Can BERT be used in a sentiment classification?
  4. 4. 1. conda create -n bert python pytorch pandas tqdm conda install -c anaconda scikit-learn (Just for safeness..) 1. pip install pytorch-pretrained-bert (Thank you hugging face 🤗) 1. Prepare your data 2. Fine-tune the model with your data Yes, of course Just your target data, e.g., Yelp polarity data
  5. 5. Can BERT be used in a Question-Answering?
  6. 6. Definitely feat, pragnakalp 1. conda create -n bert python pytorch pandas tqdm conda install -c anaconda scikit-learn (Just for safeness..) 1. pip install pytorch-pretrained-bert (Thank you hugging face 🤗) 1. Prepare your data 2. Fine-tune the model with your data
  7. 7. Can BERT be used in a chatbot?
  8. 8. 1. conda create -n bert python pytorch pandas tqdm conda install -c anaconda scikit-learn (Just for safeness..) 1. pip install pytorch-pretrained-bert (Thank you hugging face 🤗) 1. Prepare your data 2. Fine-tune the model with your data Again….
  9. 9. Definitely feat, pragnakalp
  10. 10. Can BERT be used in sentence auto-completion?
  11. 11. Gmail’s Suggested Replies, Smart Compose & Google Search Autocomplete What is autocompletion?
  12. 12. Sure you can use BERT! 1. conda create -n bert python pytorch pandas tqdm conda install -c anaconda scikit-learn (Just for safeness..) 1. pip install pytorch-pretrained-bert (Thank you hugging face 🤗) 1. Prepare your data 2. Fine-tune the model with your data feat, Hugging face
  13. 13. Why suddenly GPT-2???? 🤗🤗🤗🤗 🤗
  14. 14. € 40K
  15. 15. Review BERT can be used as ● Sentiment(emotion) classification ● Question-Answering ● Chatbot ● Sentence auto-completion
  16. 16. SO………. What is a BERT?
  17. 17. It would be way more long story…... Concepts that you should know... ● Transformer, ELMo, GPT ● Transfer Learning from Supervised Data ● Embedding model(position embedding, segment embedding, token embedding) ● Pre-trained model ● Attention mechanism(e.g., Scaled Dot-Product Attention Multi-Head Attention) ● Semi-supervised sequence learning ● Position-wise Feed-Forward Networks ● And so on…... Paper link 1 Paper link 2
  18. 18. Henry AI Labs
  19. 19. ● BERT can be used at least…. ○ Sentiment(emotion) classification ○ Question-Answering ○ Chatbot ○ Sentence auto-completion ● Feel free to ask me if you can’t find out the pre-made code. ● “Don’t try to understand everything what you use… just run the code, and don’t go deep on theory.” - Jeremy Howard ● Still you want to understand theoretical background, let’s study. Conclusion

×