Scaling automated quality text generation for enterprise sites

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

@hamletbatista
SCALING AUTOMATED
QUALITY TEXT GENERATION
FOR ENTERPRISE SITES
HAMLET BATISTA

@hamletbatista
LET’S CRAWL A FEW PAGES FROM THIS SITE

@hamletbatista
WE ARE MISSING KEY META TAGS!

@hamletbatista
AND SOME PAGES ARE LACKING CONTENT

@hamletbatista
LET’S FIX THAT WITH AUTOMATION!
We want to address 4 scenarios common in enterprise websites.
For large eco...

@hamletbatista
AGENDA
We are going to explore different text generation strategies and
recommend the best one for each pro...

@hamletbatista
AGENDA
We are going to build two models from scratch:
1. We will build an image captioning and visual quest...

@hamletbatista
HOW TO FIND RELEVANT RESEARCH AND CODE
Papers
with Code

@hamletbatista
HOW TO FIND RELEVANT RESEARCH AND CODE
Papers
with Code
(SOTA)

@hamletbatista
TEXT GENERATION FOR ECOMMERCE SITES

@hamletbatista
IMAGE CAPTIONING AND VISUAL QUESTION ANSWERING
Bottom-
Up and
Top-
Down
Attention
for Image
Captionin
g and...

@hamletbatista
THE PYTHIA MODULAR FRAMEWORK
Pythia
Github

@hamletbatista
THE PYTHIA MODULAR FRAMEWORK
Pythia

@hamletbatista
IMAGE CAPTIONING AND VISUAL ANSWERING RESULTS
Pythia

@hamletbatista
LET’S BUILD A CAPTIONING MODEL!
Pythia
captionin
g demo

@hamletbatista
RUN ALL CELLS
Pythia
captionin
g demo

@hamletbatista
USE THE LAST SECTION TO TEST IMAGES
Pythia
captionin
g demo

@hamletbatista
USE THE LAST SECTION TO CAPTION IMAGES
“a group
of people
in a boat
on the
water”

@hamletbatista
THERE IS MORE TEXT HIDDEN IN THE OTHER IMAGES!

@hamletbatista
TRY THE NEW TITLES FROM CAPTIONS IN CLOUDFLARE
FIRST
Cloudflare
App

@hamletbatista
YOU CAN ALSO ASK QUESTIONS ABOUT IMAGES
“how
many
people?”
“4 with
62.9
confidenc
e”

@hamletbatista
YOU CAN ALSO ASK QUESTIONS ABOUT IMAGES
“what are
these
people
riding?”
“boat with
99.96
confidenc
e”

@hamletbatista
CAPTIONING AND VISUAL QUESTION ANSWERING PAPER
Bottom-
Up and
Top-
Down
Attention
for Image
Captionin
g and...

@hamletbatista
CAPTIONING AND VISUAL QUESTION ANSWERING
RESULTS
Bottom-
Up and
Top-
Down
Attention
for Image
Captionin
g a...

@hamletbatista
WHERE DID I LEARN ABOUT THIS?
Advanced
Machine
Learning
Specializa
tion

@hamletbatista
WHERE DID I LEARN ABOUT THIS?
Introducti
on to
Deep
Learning

@hamletbatista
TEXT GENERATION FOR WEB PUBLISHERS

@hamletbatista
AI TEXT GENERATOR: TALKTOTRANSFORMER.COM
Talk to
transform
er

@hamletbatista
TEXT SUMMARIZATION
Papers
with Code
(Text
Summariz
ation)

@hamletbatista
TEXT SUMMARIZATION PAPER (EXTRACTIVE)
Papers
with Code
(Extractiv
e Text
Summariz
ation)

@hamletbatista
TEXT SUMMARIZATION RESULTS (EXTRACTIVE)
Fine-tune
BERT for
Extractive
Summariz
ation

@hamletbatista
TEXT SUMMARIZATION PAPER (ABSTRACTIVE)
Papers
with Code
(Abstracti
ve Text
Summariz
ation)

@hamletbatista
TEXT SUMMARIZATION RESULTS (ABSTRACTIVE)
MASS:
Masked
Sequence
to
Sequence
Pre-
training
for
Language
Gener...

@hamletbatista
LET’S BUILD AN EXTRACTIVE TEXT SUMMARIZATION
MODEL!
https://github.
com/nlpyang/B
ertSum

@hamletbatista
LET’S BUILD AN EXTRACTIVE TEXT SUMMARIZATION
MODEL!
https://github.
com/nlpyang/B
ertSum

@hamletbatista
BERTSUM MODEL OVERVIEW
"Meanwhile,
although BERT has
segmentation
embeddings for
indicating different
sente...

@hamletbatista
BERTSUM DOWNLOAD AND SETUP
BERTSUM Colab
notebook

@hamletbatista
BERTSUM TRAINING BERT+TRANSFORMER MODEL
BERTSUM Colab
notebook

@hamletbatista
BERTSUM TRAINING BERT+TRANSFORMER MODEL
BERTSUM Colab
notebook
#first run
#Change -visible_gpus 0,1,2 -gpu_...

@hamletbatista
BERTSUM TRAINING BERT+TRANSFORMER MODEL
We are simply
following the
instructions in
the Github
repository

@hamletbatista
BERTSUM TRAINING BERT+TRANSFORMER MODEL
1. Training takes two days on Colab (with interruptions)
2. Saving ...

@hamletbatista
BERTSUM TRAINING BERT+TRANSFORMER MODEL
BERTSUM Colab
notebook

@hamletbatista
BERTSUM RESUMING TRAINING
BERTSUM Colab
notebook
#resume run
#Change -visible_gpus 0,1,2 -gpu_ranks 0,1,2 -...

@hamletbatista
SIMPLER: JUST GET A TRAINED MODEL FROM THE
INVENTOR!
BERTSUM Colab
notebook

@hamletbatista
BERTSUM TESTING BERT+TRANSFORMER MODEL
BERTSUM Colab
notebook

@hamletbatista
BERTSUM TESTING RESULTS
BERTSUM Colab
notebook
Gold Summary: 'click on the brilliant
interactive graphic be...

@hamletbatista
TEXT GENERATION FOR WEB PUBLISHERS

@hamletbatista
QUESTION ANSWERING
Papers
with Code
(Question
Answering
)

@hamletbatista
QUESTION ANSWERING PAPER: XLNET
Papers
with Code
(Question
Answering
)

@hamletbatista
QUESTION ANSWERING RESULTS: XLNET
XLNet:
Generalized
Autoregressi
ve
Pretraining
for Language
Understandi
ng

@hamletbatista
XLNET CODE
https://github.
com/zihangdai
/xlnet

@hamletbatista
LONG-FORM QUESTION ANSWERING
Introducin
g long-
form
question
answering

@hamletbatista
LONG-FORM QUESTION ANSWERING
Subreddit:
Explain it
Like I'm
Five

@hamletbatista
LONG-FORM QUESTION ANSWERING
Scripts
and links
to
recreate
the ELI5
dataset.

@hamletbatista
LONG-FORM QUESTION ANSWERING BASELINE

@hamletbatista
FINALLY, LET’S GO FOR SOMETHING MORE AMBITIOUS
Generatin
g
Wikipedia
by
Summarizi
ng Long
Sequences

@hamletbatista
GENERATING WIKIPEDIA BY SUMMARIZING LONG
SEQUENCES
Generatin
g
Wikipedia
by
Summarizi
ng Long
Sequences

@hamletbatista
GENERATING WIKIPEDIA BY SUMMARIZING LONG
SEQUENCES
CONCLUSION
“We have shown that generating Wikipedia can ...

@hamletbatista
CAN WE HAVE THE SOURCE CODE? YES!
Github
link

@hamletbatista
HERE ARE SOME TRAINING COST ESTIMATES
Github
link

@hamletbatista
RESOURCES TO LEARN MORE
Faster Data Science Education
https://www.kaggle.com/learn/overview
Data Scientist’...

@hamletbatista
RESOURCES TO LEARN MORE
Build an Abstractive Text Summarizer in 94 Lines of Tensorflow !! (Tutorial 6)
http...

Share Slideshare

LinkedIn
Facebook
Twitter

Embed

Size (px)

Show related Slideshows at end

WordPress Shortcode

Link

Share
Email

2024 Trend Updates: What Really Wor... by Search Engine Jou... 1012946 views
Storytelling For The Web: Integrate... by Chiara Aliotta 921259 views
Artificial Intelligence, Data and C... by OECD Directorate ... 824586 views
How to Leverage AI to Boost Employe... by SocialHRCamp 344056 views
2024 State of Marketing Report – by... by Marius Sescu 196216 views
Everything You Need To Know About C... by Expeed Software 219258 views

View on Slideshare

10 of 62 Ad

View on Slideshare

10 of 62 Ad