SlideShare a Scribd company logo
1 of 21
Download to read offline
東京大学工学系研究科技術経営戦略学専攻松尾研究室
Generating Sentences from a Continuous Space
Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Jozefowicz and Samy Bengio
Presenter: Shuhei Iitsuka
東京大学工学系研究科技術経営戦略学専攻松尾研究室
Introduction
Variational Autoencoder for Natural Language Sentences.
● Recurrent Neural Network Language Model (RNNLM): the state-of-the-art generative model for
natural language.
● Drawback: word-by-word generation ---> cannot capture the global characteristics of a sentence.
● Brings the idea of Variational Autoencoder (VAE) into natural language sentences.
Not a representation vector,
but a continuous latent space.
東京大学工学系研究科技術経営戦略学専攻松尾研究室
Introduction: Contribution
● Proposed the variational autoencoder architecture for text sentence.
● Evaluated the performance on the language modeling task and the missing word imputing task
comparing to RNNLM.
● Conducted qualitative / quantitative analysis on the proposed model.
東京大学工学系研究科技術経営戦略学専攻松尾研究室
Google Poetry
“there is no one else in the world.
there is no one else in sight.
they were the only ones who mattered.
they were the only ones left.
he had to be with me.
she had to be with him.
i had to do this.
i wanted to kill him.
i started to cry.
i turned to him. --”
東京大学工学系研究科技術経営戦略学専攻松尾研究室
Agenda
● Introduction
● Background
○ Mapping a sentence to a latent space
○ Variational Autoencoder
● VAE for sentences
● Results
○ Language modeling
○ Imputing missing words
● Analysis
○ Dropout effect
○ Sampling from the posterior
○ Homotopies
● Conclusion
東京大学工学系研究科技術経営戦略学専攻松尾研究室
Background: Mapping a sentence to a latent space
Sequence Autoencoders
● Using seq2seq architecture as an autoencoder.
● Both encoder and decoder are RNNs.
Skip Thought Vector
● One encoder and two decoder
to predict previous sentence
and next sentence.
Paragraph Vector
● non-RNN model.
● Paragraph matrix D → sentence,
Vocabulary matrix W → word.
● Trains W to predict next words using D as contexts / memory.
[Kiros et al., 2015]
[Dai and Le, 2015]
[Le and Mikolov, 2014]
sentence_t
sentence_{t-1}
sentence_{t+1}
東京大学工学系研究科技術経営戦略学専攻松尾研究室
Background: Variational autoencoder
● A regularized version of the standard autoencoder.
● Learns inputs not as single points, but as regions in the latent space.
○ Replace the deterministic encoder φ_{enc} with a probability
distribution, q(z|x).
● KL divergence is introduced to make q(z|x) close to a prior p(z).
○ If learned with just a reconstruction loss, variances in q(z|x) get
extremely small (=q(z|x) becomes deterministic).
x^
x
z
encoder φ
decoder
x^
x
μ
encoder q(z|x)
decoder
σ
Standard autoencoder
Variational autoencoder
[Presenter Illustrated]
東京大学工学系研究科技術経営戦略学専攻松尾研究室
VAE for sentences
● Authors introduce the VAE for text which uses single-layer LSTM RNNs for both the encoder and
decoder.
● The decoder can be regarded as a special version of RNNLM conditioned by the hidden code.
● Explored some variations → no significant improvements.
○ Concatenating z every time step.
○ Soft-plus activation
○ Feedforward between (encoder, latent variable) and (decoder, latent variable). (?)
st-2 st-1 st
wt-2 wt-1 wt
U U U
W W
yt
V
Variational Sequence Autoencoder RNNLM [Presenter Illustrated]
東京大学工学系研究科技術経営戦略学専攻松尾研究室
VAE for sentences: similar models
● Variational Recurrent Autoencoder (VRAE) [Fabious and van Amersfoort, 2014]
● Continuous latent variables with RNN-style modeling [Bayer and Osendorfer, 2015][Chung et al., 2015]
→ Latent variables are separated per timestep.
→ Not suitable to capture the general sentence characteristics.
● VAE-based document-level language model [Miao et al., 2015]
→ Input texts are models as bags of words.
→ Not suitable to capture the sequential characteristics.
東京大学工学系研究科技術経営戦略学専攻松尾研究室
Optimization Challenges
● KL divergence tends to become zero. (= q(z|x) ~ p(x), which equals to RNNLM)
● Gives up capturing z and goes after explaining each sample with an optimized decoder.
● Authors introduce two techniques to migrate this issue.
○ KL cost annealing
○ Word dropout and historyless decoding
東京大学工学系研究科技術経営戦略学専攻松尾研究室
Optimization technique 1: KL cost annealing
Spike: encodes as much
information into z as it can.
Drop: starts to
smooth out its
encodings.
Rise: learns to
condense more
information into z.
東京大学工学系研究科技術経営戦略学専攻松尾研究室
Optimization technique 2: Word dropout and historyless decoding
Weaken the decoder by replacing a fraction of words with the generic unknown word token UNK.
<UNK> <UNK>
Forces the model
to rely on z for
good prediction.
Parameterized as a keep rate k ∈ [0, 1].
東京大学工学系研究科技術経営戦略学専攻松尾研究室
Results: Language modeling
Objective: Is a continuous latent variable helpful for a standard task (language modeling)?
Methods:
Baseline: RNNLM (non-variational)
Proposal: VAE (variational)
Dataset: Penn Treebank
Measurement: Likelihood
Baseline: True test likelihood
Proposal: Variational lower bound of likelihood
Disadvantage for VAE
Penn Treebank output example [Bui et al., 2010]
東京大学工学系研究科技術経営戦略学専攻松尾研究室
Results: Language modeling
NLL: negative log-likelihoods, PPL: perplexities. Lower is better for both. () shows the KL loss.
● VAE’s performance is slightly worse than RNNLM.
● Without optimization technique (KL annealing and
dropout), the performance becomes the same as RNNLM.
Inputless Decoder (== dropout keep rate = 0)
● KL loss becomes huge, but VAE shows better performance than RNNLM.
Note: 101 - 2 = 99 (<100)
for reconstruction loss.
東京大学工学系研究科技術経営戦略学専攻松尾研究室
Results: Imputing missing words
Claim: VAE’s global sentence features are suited to do the task of imputing missing words.
Computing the likelihood is intractable. (Need to calculate for every vocabulary V each step!)
→ Introduces adversarial evaluation for quantitative evaluation.
Adversarial evaluation: How well the output can deceive the classifier to train.
Example of imputing task and outputs from each model.
東京大学工学系研究科技術経営戦略学専攻松尾研究室
Results: Imputing missing words
Dataset: Book corpus (mostly fiction)... 80M sentences after pruning.
Classifier to train: A bag-of-unigrams & LSTM logistic regression classifier.
● VAE shows lower adversarial error (= much deceptive) than RNNLM.
● Negative log-likelihood is comparable.
○ RNNLM can make NLL lower by making “safe” answers.
(e.g. complete the sentence with “, he said.”)
→ RNNLM yields less diverse samples than VAE, but produces natural sentences by favoring generic
high-probability endings.
Adv. Error ∈ [0, 0.5]. 0.5 means all
imputed samples are detected (not
deceptive at all). 0 means all imputed
samples are not detected. Lower is
better.
東京大学工学系研究科技術経営戦略学専攻松尾研究室
Analysis: The impact of word dropout
The information stored in the
latent variable increases.
With the extreme
settings, the likelihood
decreases.
Cost
Too typical, not
much topics are
captured.
Far less typical.
Has a clear topic.
Ceases to be
grammatically
correct.
東京大学工学系研究科技術経営戦略学専攻松尾研究室
Analysis: Sampling from the posterior
● Analysis on reconstructed input texts.
○ Note: This is variational. It’s not re-outputting memorized samples.
● It seems capturing the length and topic of given sentences.
● As the sentence gets longer, the output becomes various.
東京大学工学系研究科技術経営戦略学専攻松尾研究室
Analysis: Homotopies
● Homotopy = linear interpolation. VAE can generate a sentence between two points in the latent
space!
● Outputs are grammatical and have consistent topic.
● Similar syntax and topic but flipped sentiment can be problematic sometimes.
Sequence autoencoder outputs: Variational autoencoder outputs:
Grammatically
incorrect
Grammatically
correct
Grammatically
correct
東京大学工学系研究科技術経営戦略学専攻松尾研究室
Analysis: Homotopies
convinced,
understood
unconvinced,
suspicious
[presenter interpretation]
東京大学工学系研究科技術経営戦略学専攻松尾研究室
Conclusion
This paper introduces the use of variational autoencoder for natural language sentences.
● Evaluated that novel techniques are useful for successful training
● Found that the model is effective for imputing missing words
● Showed the model can produce diverse sentences with smooth interoperation.
Future work
● Factorization of the latent space
● Incorporating conditional settings
● Learning sentence embeddings with semi-supervised fashion learning
● Going beyond adversarial evaluation to a fully adversarial training objective

More Related Content

What's hot

MLP-Mixer: An all-MLP Architecture for Vision
MLP-Mixer: An all-MLP Architecture for VisionMLP-Mixer: An all-MLP Architecture for Vision
MLP-Mixer: An all-MLP Architecture for VisionKazuyuki Miyazawa
 
Semi supervised, weakly-supervised, unsupervised, and active learning
Semi supervised, weakly-supervised, unsupervised, and active learningSemi supervised, weakly-supervised, unsupervised, and active learning
Semi supervised, weakly-supervised, unsupervised, and active learningYusuke Uchida
 
深層学習によるHuman Pose Estimationの基礎
深層学習によるHuman Pose Estimationの基礎深層学習によるHuman Pose Estimationの基礎
深層学習によるHuman Pose Estimationの基礎Takumi Ohkuma
 
【DL輪読会】言語以外でのTransformerのまとめ (ViT, Perceiver, Frozen Pretrained Transformer etc)
【DL輪読会】言語以外でのTransformerのまとめ (ViT, Perceiver, Frozen Pretrained Transformer etc)【DL輪読会】言語以外でのTransformerのまとめ (ViT, Perceiver, Frozen Pretrained Transformer etc)
【DL輪読会】言語以外でのTransformerのまとめ (ViT, Perceiver, Frozen Pretrained Transformer etc)Deep Learning JP
 
2022年度秋学期 画像情報処理 第9回 離散フーリエ変換と離散コサイン変換 (2022. 11. 25)
2022年度秋学期 画像情報処理 第9回 離散フーリエ変換と離散コサイン変換 (2022. 11. 25) 2022年度秋学期 画像情報処理 第9回 離散フーリエ変換と離散コサイン変換 (2022. 11. 25)
2022年度秋学期 画像情報処理 第9回 離散フーリエ変換と離散コサイン変換 (2022. 11. 25) Akira Asano
 
JVS:フリーの日本語多数話者音声コーパス
JVS:フリーの日本語多数話者音声コーパス JVS:フリーの日本語多数話者音声コーパス
JVS:フリーの日本語多数話者音声コーパス Shinnosuke Takamichi
 
[DL輪読会]Reward Augmented Maximum Likelihood for Neural Structured Prediction
[DL輪読会]Reward Augmented Maximum Likelihood for Neural Structured Prediction[DL輪読会]Reward Augmented Maximum Likelihood for Neural Structured Prediction
[DL輪読会]Reward Augmented Maximum Likelihood for Neural Structured PredictionDeep Learning JP
 
[DL輪読会]NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
[DL輪読会]NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis[DL輪読会]NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
[DL輪読会]NeRF: Representing Scenes as Neural Radiance Fields for View SynthesisDeep Learning JP
 
2022年度調和系工学研究室配属説明会資料
2022年度調和系工学研究室配属説明会資料2022年度調和系工学研究室配属説明会資料
2022年度調和系工学研究室配属説明会資料harmonylab
 
makoto shing (stability ai) - image model fine-tuning - wandb_event_230525.pdf
makoto shing (stability ai) - image model fine-tuning - wandb_event_230525.pdfmakoto shing (stability ai) - image model fine-tuning - wandb_event_230525.pdf
makoto shing (stability ai) - image model fine-tuning - wandb_event_230525.pdfAkira Shibata
 
論文紹介 wav2vec: Unsupervised Pre-training for Speech Recognition
論文紹介  wav2vec: Unsupervised Pre-training for Speech Recognition論文紹介  wav2vec: Unsupervised Pre-training for Speech Recognition
論文紹介 wav2vec: Unsupervised Pre-training for Speech RecognitionYosukeKashiwagi1
 
競輪におけるレーティングシステムを用いた予想記事生成に関する研究
競輪におけるレーティングシステムを用いた予想記事生成に関する研究競輪におけるレーティングシステムを用いた予想記事生成に関する研究
競輪におけるレーティングシステムを用いた予想記事生成に関する研究harmonylab
 
Domain Adaptation 発展と動向まとめ(サーベイ資料)
Domain Adaptation 発展と動向まとめ(サーベイ資料)Domain Adaptation 発展と動向まとめ(サーベイ資料)
Domain Adaptation 発展と動向まとめ(サーベイ資料)Yamato OKAMOTO
 
Trend of 3D object detections
Trend of 3D object detectionsTrend of 3D object detections
Trend of 3D object detectionsEiji Sekiya
 
“機械学習の説明”の信頼性
“機械学習の説明”の信頼性“機械学習の説明”の信頼性
“機械学習の説明”の信頼性Satoshi Hara
 
【DL輪読会】Flow Matching for Generative Modeling
【DL輪読会】Flow Matching for Generative Modeling【DL輪読会】Flow Matching for Generative Modeling
【DL輪読会】Flow Matching for Generative ModelingDeep Learning JP
 
【論文読み会】BEiT_BERT Pre-Training of Image Transformers.pptx
【論文読み会】BEiT_BERT Pre-Training of Image Transformers.pptx【論文読み会】BEiT_BERT Pre-Training of Image Transformers.pptx
【論文読み会】BEiT_BERT Pre-Training of Image Transformers.pptxARISE analytics
 
[DL輪読会]Glow: Generative Flow with Invertible 1×1 Convolutions
[DL輪読会]Glow: Generative Flow with Invertible 1×1 Convolutions[DL輪読会]Glow: Generative Flow with Invertible 1×1 Convolutions
[DL輪読会]Glow: Generative Flow with Invertible 1×1 ConvolutionsDeep Learning JP
 
【DL輪読会】時系列予測 Transfomers の精度向上手法
【DL輪読会】時系列予測 Transfomers の精度向上手法【DL輪読会】時系列予測 Transfomers の精度向上手法
【DL輪読会】時系列予測 Transfomers の精度向上手法Deep Learning JP
 

What's hot (20)

MLP-Mixer: An all-MLP Architecture for Vision
MLP-Mixer: An all-MLP Architecture for VisionMLP-Mixer: An all-MLP Architecture for Vision
MLP-Mixer: An all-MLP Architecture for Vision
 
Semi supervised, weakly-supervised, unsupervised, and active learning
Semi supervised, weakly-supervised, unsupervised, and active learningSemi supervised, weakly-supervised, unsupervised, and active learning
Semi supervised, weakly-supervised, unsupervised, and active learning
 
深層学習によるHuman Pose Estimationの基礎
深層学習によるHuman Pose Estimationの基礎深層学習によるHuman Pose Estimationの基礎
深層学習によるHuman Pose Estimationの基礎
 
【DL輪読会】言語以外でのTransformerのまとめ (ViT, Perceiver, Frozen Pretrained Transformer etc)
【DL輪読会】言語以外でのTransformerのまとめ (ViT, Perceiver, Frozen Pretrained Transformer etc)【DL輪読会】言語以外でのTransformerのまとめ (ViT, Perceiver, Frozen Pretrained Transformer etc)
【DL輪読会】言語以外でのTransformerのまとめ (ViT, Perceiver, Frozen Pretrained Transformer etc)
 
2022年度秋学期 画像情報処理 第9回 離散フーリエ変換と離散コサイン変換 (2022. 11. 25)
2022年度秋学期 画像情報処理 第9回 離散フーリエ変換と離散コサイン変換 (2022. 11. 25) 2022年度秋学期 画像情報処理 第9回 離散フーリエ変換と離散コサイン変換 (2022. 11. 25)
2022年度秋学期 画像情報処理 第9回 離散フーリエ変換と離散コサイン変換 (2022. 11. 25)
 
JVS:フリーの日本語多数話者音声コーパス
JVS:フリーの日本語多数話者音声コーパス JVS:フリーの日本語多数話者音声コーパス
JVS:フリーの日本語多数話者音声コーパス
 
[DL輪読会]Reward Augmented Maximum Likelihood for Neural Structured Prediction
[DL輪読会]Reward Augmented Maximum Likelihood for Neural Structured Prediction[DL輪読会]Reward Augmented Maximum Likelihood for Neural Structured Prediction
[DL輪読会]Reward Augmented Maximum Likelihood for Neural Structured Prediction
 
[DL輪読会]NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
[DL輪読会]NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis[DL輪読会]NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
[DL輪読会]NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
 
2022年度調和系工学研究室配属説明会資料
2022年度調和系工学研究室配属説明会資料2022年度調和系工学研究室配属説明会資料
2022年度調和系工学研究室配属説明会資料
 
makoto shing (stability ai) - image model fine-tuning - wandb_event_230525.pdf
makoto shing (stability ai) - image model fine-tuning - wandb_event_230525.pdfmakoto shing (stability ai) - image model fine-tuning - wandb_event_230525.pdf
makoto shing (stability ai) - image model fine-tuning - wandb_event_230525.pdf
 
論文紹介 wav2vec: Unsupervised Pre-training for Speech Recognition
論文紹介  wav2vec: Unsupervised Pre-training for Speech Recognition論文紹介  wav2vec: Unsupervised Pre-training for Speech Recognition
論文紹介 wav2vec: Unsupervised Pre-training for Speech Recognition
 
競輪におけるレーティングシステムを用いた予想記事生成に関する研究
競輪におけるレーティングシステムを用いた予想記事生成に関する研究競輪におけるレーティングシステムを用いた予想記事生成に関する研究
競輪におけるレーティングシステムを用いた予想記事生成に関する研究
 
Domain Adaptation 発展と動向まとめ(サーベイ資料)
Domain Adaptation 発展と動向まとめ(サーベイ資料)Domain Adaptation 発展と動向まとめ(サーベイ資料)
Domain Adaptation 発展と動向まとめ(サーベイ資料)
 
Trend of 3D object detections
Trend of 3D object detectionsTrend of 3D object detections
Trend of 3D object detections
 
“機械学習の説明”の信頼性
“機械学習の説明”の信頼性“機械学習の説明”の信頼性
“機械学習の説明”の信頼性
 
【DL輪読会】Flow Matching for Generative Modeling
【DL輪読会】Flow Matching for Generative Modeling【DL輪読会】Flow Matching for Generative Modeling
【DL輪読会】Flow Matching for Generative Modeling
 
【論文読み会】BEiT_BERT Pre-Training of Image Transformers.pptx
【論文読み会】BEiT_BERT Pre-Training of Image Transformers.pptx【論文読み会】BEiT_BERT Pre-Training of Image Transformers.pptx
【論文読み会】BEiT_BERT Pre-Training of Image Transformers.pptx
 
[DL輪読会]Glow: Generative Flow with Invertible 1×1 Convolutions
[DL輪読会]Glow: Generative Flow with Invertible 1×1 Convolutions[DL輪読会]Glow: Generative Flow with Invertible 1×1 Convolutions
[DL輪読会]Glow: Generative Flow with Invertible 1×1 Convolutions
 
実装レベルで学ぶVQVAE
実装レベルで学ぶVQVAE実装レベルで学ぶVQVAE
実装レベルで学ぶVQVAE
 
【DL輪読会】時系列予測 Transfomers の精度向上手法
【DL輪読会】時系列予測 Transfomers の精度向上手法【DL輪読会】時系列予測 Transfomers の精度向上手法
【DL輪読会】時系列予測 Transfomers の精度向上手法
 

Similar to VAE for Natural Language Sentences

[Paper Reading] Unsupervised Learning of Sentence Embeddings using Compositi...
[Paper Reading]  Unsupervised Learning of Sentence Embeddings using Compositi...[Paper Reading]  Unsupervised Learning of Sentence Embeddings using Compositi...
[Paper Reading] Unsupervised Learning of Sentence Embeddings using Compositi...Hiroki Shimanaka
 
Tiancheng Zhao - 2017 - Learning Discourse-level Diversity for Neural Dialog...
Tiancheng Zhao - 2017 -  Learning Discourse-level Diversity for Neural Dialog...Tiancheng Zhao - 2017 -  Learning Discourse-level Diversity for Neural Dialog...
Tiancheng Zhao - 2017 - Learning Discourse-level Diversity for Neural Dialog...Association for Computational Linguistics
 
Detecting paraphrases using recursive autoencoders
Detecting paraphrases using recursive autoencodersDetecting paraphrases using recursive autoencoders
Detecting paraphrases using recursive autoencodersFeynman Liang
 
Understanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM ArchitectureUnderstanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM Architecturerahul_net
 
Word representations in vector space
Word representations in vector spaceWord representations in vector space
Word representations in vector spaceAbdullah Khan Zehady
 
Topic model an introduction
Topic model an introductionTopic model an introduction
Topic model an introductionYueshen Xu
 
Master defence 2020 - Anastasiia Khaburska - Statistical and Neural Language ...
Master defence 2020 - Anastasiia Khaburska - Statistical and Neural Language ...Master defence 2020 - Anastasiia Khaburska - Statistical and Neural Language ...
Master defence 2020 - Anastasiia Khaburska - Statistical and Neural Language ...Lviv Data Science Summer School
 
Csr2011 june17 15_15_kaminski
Csr2011 june17 15_15_kaminskiCsr2011 june17 15_15_kaminski
Csr2011 june17 15_15_kaminskiCSR2011
 
PL Lecture 01 - preliminaries
PL Lecture 01 - preliminariesPL Lecture 01 - preliminaries
PL Lecture 01 - preliminariesSchwannden Kuo
 
Enhancing Xtext for General Purpose Languages
Enhancing Xtext for General Purpose LanguagesEnhancing Xtext for General Purpose Languages
Enhancing Xtext for General Purpose LanguagesUniversity of York
 
Talk from NVidia Developer Connect
Talk from NVidia Developer ConnectTalk from NVidia Developer Connect
Talk from NVidia Developer ConnectAnuj Gupta
 
Generating Assertion Code from OCL: A Transformational Approach Based on Simi...
Generating Assertion Code from OCL: A Transformational Approach Based on Simi...Generating Assertion Code from OCL: A Transformational Approach Based on Simi...
Generating Assertion Code from OCL: A Transformational Approach Based on Simi...Shinpei Hayashi
 
Representation Learning of Vectors of Words and Phrases
Representation Learning of Vectors of Words and PhrasesRepresentation Learning of Vectors of Words and Phrases
Representation Learning of Vectors of Words and PhrasesFelipe Moraes
 
Contrastive Learning with Adversarial Perturbations for Conditional Text Gene...
Contrastive Learning with Adversarial Perturbations for Conditional Text Gene...Contrastive Learning with Adversarial Perturbations for Conditional Text Gene...
Contrastive Learning with Adversarial Perturbations for Conditional Text Gene...MLAI2
 

Similar to VAE for Natural Language Sentences (20)

[Paper Reading] Unsupervised Learning of Sentence Embeddings using Compositi...
[Paper Reading]  Unsupervised Learning of Sentence Embeddings using Compositi...[Paper Reading]  Unsupervised Learning of Sentence Embeddings using Compositi...
[Paper Reading] Unsupervised Learning of Sentence Embeddings using Compositi...
 
Tiancheng Zhao - 2017 - Learning Discourse-level Diversity for Neural Dialog...
Tiancheng Zhao - 2017 -  Learning Discourse-level Diversity for Neural Dialog...Tiancheng Zhao - 2017 -  Learning Discourse-level Diversity for Neural Dialog...
Tiancheng Zhao - 2017 - Learning Discourse-level Diversity for Neural Dialog...
 
Detecting paraphrases using recursive autoencoders
Detecting paraphrases using recursive autoencodersDetecting paraphrases using recursive autoencoders
Detecting paraphrases using recursive autoencoders
 
Introduction to Prolog
Introduction to PrologIntroduction to Prolog
Introduction to Prolog
 
Understanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM ArchitectureUnderstanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM Architecture
 
Word representations in vector space
Word representations in vector spaceWord representations in vector space
Word representations in vector space
 
CNN for modeling sentence
CNN for modeling sentenceCNN for modeling sentence
CNN for modeling sentence
 
Topic model an introduction
Topic model an introductionTopic model an introduction
Topic model an introduction
 
Master defence 2020 - Anastasiia Khaburska - Statistical and Neural Language ...
Master defence 2020 - Anastasiia Khaburska - Statistical and Neural Language ...Master defence 2020 - Anastasiia Khaburska - Statistical and Neural Language ...
Master defence 2020 - Anastasiia Khaburska - Statistical and Neural Language ...
 
Csr2011 june17 15_15_kaminski
Csr2011 june17 15_15_kaminskiCsr2011 june17 15_15_kaminski
Csr2011 june17 15_15_kaminski
 
PL Lecture 01 - preliminaries
PL Lecture 01 - preliminariesPL Lecture 01 - preliminaries
PL Lecture 01 - preliminaries
 
Enhancing Xtext for General Purpose Languages
Enhancing Xtext for General Purpose LanguagesEnhancing Xtext for General Purpose Languages
Enhancing Xtext for General Purpose Languages
 
Talk from NVidia Developer Connect
Talk from NVidia Developer ConnectTalk from NVidia Developer Connect
Talk from NVidia Developer Connect
 
Generating Assertion Code from OCL: A Transformational Approach Based on Simi...
Generating Assertion Code from OCL: A Transformational Approach Based on Simi...Generating Assertion Code from OCL: A Transformational Approach Based on Simi...
Generating Assertion Code from OCL: A Transformational Approach Based on Simi...
 
Efficient projections
Efficient projectionsEfficient projections
Efficient projections
 
Efficient projections
Efficient projectionsEfficient projections
Efficient projections
 
Language models
Language modelsLanguage models
Language models
 
Representation Learning of Vectors of Words and Phrases
Representation Learning of Vectors of Words and PhrasesRepresentation Learning of Vectors of Words and Phrases
Representation Learning of Vectors of Words and Phrases
 
Contrastive Learning with Adversarial Perturbations for Conditional Text Gene...
Contrastive Learning with Adversarial Perturbations for Conditional Text Gene...Contrastive Learning with Adversarial Perturbations for Conditional Text Gene...
Contrastive Learning with Adversarial Perturbations for Conditional Text Gene...
 
slides.pdf
slides.pdfslides.pdf
slides.pdf
 

More from Shuhei Iitsuka

Online and offline handwritten chinese character recognition a comprehensive...
Online and offline handwritten chinese character recognition  a comprehensive...Online and offline handwritten chinese character recognition  a comprehensive...
Online and offline handwritten chinese character recognition a comprehensive...Shuhei Iitsuka
 
Inferring win–lose product network from user behavior
Inferring win–lose product network from user behaviorInferring win–lose product network from user behavior
Inferring win–lose product network from user behaviorShuhei Iitsuka
 
バリエーションの提示がもたらす長期的効果に着目したウェブサイト最適化手法 @第31回人工知能学会全国大会
バリエーションの提示がもたらす長期的効果に着目したウェブサイト最適化手法 @第31回人工知能学会全国大会バリエーションの提示がもたらす長期的効果に着目したウェブサイト最適化手法 @第31回人工知能学会全国大会
バリエーションの提示がもたらす長期的効果に着目したウェブサイト最適化手法 @第31回人工知能学会全国大会Shuhei Iitsuka
 
Procedural modeling using autoencoder networks
Procedural modeling using autoencoder networksProcedural modeling using autoencoder networks
Procedural modeling using autoencoder networksShuhei Iitsuka
 
ウェブサイト最適化のためのバリエーション自動生成システム
ウェブサイト最適化のためのバリエーション自動生成システムウェブサイト最適化のためのバリエーション自動生成システム
ウェブサイト最適化のためのバリエーション自動生成システムShuhei Iitsuka
 
Improving the Sensitivity of Online Controlled Experiments by Utilizing Pre-E...
Improving the Sensitivity of Online Controlled Experiments by Utilizing Pre-E...Improving the Sensitivity of Online Controlled Experiments by Utilizing Pre-E...
Improving the Sensitivity of Online Controlled Experiments by Utilizing Pre-E...Shuhei Iitsuka
 
Machine learning meets web development
Machine learning meets web developmentMachine learning meets web development
Machine learning meets web developmentShuhei Iitsuka
 
Python と Xpath で ウェブからデータをあつめる
Python と Xpath で ウェブからデータをあつめるPython と Xpath で ウェブからデータをあつめる
Python と Xpath で ウェブからデータをあつめるShuhei Iitsuka
 
リミックスからはじめる DTM 入門
リミックスからはじめる DTM 入門リミックスからはじめる DTM 入門
リミックスからはじめる DTM 入門Shuhei Iitsuka
 
【DBDA 勉強会 2013 夏】Chapter 12: Bayesian Approaches to Testing a Point (‘‘Null’’...
【DBDA 勉強会 2013 夏】Chapter 12: Bayesian Approaches to Testing a Point (‘‘Null’’...【DBDA 勉強会 2013 夏】Chapter 12: Bayesian Approaches to Testing a Point (‘‘Null’’...
【DBDA 勉強会 2013 夏】Chapter 12: Bayesian Approaches to Testing a Point (‘‘Null’’...Shuhei Iitsuka
 
Asia Trend Map: Forecasting “Cool Japan” Content Popularity on Web Data
Asia Trend Map: Forecasting “Cool Japan” Content Popularity on Web DataAsia Trend Map: Forecasting “Cool Japan” Content Popularity on Web Data
Asia Trend Map: Forecasting “Cool Japan” Content Popularity on Web DataShuhei Iitsuka
 
【DBDA 勉強会 2013 夏】Doing Bayesian Data Analysis Chapter 4: Bayes’ Rule
【DBDA 勉強会 2013 夏】Doing Bayesian Data Analysis Chapter 4: Bayes’ Rule【DBDA 勉強会 2013 夏】Doing Bayesian Data Analysis Chapter 4: Bayes’ Rule
【DBDA 勉強会 2013 夏】Doing Bayesian Data Analysis Chapter 4: Bayes’ RuleShuhei Iitsuka
 
UT Startup Gym で人生が変わった話
UT Startup Gym で人生が変わった話UT Startup Gym で人生が変わった話
UT Startup Gym で人生が変わった話Shuhei Iitsuka
 
ウェブサイトで収益を得る
ウェブサイトで収益を得るウェブサイトで収益を得る
ウェブサイトで収益を得るShuhei Iitsuka
 
HTML で自己紹介ページをつくる
HTML で自己紹介ページをつくるHTML で自己紹介ページをつくる
HTML で自己紹介ページをつくるShuhei Iitsuka
 
データベースを使おう
データベースを使おうデータベースを使おう
データベースを使おうShuhei Iitsuka
 
ウェブサービスの企画とデザイン
ウェブサービスの企画とデザインウェブサービスの企画とデザイン
ウェブサービスの企画とデザインShuhei Iitsuka
 
データベースを使おう
データベースを使おうデータベースを使おう
データベースを使おうShuhei Iitsuka
 
第3期キックオフ説明会+勉強会
第3期キックオフ説明会+勉強会 第3期キックオフ説明会+勉強会
第3期キックオフ説明会+勉強会 Shuhei Iitsuka
 
かんたん Twitter アプリをつくろう
かんたん Twitter アプリをつくろう かんたん Twitter アプリをつくろう
かんたん Twitter アプリをつくろう Shuhei Iitsuka
 

More from Shuhei Iitsuka (20)

Online and offline handwritten chinese character recognition a comprehensive...
Online and offline handwritten chinese character recognition  a comprehensive...Online and offline handwritten chinese character recognition  a comprehensive...
Online and offline handwritten chinese character recognition a comprehensive...
 
Inferring win–lose product network from user behavior
Inferring win–lose product network from user behaviorInferring win–lose product network from user behavior
Inferring win–lose product network from user behavior
 
バリエーションの提示がもたらす長期的効果に着目したウェブサイト最適化手法 @第31回人工知能学会全国大会
バリエーションの提示がもたらす長期的効果に着目したウェブサイト最適化手法 @第31回人工知能学会全国大会バリエーションの提示がもたらす長期的効果に着目したウェブサイト最適化手法 @第31回人工知能学会全国大会
バリエーションの提示がもたらす長期的効果に着目したウェブサイト最適化手法 @第31回人工知能学会全国大会
 
Procedural modeling using autoencoder networks
Procedural modeling using autoencoder networksProcedural modeling using autoencoder networks
Procedural modeling using autoencoder networks
 
ウェブサイト最適化のためのバリエーション自動生成システム
ウェブサイト最適化のためのバリエーション自動生成システムウェブサイト最適化のためのバリエーション自動生成システム
ウェブサイト最適化のためのバリエーション自動生成システム
 
Improving the Sensitivity of Online Controlled Experiments by Utilizing Pre-E...
Improving the Sensitivity of Online Controlled Experiments by Utilizing Pre-E...Improving the Sensitivity of Online Controlled Experiments by Utilizing Pre-E...
Improving the Sensitivity of Online Controlled Experiments by Utilizing Pre-E...
 
Machine learning meets web development
Machine learning meets web developmentMachine learning meets web development
Machine learning meets web development
 
Python と Xpath で ウェブからデータをあつめる
Python と Xpath で ウェブからデータをあつめるPython と Xpath で ウェブからデータをあつめる
Python と Xpath で ウェブからデータをあつめる
 
リミックスからはじめる DTM 入門
リミックスからはじめる DTM 入門リミックスからはじめる DTM 入門
リミックスからはじめる DTM 入門
 
【DBDA 勉強会 2013 夏】Chapter 12: Bayesian Approaches to Testing a Point (‘‘Null’’...
【DBDA 勉強会 2013 夏】Chapter 12: Bayesian Approaches to Testing a Point (‘‘Null’’...【DBDA 勉強会 2013 夏】Chapter 12: Bayesian Approaches to Testing a Point (‘‘Null’’...
【DBDA 勉強会 2013 夏】Chapter 12: Bayesian Approaches to Testing a Point (‘‘Null’’...
 
Asia Trend Map: Forecasting “Cool Japan” Content Popularity on Web Data
Asia Trend Map: Forecasting “Cool Japan” Content Popularity on Web DataAsia Trend Map: Forecasting “Cool Japan” Content Popularity on Web Data
Asia Trend Map: Forecasting “Cool Japan” Content Popularity on Web Data
 
【DBDA 勉強会 2013 夏】Doing Bayesian Data Analysis Chapter 4: Bayes’ Rule
【DBDA 勉強会 2013 夏】Doing Bayesian Data Analysis Chapter 4: Bayes’ Rule【DBDA 勉強会 2013 夏】Doing Bayesian Data Analysis Chapter 4: Bayes’ Rule
【DBDA 勉強会 2013 夏】Doing Bayesian Data Analysis Chapter 4: Bayes’ Rule
 
UT Startup Gym で人生が変わった話
UT Startup Gym で人生が変わった話UT Startup Gym で人生が変わった話
UT Startup Gym で人生が変わった話
 
ウェブサイトで収益を得る
ウェブサイトで収益を得るウェブサイトで収益を得る
ウェブサイトで収益を得る
 
HTML で自己紹介ページをつくる
HTML で自己紹介ページをつくるHTML で自己紹介ページをつくる
HTML で自己紹介ページをつくる
 
データベースを使おう
データベースを使おうデータベースを使おう
データベースを使おう
 
ウェブサービスの企画とデザイン
ウェブサービスの企画とデザインウェブサービスの企画とデザイン
ウェブサービスの企画とデザイン
 
データベースを使おう
データベースを使おうデータベースを使おう
データベースを使おう
 
第3期キックオフ説明会+勉強会
第3期キックオフ説明会+勉強会 第3期キックオフ説明会+勉強会
第3期キックオフ説明会+勉強会
 
かんたん Twitter アプリをつくろう
かんたん Twitter アプリをつくろう かんたん Twitter アプリをつくろう
かんたん Twitter アプリをつくろう
 

Recently uploaded

Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESmohitsingh558521
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demoHarshalMandlekar2
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionDilum Bandara
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 

Recently uploaded (20)

Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demo
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 

VAE for Natural Language Sentences