SlideShare a Scribd company logo
A Syntactic Neural Model
for General-Purpose Code
Generation
Pengcheng Yin, Graham Neubig
ACL2017
M1 Tomoya Ogata
1
Abstract
• Input
• natural language descriptions
• Output
• source code written in a general-purpose programming
language
• Existing data-driven methods treat this problem as a
language generation task without considering the
underlying syntax of the target programming language
• propose a novel neural architecture powered by a
grammar model to explicitly capture the target syntax
as prior knowledge
2
Given an NL description x, our task is to generate the
code snippet c in a modern PL based on the intent of x.
We define a probabilistic grammar model of generating
an AST y given x
𝑦 is then deterministically converted to the
corresponding surface code
an AST is generated by applying several production rules
composed of a head node and multiple child nodes
The Code Generation Problem
3
Grammar Model
• APPLYRULE Actions
• GENTOKEN Actions
4
APPLYRULE Actions
• APPLYRULE chooses a rule from the subset that has
a head matching the type of 𝑛 𝑓𝑡
• uses r to expand 𝑛 𝑓𝑡
by appending all child nodes
specified by the selected production
• When a variable terminal node is added to the
derivation and becomes the frontier node, the
grammar model then switches to GENTOKEN
actions to populate the variable terminal with
tokens
5
GENTOKEN Actions
Once we reach a frontier node 𝑛 𝑓𝑡
that corresponds
to a variable type, GENTOKEN actions are used to fill
this node with values.
At each time step, GENTOKEN appends one terminal
token to the current frontier variable node
6
Tracking Generation States
Action Embedding: at
Context Vector: ct
Parent Feeding: pt
7
Parent Feeding
• parent information 𝑝𝑡 from two sources
• (1) the hidden state of parent action 𝑠 𝑝 𝑡
• (2) the embedding of parent action 𝑎 𝑝 𝑡
• The parent feeding schema enables the model to
utilize the information of parent code segments to
make more confident predictions
8
Calculating Action Probabilities
g(st): tanh(W・st+b)
e(r): one-hot vector for rule r
p(gen|・), p(copy|・): tanh(W・st+b)
9
Training
• Given a dataset of pairs of NL descriptions 𝑥𝑖 and
code 𝑐𝑖 snippets
• we parse 𝑐𝑖 into its AST 𝑦𝑖
• decompose 𝑦𝑖 into a sequence of oracle actions
• The model is then optimized by maximizing the log-
likelihood of the oracle action sequence
10
Experimental Evaluation
• Datasets
• HEARTHSTONE (HS) dataset
• a collection of Python classes that implement cards for the card
game HearthStone
• DJANGO dataset
• a collection of lines of code from the Django web framework, each
with a manually annotated NL description
• IFTTT dataset
• a domain- specific benchmark that provides an interesting side
comparison
• Metrics
• accuracy
• BLUE-4
11
Experimental Evaluation
• Preprocessing
• Input are tokenized using NLTK
• replacing quoted strings in the inputs with place holders
• extract unary closures whose frequency is larger than a
threshold
• Configuration
• node type embeddings: 64
• All other embeddings: 128
• RNN states: 256
• hidden layers: 50
12
Model
• Latent Predictor Network (LPN), a state-of-the-art
sequence- to-sequence code generation model
• SEQ2TREE, a neural semantic parsing model
• NMT system using a standard encoder-decoder
architecture with attention and unknown word
replacement
13
Result (HS, DJANG0)
14
Output Example
15
Result (IFTTT)
16
Error Analysis
• randomly sampled and labeled 100 and 50 failed
examples from DJANGO and HS, respectively
• using different parameter names when defining a function
• omitting (or adding) default values of parameters in function
calls
• DJANGO
• 30%: the pointer network failed
• 25%: the generated code only partially implemented the
required functionality
• 10%: malformed English inputs
• 5%: preprocessing errors
• 30%: could not be easily categorized into the above
• HS
• partial implementation errors
17
Conclusion
• This paper proposes a syntax-driven neural code
generation approach that generates an abstract
syntax tree by sequentially applying actions from a
grammar model
18

More Related Content

What's hot

[Paper Reading] Attention is All You Need
[Paper Reading] Attention is All You Need[Paper Reading] Attention is All You Need
[Paper Reading] Attention is All You Need
Daiki Tanaka
 
CNIT 126: 13: Data Encoding
CNIT 126: 13: Data EncodingCNIT 126: 13: Data Encoding
CNIT 126: 13: Data Encoding
Sam Bowne
 
Decision Making & Loops
Decision Making & LoopsDecision Making & Loops
Decision Making & Loops
Akhil Kaushik
 
Applications of the Reverse Engineering Language REIL
Applications of the Reverse Engineering Language REILApplications of the Reverse Engineering Language REIL
Applications of the Reverse Engineering Language REIL
zynamics GmbH
 
Hd2
Hd2Hd2
Analysis of a Modified RC4
Analysis of a Modified RC4 Analysis of a Modified RC4
Analysis of a Modified RC4
Tharindu Weerasinghe
 
Fast, Private and Verifiable: Server-aided Approximate Similarity Computation...
Fast, Private and Verifiable: Server-aided Approximate Similarity Computation...Fast, Private and Verifiable: Server-aided Approximate Similarity Computation...
Fast, Private and Verifiable: Server-aided Approximate Similarity Computation...
Mateus S. H. Cruz
 
Cilk - An Efficient Multithreaded Runtime System
Cilk - An Efficient Multithreaded Runtime SystemCilk - An Efficient Multithreaded Runtime System
Cilk - An Efficient Multithreaded Runtime System
Shareek Ahamed
 
R intro
R introR intro
R intro
Vishakh Nair
 
Capturing Monotonic Components from Input Patterns
Capturing Monotonic Components from Input PatternsCapturing Monotonic Components from Input Patterns
Capturing Monotonic Components from Input Patterns
ijeukens
 
Deep Learning for Machine Translation, by Satoshi Enoue, SYSTRAN
Deep Learning for Machine Translation, by Satoshi Enoue, SYSTRANDeep Learning for Machine Translation, by Satoshi Enoue, SYSTRAN
Deep Learning for Machine Translation, by Satoshi Enoue, SYSTRAN
TAUS - The Language Data Network
 
CNIT 141: 10. RSA
CNIT 141: 10. RSACNIT 141: 10. RSA
CNIT 141: 10. RSA
Sam Bowne
 
TensorFlow Technology
TensorFlow TechnologyTensorFlow Technology
TensorFlow Technology
narayan dudhe
 
Moving to neural machine translation at google - gopro-meetup
Moving to neural machine translation at google  - gopro-meetupMoving to neural machine translation at google  - gopro-meetup
Moving to neural machine translation at google - gopro-meetup
Chester Chen
 
QVT Traceability: What does it really mean?
QVT Traceability: What does it really mean?QVT Traceability: What does it really mean?
QVT Traceability: What does it really mean?
Edward Willink
 
2017:12:06 acl読み会"Learning attention for historical text normalization by lea...
2017:12:06 acl読み会"Learning attention for historical text normalization by lea...2017:12:06 acl読み会"Learning attention for historical text normalization by lea...
2017:12:06 acl読み会"Learning attention for historical text normalization by lea...
ayaha osaki
 
Realizing Fine-Grained and Flexible Access Control to Outsourced Data with At...
Realizing Fine-Grained and Flexible Access Control to Outsourced Data with At...Realizing Fine-Grained and Flexible Access Control to Outsourced Data with At...
Realizing Fine-Grained and Flexible Access Control to Outsourced Data with At...
Mateus S. H. Cruz
 
Erlang For Five Nines
Erlang For Five NinesErlang For Five Nines
Erlang For Five Nines
Barcamp Cork
 
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Minh Pham
 
Introduction to OpenSees by Frank McKenna
Introduction to OpenSees by Frank McKennaIntroduction to OpenSees by Frank McKenna
Introduction to OpenSees by Frank McKenna
openseesdays
 

What's hot (20)

[Paper Reading] Attention is All You Need
[Paper Reading] Attention is All You Need[Paper Reading] Attention is All You Need
[Paper Reading] Attention is All You Need
 
CNIT 126: 13: Data Encoding
CNIT 126: 13: Data EncodingCNIT 126: 13: Data Encoding
CNIT 126: 13: Data Encoding
 
Decision Making & Loops
Decision Making & LoopsDecision Making & Loops
Decision Making & Loops
 
Applications of the Reverse Engineering Language REIL
Applications of the Reverse Engineering Language REILApplications of the Reverse Engineering Language REIL
Applications of the Reverse Engineering Language REIL
 
Hd2
Hd2Hd2
Hd2
 
Analysis of a Modified RC4
Analysis of a Modified RC4 Analysis of a Modified RC4
Analysis of a Modified RC4
 
Fast, Private and Verifiable: Server-aided Approximate Similarity Computation...
Fast, Private and Verifiable: Server-aided Approximate Similarity Computation...Fast, Private and Verifiable: Server-aided Approximate Similarity Computation...
Fast, Private and Verifiable: Server-aided Approximate Similarity Computation...
 
Cilk - An Efficient Multithreaded Runtime System
Cilk - An Efficient Multithreaded Runtime SystemCilk - An Efficient Multithreaded Runtime System
Cilk - An Efficient Multithreaded Runtime System
 
R intro
R introR intro
R intro
 
Capturing Monotonic Components from Input Patterns
Capturing Monotonic Components from Input PatternsCapturing Monotonic Components from Input Patterns
Capturing Monotonic Components from Input Patterns
 
Deep Learning for Machine Translation, by Satoshi Enoue, SYSTRAN
Deep Learning for Machine Translation, by Satoshi Enoue, SYSTRANDeep Learning for Machine Translation, by Satoshi Enoue, SYSTRAN
Deep Learning for Machine Translation, by Satoshi Enoue, SYSTRAN
 
CNIT 141: 10. RSA
CNIT 141: 10. RSACNIT 141: 10. RSA
CNIT 141: 10. RSA
 
TensorFlow Technology
TensorFlow TechnologyTensorFlow Technology
TensorFlow Technology
 
Moving to neural machine translation at google - gopro-meetup
Moving to neural machine translation at google  - gopro-meetupMoving to neural machine translation at google  - gopro-meetup
Moving to neural machine translation at google - gopro-meetup
 
QVT Traceability: What does it really mean?
QVT Traceability: What does it really mean?QVT Traceability: What does it really mean?
QVT Traceability: What does it really mean?
 
2017:12:06 acl読み会"Learning attention for historical text normalization by lea...
2017:12:06 acl読み会"Learning attention for historical text normalization by lea...2017:12:06 acl読み会"Learning attention for historical text normalization by lea...
2017:12:06 acl読み会"Learning attention for historical text normalization by lea...
 
Realizing Fine-Grained and Flexible Access Control to Outsourced Data with At...
Realizing Fine-Grained and Flexible Access Control to Outsourced Data with At...Realizing Fine-Grained and Flexible Access Control to Outsourced Data with At...
Realizing Fine-Grained and Flexible Access Control to Outsourced Data with At...
 
Erlang For Five Nines
Erlang For Five NinesErlang For Five Nines
Erlang For Five Nines
 
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
 
Introduction to OpenSees by Frank McKenna
Introduction to OpenSees by Frank McKennaIntroduction to OpenSees by Frank McKenna
Introduction to OpenSees by Frank McKenna
 

Similar to [論文紹介]A syntactic neural model for general purpose code generation

Trans coder
Trans coderTrans coder
Trans coder
PriyaM781673
 
Natural Language to Visualization by Neural Machine Translation
Natural Language to Visualization by Neural Machine TranslationNatural Language to Visualization by Neural Machine Translation
Natural Language to Visualization by Neural Machine Translation
ivaderivader
 
Scaling up genomic analysis with ADAM
Scaling up genomic analysis with ADAMScaling up genomic analysis with ADAM
Scaling up genomic analysis with ADAM
fnothaft
 
Natural Language Interface to Knowledge Graph
Natural Language Interface to Knowledge GraphNatural Language Interface to Knowledge Graph
Natural Language Interface to Knowledge Graph
Vaticle
 
4-BlockCipher-DES-CEN451-BSE-Spring2022-17042022-104521am.pdf
4-BlockCipher-DES-CEN451-BSE-Spring2022-17042022-104521am.pdf4-BlockCipher-DES-CEN451-BSE-Spring2022-17042022-104521am.pdf
4-BlockCipher-DES-CEN451-BSE-Spring2022-17042022-104521am.pdf
NAWAZURREHMANAWAN
 
"Source Code Abstracts Classification Using CNN", Vadim Markovtsev, Lead Soft...
"Source Code Abstracts Classification Using CNN", Vadim Markovtsev, Lead Soft..."Source Code Abstracts Classification Using CNN", Vadim Markovtsev, Lead Soft...
"Source Code Abstracts Classification Using CNN", Vadim Markovtsev, Lead Soft...
Dataconomy Media
 
Chord
ChordChord
Chord
Sutha31
 
From Hours to Minutes: The Journey of Optimizing Mask-RCNN and BERT Using MXNet
From Hours to Minutes: The Journey of Optimizing Mask-RCNN and BERT Using MXNetFrom Hours to Minutes: The Journey of Optimizing Mask-RCNN and BERT Using MXNet
From Hours to Minutes: The Journey of Optimizing Mask-RCNN and BERT Using MXNet
Eric Haibin Lin
 
Probabilistic data structures. Part 2. Cardinality
Probabilistic data structures. Part 2. CardinalityProbabilistic data structures. Part 2. Cardinality
Probabilistic data structures. Part 2. Cardinality
Andrii Gakhov
 
Presentation Slides - Genetic algorithm based key generation for fully homomo...
Presentation Slides - Genetic algorithm based key generation for fully homomo...Presentation Slides - Genetic algorithm based key generation for fully homomo...
Presentation Slides - Genetic algorithm based key generation for fully homomo...
MajedahAlkharji
 
5_RNN_LSTM.pdf
5_RNN_LSTM.pdf5_RNN_LSTM.pdf
5_RNN_LSTM.pdf
FEG
 
Recurrent Neural Networks, LSTM and GRU
Recurrent Neural Networks, LSTM and GRURecurrent Neural Networks, LSTM and GRU
Recurrent Neural Networks, LSTM and GRU
ananth
 
Kernel Recipes 2018 - 10 years of automated evolution in the Linux kernel - J...
Kernel Recipes 2018 - 10 years of automated evolution in the Linux kernel - J...Kernel Recipes 2018 - 10 years of automated evolution in the Linux kernel - J...
Kernel Recipes 2018 - 10 years of automated evolution in the Linux kernel - J...
Anne Nicolas
 
RNN & LSTM: Neural Network for Sequential Data
RNN & LSTM: Neural Network for Sequential DataRNN & LSTM: Neural Network for Sequential Data
RNN & LSTM: Neural Network for Sequential Data
Yao-Chieh Hu
 
Triantafyllia Voulibasi
Triantafyllia VoulibasiTriantafyllia Voulibasi
Triantafyllia Voulibasi
ISSEL
 
Scalable up genomic analysis with ADAM
Scalable up genomic analysis with ADAMScalable up genomic analysis with ADAM
Scalable up genomic analysis with ADAM
fnothaft
 
Introduction
IntroductionIntroduction
PD_Tcl_Examples
PD_Tcl_ExamplesPD_Tcl_Examples
PD_Tcl_Examples
Lee Johnson
 
ICSE20_Tao_slides.pptx
ICSE20_Tao_slides.pptxICSE20_Tao_slides.pptx
ICSE20_Tao_slides.pptx
DeepaGupta205807
 
Parallel Distributed Deep Learning on HPCC Systems
Parallel Distributed Deep Learning on HPCC SystemsParallel Distributed Deep Learning on HPCC Systems
Parallel Distributed Deep Learning on HPCC Systems
HPCC Systems
 

Similar to [論文紹介]A syntactic neural model for general purpose code generation (20)

Trans coder
Trans coderTrans coder
Trans coder
 
Natural Language to Visualization by Neural Machine Translation
Natural Language to Visualization by Neural Machine TranslationNatural Language to Visualization by Neural Machine Translation
Natural Language to Visualization by Neural Machine Translation
 
Scaling up genomic analysis with ADAM
Scaling up genomic analysis with ADAMScaling up genomic analysis with ADAM
Scaling up genomic analysis with ADAM
 
Natural Language Interface to Knowledge Graph
Natural Language Interface to Knowledge GraphNatural Language Interface to Knowledge Graph
Natural Language Interface to Knowledge Graph
 
4-BlockCipher-DES-CEN451-BSE-Spring2022-17042022-104521am.pdf
4-BlockCipher-DES-CEN451-BSE-Spring2022-17042022-104521am.pdf4-BlockCipher-DES-CEN451-BSE-Spring2022-17042022-104521am.pdf
4-BlockCipher-DES-CEN451-BSE-Spring2022-17042022-104521am.pdf
 
"Source Code Abstracts Classification Using CNN", Vadim Markovtsev, Lead Soft...
"Source Code Abstracts Classification Using CNN", Vadim Markovtsev, Lead Soft..."Source Code Abstracts Classification Using CNN", Vadim Markovtsev, Lead Soft...
"Source Code Abstracts Classification Using CNN", Vadim Markovtsev, Lead Soft...
 
Chord
ChordChord
Chord
 
From Hours to Minutes: The Journey of Optimizing Mask-RCNN and BERT Using MXNet
From Hours to Minutes: The Journey of Optimizing Mask-RCNN and BERT Using MXNetFrom Hours to Minutes: The Journey of Optimizing Mask-RCNN and BERT Using MXNet
From Hours to Minutes: The Journey of Optimizing Mask-RCNN and BERT Using MXNet
 
Probabilistic data structures. Part 2. Cardinality
Probabilistic data structures. Part 2. CardinalityProbabilistic data structures. Part 2. Cardinality
Probabilistic data structures. Part 2. Cardinality
 
Presentation Slides - Genetic algorithm based key generation for fully homomo...
Presentation Slides - Genetic algorithm based key generation for fully homomo...Presentation Slides - Genetic algorithm based key generation for fully homomo...
Presentation Slides - Genetic algorithm based key generation for fully homomo...
 
5_RNN_LSTM.pdf
5_RNN_LSTM.pdf5_RNN_LSTM.pdf
5_RNN_LSTM.pdf
 
Recurrent Neural Networks, LSTM and GRU
Recurrent Neural Networks, LSTM and GRURecurrent Neural Networks, LSTM and GRU
Recurrent Neural Networks, LSTM and GRU
 
Kernel Recipes 2018 - 10 years of automated evolution in the Linux kernel - J...
Kernel Recipes 2018 - 10 years of automated evolution in the Linux kernel - J...Kernel Recipes 2018 - 10 years of automated evolution in the Linux kernel - J...
Kernel Recipes 2018 - 10 years of automated evolution in the Linux kernel - J...
 
RNN & LSTM: Neural Network for Sequential Data
RNN & LSTM: Neural Network for Sequential DataRNN & LSTM: Neural Network for Sequential Data
RNN & LSTM: Neural Network for Sequential Data
 
Triantafyllia Voulibasi
Triantafyllia VoulibasiTriantafyllia Voulibasi
Triantafyllia Voulibasi
 
Scalable up genomic analysis with ADAM
Scalable up genomic analysis with ADAMScalable up genomic analysis with ADAM
Scalable up genomic analysis with ADAM
 
Introduction
IntroductionIntroduction
Introduction
 
PD_Tcl_Examples
PD_Tcl_ExamplesPD_Tcl_Examples
PD_Tcl_Examples
 
ICSE20_Tao_slides.pptx
ICSE20_Tao_slides.pptxICSE20_Tao_slides.pptx
ICSE20_Tao_slides.pptx
 
Parallel Distributed Deep Learning on HPCC Systems
Parallel Distributed Deep Learning on HPCC SystemsParallel Distributed Deep Learning on HPCC Systems
Parallel Distributed Deep Learning on HPCC Systems
 

Recently uploaded

Anti-Universe And Emergent Gravity and the Dark Universe
Anti-Universe And Emergent Gravity and the Dark UniverseAnti-Universe And Emergent Gravity and the Dark Universe
Anti-Universe And Emergent Gravity and the Dark Universe
Sérgio Sacani
 
Farming systems analysis: what have we learnt?.pptx
Farming systems analysis: what have we learnt?.pptxFarming systems analysis: what have we learnt?.pptx
Farming systems analysis: what have we learnt?.pptx
Frédéric Baudron
 
Randomised Optimisation Algorithms in DAPHNE
Randomised Optimisation Algorithms in DAPHNERandomised Optimisation Algorithms in DAPHNE
Randomised Optimisation Algorithms in DAPHNE
University of Maribor
 
Mending Clothing to Support Sustainable Fashion_CIMaR 2024.pdf
Mending Clothing to Support Sustainable Fashion_CIMaR 2024.pdfMending Clothing to Support Sustainable Fashion_CIMaR 2024.pdf
Mending Clothing to Support Sustainable Fashion_CIMaR 2024.pdf
Selcen Ozturkcan
 
IMPORTANCE OF ALGAE AND ITS BENIFITS.pptx
IMPORTANCE OF ALGAE  AND ITS BENIFITS.pptxIMPORTANCE OF ALGAE  AND ITS BENIFITS.pptx
IMPORTANCE OF ALGAE AND ITS BENIFITS.pptx
OmAle5
 
Methods of grain storage Structures in India.pdf
Methods of grain storage Structures in India.pdfMethods of grain storage Structures in India.pdf
Methods of grain storage Structures in India.pdf
PirithiRaju
 
Microbiology of Central Nervous System INFECTIONS.pdf
Microbiology of Central Nervous System INFECTIONS.pdfMicrobiology of Central Nervous System INFECTIONS.pdf
Microbiology of Central Nervous System INFECTIONS.pdf
sammy700571
 
Authoring a personal GPT for your research and practice: How we created the Q...
Authoring a personal GPT for your research and practice: How we created the Q...Authoring a personal GPT for your research and practice: How we created the Q...
Authoring a personal GPT for your research and practice: How we created the Q...
Leonel Morgado
 
Compexometric titration/Chelatorphy titration/chelating titration
Compexometric titration/Chelatorphy titration/chelating titrationCompexometric titration/Chelatorphy titration/chelating titration
Compexometric titration/Chelatorphy titration/chelating titration
Vandana Devesh Sharma
 
Physiology of Nervous System presentation.pptx
Physiology of Nervous System presentation.pptxPhysiology of Nervous System presentation.pptx
Physiology of Nervous System presentation.pptx
fatima132662
 
在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样
在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样
在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样
vluwdy49
 
Discovery of An Apparent Red, High-Velocity Type Ia Supernova at 𝐳 = 2.9 wi...
Discovery of An Apparent Red, High-Velocity Type Ia Supernova at  𝐳 = 2.9  wi...Discovery of An Apparent Red, High-Velocity Type Ia Supernova at  𝐳 = 2.9  wi...
Discovery of An Apparent Red, High-Velocity Type Ia Supernova at 𝐳 = 2.9 wi...
Sérgio Sacani
 
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Leonel Morgado
 
(June 12, 2024) Webinar: Development of PET theranostics targeting the molecu...
(June 12, 2024) Webinar: Development of PET theranostics targeting the molecu...(June 12, 2024) Webinar: Development of PET theranostics targeting the molecu...
(June 12, 2024) Webinar: Development of PET theranostics targeting the molecu...
Scintica Instrumentation
 
SDSS1335+0728: The awakening of a ∼ 106M⊙ black hole⋆
SDSS1335+0728: The awakening of a ∼ 106M⊙ black hole⋆SDSS1335+0728: The awakening of a ∼ 106M⊙ black hole⋆
SDSS1335+0728: The awakening of a ∼ 106M⊙ black hole⋆
Sérgio Sacani
 
Summary Of transcription and Translation.pdf
Summary Of transcription and Translation.pdfSummary Of transcription and Translation.pdf
Summary Of transcription and Translation.pdf
vadgavevedant86
 
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
PsychoTech Services
 
23PH301 - Optics - Optical Lenses.pptx
23PH301 - Optics  -  Optical Lenses.pptx23PH301 - Optics  -  Optical Lenses.pptx
23PH301 - Optics - Optical Lenses.pptx
RDhivya6
 
Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...
Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...
Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...
Creative-Biolabs
 
AJAY KUMAR NIET GreNo Guava Project File.pdf
AJAY KUMAR NIET GreNo Guava Project File.pdfAJAY KUMAR NIET GreNo Guava Project File.pdf
AJAY KUMAR NIET GreNo Guava Project File.pdf
AJAY KUMAR
 

Recently uploaded (20)

Anti-Universe And Emergent Gravity and the Dark Universe
Anti-Universe And Emergent Gravity and the Dark UniverseAnti-Universe And Emergent Gravity and the Dark Universe
Anti-Universe And Emergent Gravity and the Dark Universe
 
Farming systems analysis: what have we learnt?.pptx
Farming systems analysis: what have we learnt?.pptxFarming systems analysis: what have we learnt?.pptx
Farming systems analysis: what have we learnt?.pptx
 
Randomised Optimisation Algorithms in DAPHNE
Randomised Optimisation Algorithms in DAPHNERandomised Optimisation Algorithms in DAPHNE
Randomised Optimisation Algorithms in DAPHNE
 
Mending Clothing to Support Sustainable Fashion_CIMaR 2024.pdf
Mending Clothing to Support Sustainable Fashion_CIMaR 2024.pdfMending Clothing to Support Sustainable Fashion_CIMaR 2024.pdf
Mending Clothing to Support Sustainable Fashion_CIMaR 2024.pdf
 
IMPORTANCE OF ALGAE AND ITS BENIFITS.pptx
IMPORTANCE OF ALGAE  AND ITS BENIFITS.pptxIMPORTANCE OF ALGAE  AND ITS BENIFITS.pptx
IMPORTANCE OF ALGAE AND ITS BENIFITS.pptx
 
Methods of grain storage Structures in India.pdf
Methods of grain storage Structures in India.pdfMethods of grain storage Structures in India.pdf
Methods of grain storage Structures in India.pdf
 
Microbiology of Central Nervous System INFECTIONS.pdf
Microbiology of Central Nervous System INFECTIONS.pdfMicrobiology of Central Nervous System INFECTIONS.pdf
Microbiology of Central Nervous System INFECTIONS.pdf
 
Authoring a personal GPT for your research and practice: How we created the Q...
Authoring a personal GPT for your research and practice: How we created the Q...Authoring a personal GPT for your research and practice: How we created the Q...
Authoring a personal GPT for your research and practice: How we created the Q...
 
Compexometric titration/Chelatorphy titration/chelating titration
Compexometric titration/Chelatorphy titration/chelating titrationCompexometric titration/Chelatorphy titration/chelating titration
Compexometric titration/Chelatorphy titration/chelating titration
 
Physiology of Nervous System presentation.pptx
Physiology of Nervous System presentation.pptxPhysiology of Nervous System presentation.pptx
Physiology of Nervous System presentation.pptx
 
在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样
在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样
在线办理(salfor毕业证书)索尔福德大学毕业证毕业完成信一模一样
 
Discovery of An Apparent Red, High-Velocity Type Ia Supernova at 𝐳 = 2.9 wi...
Discovery of An Apparent Red, High-Velocity Type Ia Supernova at  𝐳 = 2.9  wi...Discovery of An Apparent Red, High-Velocity Type Ia Supernova at  𝐳 = 2.9  wi...
Discovery of An Apparent Red, High-Velocity Type Ia Supernova at 𝐳 = 2.9 wi...
 
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
 
(June 12, 2024) Webinar: Development of PET theranostics targeting the molecu...
(June 12, 2024) Webinar: Development of PET theranostics targeting the molecu...(June 12, 2024) Webinar: Development of PET theranostics targeting the molecu...
(June 12, 2024) Webinar: Development of PET theranostics targeting the molecu...
 
SDSS1335+0728: The awakening of a ∼ 106M⊙ black hole⋆
SDSS1335+0728: The awakening of a ∼ 106M⊙ black hole⋆SDSS1335+0728: The awakening of a ∼ 106M⊙ black hole⋆
SDSS1335+0728: The awakening of a ∼ 106M⊙ black hole⋆
 
Summary Of transcription and Translation.pdf
Summary Of transcription and Translation.pdfSummary Of transcription and Translation.pdf
Summary Of transcription and Translation.pdf
 
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
Sexuality - Issues, Attitude and Behaviour - Applied Social Psychology - Psyc...
 
23PH301 - Optics - Optical Lenses.pptx
23PH301 - Optics  -  Optical Lenses.pptx23PH301 - Optics  -  Optical Lenses.pptx
23PH301 - Optics - Optical Lenses.pptx
 
Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...
Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...
Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...
 
AJAY KUMAR NIET GreNo Guava Project File.pdf
AJAY KUMAR NIET GreNo Guava Project File.pdfAJAY KUMAR NIET GreNo Guava Project File.pdf
AJAY KUMAR NIET GreNo Guava Project File.pdf
 

[論文紹介]A syntactic neural model for general purpose code generation

  • 1. A Syntactic Neural Model for General-Purpose Code Generation Pengcheng Yin, Graham Neubig ACL2017 M1 Tomoya Ogata 1
  • 2. Abstract • Input • natural language descriptions • Output • source code written in a general-purpose programming language • Existing data-driven methods treat this problem as a language generation task without considering the underlying syntax of the target programming language • propose a novel neural architecture powered by a grammar model to explicitly capture the target syntax as prior knowledge 2
  • 3. Given an NL description x, our task is to generate the code snippet c in a modern PL based on the intent of x. We define a probabilistic grammar model of generating an AST y given x 𝑦 is then deterministically converted to the corresponding surface code an AST is generated by applying several production rules composed of a head node and multiple child nodes The Code Generation Problem 3
  • 4. Grammar Model • APPLYRULE Actions • GENTOKEN Actions 4
  • 5. APPLYRULE Actions • APPLYRULE chooses a rule from the subset that has a head matching the type of 𝑛 𝑓𝑡 • uses r to expand 𝑛 𝑓𝑡 by appending all child nodes specified by the selected production • When a variable terminal node is added to the derivation and becomes the frontier node, the grammar model then switches to GENTOKEN actions to populate the variable terminal with tokens 5
  • 6. GENTOKEN Actions Once we reach a frontier node 𝑛 𝑓𝑡 that corresponds to a variable type, GENTOKEN actions are used to fill this node with values. At each time step, GENTOKEN appends one terminal token to the current frontier variable node 6
  • 7. Tracking Generation States Action Embedding: at Context Vector: ct Parent Feeding: pt 7
  • 8. Parent Feeding • parent information 𝑝𝑡 from two sources • (1) the hidden state of parent action 𝑠 𝑝 𝑡 • (2) the embedding of parent action 𝑎 𝑝 𝑡 • The parent feeding schema enables the model to utilize the information of parent code segments to make more confident predictions 8
  • 9. Calculating Action Probabilities g(st): tanh(W・st+b) e(r): one-hot vector for rule r p(gen|・), p(copy|・): tanh(W・st+b) 9
  • 10. Training • Given a dataset of pairs of NL descriptions 𝑥𝑖 and code 𝑐𝑖 snippets • we parse 𝑐𝑖 into its AST 𝑦𝑖 • decompose 𝑦𝑖 into a sequence of oracle actions • The model is then optimized by maximizing the log- likelihood of the oracle action sequence 10
  • 11. Experimental Evaluation • Datasets • HEARTHSTONE (HS) dataset • a collection of Python classes that implement cards for the card game HearthStone • DJANGO dataset • a collection of lines of code from the Django web framework, each with a manually annotated NL description • IFTTT dataset • a domain- specific benchmark that provides an interesting side comparison • Metrics • accuracy • BLUE-4 11
  • 12. Experimental Evaluation • Preprocessing • Input are tokenized using NLTK • replacing quoted strings in the inputs with place holders • extract unary closures whose frequency is larger than a threshold • Configuration • node type embeddings: 64 • All other embeddings: 128 • RNN states: 256 • hidden layers: 50 12
  • 13. Model • Latent Predictor Network (LPN), a state-of-the-art sequence- to-sequence code generation model • SEQ2TREE, a neural semantic parsing model • NMT system using a standard encoder-decoder architecture with attention and unknown word replacement 13
  • 17. Error Analysis • randomly sampled and labeled 100 and 50 failed examples from DJANGO and HS, respectively • using different parameter names when defining a function • omitting (or adding) default values of parameters in function calls • DJANGO • 30%: the pointer network failed • 25%: the generated code only partially implemented the required functionality • 10%: malformed English inputs • 5%: preprocessing errors • 30%: could not be easily categorized into the above • HS • partial implementation errors 17
  • 18. Conclusion • This paper proposes a syntax-driven neural code generation approach that generates an abstract syntax tree by sequentially applying actions from a grammar model 18