The document presents an LSTM-based neural network architecture designed for model transformations, utilizing an encoder-decoder structure combined with attention mechanisms to enhance performance. It discusses the preprocessing and post-processing of model data to improve training efficacy, as well as preliminary results on correctness and performance metrics. Limitations such as the size and diversity of the training dataset, computational constraints, and generalization challenges are also addressed.

1. An LSTM-Based Neural Network
Architecture for
Model Transformations
Loli Burgueño1,2, Jordi Cabot1, Sébastien Gérard2
1 Open University of Catalonia, Barcelona, Spain
2 CEA LIST, Paris, France
MODELS’19
Munich, September 20th, 2019

2. 2

3. Artificial Intelligence
• Machine Learning - Supervised Learning:
Input
Output
Training Transforming
ML Input OutputML
Artificial Intelligence
Machine Learning
Artificial Neural Networks
Deep Artificial
Neural Networks
3

4. Artificial Neural Networks
• Graph structure: Neurons + directed weighted connections
• Neurons are mathematical functions
• Connections have associated weights
• Adjusted during the learning process to increase/decrease the strength of the
connection
4

5. Artificial Neural Networks
• The learning process basically means to find the right weights
• Supervised learning methods. Training phase:
• Example input-output pairs are used (Dataset)
Dataset
Training Validation Test
5

6. Artificial Neural Networks
• Combine two LSTM for better results
• Avoids fixed size input and output constraints
• MTs ≈ sequence-to-sequence arch
6

7. Architecture
• Encoder-decoder architecture
+
• Long short-term memory neural networks
Encoder
LSTM network
Decoder
LSTM network
InputModel
OutputModel
7

8. Architecture
• Sequence-to-Sequence transformations
• Tree-to-tree transformations
• Input layer to embed the input tree to a numeric vector
+
• Output layer to obtain the output model from the numeric vectors produced by the decoderInputTree
EmbeddingLayer
Encoder
LSTM network
OutputTree
ExtractionLayer
Decoder
LSTM network
InputModel
OutputModel
8

9. • Attention mechanism
• To pay more attention (remember better) to specific parts
• It automatically detects to which parts are more important
Architecture
InputTree
EmbeddingLayer
Encoder
LSTM network
OutputTree
ExtractionLayer
Decoder
LSTM network
AttentionLayer
InputModel
OutputModel
9

10. • Pre- and post-processing required to…
• represent models as trees
• reduce the size of the training dataset by using a canonical form
• rename variables to avoid the “dictionary problem”
Model pre- and post-processing
InputModel
(preprocessed)
InputTree
EmbeddingLayer
Encoder
LSTM network
OutputTree
ExtractionLayer
OutputModel
(non-postprocessed)
Decoder
LSTM network
AttentionLayer
InputModel
OutputModel
Preprocessing
Postprocessing
10

11. Preliminary results
• Class to Relational
11

12. Preliminary results
• Correctness
• Measured through the accuracy and validation loss
12

13. Preliminary results
• Performance
1. How long does it take for the
training phase to complete?
13

14. Preliminary results
• Performance
1. How long does it take for the
training phase to complete?
2. How long it takes to transform an
input model when the network is
trained?
14

15. Limitations/Discussion
• Size of the training dataset
• Diversity in the training set
• Computational limitations of ANNs
• i.e., mathematical operations
• Generalization problem
• predicting output solutions for input models very different from the training
distribution it has learn from
• Social acceptance
15