The document describes a study that analyzed code smells in popular Python projects for reinforcement learning. The study involved:
1) Filtering the 20 most popular RL repositories on GitHub that use Python.
2) Performing static analysis on the code to detect 8 different code smells based on predefined metrics and thresholds.
3) Finding that certain code smells like long methods and long classes were highly prevalent in the analyzed projects.
45min talk given at LondonR March 2014 Meetup.
The presentation describes how one might go about an insights-driven data science project using the R language and packages, using an open source dataset.
Customer Clustering for Retailer MarketingJonathan Sedar
This was a 90 min talk given to the Dublin R user group in Nov 2013. It describes how one might go about a data analysis project using the R language and packages, using an open source dataset.
Our fall 12-Week Data Science bootcamp starts on Sept 21st,2015. Apply now to get a spot!
If you are hiring Data Scientists, call us at (1)888-752-7585 or reach info@nycdatascience.com to share your openings and set up interviews with our excellent students.
---------------------------------------------------------------
Come join our meet-up and learn how easily you can use R for advanced Machine learning. In this meet-up, we will demonstrate how to understand and use Xgboost for Kaggle competition. Tong is in Canada and will do remote session with us through google hangout.
---------------------------------------------------------------
Speaker Bio:
Tong is a data scientist in Supstat Inc and also a master students of Data Mining. He has been an active R programmer and developer for 5 years. He is the author of the R package of XGBoost, one of the most popular and contest-winning tools on kaggle.com nowadays.
Pre-requisite(if any): R /Calculus
Preparation: A laptop with R installed. Windows users might need to have RTools installed as well.
Agenda:
Introduction of Xgboost
Real World Application
Model Specification
Parameter Introduction
Advanced Features
Kaggle Winning Solution
Event arrangement:
6:45pm Doors open. Come early to network, grab a beer and settle in.
7:00-9:00pm XgBoost Demo
Reference:
https://github.com/dmlc/xgboost
45min talk given at LondonR March 2014 Meetup.
The presentation describes how one might go about an insights-driven data science project using the R language and packages, using an open source dataset.
Customer Clustering for Retailer MarketingJonathan Sedar
This was a 90 min talk given to the Dublin R user group in Nov 2013. It describes how one might go about a data analysis project using the R language and packages, using an open source dataset.
Our fall 12-Week Data Science bootcamp starts on Sept 21st,2015. Apply now to get a spot!
If you are hiring Data Scientists, call us at (1)888-752-7585 or reach info@nycdatascience.com to share your openings and set up interviews with our excellent students.
---------------------------------------------------------------
Come join our meet-up and learn how easily you can use R for advanced Machine learning. In this meet-up, we will demonstrate how to understand and use Xgboost for Kaggle competition. Tong is in Canada and will do remote session with us through google hangout.
---------------------------------------------------------------
Speaker Bio:
Tong is a data scientist in Supstat Inc and also a master students of Data Mining. He has been an active R programmer and developer for 5 years. He is the author of the R package of XGBoost, one of the most popular and contest-winning tools on kaggle.com nowadays.
Pre-requisite(if any): R /Calculus
Preparation: A laptop with R installed. Windows users might need to have RTools installed as well.
Agenda:
Introduction of Xgboost
Real World Application
Model Specification
Parameter Introduction
Advanced Features
Kaggle Winning Solution
Event arrangement:
6:45pm Doors open. Come early to network, grab a beer and settle in.
7:00-9:00pm XgBoost Demo
Reference:
https://github.com/dmlc/xgboost
Data Exploration (EDA)
- Relationship between variables
- Check for
- Multi-co linearity
- Distribution of variables
- Presence of outliers and its treatment
- Statistical significance of variables
- Class imbalance and its treatment
Feature Engineering
- Whether any transformations required
- Scaling the data
- Feature selection
- Dimensionality reduction
Assumptions
- Check for the assumptions to be satisfied for each of the models in
- Regression – SLR, Multiple Linear Regression, Logistic Regression
- Classification – Decision Tree, Random Forest, SVM, Bagged and boosted models
- Clustering – PCA (multi-co linearity), K-Means (presence of outliers, scaling, conversion to
numerical, etc.)
----------------------------- Interim Presentation Checkpoint----------------------------------------------------------
Model building
- Split the data to train and test.
- Start with a simple model which satisfies all the above assumptions based on your dataset.
- Check for bias and variance errors.
- To improve the performance, try cross-validation, ensemble models, hyperparameter
tuning, grid search
Evaluation of model
- Regression – RMSE, R-Squared value,
- Classification – Classification report with precision, recall, F1-score, Support, AUC, etc.
- Clustering – Inertia value
- Comparison of different models built and discussion of the same
- Time taken for the inferences/ predictions
Business Recommendations & Future enhancements
- How to improve data collection, processing, and model accuracy?
- Commercial value/ Social value / Research value
- Recommendations based on insights
----------------------------- Final Presentation Checkpoint----------------------------------------------------------
Dashboard
- EDA – Correlation matrix, pair plots, box blots, distribution plots
- Model
- Model Parameters
- Visualization of performance of the model with varying parameters
- Visualization of model Metrics
- Testing outcome
- Failure cases and explanation for the same
- Most successful and obvious cases
- Border cases
Naive application of Machine Learning to Software DevelopmentAndriy Khavryuchenko
Naive application of Machine Learning to Software Development: get tickets from Django trac ticket tracking system and try to predict how long it will take to close the ticket.
Facts that developers aren't putting RIGHT information into their tracking systems :)
Tong is a data scientist in Supstat Inc and also a master students of Data Mining. He has been an active R programmer and developer for 5 years. He is the author of the R package of XGBoost, one of the most popular and contest-winning tools on kaggle.com nowadays.
Agenda:
Introduction of Xgboost
Real World Application
Model Specification
Parameter Introduction
Advanced Features
Kaggle Winning Solution
The `Right way` of testing and refactoring is so ambiguous to put in the real world, Especially, Business situation.
Let's look in deep the testing and refactoring at the perspective of a developer who works for a business situation.
Learning of Machine Learning Algorithms can be a tiresome and quite boring unless you can apply some of the techniques and methods into some Real-World problems and Data. Here in this document, I've used Basic Machine Learning Algorithms like Linear Regression, Random Forest, K-Means Clustering etc. using Python (v2.7) to predict the future sale of products based on the data hosted by Kaggle Competitions (Predicting Future Sale).
Going deeper into Algorithms and implementations, the next step would be to use some more advanced techniques like an application of Model Boosting Algorithms such as XGBoost, or even application of Depp Learning to predict the score more accurately.
By this basic implementation, a Score of ~1.37 (using RMSE Scoring Method) was achieved at Kaggle.
Numerical tour in the Python eco-system: Python, NumPy, scikit-learnArnaud Joly
We first present the Python programming language and the NumPy package for scientific computing. Then, we devise a digit recognition system highlighting the scikit-learn package.
Feature Engineering - Getting most out of data for predictive modelsGabriel Moreira
How should data be preprocessed for use in machine learning algorithms? How to identify the most predictive attributes of a dataset? What features can generate to improve the accuracy of a model?
Feature Engineering is the process of extracting and selecting, from raw data, features that can be used effectively in predictive models. As the quality of the features greatly influences the quality of the results, knowing the main techniques and pitfalls will help you to succeed in the use of machine learning in your projects.
In this talk, we will present methods and techniques that allow us to extract the maximum potential of the features of a dataset, increasing flexibility, simplicity and accuracy of the models. The analysis of the distribution of features and their correlations, the transformation of numeric attributes (such as scaling, normalization, log-based transformation, binning), categorical attributes (such as one-hot encoding, feature hashing, Temporal (date / time), and free-text attributes (text vectorization, topic modeling).
Python, Python, Scikit-learn, and Spark SQL examples will be presented and how to use domain knowledge and intuition to select and generate features relevant to predictive models.
Hello- I hope you are doing well- I am doing my project- which is Rans (1).pdfIan0J2Bondo
Hello, I hope you are doing well,
I am doing my project, which is Ransomware attack detection using Deep learning CNNs
I got a result of 0.93, which is not good enough, I have tried to improve my accuracy, but I gave
up. Could you modify the code feature selection or pre-processing to get a higher result? Even
for that result, I have to wait a long time to get it because I increased the epoch, and could you
explain the added part?
import os
import numpy as np
import pandas as pd
import keras
import tensorflow as tf
# os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID" # see issue #152
# os.environ['CUDA_VISIBLE_DEVICES'] = '-1'
from keras.preprocessing.image import ImageDataGenerator #, load_img
from keras.utils import to_categorical
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
import random
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D, Dropout, Flatten, Dense, Activation,
BatchNormalization
from keras.callbacks import EarlyStopping, ReduceLROnPlateau
print(os.listdir("D:\RansomSecondApproach\Ransomware_Detection_using
_CNN\MixImages"))
# Define Constants
FAST_RUN = False
IMAGE_WIDTH=128 # maybe 256
IMAGE_HEIGHT=128 # maybe 256
IMAGE_SIZE=(IMAGE_WIDTH, IMAGE_HEIGHT)
IMAGE_CHANNELS=3 # maybe not need
physical_devices = tf.config.experimental.list_physical_devices('GPU')
print(physical_devices)
if physical_devices:
tf.config.experimental.set_memory_growth(physical_devices[0], True)
# Prepare Traning Data
filenames = os.listdir("D:\RansomSecondApproach\Ransomware_Detection_using
_CNN\MixImages")
categories = []
for filename in filenames:
category = filename.split('l')[0]
if category == 'image_benign_':
categories.append(0)
else:
categories.append(1)
df = pd.DataFrame({
'filename': filenames,
'category': categories
})
print(df.head())
print(df.tail())
# in collab it will work df['category'].value_counts().plot.bar()
# See sample image
# sample = random.choice(filenames)
# image = load_img("D:\Ransomware_Detection_using _CNN\MixImages"+sample)
# plt.imshow(image)
# in collab it will work df['category'].value_counts().plot.bar()
# Build Model
#===============================================================================================================
# Testing
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(IMAGE_WIDTH,
IMAGE_HEIGHT, IMAGE_CHANNELS)))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(128, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(256, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(512, activation='re.
I am Irene M. I am a C++ Homework Expert at cpphomeworkhelp.com. I hold a Masters in Programming from from California, USA. I have been helping students with their homework for the past 10 years. I solve homework related to C++.
Visit cpphomeworkhelp.com or email info@cpphomeworkhelp.com. You can also call on +1 678 648 4277 for any assistance with C++ Homework.
As part of our team's enrollment for Data Science Super Specialization course under UpX Academy, we submitted many projects for our final assessments, one of them was Telecom Churn Analysis Model.
The input data was provided by UpX academy and language we used is R. As part of the project, our main objective was :-
-> To predict Customer Churn.
-> To Highlight the main variables/factors influencing Customer Churn.
-> To Use various ML algorithms to build prediction models, evaluate the accuracy and performance of these models.
-> Finding out the best model for our business case & providing executive Summary.
To address the mentioned business problem, we tried to follow a thorough approach. We did a detailed level Exploratory Data Analysis which consists of various Box Plots, Bar Plots etc..
Further we tried our best to build as many Classification models possible which fits our business case (Logistic Regression/kNN/Decision Trees/Random Forest/SVM) and also tried to touch Cox Hazard Survival analysis Model. Later for every model we tried to boost their performances by applying various performance tuning techniques.
As we all are still into our learning mode w.r.t these concepts & starting new, please feel free to provide feedback on our work. Any suggestions are most welcome... :)
Thanks!!
An expressive and modular layer activation mechanism for Context-Oriented Pro...Universidad de los Andes
Abstract: Context. There is a trend in the software industry towards building systems that dynamically adapt their behavior in response to their surrounding environment, given the proliferation of various technological devices, such as notebooks, smartphones, and wearables, capable of capturing their execution context. Context-oriented Programming (COP) allows developers to use layer abstractions to adapt software behavior to the context. A layer is associated with a context and can be dynamically activated in direct response to gathered information from its surrounding execution environment. However, most existing layer activation mechanisms have been tailored specifically to address a particular concern; implying that developers need to tweak layer definitions in contortive ways or create new specialized activation mechanisms altogether if their specific needs are not supported. Objective. Complementing ideas to expressively declare activation mechanism models with interfaces that define conditionals of activation mechanisms modularly, this paper proposes an Expressive and Modular Activation mechanism, named EMA. Method. To propose EMA, we analyze existing activation mechanisms in COP regarding activation features and scope strategies. After, we propose the design of EMA and validate it with a case study discussion. Results. Using a concrete JavaScript implementation of EMA, named EMAjs, we can implement two Web applications: a smartphone application as an example to illustrate EMAjs in action, and an application of home automation to discuss and compare our proposal. Conclusion. Our proposed mechanism allows developers to instantiate different activation scope strategies and interfaces to decouple the declaration of activation mechanism conditionals from the base code.
Paper presentation: https://youtu.be/76RgVB6JihQ?si=-uylVc4Xnqnh4zbl
Paper presented at the 2023 Formal Techniques for Java-like Programs.
Abstract: Static analyses, as points-to analysis, are useful to determine and assure different properties about a program, such as security or type safety. While existing analyses are effective in programs restricted to static features, precision declines in the presence of dynamic language features, and even further when the system behavior changes dynamically. As a consequence, improved points-to sets algorithms taking into account such language features and uses are required. In this paper, we present and extension of the point-to sets analysis to incorporate the language abstractions introduced by context-oriented programming adding the capability for programs to adapt their behavior dynamically to the system’s execution context. To do this, we extend WALA to detect the context-oriented language abstractions, and their representation within the system, to capture the dynamic behavior, in the particular case of the Context Traits JavaScript language extension. To prove the effectiveness of our extension, we evaluate the precision of the points-to set analysis with respect to the state of the art, over four context-oriented programs taken from the literature.
More Related Content
Similar to [CAIN'23] Prevalence of Code Smells in Reinforcement Learning Projects
Data Exploration (EDA)
- Relationship between variables
- Check for
- Multi-co linearity
- Distribution of variables
- Presence of outliers and its treatment
- Statistical significance of variables
- Class imbalance and its treatment
Feature Engineering
- Whether any transformations required
- Scaling the data
- Feature selection
- Dimensionality reduction
Assumptions
- Check for the assumptions to be satisfied for each of the models in
- Regression – SLR, Multiple Linear Regression, Logistic Regression
- Classification – Decision Tree, Random Forest, SVM, Bagged and boosted models
- Clustering – PCA (multi-co linearity), K-Means (presence of outliers, scaling, conversion to
numerical, etc.)
----------------------------- Interim Presentation Checkpoint----------------------------------------------------------
Model building
- Split the data to train and test.
- Start with a simple model which satisfies all the above assumptions based on your dataset.
- Check for bias and variance errors.
- To improve the performance, try cross-validation, ensemble models, hyperparameter
tuning, grid search
Evaluation of model
- Regression – RMSE, R-Squared value,
- Classification – Classification report with precision, recall, F1-score, Support, AUC, etc.
- Clustering – Inertia value
- Comparison of different models built and discussion of the same
- Time taken for the inferences/ predictions
Business Recommendations & Future enhancements
- How to improve data collection, processing, and model accuracy?
- Commercial value/ Social value / Research value
- Recommendations based on insights
----------------------------- Final Presentation Checkpoint----------------------------------------------------------
Dashboard
- EDA – Correlation matrix, pair plots, box blots, distribution plots
- Model
- Model Parameters
- Visualization of performance of the model with varying parameters
- Visualization of model Metrics
- Testing outcome
- Failure cases and explanation for the same
- Most successful and obvious cases
- Border cases
Naive application of Machine Learning to Software DevelopmentAndriy Khavryuchenko
Naive application of Machine Learning to Software Development: get tickets from Django trac ticket tracking system and try to predict how long it will take to close the ticket.
Facts that developers aren't putting RIGHT information into their tracking systems :)
Tong is a data scientist in Supstat Inc and also a master students of Data Mining. He has been an active R programmer and developer for 5 years. He is the author of the R package of XGBoost, one of the most popular and contest-winning tools on kaggle.com nowadays.
Agenda:
Introduction of Xgboost
Real World Application
Model Specification
Parameter Introduction
Advanced Features
Kaggle Winning Solution
The `Right way` of testing and refactoring is so ambiguous to put in the real world, Especially, Business situation.
Let's look in deep the testing and refactoring at the perspective of a developer who works for a business situation.
Learning of Machine Learning Algorithms can be a tiresome and quite boring unless you can apply some of the techniques and methods into some Real-World problems and Data. Here in this document, I've used Basic Machine Learning Algorithms like Linear Regression, Random Forest, K-Means Clustering etc. using Python (v2.7) to predict the future sale of products based on the data hosted by Kaggle Competitions (Predicting Future Sale).
Going deeper into Algorithms and implementations, the next step would be to use some more advanced techniques like an application of Model Boosting Algorithms such as XGBoost, or even application of Depp Learning to predict the score more accurately.
By this basic implementation, a Score of ~1.37 (using RMSE Scoring Method) was achieved at Kaggle.
Numerical tour in the Python eco-system: Python, NumPy, scikit-learnArnaud Joly
We first present the Python programming language and the NumPy package for scientific computing. Then, we devise a digit recognition system highlighting the scikit-learn package.
Feature Engineering - Getting most out of data for predictive modelsGabriel Moreira
How should data be preprocessed for use in machine learning algorithms? How to identify the most predictive attributes of a dataset? What features can generate to improve the accuracy of a model?
Feature Engineering is the process of extracting and selecting, from raw data, features that can be used effectively in predictive models. As the quality of the features greatly influences the quality of the results, knowing the main techniques and pitfalls will help you to succeed in the use of machine learning in your projects.
In this talk, we will present methods and techniques that allow us to extract the maximum potential of the features of a dataset, increasing flexibility, simplicity and accuracy of the models. The analysis of the distribution of features and their correlations, the transformation of numeric attributes (such as scaling, normalization, log-based transformation, binning), categorical attributes (such as one-hot encoding, feature hashing, Temporal (date / time), and free-text attributes (text vectorization, topic modeling).
Python, Python, Scikit-learn, and Spark SQL examples will be presented and how to use domain knowledge and intuition to select and generate features relevant to predictive models.
Hello- I hope you are doing well- I am doing my project- which is Rans (1).pdfIan0J2Bondo
Hello, I hope you are doing well,
I am doing my project, which is Ransomware attack detection using Deep learning CNNs
I got a result of 0.93, which is not good enough, I have tried to improve my accuracy, but I gave
up. Could you modify the code feature selection or pre-processing to get a higher result? Even
for that result, I have to wait a long time to get it because I increased the epoch, and could you
explain the added part?
import os
import numpy as np
import pandas as pd
import keras
import tensorflow as tf
# os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID" # see issue #152
# os.environ['CUDA_VISIBLE_DEVICES'] = '-1'
from keras.preprocessing.image import ImageDataGenerator #, load_img
from keras.utils import to_categorical
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
import random
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D, Dropout, Flatten, Dense, Activation,
BatchNormalization
from keras.callbacks import EarlyStopping, ReduceLROnPlateau
print(os.listdir("D:\RansomSecondApproach\Ransomware_Detection_using
_CNN\MixImages"))
# Define Constants
FAST_RUN = False
IMAGE_WIDTH=128 # maybe 256
IMAGE_HEIGHT=128 # maybe 256
IMAGE_SIZE=(IMAGE_WIDTH, IMAGE_HEIGHT)
IMAGE_CHANNELS=3 # maybe not need
physical_devices = tf.config.experimental.list_physical_devices('GPU')
print(physical_devices)
if physical_devices:
tf.config.experimental.set_memory_growth(physical_devices[0], True)
# Prepare Traning Data
filenames = os.listdir("D:\RansomSecondApproach\Ransomware_Detection_using
_CNN\MixImages")
categories = []
for filename in filenames:
category = filename.split('l')[0]
if category == 'image_benign_':
categories.append(0)
else:
categories.append(1)
df = pd.DataFrame({
'filename': filenames,
'category': categories
})
print(df.head())
print(df.tail())
# in collab it will work df['category'].value_counts().plot.bar()
# See sample image
# sample = random.choice(filenames)
# image = load_img("D:\Ransomware_Detection_using _CNN\MixImages"+sample)
# plt.imshow(image)
# in collab it will work df['category'].value_counts().plot.bar()
# Build Model
#===============================================================================================================
# Testing
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(IMAGE_WIDTH,
IMAGE_HEIGHT, IMAGE_CHANNELS)))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(128, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(256, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(512, activation='re.
I am Irene M. I am a C++ Homework Expert at cpphomeworkhelp.com. I hold a Masters in Programming from from California, USA. I have been helping students with their homework for the past 10 years. I solve homework related to C++.
Visit cpphomeworkhelp.com or email info@cpphomeworkhelp.com. You can also call on +1 678 648 4277 for any assistance with C++ Homework.
As part of our team's enrollment for Data Science Super Specialization course under UpX Academy, we submitted many projects for our final assessments, one of them was Telecom Churn Analysis Model.
The input data was provided by UpX academy and language we used is R. As part of the project, our main objective was :-
-> To predict Customer Churn.
-> To Highlight the main variables/factors influencing Customer Churn.
-> To Use various ML algorithms to build prediction models, evaluate the accuracy and performance of these models.
-> Finding out the best model for our business case & providing executive Summary.
To address the mentioned business problem, we tried to follow a thorough approach. We did a detailed level Exploratory Data Analysis which consists of various Box Plots, Bar Plots etc..
Further we tried our best to build as many Classification models possible which fits our business case (Logistic Regression/kNN/Decision Trees/Random Forest/SVM) and also tried to touch Cox Hazard Survival analysis Model. Later for every model we tried to boost their performances by applying various performance tuning techniques.
As we all are still into our learning mode w.r.t these concepts & starting new, please feel free to provide feedback on our work. Any suggestions are most welcome... :)
Thanks!!
Similar to [CAIN'23] Prevalence of Code Smells in Reinforcement Learning Projects (20)
An expressive and modular layer activation mechanism for Context-Oriented Pro...Universidad de los Andes
Abstract: Context. There is a trend in the software industry towards building systems that dynamically adapt their behavior in response to their surrounding environment, given the proliferation of various technological devices, such as notebooks, smartphones, and wearables, capable of capturing their execution context. Context-oriented Programming (COP) allows developers to use layer abstractions to adapt software behavior to the context. A layer is associated with a context and can be dynamically activated in direct response to gathered information from its surrounding execution environment. However, most existing layer activation mechanisms have been tailored specifically to address a particular concern; implying that developers need to tweak layer definitions in contortive ways or create new specialized activation mechanisms altogether if their specific needs are not supported. Objective. Complementing ideas to expressively declare activation mechanism models with interfaces that define conditionals of activation mechanisms modularly, this paper proposes an Expressive and Modular Activation mechanism, named EMA. Method. To propose EMA, we analyze existing activation mechanisms in COP regarding activation features and scope strategies. After, we propose the design of EMA and validate it with a case study discussion. Results. Using a concrete JavaScript implementation of EMA, named EMAjs, we can implement two Web applications: a smartphone application as an example to illustrate EMAjs in action, and an application of home automation to discuss and compare our proposal. Conclusion. Our proposed mechanism allows developers to instantiate different activation scope strategies and interfaces to decouple the declaration of activation mechanism conditionals from the base code.
Paper presentation: https://youtu.be/76RgVB6JihQ?si=-uylVc4Xnqnh4zbl
Paper presented at the 2023 Formal Techniques for Java-like Programs.
Abstract: Static analyses, as points-to analysis, are useful to determine and assure different properties about a program, such as security or type safety. While existing analyses are effective in programs restricted to static features, precision declines in the presence of dynamic language features, and even further when the system behavior changes dynamically. As a consequence, improved points-to sets algorithms taking into account such language features and uses are required. In this paper, we present and extension of the point-to sets analysis to incorporate the language abstractions introduced by context-oriented programming adding the capability for programs to adapt their behavior dynamically to the system’s execution context. To do this, we extend WALA to detect the context-oriented language abstractions, and their representation within the system, to capture the dynamic behavior, in the particular case of the Context Traits JavaScript language extension. To prove the effectiveness of our extension, we evaluate the precision of the points-to set analysis with respect to the state of the art, over four context-oriented programs taken from the literature.
[JIST] Programming language implementations for context-oriented self-adaptiv...Universidad de los Andes
Context
The context-oriented programming paradigm is designed to enable self-adaptation, or dynamic behavior modification of software systems, in response to changes in their surrounding environment. Contextoriented programming offers an adaptation model, from a programming language perspective, that maintains a clean modularisation between the application and adaptation logic, as well as between the components providing adaptations.
Objective
We use three implementation techniques for context-oriented programming languages to assess their appropriateness to foster self-adaptive systems. These approaches take advantage of the capabilities offered by the host programming language to realize self-adaptation as proposed by context-oriented languages.
Method
We evaluate each of these approaches by assessing their modularity and complexity when defining adaptations, and by comparing their run-time performance on a simple benchmark.
Results
Our results show a higher modularity than that for common architecture based self-adaptive systems, while maintaining comparable performance.
Conclusion
We conclude that context-oriented programming is an appropriate paradigm to realize self-adaptation.
Link to the paper: https://doi.org/10.1016/j.infsof.2021.106789
Clon detection provides insight about replicated fragments in a code base. With the rise of multi-language code bases, new techniques addressing cross-language code clone detection enable the analysis of polyglot systems. Such techniques have not yet been applied to the mobile apps’ domain, which are naturally polyglot. Native mobile app developers must synchronize their code base in at least two different programming languages. App synchronization is a difficult and time-consuming maintenance task, as features can rapidly diverge between platforms, and feature identification must be performed manually. Our goal is to provide an analysis framework to reduce the impact of app synchronization. A first step in this direction consists on a structural algorithm for cross-language clone detection exploiting the idea behind enriched concrete syntax trees. Such trees are used as a common intermediate representation built from programming languages’ grammars, to detect similarities between app code bases. Our technique finds code similarities with 79% precision for controlled tests where Type 1-3 clones are manually injected for the analysis of both single- and cross-language cases for Kotlin and Dart. We evaluate our tool on a corpus of 52 mobile apps identifying code similarities with a precision of 65% to 84% for the full application logic.
recorrido sobre las características principales de LaTeX y lagunas librerías útiles para escribir documentos más fácilmente.
Temas tratados:
Estructura de documentos
Paquetes
Elementos básicos (secciones, listas)
Figuras
Tablas
Entornos matemáticos (ecuaciones anotadas https://github.com/synercys/annotated_latex_equations)
Fragmentos de código
Referencias
Acrónimos y comandos
[JPDC,JCC@LMN22] Ad hoc systems Management and specification with distributed...Universidad de los Andes
This presents the main ideas behind Distributed ad hoc Petri nets (DaPNs). A specification and run-time model to manage and reason about ad hoc distributed systems.
The presentation describes the main contributions of our work validated through a case study on vehicular ad hoc distributed networks.
This work is available at the Journal of Parallel and Distributed Computing, and was presented at the Lo Mejor de lo nuestro #LMN2022, at the Jornadas Chilenas de Computación 2022
Link to the paper: https://doi.org/10.1016/j.jpdc.2022.06.015
This work presents the evaluation of the two classic workstealing algorithms (FIFO and LIFO) together with a new proposed implementation based on the priority of tasks calculated using the longest path as a metric
Generating Adaptations from the System Execution using Reinforcement Learning...Universidad de los Andes
Invited talk to the Context-oriented programming workshop co-located with ECOOP 2021.
This talk describes work in progress on generating adaptation dynamically with respect to the conditions of the system environment using reinforcement learning
Language Abstractions and Techniques for Developing Collective Adaptive Syste...Universidad de los Andes
Presentation at the 5th eCAS Workshop on Engineering Collective Adaptive Systems 2020, colocated with acsos.
Collective Adaptive Systems (CAS) are increasingly used as a model for building and managing large- scale cyber-physical systems, highlighting the need for a flexible approach for their definition and engineering. CAS consist of multiple individual context-aware self-adaptive systems, which need to adapt to their own environment, but also interact with each other, resulting in the global CAS behavior. Interactions can emerge as complementary or conflicting, and needs to be managed to assure a correct global behavior. Context-oriented Programming (COP) is one of the programming paradigms that has risen as a suitable approach to developing self-adaptive systems, due to the natural mapping of its programming constructs to self- adaptive systems taxonomy. Recently, multiple extensions to the COP model have been developed, which manage inter- actions between adaptive components (or systems), enabling its suitability for developing CAS as well. In this paper, we map out the COP language abstractions to the characteristics of CAS, demonstrating COP applicability, its formalization, verification tools, and adaptation composition techniques to foster CAS. Our discussion is illustrated using city-wide bus transport management system, as an example of a CAS.
Does Neuron Coverage Matter for Deep Reinforcement Learning? A preliminary studyUniversidad de los Andes
This work is presented at the 2020 Workshop on Testing for Deep Learning and Deep Learning for Testing (DeepTest) co-located with ICSE held virtually.
This work analyzes the possibility of using neuron coverage as a test adequacy metric for Deep Reinforcement Learning (DeepRL). This work spawns from the positive results in using neuron coverage to test (deep) neural networks. However, testing DeepRL systems, brings its own challenges. In the presentation, and associated paper, we discuss the characteristics of DeepRL that prevent promoting neuron coverage as an adequacy testing metric.
The paper is available at: https://deeptestconf.github.io/pdfs/2020-Trujillo-DeepTest.pdf
This work is presented at the 2020 Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS) co-located with ICSE held virtually.
In this work we present the early results of our ComInA technique to learn the way behavior adaptations interact with each other, and from there, offer the most appropriate composition of adaptations. Our technique uses a combination of Context-oriented programming and reinforcement learning to define and compose fine-grained modular adaptations, and learn their interaction respectively
Talk presented at the 2019 Context-oriented programming workshop colocated with ECOOP in london.
This work presents an extension to Context petri nets (CoPN) to enable the communication between different nodes, each defined using a CoPN. This extension demonstrates the interaction with different context dependency relations in face of unannounced connection and disconnection of nodes.
Talk presented at the 2018 Context-oriented programming workshop colocated with ECOOP in Amsterdam.
This work presents a small query language designed for the activation of context objects in response to information gathered from sensors. The objective of CQL is to enable the possibility to activate multiple contexts simultaneously, using a single query
Talk presented at the 2018 machine learning for programming languages workshop colocated with ECOOP in Amsterdam.
This work presents the use of reinforcement learning for the discovery of context objects to drive behavioral adaptations. Our technique uses information gathered from the systems environment to define context, associating the actions taken by the user in such situations to define and automate behavioral adaptations
Presentation for the bachelor thesis project at Universidad de los Andes 2020-10.
This thesis studies the use parallel programming techniques to speed-up the decomposition phase of the constraint solving problem. The technique is integrated with the Oscar-CP solver, and explores the exam assignment problem
Invited presentation at the university of Coimbra in 2016.
This presentation gives an overview of the programming language implementation effort at FLAGlab applied to dynamic adaptive software systems.
Talk presented at the 2017 Context-oriented programming workshop colocated with ECOOP in barcelona.
This work presents a reinforcement learning based solution to resolve conflicts between dynamic adaptations.
Talk presented at the 2016 Onward! conference colocated with splash in amsterdam.
This work presents a framework for the spontaneous interaction (composition) of service components by learning services APIs. Upon discovery of services with a known API the composition is seamless.
Hierarchical Digital Twin of a Naval Power SystemKerry Sado
A hierarchical digital twin of a Naval DC power system has been developed and experimentally verified. Similar to other state-of-the-art digital twins, this technology creates a digital replica of the physical system executed in real-time or faster, which can modify hardware controls. However, its advantage stems from distributing computational efforts by utilizing a hierarchical structure composed of lower-level digital twin blocks and a higher-level system digital twin. Each digital twin block is associated with a physical subsystem of the hardware and communicates with a singular system digital twin, which creates a system-level response. By extracting information from each level of the hierarchy, power system controls of the hardware were reconfigured autonomously. This hierarchical digital twin development offers several advantages over other digital twins, particularly in the field of naval power systems. The hierarchical structure allows for greater computational efficiency and scalability while the ability to autonomously reconfigure hardware controls offers increased flexibility and responsiveness. The hierarchical decomposition and models utilized were well aligned with the physical twin, as indicated by the maximum deviations between the developed digital twin hierarchy and the hardware.
An Approach to Detecting Writing Styles Based on Clustering Techniquesambekarshweta25
An Approach to Detecting Writing Styles Based on Clustering Techniques
Authors:
-Devkinandan Jagtap
-Shweta Ambekar
-Harshit Singh
-Nakul Sharma (Assistant Professor)
Institution:
VIIT Pune, India
Abstract:
This paper proposes a system to differentiate between human-generated and AI-generated texts using stylometric analysis. The system analyzes text files and classifies writing styles by employing various clustering algorithms, such as k-means, k-means++, hierarchical, and DBSCAN. The effectiveness of these algorithms is measured using silhouette scores. The system successfully identifies distinct writing styles within documents, demonstrating its potential for plagiarism detection.
Introduction:
Stylometry, the study of linguistic and structural features in texts, is used for tasks like plagiarism detection, genre separation, and author verification. This paper leverages stylometric analysis to identify different writing styles and improve plagiarism detection methods.
Methodology:
The system includes data collection, preprocessing, feature extraction, dimensional reduction, machine learning models for clustering, and performance comparison using silhouette scores. Feature extraction focuses on lexical features, vocabulary richness, and readability scores. The study uses a small dataset of texts from various authors and employs algorithms like k-means, k-means++, hierarchical clustering, and DBSCAN for clustering.
Results:
Experiments show that the system effectively identifies writing styles, with silhouette scores indicating reasonable to strong clustering when k=2. As the number of clusters increases, the silhouette scores decrease, indicating a drop in accuracy. K-means and k-means++ perform similarly, while hierarchical clustering is less optimized.
Conclusion and Future Work:
The system works well for distinguishing writing styles with two clusters but becomes less accurate as the number of clusters increases. Future research could focus on adding more parameters and optimizing the methodology to improve accuracy with higher cluster values. This system can enhance existing plagiarism detection tools, especially in academic settings.
Cosmetic shop management system project report.pdfKamal Acharya
Buying new cosmetic products is difficult. It can even be scary for those who have sensitive skin and are prone to skin trouble. The information needed to alleviate this problem is on the back of each product, but it's thought to interpret those ingredient lists unless you have a background in chemistry.
Instead of buying and hoping for the best, we can use data science to help us predict which products may be good fits for us. It includes various function programs to do the above mentioned tasks.
Data file handling has been effectively used in the program.
The automated cosmetic shop management system should deal with the automation of general workflow and administration process of the shop. The main processes of the system focus on customer's request where the system is able to search the most appropriate products and deliver it to the customers. It should help the employees to quickly identify the list of cosmetic product that have reached the minimum quantity and also keep a track of expired date for each cosmetic product. It should help the employees to find the rack number in which the product is placed.It is also Faster and more efficient way.
Final project report on grocery store management system..pdfKamal Acharya
In today’s fast-changing business environment, it’s extremely important to be able to respond to client needs in the most effective and timely manner. If your customers wish to see your business online and have instant access to your products or services.
Online Grocery Store is an e-commerce website, which retails various grocery products. This project allows viewing various products available enables registered users to purchase desired products instantly using Paytm, UPI payment processor (Instant Pay) and also can place order by using Cash on Delivery (Pay Later) option. This project provides an easy access to Administrators and Managers to view orders placed using Pay Later and Instant Pay options.
In order to develop an e-commerce website, a number of Technologies must be studied and understood. These include multi-tiered architecture, server and client-side scripting techniques, implementation technologies, programming language (such as PHP, HTML, CSS, JavaScript) and MySQL relational databases. This is a project with the objective to develop a basic website where a consumer is provided with a shopping cart website and also to know about the technologies used to develop such a website.
This document will discuss each of the underlying technologies to create and implement an e- commerce website.
Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...Dr.Costas Sachpazis
Terzaghi's soil bearing capacity theory, developed by Karl Terzaghi, is a fundamental principle in geotechnical engineering used to determine the bearing capacity of shallow foundations. This theory provides a method to calculate the ultimate bearing capacity of soil, which is the maximum load per unit area that the soil can support without undergoing shear failure. The Calculation HTML Code included.
NUMERICAL SIMULATIONS OF HEAT AND MASS TRANSFER IN CONDENSING HEAT EXCHANGERS...ssuser7dcef0
Power plants release a large amount of water vapor into the
atmosphere through the stack. The flue gas can be a potential
source for obtaining much needed cooling water for a power
plant. If a power plant could recover and reuse a portion of this
moisture, it could reduce its total cooling water intake
requirement. One of the most practical way to recover water
from flue gas is to use a condensing heat exchanger. The power
plant could also recover latent heat due to condensation as well
as sensible heat due to lowering the flue gas exit temperature.
Additionally, harmful acids released from the stack can be
reduced in a condensing heat exchanger by acid condensation. reduced in a condensing heat exchanger by acid condensation.
Condensation of vapors in flue gas is a complicated
phenomenon since heat and mass transfer of water vapor and
various acids simultaneously occur in the presence of noncondensable
gases such as nitrogen and oxygen. Design of a
condenser depends on the knowledge and understanding of the
heat and mass transfer processes. A computer program for
numerical simulations of water (H2O) and sulfuric acid (H2SO4)
condensation in a flue gas condensing heat exchanger was
developed using MATLAB. Governing equations based on
mass and energy balances for the system were derived to
predict variables such as flue gas exit temperature, cooling
water outlet temperature, mole fraction and condensation rates
of water and sulfuric acid vapors. The equations were solved
using an iterative solution technique with calculations of heat
and mass transfer coefficients and physical properties.
Student information management system project report ii.pdfKamal Acharya
Our project explains about the student management. This project mainly explains the various actions related to student details. This project shows some ease in adding, editing and deleting the student details. It also provides a less time consuming process for viewing, adding, editing and deleting the marks of the students.
Student information management system project report ii.pdf
[CAIN'23] Prevalence of Code Smells in Reinforcement Learning Projects
1. CAIN 2023
Prevalence of Code Smells in
Reinforcement Learning Projects
Nicolás Cardozo, Ivana Dusparic, Christian Cabrera
Systems an Computing Engineering - Universidad de los Andes, Bogotá - Colombia
Trinity College Dublin - Ireland
University of Cambridge - UK
n.cardozo@uniandes.edu.co, ivana.dusparic@tcd.ie, chc79@cam.ac.uk
@ncardoz
17. Metrics
5
Code smell Métric Threshold
Long Method
(LM)
Function LOC 38
Long Class
(LC)
Class LOC 29
Long Parameter List
(LPL)
Number of parameters 5
Long Method Chain
(LMC)
Length of message chain 5
Long Scope Chaining
(LSC)
Depth of closure 3
Long Ternary Conditional Expression
(LTCE)
Number of characters 54
Multiply-Nested Container
(MNC)
Depth of nested container 3
Long Lambda Function
(LLF)
Number of characters 48
18. Metrics
6
def update(self, msg_env):
'''
Update the state of the agent
:param msg_env: dict. A message generated by the order matching
'''
# check if should update, if it is not a trade
# if not isinstance(msg_env, type(None)):
if not msg_env:
if not self.should_update():
return None
# recover basic infos
inputs = self.env.sense(self)
state = self.env.agent_states[self]
s_cmm = self.env.s_main_intrument
# Update state (position ,volume and if has an order in bid or ask)
self.state = self.get_intern_state(inputs, state)
# Select action according to the agent's policy
s_action = None
s_action, l_msg = self.take_action(self.state, msg_env)
s_action2 = s_action
# Execute action and get reward
reward = 0.
self.env.update_order_book(l_msg)
l_prices_to_print = []
if len(l_msg) == 0:
reward += self.env.act(self, None)
self.b_new_reward = False
for msg in l_msg:
if msg['agent_id'] == self.i_id:
# check if should hedge the position
self.should_change_stoptime(msg)
# form log message
s_action = msg['action']
s_action2 = s_action
s_side_msg = msg['order_side'].split()[0]
s_indic = msg['agressor_indicator']
s_cmm = msg['instrumento_symbol']
d_aux = {'A': msg['order_status'],
# log just the last 4 digits of the order
'I': msg['order_id'] % 10**4,
'Q': msg['order_qty'],
'C': msg['instrumento_symbol'],
'S': s_side_msg,
'P': '{:0.2f}'.format(msg['order_price'])}
l_prices_to_print.append(d_aux)
#
l_prices_to_print.append('{:0.2f}'.format(msg['order_price']))
if s_indic == 'Agressor' and s_action == 'SELL':
s_action2 = 'HIT' # hit the bid
elif s_indic == 'Agressor' and s_action == 'BUY':
s_action2 = 'TAKE' # take the offer
try:
# the agent's positions and orders list are update here
# TODO: The reward really should be collect at this
point?
reward += self.env.act(self, msg)
self.b_new_reward = False
except:
print 'BasicAgent.update(): Message with error at
reward:'
pprint.pprint(msg)
raise
# check if should cancel any order due to excess
l_msg1 = self.could_include_new(s_action)
self.env.update_order_book(l_msg1)
for msg in l_msg1:
if msg['agent_id'] == self.i_id:
s_indic = msg['agressor_indicator']
d_aux = {'A': msg['order_status'],
'I': msg['order_id'],
'C': msg['instrumento_symbol'],
'S': msg['order_side'].split()[0],
'P': '{:0.2f}'.format(msg['order_price'])}
l_prices_to_print.append(d_aux)
try:
# the agent's positions and orders list are update here
# there is no meaning in colecting reward here
self.env.act(self, msg)
except:
print 'BasicAgent.update(): Message with error at
reward:'
pprint.pprint(msg)
raise
# === DEBUG ====
# if len(l_msg1) > 0:
# print 'n====CANCEL ORDER DUE TO EXCESS======n'
# pprint.pprint(l_msg1)
# ==============
# NOTE: I am not sure about that, but at least makes sense... I guess
# I should have to apply the reward to the action that has generated
# the trade (when my order was hit, I was in the book before)
if s_action2 == s_action:
if s_action == 'BUY':
s_action = 'BEST_BID'
elif s_action == 'SELL':
s_action = 'BEST_OFFER'
if s_action in ['correction_by_trade', 'crossed_prices']:
if s_side_msg == 'Buy':
s_action = 'BEST_BID'
elif s_side_msg == 'Sell':
s_action = 'BEST_OFFER'
# Learn policy based on state, action, reward
if s_cmm == self.env.s_main_intrument:
if self.policy_update(self.state, s_action, reward):
self.k_steps += 1
self.n_steps += 1
# print 'new step: {}n'.format(self.n_steps)
# calculate the next time that the agent will react
if not isinstance(msg_env, type(dict)):
self.next_time = self.env.order_matching.last_date
f_delta_time = self.f_min_time
# add additional miliseconds to the next_time to act
if self.f_min_time > 0.004:
if np.random.rand() > 0.4:
i_mult = 1
if np.random.rand() < 0.5:
i_mult = -1
f_add = min(1., self.f_min_time*100)
f_add *= np.random.rand()
f_delta_time += (int(np.ceil(f_add))*i_mult)/1000.
self.next_time += f_delta_time
self.last_delta_time = int(f_delta_time * 1000)
# print agent inputs
self._pnl_information_update()
self.log_step(state, inputs, s_action2, l_prices_to_print, reward)
Long method
19. Metrics
6
def update(self, msg_env):
'''
Update the state of the agent
:param msg_env: dict. A message generated by the order matching
'''
# check if should update, if it is not a trade
# if not isinstance(msg_env, type(None)):
if not msg_env:
if not self.should_update():
return None
# recover basic infos
inputs = self.env.sense(self)
state = self.env.agent_states[self]
s_cmm = self.env.s_main_intrument
# Update state (position ,volume and if has an order in bid or ask)
self.state = self.get_intern_state(inputs, state)
# Select action according to the agent's policy
s_action = None
s_action, l_msg = self.take_action(self.state, msg_env)
s_action2 = s_action
# Execute action and get reward
reward = 0.
self.env.update_order_book(l_msg)
l_prices_to_print = []
if len(l_msg) == 0:
reward += self.env.act(self, None)
self.b_new_reward = False
for msg in l_msg:
if msg['agent_id'] == self.i_id:
# check if should hedge the position
self.should_change_stoptime(msg)
# form log message
s_action = msg['action']
s_action2 = s_action
s_side_msg = msg['order_side'].split()[0]
s_indic = msg['agressor_indicator']
s_cmm = msg['instrumento_symbol']
d_aux = {'A': msg['order_status'],
# log just the last 4 digits of the order
'I': msg['order_id'] % 10**4,
'Q': msg['order_qty'],
'C': msg['instrumento_symbol'],
'S': s_side_msg,
'P': '{:0.2f}'.format(msg['order_price'])}
l_prices_to_print.append(d_aux)
#
l_prices_to_print.append('{:0.2f}'.format(msg['order_price']))
if s_indic == 'Agressor' and s_action == 'SELL':
s_action2 = 'HIT' # hit the bid
elif s_indic == 'Agressor' and s_action == 'BUY':
s_action2 = 'TAKE' # take the offer
try:
# the agent's positions and orders list are update here
# TODO: The reward really should be collect at this
point?
reward += self.env.act(self, msg)
self.b_new_reward = False
except:
print 'BasicAgent.update(): Message with error at
reward:'
pprint.pprint(msg)
raise
# check if should cancel any order due to excess
l_msg1 = self.could_include_new(s_action)
self.env.update_order_book(l_msg1)
for msg in l_msg1:
if msg['agent_id'] == self.i_id:
s_indic = msg['agressor_indicator']
d_aux = {'A': msg['order_status'],
'I': msg['order_id'],
'C': msg['instrumento_symbol'],
'S': msg['order_side'].split()[0],
'P': '{:0.2f}'.format(msg['order_price'])}
l_prices_to_print.append(d_aux)
try:
# the agent's positions and orders list are update here
# there is no meaning in colecting reward here
self.env.act(self, msg)
except:
print 'BasicAgent.update(): Message with error at
reward:'
pprint.pprint(msg)
raise
# === DEBUG ====
# if len(l_msg1) > 0:
# print 'n====CANCEL ORDER DUE TO EXCESS======n'
# pprint.pprint(l_msg1)
# ==============
# NOTE: I am not sure about that, but at least makes sense... I guess
# I should have to apply the reward to the action that has generated
# the trade (when my order was hit, I was in the book before)
if s_action2 == s_action:
if s_action == 'BUY':
s_action = 'BEST_BID'
elif s_action == 'SELL':
s_action = 'BEST_OFFER'
if s_action in ['correction_by_trade', 'crossed_prices']:
if s_side_msg == 'Buy':
s_action = 'BEST_BID'
elif s_side_msg == 'Sell':
s_action = 'BEST_OFFER'
# Learn policy based on state, action, reward
if s_cmm == self.env.s_main_intrument:
if self.policy_update(self.state, s_action, reward):
self.k_steps += 1
self.n_steps += 1
# print 'new step: {}n'.format(self.n_steps)
# calculate the next time that the agent will react
if not isinstance(msg_env, type(dict)):
self.next_time = self.env.order_matching.last_date
f_delta_time = self.f_min_time
# add additional miliseconds to the next_time to act
if self.f_min_time > 0.004:
if np.random.rand() > 0.4:
i_mult = 1
if np.random.rand() < 0.5:
i_mult = -1
f_add = min(1., self.f_min_time*100)
f_add *= np.random.rand()
f_delta_time += (int(np.ceil(f_add))*i_mult)/1000.
self.next_time += f_delta_time
self.last_delta_time = int(f_delta_time * 1000)
# print agent inputs
self._pnl_information_update()
self.log_step(state, inputs, s_action2, l_prices_to_print, reward)
Long method
Long class
class
QLearningAgent(BasicAgent):
'''
A representation of an agent
that learns using Q-learning
with linear
parametrization and e-greedy
exploration described at p.60 ~
p.61 form
Busoniu at al., 2010. The
approximator used is the
implementation of tile
coding, described at Sutton
and Barto, 2016 (draft).
'''
actions_to_open = [None,
'BEST_BID', 'BEST_OFFER',
'BEST_BOTH']
actions_to_close_when_short
= [None, 'BEST_BID']
actions_to_close_when_long
= [None, 'BEST_OFFER']
actions_to_stop_when_short
= [None, 'BEST_BID', 'BUY']
actions_to_stop_when_long
= [None, 'BEST_OFFER',
'SELL']
FROZEN_POLICY = False
def __init__(self, env, i_id,
d_normalizers, d_o
fi
_scale,
f_min_time=3600.,
f_gamma=0.5,
f_alpha=0.5, i_numOfTilings=16,
s_decay_fun=None,
f_ttoupdate=5.,
d_initial_pos={},
s_hedging_on='DI1F19',
b_hedging=True,
b_keep_pos=True):
'''
Initialize a
QLearningAgent. Save all
parameters as attributes
:param env: Environment
Object. The Environment where
the agent acts
:param i_id: integer. Agent
id
:param d_normalizers:
dictionary. The maximum range
of each feature
:param f_min_time*:
fl
oat.
Minimum time in seconds to the
agent react
:param f_gamma*:
fl
oat.
weight of delayed versus
immediate rewards
:param f_alpha*: the initial
learning rate used
:param i_numOfTilings*:
unmber of tiling desired
:param s_decay_fun*:
string. The exploration factor
decay function
:param f_ttoupdate*.
fl
oat.
time in seconds to choose a
diferent action
'''
f_aux = f_ttoupdate
super(QLearningAgent,
self).__init__(env, i_id,
f_min_time, f_aux,
d_initial_pos=d_initial_pos)
self.learning = True # this
agent is expected to learn
self.decayfun =
s_decay_fun
# Initialize any additional
variables here
self.max_pos = 100.
self.max_disclosed_pos =
10.
self.orders_lim = 4
self.order_size = 5
self.s_agent_name =
'QLearningAgent'
# control hedging
obj_aux =
risk_model.GreedyHedgeModel
self.s_hedging_on =
s_hedging_on
self.risk_model =
obj_aux(env,
s_instrument=s_hedging_on,
s_fairness='closeout')
self.last_spread = [0.0, 0.0]
self.f_spread = [0.0, 0.0]
self.f_gamma = f_gamma
self.f_alpha = f_alpha
self.f_epsilon = 1.0
self.b_hedging =
b_hedging
self.current_open_price =
None
self.current_max_price =
-9999.
self.current_min_price =
9999.
self.b_keep_pos =
b_keep_pos
# Initialize any additional
variables here
self.f_time_to_buy = 0.
self.f_time_to_sell = 0.
self.b_print_always = False
self.d_normalizers =
d_normalizers
self.d_o
fi
_scale =
d_o
fi
_scale
self.numOfTilings =
i_numOfTilings
self.alpha = f_alpha
i_nTiling = i_numOfTilings
value_fun =
ValueFunction(f_alpha,
d_normalizers, i_nTiling)
self.value_function =
value_fun
self.old_state = None
self.last_action = None
self.last_reward = None
self.disclosed_position = {}
self.f_stop_time =
STOP_MKT_TIME - 1 + 1
# self.features_names =
['position', 'o
fi
_new',
'spread_longo',
#
'ratio_longo', 'ratio_curto',
#
'size_bid_longo',
'size_bid_curto',
#
'spread_curto', 'high_low',
'rel_price']
self.features_names =
['position', 'o
fi
_new',
'ratio_longo',
'ratio_curto',
'spread_longo',
'rel_price']
def
reset_additional_variables(self,
testing):
'''
Reset the state and the
agent's memory about its
positions
:param testing: boolean. If
should freeze policy
'''
self.risk_model.reset()
self.f_time_to_buy = 0.
self.f_time_to_sell = 0.
self.last_reward = None
self.current_open_price =
None
self.current_max_price =
-9999.
self.current_min_price =
9999.
self.spread_position = {}
self.disclosed_position = {}
self.env.reward_fun.reset()
for s_instr in
self.env.l_instrument:
self.disclosed_position[s_instr]
= {'qAsk': 0.,
'Ask': 0.,
'qBid': 0.,
'Bid': 0.}
if testing:
self.freeze_policy()
def
additional_actions_when_exec(
self, s_instr, s_side, msg):
'''
Execute additional action
when execute a trade
:param s_instr: string.
:param s_side: string.
:param msg: dictionary.
Last trade message
'''
# check if the main
intrument was traded
s_main =
self.env.s_main_intrument
if
msg['instrumento_symbol'] ==
s_main:
self.b_has_traded = True
# check if it open or close
a pos
f_pos =
self.position[s_instr]['qBid']
f_pos -=
self.position[s_instr]['qAsk']
b_zeroout_buy = f_pos ==
0 and s_side == 'ASK'
b_zeroout_sell = f_pos ==
0 and s_side == 'BID'
b_new_buy = f_pos > 0
and s_side == 'BID'
b_new_sell = f_pos < 0
and s_side == 'ASK'
b_close_buy = f_pos > 0
and s_side == 'ASK'
b_close_sell = f_pos < 0
and s_side == 'BID'
s_other_side = 'BID'
# set the time to open
position if it just close it
if b_close_buy or
b_zeroout_buy:
self.f_time_to_buy =
self.env.order_matching.f_time
+ 60.
elif b_close_sell or
b_zeroout_sell:
self.f_time_to_sell =
self.env.order_matching.f_time
+ 60.
# print when executed
f_pnl = self.log_info['pnl']
s_time =
self.env.order_matching.s_time
s_err = '{}: {} - current
position {:0.2f}, PnL: {:0.2f}n'
print s_err.format(s_time,
s_instr, f_pos, f_pnl)
# keep a list of the opened
positions
if s_side == 'BID':
s_other_side = 'ASK'
if b_zeroout_buy or
b_zeroout_sell:
self.current_open_price
= None # update by risk model
self.current_max_price =
-9999.
self.current_min_price =
9999.
self.d_trades[s_instr]
[s_side] = []
self.d_trades[s_instr]
[s_other_side] = []
elif b_new_buy or
b_new_sell:
if b_new_buy:
self.risk_model.price_stop_sell
= None
if b_new_sell:
self.risk_model.price_stop_buy
= None
# log more information
l_info_to_hold =
[msg['order_price'],
msg['order_qty'], None]
if 'last_inputs' in
self.log_info:
l_info_to_hold[2] =
self.log_info['last_inputs']['TOB']
self.d_trades[s_instr]
[s_side].append(l_info_to_hold)
self.d_trades[s_instr]
[s_other_side] = []
elif b_close_buy or
b_close_sell:
f_qty_to_match =
msg['order_qty']
l_aux = []
for f_price, f_qty, d_tob
in self.d_trades[s_instr]
[s_other_side]:
if f_qty_to_match ==
0:
l_aux.append([f_price, f_qty,
d_tob])
elif f_qty <=
f_qty_to_match:
f_qty_to_match -=
f_qty
elif f_qty >
f_qty_to_match:
f_qty -=
f_qty_to_match
f_qty_to_match = 0
l_aux.append([f_price, f_qty,
d_tob])
self.d_trades[s_instr]
[s_other_side] = l_aux
if abs(f_qty_to_match) >
0:
l_info_to_hold =
[msg['order_price'],
f_qty_to_match, None]
if 'last_inputs' in
self.log_info:
l_info_to_hold[2] =
self.log_info['last_inputs']['TOB']
self.d_trades[s_instr]
[s_side].append(l_info_to_hold)
def need_to_hedge(self):
'''
Return if the agent need to
hedge position
'''
# ask risk model if should
hedge
if not self.b_hedging:
return False
if not self.b_keep_pos:
if
self.env.order_matching.last_da
te > self.f_stop_time:
if
abs(self.log_info['duration']) >
0.01:
self.b_need_hedge
= True
# print
'need_to_hedge(): HERE'
return
self.b_need_hedge
if
self.risk_model.should_stop_dis
closed(self):
return True
if
self.risk_model.should_hedge_o
pen_position(self):
# check if should hedge
position
if
abs(self.log_info['duration']) >
1.:
self.b_need_hedge =
True
return
self.b_need_hedge
return False
def
get_valid_actions_old(self):
'''
Return a list of valid
actions based on the current
position
'''
# b_stop = False
valid_actions =
list(self.actions_to_open)
if not
self.risk_model.can_open_positi
on('ASK', self):
valid_actions =
list(self.actions_to_close_when_
short) # copy
if
self.risk_model.should_stop_dis
closed(self):
# b_stop = True
valid_actions =
list(self.actions_to_stop_when_s
hort)
elif not
self.risk_model.can_open_positi
on('BID', self):
valid_actions =
list(self.actions_to_close_when_
long)
if
self.risk_model.should_stop_dis
closed(self):
# b_stop = True
valid_actions =
list(self.actions_to_stop_when_l
ong)
return valid_actions
def get_valid_actions(self):
'''
Return a list of valid
actions based on the current
position
'''
# b_stop = False
valid_actions =
list(self.actions_to_open)
return valid_actions
def get_intern_state(self,
inputs, state):
'''
Return a dcitionary
representing the intern state of
the agent
:param inputs: dictionary.
what the agent can sense from
env
:param state: dictionary.
the current state of the agent
'''
d_data = {}
s_main =
self.env.s_main_intrument
d_data['OFI'] =
inputs['qO
fi
']
d_data['qBID'] =
inputs['qBid']
d_data['BOOK_RATIO'] =
0.
d_data['LOG_RET'] =
inputs['logret']
d_rtn = {}
d_rtn['cluster'] = 0
d_rtn['Position'] =
fl
oat(state[s_main]['Position'])
d_rtn['best_bid'] =
state['best_bid']
d_rtn['best_o
ff
er'] =
state['best_o
ff
er']
# calculate the current
position in the main instrument
f_pos =
self.position[s_main]['qBid']
f_pos -=
self.position[s_main]['qAsk']
f_pos +=
self.disclosed_position[s_main]
['qBid']
f_pos -=
self.disclosed_position[s_main]
['qAsk']
# calculate the duration
exposure
# f_duration =
self.risk_model.portfolio_duratio
n(self.position)
# measure the OFI index
f_last_o
fi
= 0.
# if self.logged_action:
# compare with the last
data
# if 'to_delta' in
self.log_info:
# # measure the change
in OFI from he last sction taken
# for s_key in [s_main]:
# i_o
fi
_now =
inputs['OFI'][s_key]
# i_o
fi
_old =
self.log_info['to_delta']['OFI']
[s_key]
# f_aux = i_o
fi
_now -
i_o
fi
_old
# f_last_o
fi
+= f_aux
f_last_o
fi
= inputs['dOFI']
[s_main]
# for the list to be used as
features
fun = self.bound_values
s_lng =
self.env.s_main_intrument
s_crt = self.s_hedging_on
l_values = [fun(f_pos * 1.,
'position'),
fun(f_last_o
fi
,
'o
fi
_new', s_main),
fun(inputs['ratio']
[s_lng]['BID'], 'ratio_longo'),
fun(inputs['ratio']
[s_crt]['BID'], 'ratio_curto'),
fun(inputs['spread']
[s_lng], 'spread_longo'),
#
fun(inputs['spread'][s_crt],
'spread_curto'),
# fun(inputs['size']
[s_lng]['BID'], 'size_bid_longo'),
# fun(inputs['size']
[s_crt]['BID'], 'size_bid_curto'),
#
fun(inputs['HighLow'][s_lng],
'high_low'),
fun(inputs['reallAll']
[s_lng], 'rel_price')]
d_rtn['features'] =
dict(zip(self.features_names,
l_values))
return d_rtn
def bound_values(self,
f_value, s_feature_name,
s_cmm=None):
'''
Return the value bounded
by the maximum and minimum
values predicted.
Also apply nomalizations
functions if it is de
fi
ned and
d_normalizers,
in the FUN key.
:param f_value:
fl
oat. value
to be bounded
:param s_feature_name:
string. the name of the feature
in d_normalizers
:param s_cmm*: string.
Name of the instrument
'''
f_max =
self.d_normalizers[s_feature_na
me]['MAX']
f_min =
self.d_normalizers[s_feature_na
me]['MIN']
f_value2 = max(f_min,
f_value)
f_value2 = min(f_max,
f_value)
if 'FUN' in
self.d_normalizers[s_feature_na
me]:
if s_feature_name ==
'o
fi
_new':
f =
self.d_normalizers[s_feature_na
me]['FUN'](f_value, s_cmm)
f = max(f_min, f)
f = min(f_max, f)
f_value2 = f
else:
f_value2 =
self.d_normalizers[s_feature_na
me]['FUN'](f_value2)
return f_value2
def get_epsilon_k(self):
'''
Get $epsilon_k$
according to the exploration
schedule
'''
trial = self.env.count_trials
- 2 # ?
if self.decayfun ==
'tpower':
# e = a^t, where 0 < z <
1
# self.f_epsilon =
math.pow(0.9675, trial) # for
100 trials
self.f_epsilon =
math.pow(0.9333, trial) # for 50
trials
elif self.decayfun == 'trig':
# e = cos(at), where 0 <
z < 1
# self.f_epsilon =
math.cos(0.0168 * trial) # for
100 trials
self.f_epsilon =
math.cos(0.03457 * trial) # for
50 trials
else:
# self.f_epsilon =
max(0., 1. - (1./45. * trial)) # for
50 trials
self.f_epsilon = max(0.,
1. - (1./95. * trial)) # for 100
trials
return self.f_epsilon
def choose_an_action(self,
d_state, valid_actions):
'''
Return an action from a list
of allowed actions according to
the
agent's policy based on
epsilon greedy policy and
valueFunction
:param d_state: dictionary.
The inputs to be considered by
the agent
:param valid_actions: list.
List of the allowed actions
'''
# return a uniform random
action with prob $epsilon_k$
(exploration)
state_ = d_state['features']
best_Action =
random.choice(valid_actions)
if not
self.FROZEN_POLICY:
if np.random.binomial(1,
self.get_epsilon_k()) == 1:
return best_Action
# apply: arg max_{u'}
( phi^T (x_k, u') theta_k)
values = []
for action in valid_actions:
values.append(self.value_functi
on.value(state_, action, self))
# return
self.d_value_to_action[argmax(v
alues)]
return
valid_actions[argmax(values)]
def apply_policy(self, state,
action, reward):
'''
Learn policy based on
state, action, reward. The algo
part of "apply
action u_k" is in the update
method from agent frmk as the
update just
occur after one trial, state
and reward are at the next step.
Return
True if the policy was
updated
:param state: dictionary.
The current state of the agent
:param action: string. the
action selected at this time
:param reward: integer. the
rewards received due to the
action
'''
# check if there is some
state in cache
state_ = state['features']
valid_actions =
self.get_valid_actions()
if self.old_state and not
self.FROZEN_POLICY:
# TD Update
q_values_next = []
for act in valid_actions:
# for the vector: $
(phi^t (x_{k+1}, u') *
theta_k)_{u'}$
# state here plays the
role of next state x_{k+1}. act
are u's
f_value =
self.value_function.value(state_,
act, self)
q_values_next.append(f_value)
# Q-Value TD Target
# apply: Qhat <- r_{k+1}
+ y max_u' (phi^T(x_{k+1}, u')
theta_k)
# note that u' is the
result of apply u in x. u' is the
action that
# would maximize the
estimated Q-value for the state
x'
td_target =
self.last_reward + self.f_gamma
* np.max(q_values_next)
# Update the state value
function using our target
# apply: $theta_{k+1}
<- alpha_k (Q_ - Qhat)
theta(x_k, u_k)$
# the remain part of the
update is inside the method
learn
# use last_action here
because it generated the
curremt reward
self.value_function.learn(self.old
_state, self.last_action,
td_target, self)
# save current state, action
and reward to use in the next
run
self.old_state = state_ # in
the next run it is x_k <- x_{k+1}
self.last_action = action #
in the next run it is u_k
self.last_reward = reward
# in the next run it is r_{k+1}
if action in ['SELL', 'BUY']:
print '=',
return True
def set_qtable(self, s_fname,
b_freezy_policy=True):
'''
Set up the q-table to be
used in testing simulation and
freeze policy
:param s_fname: string.
Path to the qtable to be used
'''
# freeze policy if it is for
test and not for traning
if b_freezy_policy:
self.freeze_policy()
# load qtable and
transform in a dictionary
value_fun =
pickle.load(open(s_fname, 'r'))
self.value_function =
value_fun
# log
fi
le used
s_print = '{}.set_qtable():
Setting up the agent to use'
s_print =
s_print.format(self.s_agent_nam
e)
s_print += ' the Value
Function at {}'.format(s_fname)
# DEBUG
logging.info(s_print)
def stop_on_main(self, l_msg,
l_spread):
'''
Stop on the main
instrument
:param l_msg: list.
:param l_spread: list.
'''
s_main_action = ''
if
self.risk_model.should_stop_dis
closed(self):
if self.log_info['duration']
< 0.:
print '=',
s_main_action =
'SELL'
if self.log_info['duration']
> 0.:
print '=',
s_main_action =
'BUY'
if
self.env.order_matching.last_da
te > self.f_stop_time:
if not self.b_keep_pos:
if
self.log_info['duration'] < 0.:
print '>',
s_main_action =
'SELL'
if
self.log_info['duration'] > 0.:
print '>',
s_main_action =
'BUY'
# place orders in the best
price will be handle by the
spread
# in the next time the
agent updates its orders
# l_spread_main =
self._select_spread(self.state,
s_code)
if s_main_action in ['BUY',
'SELL']:
self.b_need_hedge =
False
l_msg +=
self.cancel_all_hedging_orders()
l_msg +=
self.translate_action(self.state,
s_main_action,
l_spread=l_spread)
return l_msg
return []
def msgs_due_hedge(self):
'''
Return messages given
that the agent needs to hedge
its positions
'''
# check if there are
reasons to hedge
l_aux =
self.risk_model.get_instruments
_to_hedge(self)
l_msg = []
if l_aux:
# print 'nHedging {} ...
n'.format(self.position['DI1F21']
)
s_, l_spread =
self._select_spread(self.state,
None)
s_action, s_instr, i_qty =
random.choice(l_aux)
# generate the
messages to the environment
my_book =
self.env.get_order_book(s_instr,
False)
row = {}
row['order_side'] = ''
row['order_price'] = 0.0
row['total_qty_order'] =
abs(i_qty)
row['instrumento_symbol'] =
s_instr
row['agent_id'] =
self.i_id
# check if should send
mkt orders in the main
instrument
l_rtn =
self.stop_on_main(l_msg,
l_spread)
if len(l_rtn) > 0:
# print 'stop on main'
s_time =
self.env.order_matching.s_time
print '{}: Stop loss.
{}'.format(s_time, l_aux)
return l_rtn
# generate trade and the
hedge instruments
s_time =
self.env.order_matching.s_time
print '{}: Stop gain.
{}'.format(s_time, l_aux)
if s_action == 'BUY':
self.b_need_hedge =
False
row['order_side'] =
'Buy Order'
row['order_price'] =
my_book.best_ask[0]
l_msg +=
self.cancel_all_hedging_orders()
l_msg +=
translator.translate_trades_to_a
gent(row, my_book)
return l_msg
elif s_action == 'SELL':
self.b_need_hedge =
False
row['order_side'] =
'Sell Order'
row['order_price'] =
my_book.best_bid[0]
l_msg +=
self.cancel_all_hedging_orders()
l_msg +=
translator.translate_trades_to_a
gent(row, my_book)
return l_msg
# generate limit order or
cancel everything
elif s_action ==
'BEST_BID':
f_curr_price,
i_qty_book = my_book.best_bid
l_spread = [0.,
self.f_spread_to_cancel]
elif s_action ==
'BEST_OFFER':
f_curr_price,
i_qty_book =
my_book.best_ask
l_spread =
[self.f_spread_to_cancel, 0.]
if s_action in
['BEST_BID', 'BEST_OFFER']:
i_order_size =
row['total_qty_order']
l_msg +=
translator.translate_to_agent(sel
f,
s_action,
my_book,
# worst t/TOB
l_spread,
i_qty=i_order_size)
return l_msg
else:
# if there is not need to
send any order, so there is no
# reason to hedge
self.b_need_hedge =
False
l_msg +=
self.cancel_all_hedging_orders()
return l_msg
self.b_need_hedge = False
return l_msg
def
cancel_all_hedging_orders(self):
'''
Cancel all hedging orders
that might be in the books
'''
l_aux = []
for s_instr in
self.risk_model.l_hedging_instr:
my_book =
self.env.get_order_book(s_instr,
False)
f_aux =
self.f_spread_to_cancel
l_aux +=
translator.translate_to_agent(sel
f,
None,
my_book,
# worst t/TOB
[f_aux, f_aux])
return l_aux
def _select_spread(self,
t_state, s_code=None):
'''
Select the spread to use in
a new order. Return the
criterium and
a list od spread
:param t_state: tuple. The
inputs to be considered by the
agent
'''
l_spread = [0.0, 0.0]
s_main =
self.env.s_main_intrument
my_book =
self.env.get_order_book(s_main,
False)
# check if it is a valid book
if abs(my_book.best_ask[0]
- my_book.best_bid[0]) <= 1e-6:
return s_code, [0.02,
0.02]
elif my_book.best_ask[0] -
my_book.best_bid[0] <= -0.01:
return s_code, [0.15,
0.15]
# check if should stop to
trade
if
self.risk_model.b_stop_trading:
return s_code, [0.04,
0.04]
# check if it is time to get
agressive due to closing market
if
self.env.order_matching.last_da
te > STOP_MKT_TIME:
if self.log_info['pos']
[s_main] < -0.01:
return s_code, [0.0,
0.04]
elif self.log_info['pos']
[s_main] > 0.01:
return s_code, [0.04,
0.0]
else:
return s_code, [0.04,
0.04]
# change spread
if not
self.risk_model.should_open_at
_current_price('ASK', self):
l_spread[1] = 0.01
elif not
self.risk_model.should_open_at
_current_price('BID', self):
l_spread[0] = 0.01
# if it just have close a
position at the speci
fi
c side
if
self.env.order_matching.f_time
< self.f_time_to_buy:
l_spread[0] = 0.01
if
self.env.order_matching.f_time
< self.f_time_to_sell:
l_spread[1] = 0.01
# check if can not open
positions due to limits
if not
self.risk_model.can_open_positi
on('ASK', self):
l_spread[1] = 0.02
elif not
self.risk_model.can_open_positi
on('BID', self):
l_spread[0] = 0.02
return s_code, l_spread
def should_print_logs(self,
s_question):
'''
Return if should print the
log based on s_question:
:param s_question: string.
All or 5MIN
'''
if self.b_print_always:
return True
if s_question == 'ALL':
return PRINT_ALL
elif s_question == '5MIN':
return PRINT_5MIN
return False
def set_to_print_always(self):
'''
'''
self.b_print_always = True
20. Metrics
7
state = [
(player.x_change == 20 and player.y_change == 0 and ((list(map(add, player.position[-1], [20, 0])) in player.position) or
player.position[-1][0] + 20 >= (game.game_width - 20))) or (player.x_change == -20 and player.y_change == 0 and ((list(map(add, player.position[-1], [-20, 0])) in player.position) or
player.position[-1][0] - 20 < 20)) or (player.x_change == 0 and player.y_change == -20 and ((list(map(add, player.position[-1], [0, -20])) in player.position) or
player.position[-1][-1] - 20 < 20)) or (player.x_change == 0 and player.y_change == 20 and ((list(map(add, player.position[-1], [0, 20])) in player.position) or
player.position[-1][-1] + 20 >= (game.game_height-20))), # danger straight
(player.x_change == 0 and player.y_change == -20 and ((list(map(add,player.position[-1],[20, 0])) in player.position) or
player.position[ -1][0] + 20 > (game.game_width-20))) or (player.x_change == 0 and player.y_change == 20 and ((list(map(add,player.position[-1],
[-20,0])) in player.position) or player.position[-1][0] - 20 < 20)) or (player.x_change == -20 and player.y_change == 0 and ((list(map(
add,player.position[-1],[0,-20])) in player.position) or player.position[-1][-1] - 20 < 20)) or (player.x_change == 20 and player.y_change == 0 and (
(list(map(add,player.position[-1],[0,20])) in player.position) or player.position[-1][
-1] + 20 >= (game.game_height-20))), # danger right
(player.x_change == 0 and player.y_change == 20 and ((list(map(add,player.position[-1],[20,0])) in player.position) or
player.position[-1][0] + 20 > (game.game_width-20))) or (player.x_change == 0 and player.y_change == -20 and ((list(map(
add, player.position[-1],[-20,0])) in player.position) or player.position[-1][0] - 20 < 20)) or (player.x_change == 20 and player.y_change == 0 and (
(list(map(add,player.position[-1],[0,-20])) in player.position) or player.position[-1][-1] - 20 < 20)) or (
player.x_change == -20 and player.y_change == 0 and ((list(map(add,player.position[-1],[0,20])) in player.position) or
player.position[-1][-1] + 20 >= (game.game_height-20))), #danger left
player.x_change == -20, # move left
player.x_change == 20, # move right
player.y_change == -20, # move up
player.y_change == 20, # move down
food.x_food < player.x, # food left
food.x_food > player.x, # food right
food.y_food < player.y, # food up
food.y_food > player.y # food down
]
Multiply-nested container
State space: 11
depth: 5
24. Conclusion
9
RL projects contain many code smells - 3.15 average per file
- can be up to 8 per file (or 1 code smell every 27 lines)
25. Conclusion
9
RL projects contain many code smells - 3.15 average per file
- can be up to 8 per file (or 1 code smell every 27 lines)
From top 4 most common code smells 3 are shared across the 2 data sets
(Multiply-Nested Container, Long Method, Long Parameter List)
26. Conclusion
9
RL projects contain many code smells - 3.15 average per file
- can be up to 8 per file (or 1 code smell every 27 lines)
From top 4 most common code smells 3 are shared across the 2 data sets
(Multiply-Nested Container, Long Method, Long Parameter List)
State representations
are inherently complex
27. Conclusion
9
RL projects contain many code smells - 3.15 average per file
- can be up to 8 per file (or 1 code smell every 27 lines)
From top 4 most common code smells 3 are shared across the 2 data sets
(Multiply-Nested Container, Long Method, Long Parameter List)
State representations
are inherently complex
Functionality is presented
as a code block
28. Conclusion
9
RL projects contain many code smells - 3.15 average per file
- can be up to 8 per file (or 1 code smell every 27 lines)
From top 4 most common code smells 3 are shared across the 2 data sets
(Multiply-Nested Container, Long Method, Long Parameter List)
State representations
are inherently complex
Functionality is presented
as a code block
RL algorithms are riddle
with learning parameters
29. Conclusion
9
RL projects contain many code smells - 3.15 average per file
- can be up to 8 per file (or 1 code smell every 27 lines)
From top 4 most common code smells 3 are shared across the 2 data sets
(Multiply-Nested Container, Long Method, Long Parameter List)
State representations
are inherently complex
Functionality is presented
as a code block
RL algorithms are riddle
with learning parameters
Code smells point to a violation of the design principles
(coupling, cohesion, single responsibility)
32. Future perspectives
10
specific metrics and
code smells for RL
we need specific metrics, thresholds, and tools
to capture the complexity of RL algorithms
The complexity of RL can be managed with the
creation of dedicated data structures or express
relations between entities more ergonomically