Models in Genetic Based Machine Learning (GBML) systems are commonly used to gain understanding of how the system works and, as a consequence, adjust it better. In this paper we propose models for the probability of having a good initial population using the Attribute List Knowledge Representation (ALKR) for discrete inputs using the GABIL encoding. We base our work in the schema and covering bound models previously proposed for XCS. The models are extended to (a) deal with the combination of ALKR+GABIL representation, (b) explicitly handle datasets with niche overlap and (c) model the impact of using covering and a default rule in the representation. The models are designed and evaluated within the framework of the BioHEL GBML system and are empirically evaluated using first boolean datasets and later also nominal datasets of higher cardinality. The models in this paper allow us to evaluate the challenges presented by problems with high cardinality (in terms of number of attributes and values of the attributes) as well as the benefits contributed by each of the components of BioHEL's representation and initialisation operators.
Neural object classification by pattern recognition of one dimensional data...naschibo
Pattern
Groups of measurements or observations, defining points in an appropriate multidimensional space.
Pattern recognition
Aims to classify data (patterns) based on either a priori knowledge or on statistical information extracted from the patterns.
My aim is to classify 2-dimensional objects with the help of manipulator and by the use of pattern recognition.
Neural object classification by pattern recognition of one dimensional data...naschibo
Pattern
Groups of measurements or observations, defining points in an appropriate multidimensional space.
Pattern recognition
Aims to classify data (patterns) based on either a priori knowledge or on statistical information extracted from the patterns.
My aim is to classify 2-dimensional objects with the help of manipulator and by the use of pattern recognition.
Artificial intelligence (AI) is everywhere, promising self-driving cars, medical breakthroughs, and new ways of working. But how do you separate hype from reality? How can your company apply AI to solve real business problems?
Here’s what AI learnings your business should keep in mind for 2017.
Study: The Future of VR, AR and Self-Driving CarsLinkedIn
We asked LinkedIn members worldwide about their levels of interest in the latest wave of technology: whether they’re using wearables, and whether they intend to buy self-driving cars and VR headsets as they become available. We asked them too about their attitudes to technology and to the growing role of Artificial Intelligence (AI) in the devices that they use. The answers were fascinating – and in many cases, surprising.
This SlideShare explores the full results of this study, including detailed market-by-market breakdowns of intention levels for each technology – and how attitudes change with age, location and seniority level. If you’re marketing a tech brand – or planning to use VR and wearables to reach a professional audience – then these are insights you won’t want to miss.
Substructrual surrogates for learning decomposable classification problems: i...kknsastry
This paper presents a learning methodology based on a substructural classification model to solve decomposable classification problems. The proposed method consists of three important components: (1) a structural model that represents salient interactions between attributes for a given data, (2) a surrogate model which provides a functional approximation of the output as a function of attributes, and (3) a classification model which predicts the class for new inputs. The structural model is used to infer the functional form of the surrogate and its coefficients are estimated using linear regression methods. The classification model uses a maximally-accurate, least-complex surrogate to predict the output for given inputs. The structural model that yields an optimal classification model is searched using an iterative greedy search heuristic. Results show that the proposed method successfully detects the interacting variables in hierarchical problems, group them in linkages groups, and build maximally accurate classification models. The initial results on non-trivial hierarchical test problems indicate that the proposed method holds promise and have also shed light on several improvements to enhance the capabilities of the proposed method.
Low Power High-Performance Computing on the BeagleBoard Platforma3labdsp
The ever increasing energy requirements of supercomputers and server farms is driving the scientific and industrial communities to take in deeper consideration the energy efficiency of computing equipments. This contribution addresses the issue proposing a cluster of ARM processors for high-performance computing. The cluster is composed of five BeagleBoard-xM, with one board managing the cluster, and the other boards executing the actual processing. The software platform is based on the Angstrom GNU/Linux distribution and is equipped with a distributed file system to ease sharing data and code among the nodes of the cluster, and with tools for managing tasks and monitoring the status of each node. The computational capabilities of the cluster have been assessed through High-Performance Linpack and a cluster-wide speaker diarization algorithm, while power consumption has been measured using a clamp meter. Experimental results obtained in the speaker diarization task showed that the energy efficiency of the BeagleBoard-xM cluster is comparable to the one of a laptop computer equipped with a Intel Core2 Duo T8300 running at 2.4 GHz. Furthermore, removing the bottleneck due to the Ethernet interface, the BeagleBoard-xM cluster is able to achieve a superior energy efficiency.
Model Agnostic Meta-Learning (MAML) is one of the most representative of gradient-based meta-learning algorithms. MAML learns new tasks with a few data samples using inner updates from a meta-initialization point and learns the meta-initialization parameters with outer updates. It has recently been hypothesized that representation reuse, which makes little change in efficient representations, is the dominant factor in the performance of the meta-initialized model through MAML in contrast to representation change, which causes a significant change in representations. In this study, we investigate the necessity of representation change for the ultimate goal of few-shot learning, which is solving domain-agnostic tasks. To this aim, we propose a novel meta-learning algorithm, called BOIL (Body Only update in Inner Loop), which updates only the body (extractor) of the model and freezes the head (classifier) during inner loop updates. BOIL leverages representation change rather than representation reuse. This is because feature vectors (representations) have to move quickly to their corresponding frozen head vectors. We visualize this property using cosine similarity, CKA, and empirical results without the head. BOIL empirically shows significant performance improvement over MAML, particularly on cross-domain tasks. The results imply that representation change in gradient-based meta-learning approaches is a critical component.
Meta Dropout: Learning to Perturb Latent Features for Generalization MLAI2
A machine learning model that generalizes well should obtain low errors on unseen test examples. Thus, if we know how to optimally perturb training examples to account for test examples, we may achieve better generalization performance. However, obtaining such perturbation is not possible in standard machine learning frameworks as the distribution of the test data is unknown. To tackle this challenge, we propose a novel regularization method, meta-dropout, which learns to perturb the latent features of training examples for generalization in a meta-learning framework. Specifically, we meta-learn a noise generator which outputs a multiplicative noise distribution for latent features, to obtain low errors on the test instances in an input-dependent manner. Then, the learned noise generator can perturb the training examples of unseen tasks at the meta-test time for improved generalization. We validate our method on few-shot classification datasets, whose results show that it significantly improves the generalization performance of the base model, and largely outperforms existing regularization methods such as information bottleneck, manifold mixup, and information dropout.
Artificial intelligence (AI) is everywhere, promising self-driving cars, medical breakthroughs, and new ways of working. But how do you separate hype from reality? How can your company apply AI to solve real business problems?
Here’s what AI learnings your business should keep in mind for 2017.
Study: The Future of VR, AR and Self-Driving CarsLinkedIn
We asked LinkedIn members worldwide about their levels of interest in the latest wave of technology: whether they’re using wearables, and whether they intend to buy self-driving cars and VR headsets as they become available. We asked them too about their attitudes to technology and to the growing role of Artificial Intelligence (AI) in the devices that they use. The answers were fascinating – and in many cases, surprising.
This SlideShare explores the full results of this study, including detailed market-by-market breakdowns of intention levels for each technology – and how attitudes change with age, location and seniority level. If you’re marketing a tech brand – or planning to use VR and wearables to reach a professional audience – then these are insights you won’t want to miss.
Substructrual surrogates for learning decomposable classification problems: i...kknsastry
This paper presents a learning methodology based on a substructural classification model to solve decomposable classification problems. The proposed method consists of three important components: (1) a structural model that represents salient interactions between attributes for a given data, (2) a surrogate model which provides a functional approximation of the output as a function of attributes, and (3) a classification model which predicts the class for new inputs. The structural model is used to infer the functional form of the surrogate and its coefficients are estimated using linear regression methods. The classification model uses a maximally-accurate, least-complex surrogate to predict the output for given inputs. The structural model that yields an optimal classification model is searched using an iterative greedy search heuristic. Results show that the proposed method successfully detects the interacting variables in hierarchical problems, group them in linkages groups, and build maximally accurate classification models. The initial results on non-trivial hierarchical test problems indicate that the proposed method holds promise and have also shed light on several improvements to enhance the capabilities of the proposed method.
Low Power High-Performance Computing on the BeagleBoard Platforma3labdsp
The ever increasing energy requirements of supercomputers and server farms is driving the scientific and industrial communities to take in deeper consideration the energy efficiency of computing equipments. This contribution addresses the issue proposing a cluster of ARM processors for high-performance computing. The cluster is composed of five BeagleBoard-xM, with one board managing the cluster, and the other boards executing the actual processing. The software platform is based on the Angstrom GNU/Linux distribution and is equipped with a distributed file system to ease sharing data and code among the nodes of the cluster, and with tools for managing tasks and monitoring the status of each node. The computational capabilities of the cluster have been assessed through High-Performance Linpack and a cluster-wide speaker diarization algorithm, while power consumption has been measured using a clamp meter. Experimental results obtained in the speaker diarization task showed that the energy efficiency of the BeagleBoard-xM cluster is comparable to the one of a laptop computer equipped with a Intel Core2 Duo T8300 running at 2.4 GHz. Furthermore, removing the bottleneck due to the Ethernet interface, the BeagleBoard-xM cluster is able to achieve a superior energy efficiency.
Model Agnostic Meta-Learning (MAML) is one of the most representative of gradient-based meta-learning algorithms. MAML learns new tasks with a few data samples using inner updates from a meta-initialization point and learns the meta-initialization parameters with outer updates. It has recently been hypothesized that representation reuse, which makes little change in efficient representations, is the dominant factor in the performance of the meta-initialized model through MAML in contrast to representation change, which causes a significant change in representations. In this study, we investigate the necessity of representation change for the ultimate goal of few-shot learning, which is solving domain-agnostic tasks. To this aim, we propose a novel meta-learning algorithm, called BOIL (Body Only update in Inner Loop), which updates only the body (extractor) of the model and freezes the head (classifier) during inner loop updates. BOIL leverages representation change rather than representation reuse. This is because feature vectors (representations) have to move quickly to their corresponding frozen head vectors. We visualize this property using cosine similarity, CKA, and empirical results without the head. BOIL empirically shows significant performance improvement over MAML, particularly on cross-domain tasks. The results imply that representation change in gradient-based meta-learning approaches is a critical component.
Meta Dropout: Learning to Perturb Latent Features for Generalization MLAI2
A machine learning model that generalizes well should obtain low errors on unseen test examples. Thus, if we know how to optimally perturb training examples to account for test examples, we may achieve better generalization performance. However, obtaining such perturbation is not possible in standard machine learning frameworks as the distribution of the test data is unknown. To tackle this challenge, we propose a novel regularization method, meta-dropout, which learns to perturb the latent features of training examples for generalization in a meta-learning framework. Specifically, we meta-learn a noise generator which outputs a multiplicative noise distribution for latent features, to obtain low errors on the test instances in an input-dependent manner. Then, the learned noise generator can perturb the training examples of unseen tasks at the meta-test time for improved generalization. We validate our method on few-shot classification datasets, whose results show that it significantly improves the generalization performance of the base model, and largely outperforms existing regularization methods such as information bottleneck, manifold mixup, and information dropout.
Labelled Variables in Logic Programming: A First Prototipe in tuPrologRoberta Calegari
We present the first prototype of Labelled tuProlog, an extension of tuProlog exploiting labelled variables to enable a sort of multi- paradigm / multi-language programming aimed at pervasive systems.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™UiPathCommunity
In questo evento online gratuito, organizzato dalla Community Italiana di UiPath, potrai esplorare le nuove funzionalità di Autopilot, il tool che integra l'Intelligenza Artificiale nei processi di sviluppo e utilizzo delle Automazioni.
📕 Vedremo insieme alcuni esempi dell'utilizzo di Autopilot in diversi tool della Suite UiPath:
Autopilot per Studio Web
Autopilot per Studio
Autopilot per Apps
Clipboard AI
GenAI applicata alla Document Understanding
👨🏫👨💻 Speakers:
Stefano Negro, UiPath MVPx3, RPA Tech Lead @ BSP Consultant
Flavio Martinelli, UiPath MVP 2023, Technical Account Manager @UiPath
Andrei Tasca, RPA Solutions Team Lead @NTT Data
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx
Modelling the Initialisation Stage of the ALKR Representation for Discrete Domains and GABIL Encoding
1. Modelling the Initialisation Stage of the ALKR
Representation for Discrete Domains and
GABIL Encoding
María A. Franco, Natalio Krasnogor, Jaume Bacardit
University of Nottingham, UK.
ASAP Research Group,
School of Computer Science
mxf@cs.nott.ac.uk
July 14, 2011
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 1 / 25
2. Problem definition
BioHEL[Bacardit et al., 2009a] is a Genetic Based Machine
Learning (GBML) designed to cope with large scale
datasets[Bacardit et al., 2009b].
Iterative Rule Learning approach
Attribute List Knowledge Representation (ALKR)
ILAS Windowing scheme
Default rule
Smart initialisation mechanisms (covering)
GPU-based evaluation process
Problem
The system obtains good results [Stout et al., 2008], but we do not
have a formal understanding of why, when and how this happens.
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 2 / 25
3. Problem definition
BioHEL[Bacardit et al., 2009a] is a Genetic Based Machine
Learning (GBML) designed to cope with large scale
datasets[Bacardit et al., 2009b].
Iterative Rule Learning approach
Attribute List Knowledge Representation (ALKR)
ILAS Windowing scheme
Default rule
Smart initialisation mechanisms (covering)
GPU-based evaluation process
Problem
The system obtains good results [Stout et al., 2008], but we do not
have a formal understanding of why, when and how this happens.
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 2 / 25
4. What is the aim of this work?
The aim of this work is to model the initialisation stage of the BioHEL
system and calculate the probability of having a good initial
population. Two conditions should be meet[Goldberg, 2002]:
A good individual exists in an initial population (building blocks)
The initial population covers the whole search space
Background
These probabilities are also know as schema and covering bound.
This have already being determined for XCS and the ternary
representation {1,0,#} by [Butz, 2006].
Problem
Models need to be adapted for our ALKR+GABIL representation.
Moreover, we want to model the impact of the BioHEL mechanisms
that are relevant in initialisation: covering and default rule.
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 3 / 25
5. What is the aim of this work?
The aim of this work is to model the initialisation stage of the BioHEL
system and calculate the probability of having a good initial
population. Two conditions should be meet[Goldberg, 2002]:
A good individual exists in an initial population (building blocks)
The initial population covers the whole search space
Background
These probabilities are also know as schema and covering bound.
This have already being determined for XCS and the ternary
representation {1,0,#} by [Butz, 2006].
Problem
Models need to be adapted for our ALKR+GABIL representation.
Moreover, we want to model the impact of the BioHEL mechanisms
that are relevant in initialisation: covering and default rule.
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 3 / 25
6. What is the aim of this work?
The aim of this work is to model the initialisation stage of the BioHEL
system and calculate the probability of having a good initial
population. Two conditions should be meet[Goldberg, 2002]:
A good individual exists in an initial population (building blocks)
The initial population covers the whole search space
Background
These probabilities are also know as schema and covering bound.
This have already being determined for XCS and the ternary
representation {1,0,#} by [Butz, 2006].
Problem
Models need to be adapted for our ALKR+GABIL representation.
Moreover, we want to model the impact of the BioHEL mechanisms
that are relevant in initialisation: covering and default rule.
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 3 / 25
7. 1 Background
GABIL Representation
Attribute List Knowledge Representation (ALKR)
2 Probabilistic models
Initial considerations
Schema bound
How does the overlapping affects?
Covering bound
3 Generalised model for x-ary attributes
Schema and Covering bound
4 Conclusions and Further Work
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 4 / 25
8. How does GABIL works?
The GABIL representation[Jong and Spears, 1991] is used inside
ALKR to represent nominal attributes.
Example
F1 ={A,B,C} F2={O,P} F3={W,Z,X,Y}
F1 F2 F3
100 01 1101
F1 is A ∧ F2 is P ∧ (F3 is W ∨ F3 is Z ∨ F3 is Y)
In GABIL, when initialising the attribute values we set the bit to 1 with
probability p and to 0 with probability 1 − p
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 5 / 25
9. How does GABIL works?
The GABIL representation[Jong and Spears, 1991] is used inside
ALKR to represent nominal attributes.
Example
F1 ={A,B,C} F2={O,P} F3={W,Z,X,Y}
F1 F2 F3
100 01 1101
F1 is A ∧ F2 is P ∧ (F3 is W ∨ F3 is Z ∨ F3 is Y)
In GABIL, when initialising the attribute values we set the bit to 1 with
probability p and to 0 with probability 1 − p
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 5 / 25
10. How does Attribute List Knowledge Representation works?
ALKR Classifier Example
numAtt 3
whichAtt 0
predicates 0.5 0.7 0.3
offsetPred 0
class 1
How do we select the attributes in the list?
1 d <= ExpAtts
ld = ExpAtts
d d > ExpAtts
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 6 / 25
11. Initial considerations for the probabilistic models
Mechanisms involved in initialisation
Covering ⇒ We have to consider 4
initialisation scenarios
Default Rule
Types of attributes
Fully mapped attributes
Partially mapped attributes.
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 7 / 25
12. Initial considerations for the probabilistic models
Mechanisms involved in initialisation
Covering ⇒ We have to consider 4
initialisation scenarios
Default Rule
Types of attributes
Fully mapped attributes
Partially mapped attributes.
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 7 / 25
13. Initial considerations for the probabilistic models
Mechanisms involved in initialisation
Covering ⇒ We have to consider 4
initialisation scenarios
Default Rule
Types of attributes
Fully mapped attributes
Partially mapped attributes.
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 7 / 25
14. Schema bound
Problem
We want to calculate the probability of having good classifiers or
representatives in an initial population. Classifiers that do not make
mistakes, since they represent correctly all the specified bits in an
original problem rule.
Example
Considering the rule #10#1 with 3 values specified (k=3), the following
classifiers are representatives: 110*1, 11011, 010*1.
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 8 / 25
15. Schema bound
Problem
We want to calculate the probability of having good classifiers or
representatives in an initial population. Classifiers that do not make
mistakes, since they represent correctly all the specified bits in an
original problem rule.
Example
Considering the rule #10#1 with 3 values specified (k=3), the following
classifiers are representatives: 110*1, 11011, 010*1.
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 8 / 25
16. Schema bound
Question
What is the probability of obtaining a representative with at least k
values specified?
To become a representative the rule should:
1 Specify at least k attributes correctly.
2 The rest of the attributes should not have all 0’s.
k d−k
2 f (ld p(1−p))k (1−ld (1−p)2 )
P(rep) =
where kf is the number of fully map attributes
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 9 / 25
17. Schema bound
Question
What is the probability of obtaining a representative with at least k
values specified?
To become a representative the rule should:
1 Specify at least k attributes correctly.
2 The rest of the attributes should not have all 0’s.
Without using any of the mechanisms:
k d−k
2 f (ld p(1−p))k (1−ld (1−p)2 )
P(rep) = n
where kf is the number of fully map attributes
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 9 / 25
18. Schema bound
Question
What is the probability of obtaining a representative with at least k
values specified?
To become a representative the rule should:
1 Specify at least k attributes correctly.
2 The rest of the attributes should not have all 0’s.
Using default rule:
k d−k
2 f (ld p(1−p))k (1−ld (1−p)2 )
P(rep) = n−1
where kf is the number of fully map attributes
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 9 / 25
19. Schema bound
Question
What happens when we use covering?
1 We sample an instance with uniform probabilities for all classes.
2 We set the bits corresponding to the instance values to 1.
It is not possible to have all 0’s anymore.
P(rep) = m
(ld (1 − p))k
where m is the number of classes mapped by the problem rules
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 10 / 25
20. Schema bound
Question
What happens when we use covering?
1 We sample an instance with uniform probabilities for all classes.
2 We set the bits corresponding to the instance values to 1.
It is not possible to have all 0’s anymore.
P(rep) = m
n (ld (1 − p))k
where m is the number of classes mapped by the problem rules
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 10 / 25
21. Schema bound
Question
What happens when we use covering and default rule?
1 We sample an instance with uniform probabilities for all classes.
2 We set the bits corresponding to the instance values to 1.
It is not possible to have all 0’s anymore.
P(rep) = m
n−1 (ld (1 − p))k
where m is the number of classes mapped by the problem rules
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 10 / 25
22. Problems used for model validation
Binary and Ternary Multiplexer problems
k address bits
2k string bits (3k for ternary case)
k-Disjuntive Normal Functions
[Butz and Pelikan, 2006, Franco et al., 2010].
r disjunctive terms
d possible attributes
k represented attributes in each term
Example kDNF: d = 10, k = 3, r = 3
(¬x1 ∧ x5 ∧ x7 ) ∨ (x1 ∧ ¬x2 ∧ x8 ) ∨ (x4 ∧ ¬x5 ∧ ¬x9 )
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 11 / 25
23. Problems used for model validation
Binary and Ternary Multiplexer problems
k address bits
2k string bits (3k for ternary case)
k-Disjuntive Normal Functions
[Butz and Pelikan, 2006, Franco et al., 2010].
r disjunctive terms
d possible attributes
k represented attributes in each term
Example kDNF: d = 10, k = 3, r = 3
(¬x1 ∧ x5 ∧ x7 ) ∨ (x1 ∧ ¬x2 ∧ x8 ) ∨ (x4 ∧ ¬x5 ∧ ¬x9 )
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 11 / 25
25. What have we calculated so far?
These models so far only hold for:
Problems with Problems that have just
no-overlapping one rule
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 13 / 25
26. What happens here?
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 14 / 25
27. How does the overlapping affects the probability of a
representative?
P(rep)
P(niche) =
r
1 ExamplesNiche (EN)
=
? ExamplesCovered (EC)
r
EC = 2d 1 − 1 − 2−k
2d
EN =
2k
P (rep) = 1 − (1 − P(niche))r
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 15 / 25
28. How does the overlapping affects the probability of a
representative?
P(rep)
P(niche) =
r
1 ExamplesNiche (EN)
=
? ExamplesCovered (EC)
r
EC = 2d 1 − 1 − 2−k
2d
EN =
2k
P (rep) = 1 − (1 − P(niche))r
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 15 / 25
29. How does the overlapping affects the probability of a
representative?
P(rep)
P(niche) =
?
1 ExamplesNiche (EN)
=
? ExamplesCovered (EC)
r
EC = 2d 1 − 1 − 2−k
2d
EN =
2k
P (rep) = 1 − (1 − P(niche))r
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 15 / 25
30. How does the overlapping affects the probability of a
representative?
P(rep)
P(niche) =
?
1 ExamplesNiche (EN)
=
? ExamplesCovered (EC)
r
EC = 2d 1 − 1 − 2−k
2d
EN =
2k
P (rep) = 1 − (1 − P(niche))r
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 15 / 25
31. How does the overlapping affects the probability of a
representative?
P(rep)
P(niche) =
?
1 ExamplesNiche (EN)
=
? ExamplesCovered (EC)
r
EC = 2d 1 − 1 − 2−k
2d
EN =
2k
P (rep) = 1 − (1 − P(niche))r
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 15 / 25
32. How does the overlapping affects the probability of a
representative?
P(rep)
P(niche) =
?
1 ExamplesNiche (EN)
=
? ExamplesCovered (EC)
r
EC = 2d 1 − 1 − 2−k
2d
EN =
2k
P (rep) = 1 − (1 − P(niche))r
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 15 / 25
33. How does the overlapping affects the probability of a
representative?
P(rep)
P(niche) =
2k 1 − (1 − 2−k )r
1 ExamplesNiche (EN)
=
? ExamplesCovered (EC)
r
EC = 2d 1 − 1 − 2−k
2d
EN =
2k
P (rep) = 1 − (1 − P(niche))r
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 15 / 25
34. Validation of models considering overlapping
Teoretical Teoretical
P(rep) Empirical r=1 P(rep) Empirical r=1
Empirical r=5 Empirical r=5
1 Empirical r=10 1 Empirical r=10
Empirical r=20 Empirical r=20
0.8 Empirical r=40 0.8 Empirical r=40
0.6 0.6
0.4 0.4
0.2 0.2
0 0
25 25
0 2 5 0 2 5
4 # of rules 4 # of rules
Atts esp (k) 6 8 Atts esp (k) 6 8
10 1 10 1
(e) Base Case (f) Covering and Default Class
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 16 / 25
35. Covering bound
Problem
How can we calculate the probability of covering the whole search
space?
We need to calculate the probability of matching an instance
d
Base case P(match) = (1 − ld + ld p)
d
1+p
Covering case P(match) = 1 − ld + ld 2
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 17 / 25
36. Covering bound
Problem
How can we calculate the probability of covering the whole search
space?
We need to calculate the probability of matching an instance
d
Base case P(match) = (1 − ld + ld p)
d
1+p
Covering case P(match) = 1 − ld + ld 2
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 17 / 25
37. Covering bound
Problem
How can we calculate the probability of covering the whole search
space?
We need to calculate the probability of matching an instance
d
Base case P(match) = (1 − ld + ld p)
d
1+p
Covering case P(match) = 1 − ld + ld 2
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 17 / 25
38. Covering bound - Model validation
(g) No covering (h) Covering
1 1
Empirical p=0.75 Empirical p=0.75
Model p=0.75 Model p=0.75
Empirical p=0.50 Empirical p=0.50
0.8 Model p=0.50 0.8 Model p=0.50
Empirical p=0.25 Empirical p=0.25
Model p=0.25 Model p=0.25
0.6 0.6
P(match)
P(match)
0.4 0.4
0.2 0.2
0 0
0 5 10 15 20 0 5 10 15 20
k - Number of Attributes k - Number of Attributes
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 18 / 25
39. What happens with x-ary attributes?
What happens when the problem is not binary but has more than 2
values per attribute?
Generalised models for x-ary attributes
Where t is the number of values per attribute and e is the number of
active bits per attribute.
Example 1: 101|110|011:0 ⇒ t=3 e=2
Example 2: 001|100|010:1 ⇒ t=3 e=1
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 19 / 25
40. What happens with x-ary attributes?
What happens when the problem is not binary but has more than 2
values per attribute?
Generalised models for x-ary attributes
Where t is the number of values per attribute and e is the number of
active bits per attribute.
Example 1: 101|110|011:0 ⇒ t=3 e=2
Example 2: 001|100|010:1 ⇒ t=3 e=1
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 19 / 25
41. Generalised model for x-ary attributes
Schema bound
k d−k
tkf (ld pe (1−p)t−e ) (1−ld (1−p)t )
Base case P(rep) = n
k
m t−e−1
Covering case P(rep) = n ld pe−1 (1 − p)
Covering bound
d
Base case P(match) = (1 − ld + ld p)
d
1+(t−1)p
Covering case P(match) = 1 − ld + ld t
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 20 / 25
42. Generalised model for x-ary attributes
Schema bound with Default Rule
k d−k
tkf (ld pe (1−p)t−e ) (1−ld (1−p)t )
Base case P(rep) = n−1
k
m t−e−1
Covering case P(rep) = n−1 ld pe−1 (1 − p)
Covering bound
d
Base case P(match) = (1 − ld + ld p)
d
1+(t−1)p
Covering case P(match) = 1 − ld + ld t
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 20 / 25
43. Generalised model for x-ary attributes
Schema bound validation (with ternary multiplexer problems)
(i) No covering (j) Covering
0.16 0.6
Empirical p=0.75 Empirical p=0.75
0.14 Model p=0.75 Model p=0.75
Empirical p=0.50 0.5 Empirical p=0.50
0.12 Model p=0.50 Model p=0.50
Empirical p=0.25 Empirical p=0.25
Model p=0.25 0.4 Model p=0.25
0.1
P(rep)
P(rep)
0.08 0.3
0.06
0.2
0.04
0.1
0.02
0 0
1 2 3 4 5 6 1 2 3 4 5 6
k - Number of Attributes k - Number of Attributes
≈ 5 times more probability of generating
a good individual when using covering
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 21 / 25
44. Generalised model for x-ary attributes
Covering bound validation (with ternary multiplexer problems)
(k) No covering (l) Covering
1 1
Empirical p=0.75 Empirical p=0.75
Model p=0.75 Model p=0.75
Empirical p=0.50 Empirical p=0.50
0.8 Model p=0.50 0.8 Model p=0.50
Empirical p=0.25 Empirical p=0.25
Model p=0.25 Model p=0.25
0.6 0.6
P(match)
P(match)
0.4 0.4
0.2 0.2
0 0
2 4 6 8 10 12 14 2 4 6 8 10 12 14
k - Number of Attributes k - Number of Attributes
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 22 / 25
45. Conclusions
The presented models explains what is the probability of having
a good initial population in BioHEL considering de ALKR
representation and other initialisation mechanisms.
We also presented a generalisation of the model for x-ary
attributes and adjusted the probability for problems with
overlapping.
These models explain the benefits of BioHEL initialisation
mechanisms giving a further understanding of how the BioHEL
system works.
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 23 / 25
46. Further Work
Simplify the current models to make them less dependent on
problem parameters not known beforehand.
Model the reproductive opportunity and learning time of BioHEL.
Derive boundaries for the population size and other user-defined
parameters in BioHEL.
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 24 / 25
47. Modelling the Initialisation Stage of the ALKR
Representation for Discrete Domains and
GABIL Encoding
María A. Franco, Natalio Krasnogor, Jaume Bacardit
University of Nottingham, UK.
ASAP Research Group,
School of Computer Science
mxf@cs.nott.ac.uk
July 14, 2011
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 25 / 25
48. Bacardit, J., Burke, E., and Krasnogor, N. (2009a).
Improving the scalability of rule-based evolutionary learning.
Memetic Computing, 1(1):55–67.
Bacardit, J., Stout, M., Hirst, J. D., Valencia, A., Smith, R., and Krasnogor, N. (2009b).
Automated alphabet reduction for protein datasets.
BMC Bioinformatics, 10(1):6.
Butz, M. V. (2006).
Rule-Based Evolutionary Online Learning Systems: A Principled Approach to LCS Analysis and Design, volume 109 of
Studies in Fuzziness and Soft Computing.
Springer.
Butz, M. V. and Pelikan, M. (2006).
Studying XCS/BOA learning in boolean functions: structure encoding and random boolean functions.
In GECCO ’06: Proceedings of the 8th annual conference on Genetic and evolutionary computation, pages 1449–456,
New York, NY, USA. ACM.
Franco, M. A., Krasnogor, N., and Bacardit, J. (2010).
Analysing biohel using challenging boolean functions.
In GECCO ’10: Proceedings of the 12th annual conference comp on Genetic and evolutionary computation, pages
1855–1862, New York, NY, USA. ACM.
Goldberg, D. E. (2002).
The Design of Innovation: Lessons from and for Competent Genetic Algorithms.
Kluwer Academic Publishers, Norwell, MA, USA.
Jong, K. D. and Spears, W. M. (1991).
Learning concept classification rules using genetic algorithms.
In Proceedings of the 12th international joint conference on Artificial intelligence - Volume 2, pages 651–656, Sydney,
New South Wales, Australia. Morgan Kaufmann Publishers Inc.
Stout, M., Bacardit, J., Hirst, J. D., and Krasnogor, N. (2008).
Prediction of recursive convex hull class assignments for protein residues.
Bioinformatics, 24(7):916–923.
Franco et al. (University of Nottingham) Modelling Initialisation using ALKR+GABIL July 14, 2011 25 / 25