https://telecombcn-dl.github.io/2018-dlai/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
Reinforcement Learning (RL) approaches to deal with finding an optimal reward based policy to act in an environment (Charla en Inglés)
However, what has led to their widespread use is its combination with deep neural networks (DNN) i.e., deep reinforcement learning (Deep RL). Recent successes on not only learning to play games but also superseding humans in it and academia-industry research collaborations like for manipulation of objects, locomotion skills, smart grids, etc. have surely demonstrated their case on a wide variety of challenging tasks.
With application spanning across games, robotics, dialogue, healthcare, marketing, energy and many more domains, Deep RL might just be the power that drives the next generation of Artificial Intelligence (AI) agents!
Machine Learning and Data Mining: 05 Advanced Association Rule MiningPier Luca Lanzi
Course "Machine Learning and Data Mining" for the degree of Computer Engineering at the Politecnico di Milano. This lecture overviews the FP-growth algorithm, methods for multilevel rules, correlation rules, and sequential rules.
Machine Learning and Data Mining: 09 Clustering: Density-based, Grid-based, M...Pier Luca Lanzi
Course "Machine Learning and Data Mining" for the degree of Computer Engineering at the Politecnico di Milano. This lecture introduces density-based, grid-based, and model-based clustering
Machine Learning and Data Mining: 08 Clustering: Hierarchical Pier Luca Lanzi
Course "Machine Learning and Data Mining" for the degree of Computer Engineering at the Politecnico di Milano. This lecture introduces hierarchical clustering.
Machine Learning and Data Mining: 04 Association Rule MiningPier Luca Lanzi
Course "Machine Learning and Data Mining" for the degree of Computer Engineering at the Politecnico di Milano. This lecture introduces association rule mining and the Apriori algorithm
https://telecombcn-dl.github.io/2018-dlai/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
Reinforcement Learning (RL) approaches to deal with finding an optimal reward based policy to act in an environment (Charla en Inglés)
However, what has led to their widespread use is its combination with deep neural networks (DNN) i.e., deep reinforcement learning (Deep RL). Recent successes on not only learning to play games but also superseding humans in it and academia-industry research collaborations like for manipulation of objects, locomotion skills, smart grids, etc. have surely demonstrated their case on a wide variety of challenging tasks.
With application spanning across games, robotics, dialogue, healthcare, marketing, energy and many more domains, Deep RL might just be the power that drives the next generation of Artificial Intelligence (AI) agents!
Machine Learning and Data Mining: 05 Advanced Association Rule MiningPier Luca Lanzi
Course "Machine Learning and Data Mining" for the degree of Computer Engineering at the Politecnico di Milano. This lecture overviews the FP-growth algorithm, methods for multilevel rules, correlation rules, and sequential rules.
Machine Learning and Data Mining: 09 Clustering: Density-based, Grid-based, M...Pier Luca Lanzi
Course "Machine Learning and Data Mining" for the degree of Computer Engineering at the Politecnico di Milano. This lecture introduces density-based, grid-based, and model-based clustering
Machine Learning and Data Mining: 08 Clustering: Hierarchical Pier Luca Lanzi
Course "Machine Learning and Data Mining" for the degree of Computer Engineering at the Politecnico di Milano. This lecture introduces hierarchical clustering.
Machine Learning and Data Mining: 04 Association Rule MiningPier Luca Lanzi
Course "Machine Learning and Data Mining" for the degree of Computer Engineering at the Politecnico di Milano. This lecture introduces association rule mining and the Apriori algorithm
Abstract—In evolutionary high-level synthesis, design solutions
have to be evaluated to extract information about some
figures of merit (such as performance, area, etc.) and to allow
the genetic algorithm to evolve and converge to Pareto-optimal
solutions. Since the execution time of such evaluations increases
with the complexity of the specification, this could lead to
unacceptable execution time of the overall methodology. This
paper presents a model to exploit fitness inheritance in a multiobjective
optimization algorithm (i.e. NSGA-II [1]) by substituting
the expensive real evaluations with an estimation based
on neighbors in an hypothetical design space. The estimations
are based on a measure of distance between individuals and
a weighted average on fitnesses of closer ones. The results
shows that the Pareto-optimal set obtained by applying the
proposed model good approximates the set obtained without
fitness inheritance and overall execution time is reduced more
than 25% in average.
Machine Learning and Data Mining: 02 Machine LearningPier Luca Lanzi
Course "Machine Learning and Data Mining" for the degree of Computer Engineering at the Politecnico di Milano. This lecture gives a very short introduction to the three main machine learning paradigms.
Video game design and programming course for the Master in Computer Engineering at the Politecnico di Milano.
http://www.facebook.com/polimigamecollective
https://twitter.com/@POLIMIGC
http://www.youtube.com/PierLucaLanzi
http://www.polimigamecollective.org
Politecnico di Milano, Videogiochi, Video Games, Computer Engineering, game design, game development, sviluppo videogiochi
Video game design and programming course for the Master in Computer Engineering at the Politecnico di Milano.
http://www.facebook.com/polimigamecollective
https://twitter.com/@POLIMIGC
http://www.youtube.com/PierLucaLanzi
http://www.polimigamecollective.org
Politecnico di Milano, Videogiochi, Video Games, Computer Engineering, game design, game development, sviluppo videogiochi
Abstract—In evolutionary high-level synthesis, design solutions
have to be evaluated to extract information about some
figures of merit (such as performance, area, etc.) and to allow
the genetic algorithm to evolve and converge to Pareto-optimal
solutions. Since the execution time of such evaluations increases
with the complexity of the specification, this could lead to
unacceptable execution time of the overall methodology. This
paper presents a model to exploit fitness inheritance in a multiobjective
optimization algorithm (i.e. NSGA-II [1]) by substituting
the expensive real evaluations with an estimation based
on neighbors in an hypothetical design space. The estimations
are based on a measure of distance between individuals and
a weighted average on fitnesses of closer ones. The results
shows that the Pareto-optimal set obtained by applying the
proposed model good approximates the set obtained without
fitness inheritance and overall execution time is reduced more
than 25% in average.
Machine Learning and Data Mining: 02 Machine LearningPier Luca Lanzi
Course "Machine Learning and Data Mining" for the degree of Computer Engineering at the Politecnico di Milano. This lecture gives a very short introduction to the three main machine learning paradigms.
Video game design and programming course for the Master in Computer Engineering at the Politecnico di Milano.
http://www.facebook.com/polimigamecollective
https://twitter.com/@POLIMIGC
http://www.youtube.com/PierLucaLanzi
http://www.polimigamecollective.org
Politecnico di Milano, Videogiochi, Video Games, Computer Engineering, game design, game development, sviluppo videogiochi
Video game design and programming course for the Master in Computer Engineering at the Politecnico di Milano.
http://www.facebook.com/polimigamecollective
https://twitter.com/@POLIMIGC
http://www.youtube.com/PierLucaLanzi
http://www.polimigamecollective.org
Politecnico di Milano, Videogiochi, Video Games, Computer Engineering, game design, game development, sviluppo videogiochi
Extending and integrating a hybrid knowledge representation system into the c...Valentina Rho
Extending and integrating a hybrid knowledge representation system into the cognitive architecture ACT-R - 15th International Conference of the Italian Association for Artificial Intelligence - 1 December 2016
Machine Learning and Data Mining: 16 Classifiers EnsemblesPier Luca Lanzi
Course "Machine Learning and Data Mining" for the degree of Computer Engineering at the Politecnico di Milano. In this lecture we introduce classifiers ensembles.
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
Deep Learning and Reinforcement Learning summer schools summary
26th June-6th July 2017, Montreal, Quebec
Things I learned. What was your favourite lesson?
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
DMTM Lecture 13 Representative based clusteringPier Luca Lanzi
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
GridMate - End to end testing is a critical piece to ensure quality and avoid...ThomasParaiso2
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
3. Early Evolutionary Research 3
Box (1957). Evolutionary operations.
Led to simplex methods, Nelder-Mead.
Other Evolutionaries: Friedman (1959),
Bledsoe (1961), Bremermann (1961)
Rechenberg (1964), Schwefel (1965).
Evolution Strategies.
Fogel, Owens & Walsh (1966).
Evolutionary programming.
Common view
Evolution = Random mutation + Save the best.
Pier Luca Lanzi
4. Early intuitions 4
“There is the genetical or evolutionary search
by which a combination of genes is looked for,
the criterion being the survival value.”
Alan M. Turing, Intelligent Machinery, 1948
“We cannot expect to find a good child-machine
at the first attempt. One must experiment with
teaching one such machine and see how well it
learns. One can then try another and see if it is
better or worse. There is an obvious connection
between this process and evolution, by the identifications
“Structure of the child machine = Hereditary material
“Changes of the child machine = Mutations
“Natural selection = Judgment of the experimenter”
Alan M. Turing, “Computing Machinery and Intelligence” 1950.
Pier Luca Lanzi
5. Meanwhile… in Ann Arbor… 5
Holland (1959).
Iterative circuit computers.
Holland (1962). Outline for a
logical theory of adaptive systems.
Role of recombination (Holland 1965)
Role of schemata (Holland 1968, 1971)
Two-armed bandit (Holland 1973, 1975)
First dissertations (Bagley, Rosenberg 1967)
Simple Genetic Algorithm(De Jong 1975)
Pier Luca Lanzi
6. What are Genetic Algorithms? 6
Genetic algorithms (GAs) are search algorithms based on the
mechanics of natural selection and genetics
Two components
Natural selection: survival of the fittest
Genetics: recombination of structures, variation
Underlying methaphor
Individuals in a population must be adapted
to the environment to survive and reproduce
A problem can be viewed as an environment,
we evolve a population of solutions to solve it
Different individuals are differently adapted
To survive a solution must be “adapted” to the problem
Pier Luca Lanzi
7. A Peek into Genetic Algorithms 7
Population Representation
A set of candidate The coding of solutions
solutions Originally, binary strings
Fitness function Operators inspired by Nature
Evaluates candidate Selection
solutions Recombination
Mutation
Genetic Algorithm
Generate an initial random population
Repeat
Select promising solutions
Create new solutions by applying variation
Incorporate new solutions into original population
Until stop criterion met
Pier Luca Lanzi
9. Holland’s Vision, Cognitive System One 9
To state, in concrete technical form, a model of
a complete mind and its several aspects
A cognitive system interacting
with an environment
Binary detectors and effectors
Knowledge = set of classifiers
Condition-action rules that
recognize a situation and
propose an action
Payoff reservoir for
the system’s needs
Payoff distributed through
an epochal algorithm
Internal memory as
message list
Genetic search of classifiers
Pier Luca Lanzi
10. What was the goal? 10
1#11:buy ⇒30
0#0#:sell ⇒-2
…
A real system with an unknown underlying dynamics
Use a classifier system online to generate a behavior
that matched the real system.
The evolved rules would provide a plausible,
human readable, model of the unknown system
Pier Luca Lanzi
11. Holland’s Learning Classifier Systems 11
Explicit representation of the incoming reward
Good classifiers are the
ones that predict
high rewards
Credit Assignment using
Bucket Brigade
Rule Discovery through
a genetic algorithm on
all the rule base (on the
whole solution)
Description was vast
It did not work right off!
Very limited success
David E. Goldberg: Computer-aided gas pipeline operation using
genetic algorithms and rule learning, PhD thesis. University of
Michigan. Ann Arbor, MI.
Pier Luca Lanzi
12. Learning System LS-1 & 12
Pittsburgh Classifier Systems
Holland models learning as an adaptation process
De Jong models learning as an optimization process
Genetic algorithm applied to a population of rule sets
1. t := 0
2. Initialize the population P(t)
3. Evaluate the rules sets in P(t)
4. While the termination condition is not satisfied
5. Begin
6. Select the rule sets in P(t) and generate Ps(t)
7. Recombine and mutate the rule sets in Ps(t)
8. P(t+1) := Ps(t)
9. t := t+1
No apportionment of credit
10. Evaluate the rules sets in P(t)
Offline evaluation of rule sets
11. End
Pier Luca Lanzi
13. As time goes by… 13
Genetic algorithms and CS-1
1970’s Research flourishes
Success is limited
Reinforcement
Learning
Evolving rules as optimization
Machine
1980’s Research follows Holland’s vision
Learning
Success is still limited
Stewart Wilson Robotics applications
1990’s
creates XCS First results on classification
But the interest fades away
Classifier systems finally work
2000’s Large development of models,
facetwise theory, and applications
Pier Luca Lanzi
14. Stewart W. Wilson & 14
The XCS Classifier System
1. Simplify the model
2. Go for accurate predictions
not high payoffs
3. Apply the genetic algorithm
to subproblems not to
the whole problem
4. Focus on classifier systems as
reinforcement learning
with rule-based generalization
5. Use reinforcement learning (Q-learning) to distribute reward
Most successfull model developed so far
Wilson, S.W.: Classifier Fitness Based on Accuracy. Evolutionary
Computation 3(2), 149-175 (1995).
Pier Luca Lanzi
16. Learning Classifier Systems as 16
Reinforcement Learning Methods
System
stt+1 at
rt+1
Environment
The goal: maximize the amount of reward received
How much future reward when at is performed in st?
What is the expected payoff for st and at?
Need to compute a value function, Q(st,at)→ payoff
Pier Luca Lanzi
17. 17
Define the inputs, the actions,
and how the reward is determined
HowDefine the expected payoff
does reinforcement
learning work?
Compute a value function Q(st,at) mapping
state-action pairs into expected payoffs
Pier Luca Lanzi
18. How does reinforcement learning work? 18
First we define the expected payoff
First we define the expected payoff as
γ is the discount factor
Pier Luca Lanzi
19. How does reinforcement learning work? 19
Then, Q-learning is an option.
At the beginning, is initialized with random values
At time t,
previous value
new estimate incoming reward
new estimate
Parameters,
Discount factor γ
The learning rate β
The action selection strategy
Pier Luca Lanzi
20. This looks simple… 20
Let’s bring RL to the real world!
Reinforcement learning assumes
that Q(st,at) is represented as a table
But the real world is complex,
the number of possible inputs can be huge!
We cannot afford an exact Q(st,at)
Pier Luca Lanzi
21. Example: The Mountain Car 21
rt = 0 when goal is
reached, -1 otherwise.
GOAL
Value Function
Q(st,at)
st = position,
acc .
acc
velocity
.
ht, . left,
no
a cc
a=
r ig
t
Task: drive an underpowered
car up a steep mountain road
Pier Luca Lanzi
22. What are the issues? 22
Exact representation infeasible
Approximation mandatory
The function is unknown,
it is learnt online from experience
Pier Luca Lanzi
23. What are the issues? 23
Learning an unknown payoff function
while also trying to approximate it
Approximator works on intermediate estimates
While also providing information for the learning
Convergence is not guaranteed
Pier Luca Lanzi
24. Whats does this have to do with 24
Learning Classifier Systems?
They solve reinforcement learning problems
Represent the payoff function Q(st, at) as
a population of rules, the classifiers
Classifiers are evolved while
Q(st, at) is learnt online
Pier Luca Lanzi
25. What is a classifier? 25
IF condition C is true for input s
THEN the payoff of action A is p
Accurate
approximations
payoff
payoff
surface for A
p
General conditions
covering large portions
Condition
of the problem space
C(s)=l≤s≤u
s
l u
Pier Luca Lanzi
27. How do learning classifier systems work? 27
The main performance cycle
Pier Luca Lanzi
28. How do learning classifier systems work? 28
The main performance cycle
The classifiers predict an expected payoff
The incoming reward is used to update
the rules which helped in getting the reward
Any reinforcement learning algorithm can be used
to estimate the classifier prediction.
Pier Luca Lanzi
29. How do learning classifier systems work? 29
The main performance cycle
Pier Luca Lanzi
30. 30
In principle, any search method may be used
I prefer genetic algorithms
Where do classifiers come from?
because they are representation independent
A genetic algorithm select, recombines,
mutate existing classifiers to search for better ones
Pier Luca Lanzi
31. What are the good classifiers? 31
What is the classifier fitness?
The goal is to approximate a target value function
with as few classifiers as possible
We wish to have an accurate approximation
One possible approach is to define fitness
as a function of the classifier prediction accuracy
Pier Luca Lanzi
32. What about getting as 32
few classifiers as possible?
The genetic algorithm can take care of this
General classifiers apply more often,
thus they are reproduced more
But since fitness is based on classifiers accuracy
only accurate classifiers are likely to be reproduced
The genetic algorithm evolves
maximally general maximally accurate classifiers
Pier Luca Lanzi
33. How to apply learning classifier systems 33
Environment
Determine the inputs, the actions,
and how reward is distributed
Determine what is the expected payoff
that must be maximized
st rt at
Decide an action selection strategy
Set up the parameters β and γ
Learning Classifier System
Select a representation for conditions,
the recombination and the mutation operators
Select a reinforcement learning algorithm
Setup the parameters, mainly the population size,
the parameters for the genetic algorithm, etc.
Pier Luca Lanzi
34. Things can be extremely simple! 34
For instance in supervised classification
Environment
1 if the class is correct
0 if the class is not correct
example class
Select a representation for conditions and
the recombination and mutation operators
Setup the parameters, mainly the population size,
the parameters for the genetic algorithm, etc.
Learning Classifier System
Pier Luca Lanzi
35. 35
Genetics-Based
Accurate Estimates
Generalization
About Classifiers
(Powerful RL)
Classifier
Representation
Pier Luca Lanzi
36. One Representation, 36
One Principle
Data described by 6 variables a1, …, a6
They represents the simple concept “a1=a2 Ç a5=1”
A rather typical approach
Select a representation
Select an algorithm which
produces such a representation
Apply the algorithm
Decision Rules (attribute-value)
if (a5 = 1) then class 1 [95.3%]
If (a1=3 Æ a2=3) then class = 1 [92.2%]
…
FOIL
Clause 0: is_0(a1,a2,a3,a4,a5,a6) :- a1≠a2, a5≠ 1
Pier Luca Lanzi
37. Learning Classifier Systems: 37
One Principle Many Representations
Ternary rules ####1#:1 if a5<2, class=1
0, 1, # 22####:1 if a1=a2, class=1
Learning Classifier System
Genetic Estimates
Knowledge
Search RL & ML
Representation
Conditions &
Ternary Conditions
Attribute-Value
Symbolic
Prediction
Conditions
0, 1, #
No need to change the framework
Ternary Conditions Attribute-Value Symbolic
Just plug-in your favourite representation
0, 1, # Conditions Conditions
Pier Luca Lanzi
39. What is computed prediction? 39
Replace the prediction p by
a parametrized function p(x,w)
Which type of
approximation?
payoff
payoff
p(x,w)=w0+xw1
landscape of A
Which Representation?
Condition
C(s)=l≤s≤u
x
l u
Pier Luca Lanzi
40. Same example with computed prediction 40
Again, no need to change the framework
Just plug-in your favourite estimator
Linear, Polynomial, NNs, SVMs, tile-coding
Pier Luca Lanzi
42. 42
Learning Classifier Systems involve
Representation, Reinforcement Learning,
& Genetics-based Search
Unified theory is impractical
Develop facetwise models
Pier Luca Lanzi
43. Facetwise Models for a Theory of 43
Evolution and Learning
Prof. David E. Goldberg
University of Illinois at Urbana Champaign
David Goldberg & Kumara Sastry
Genetic Algorithms: The Design of Innovation
Springer-Verlag May 2008
Facetwise approach to analysis and
design for genetic algorithms
In learning classifier systems
Separate learning from evolution
Simplify the problem by focusing
only on relevant aspect
Derive facetwise models
Applied to model several aspects of evolution
E.g., the time to convergence is O(L 2k)
Pier Luca Lanzi
47. What Applications? 47
Computational Models of Cognition
Learning classifier system model
certain aspects of cognition
Human language learning
Perceptual category learning
Affect theory
Anticipatory and latent learning
Learning classifier systems provide good
models for animals in experiments
in which the subjects must learn internal
models to perform as well as they do
Martin V. Butz, University of Würzburg,
Department of Cognitive Psychology III
Cognitive Bodyspaces:
Learning and Behavior (COBOSLAB)
Wolfgang Stolzmann, Daimler Chrysler
Rick R. Riolo, University of Michigan,
Center for the Study of Complex Systems
Pier Luca Lanzi
48. References 48
Butz, M.V.: Anticipatory Learning Classifier Systems, Genetic
Algorithms and Evolutionary Computation, vol. 4.
Springer-Verlag (2000)
Riolo, R.L.: Lookahead Planning and
Latent Learning in a Classifier System.
In: J.A. Meyer, S.W. Wilson (eds.)
From Animals to Animats 1.
Proceedings of the First International
Conferenceon Simulation of Adaptive
Behavior (SAB90), pp. 316{326.
A Bradford Book. MIT Press (1990)
Stolzmann, W. and Butz, M.V. and Hoffman, J. and Goldberg,
D.E.: First Cognitive Capabilities in the Anticipatory Classifier
System. In: From Animals to Animats: Proceedings of the
Sixth International Conference on Simulation of Adaptive
Behavior. MIT Press (2000)
Pier Luca Lanzi
50. What Applications? 50
Computational Economics
To models one single agent acting in
the market (BW Arthur, JH Holland, B LeBaron)
To model many interactive agents each one
controlled by its own classifier system.
Modeling the behavior of agents trading
risk free bonds and risky assets
Different trader types modeled by supplying
different input information sets
to a group of homogenous agents
Later extended to a multi-LCS architecture
applied to portfolio optimization
Technology startup company
founded in March 2005
Pier Luca Lanzi
51. References 51
Sor Ying (Byron) Wong, Sonia Schulenburg: Portfolio
allocation using XCS experts in technical analysis, market
conditions and options market. GECCO (Companion) 2007:
2965-2972
Sonia Schulenburg, Peter Ross: An Adaptive Agent Based
Economic Model. Learning Classifier Systems 1999: 263-282
BW Arthur, J.H. Holland, B. LeBaron, R. Palmer, and P.
Tayler: quot;Asset Pricing Under Endogenous Expectations in an
Artificial Stock Market,“ in The Economy as an Evolving
Complex System II. Edited (with S. Durlauf and D. Lane),
Addison-Wesley, 1997.
BW Arthur, R. Palmer, J. Holland, B. LeBaron, and P. Taylor:
quot;Artificial Economic Life: a Simple Model of a Stockmarket,“
Physica D, 75, 264-274, 1994
Pier Luca Lanzi
53. What Applications? 53
Classification and Data Mining
Bull, L. (ed) Applications of Learning
Classifier Systems. Springer. (2004)
Bull, L., Bernado Mansilla, E. & Holmes, J.
(eds) Learning Classifier Systems in
Data Mining. Springer. (2008)
Nowadays, by far the most important
application domain for LCSs
Many models GA-Miner, REGAL, GALE GAssist
Performance comparable to state of the art machine learning
Human Competitive Results 2007
X Llorà, R Reddy, B Matesic, R Bhargava: Towards Better than
Human Capability in Diagnosing Prostate Cancer Using Infrared
Spectroscopic Imaging
Pier Luca Lanzi
55. What Applications? 55
Hyper-Heuristics
Ross P., Marin-Blazquez J., Schulenburg S.,
and Hart E., Learning a Procedure that can
Solve Hard Bin-packing Problems: A New
GA-Based Approach to Hyper-Heuristics.
In Proceedings of GECCO 2003
Bin-packing and timetabling problems
Pick a set of non-evolutionary heuristics
Use classifier system to learn
a solution process not a solution
The classifier system learns a sequence of heuristics which
should be applied to gradually transform the problem from
its initial state to its final solved state.
Pier Luca Lanzi
57. What Applications? 57
Epidemiologic Surveillance
John H. Holmes
Center for Clinical Epidemiology & Biostatistics
Department of Biostatistics & Epidemiology
University of Pennsylvania - School of Medicine
Epidemiologic surveillance data
need adaptivity to abrupt changes
Readable rules are attractive
Performance similar to state
of the art machine learning
But several important
feature-outcome relationships
missed by other methods were discovered
Similar results were reported by
Stewart Wilson for breast cancer data
Pier Luca Lanzi
58. References 58
John H. Holmes, Jennifer A. Sager: Rule Discovery in
Epidemiologic Surveillance Data Using EpiXCS: An
Evolutionary Computation Approach. AIME 2005: 444-452
John H. Holmes, Dennis R. Durbin, Flaura K. Winston: A New
Bootstrapping Method to Improve Classification Performance
in Learning Classifier Systems. PPSN 2000: 745-754
John H. Holmes, Dennis R. Durbin, Flaura K. Winston: The
learning classifier system: an evolutionary computation
approach to knowledge discovery in epidemiologic
surveillance. Artificial Intelligence in Medicine 19(1): 53-74
(2000)
Pier Luca Lanzi
60. What Applications? 60
Autonomous Robotics
In the 1990's, a major testbed
for learning classifier systems.
Marco Dorigo and Marco Colombetti:
Robot Shaping An Experiment in Behavior
Engineering, 1997
They introduced the concept of
robot shaping defined as the
incremental training of an autonomous agent.
Behavior engineering methodology named BAT:
Behavior Analysis and Training.
Recently, University of West England
applied several learning classifier system
models to several robotics problems
Pier Luca Lanzi
62. What Applications? 62
Modeling Artificial Ecosystems
Jon McCormack, Monash University
Eden: an interactive, self-generating, artificial ecosystem.
World populated by collections of evolving virtual creatures
Creatures move about
the environment,
Make and listen to sounds,
Foraging for food,
Encountering predators,
Mating with each other.
Creatures evolve to
fit their landscape
Eden has four seasons per year (15mins) Jon McCormack
Simple physics for rocks, biomass and sonic animals.
Pier Luca Lanzi
63. References 63
McCormack, J. Impossible Nature The Art of Jon McCormack
Published by the Australian Centre for the Moving Image
ISBN 1 920805 08 7, ISBN 1 920805 09 5 (DVD)
J. McCormack: New Challenges
for Evolutionary Music and Art,
ACM SIGEVOlution Newsletter,
Vol. 1(1), April 2006, pp. 5-11.
McCormack, J. 2005, 'On the
Evolution of Sonic Ecosystems'
in Adamatzky, et al. (eds),
Artificial Life Models in Software,
Springer, Berlin.
McCormack, J. 2003, 'Evolving Sonic Ecosystems',
Kybernetes, 32(1/2), pp. 184-202.
Pier Luca Lanzi
65. What Applications? 65
Chemical and Neuronal Networks
L. Bull, A. Budd, C. Stone, I. Uroukov,
B. De Lacy Costello and A. Adamatzky
University of the West of England
Behaviour of non-linear media
controlled automatically through
evolutionary learning
Unconventional computing
realised by such an approach.
Learning classifier systems
Control a light-sensitive sub-excitable
Belousov-Zhabotinski reaction
Control the electrical stimulation of
cultured neuronal networks
Pier Luca Lanzi
66. What Applications? 66
Chemical and Neuronal Networks
To control a light-sensitive sub-excitable BZ reaction, pulses of
wave fragments are injected into the checkerboard grid resulting in
rich spatio-temporal behaviour
Learning classifier system can direct the fragments to an arbitrary
position through control of the light intensity within each cell
Learning Classifier Systems control the electrical stimulation of
cultured neuronal networks such that they display elementary
learning, respond to a given input signal in a pre-specified way
Results indicate that the learned stimulation protocols identify
seemingly fundamental properties of in vitro neuronal networks
Pier Luca Lanzi
67. References 67
Larry Bull, Adam Budd, Christopher Stone, Ivan Uroukov,
Ben De Lacy Costello and Andrew Adamatzky: Towards
Unconventional Computing through Simulated Evolution:
Learning Classifier System Control of Non-Linear Media
Artificial Life (to appear)
Budd, A., Stone, C., Masere, J., Adamatzky, A.,
DeLacyCostello, B., Bull, L.: Towards machine learning
control of chemical computers. In: A. Adamatzky, C.
Teuscher (eds.) From Utopian to Genuine Unconventional
Computers, pp. 17-36. Luniver Press
Bull, L., Uroukov, I.S.: Initial results from the use of learning
classier systems to control n vitro neuronal networks. In:
Lipson [189], pp. 369-376
Pier Luca Lanzi
69. Conclusions 69
Cognitive Modeling
Complex Adaptive Systems
Machine Learning
Reinforcement Learning
Metaheuristics
…
Many blocks to plug-in
Several Representations
Several RL algorithms
Several evolutionary methods
…
Pier Luca Lanzi
70. Further Readings 70
Martin V. Butz “Rule-Based Evolutionary Online Learning
Systems: A Principled Approach to LCS Analysis and Design”
Studies in Fuzziness and Soft Computing, Springer-Verlag
2005
Proceedings of IWLCS from 2000 to 2007 published by
Springer Verlag
Pier Luca Lanzi