Dana Simian, Florin Stoica, Generic Reinforcement Schemes and Their Optimization, Proceedings of the 5th European Computing Conference (ECC ’11), Paris, France, April 28-30, 2011, pp. 332-337
Optimizing a New Nonlinear Reinforcement Scheme with Breeder genetic algorithminfopapers
Florin Stoica, Dana Simian, Optimizing a New Nonlinear Reinforcement Scheme with Breeder genetic algorithm, Proceedings of the Recent Advances in Neural Networks, Fuzzy Systems & Evolutionary Computing,13-15 June 2010, Iasi, Romania, ISSN: 1790-2769, ISBN: 978-960-474-194-6, pp. 273-278
A New Nonlinear Reinforcement Scheme for Stochastic Learning Automatainfopapers
Dana Simian, Florin Stoica, A New Nonlinear Reinforcement Scheme for Stochastic Learning Automata, Proceedings of the 12th WSEAS International Conference on AUTOMATIC CONTROL, MODELLING & SIMULATION, 29-31 May 2010, Catania, Italy, ISSN 1790-5117, ISBN 978-954-92600-5-2, pp. 450-454
MATHEMATICAL MODELING OF COMPLEX REDUNDANT SYSTEM UNDER HEAD-OF-LINE REPAIREditor IJMTER
Suppose a composite system consisting of two subsystems designated as ‘P’ and
‘Q’ connected in series. Subsystem ‘P’ consists of N non-identical units in series, while the
subsystem ‘Q’ consists of three identical components in parallel redundancy.
Optimizing a New Nonlinear Reinforcement Scheme with Breeder genetic algorithminfopapers
Florin Stoica, Dana Simian, Optimizing a New Nonlinear Reinforcement Scheme with Breeder genetic algorithm, Proceedings of the Recent Advances in Neural Networks, Fuzzy Systems & Evolutionary Computing,13-15 June 2010, Iasi, Romania, ISSN: 1790-2769, ISBN: 978-960-474-194-6, pp. 273-278
A New Nonlinear Reinforcement Scheme for Stochastic Learning Automatainfopapers
Dana Simian, Florin Stoica, A New Nonlinear Reinforcement Scheme for Stochastic Learning Automata, Proceedings of the 12th WSEAS International Conference on AUTOMATIC CONTROL, MODELLING & SIMULATION, 29-31 May 2010, Catania, Italy, ISSN 1790-5117, ISBN 978-954-92600-5-2, pp. 450-454
MATHEMATICAL MODELING OF COMPLEX REDUNDANT SYSTEM UNDER HEAD-OF-LINE REPAIREditor IJMTER
Suppose a composite system consisting of two subsystems designated as ‘P’ and
‘Q’ connected in series. Subsystem ‘P’ consists of N non-identical units in series, while the
subsystem ‘Q’ consists of three identical components in parallel redundancy.
Understanding Blackbox Prediction via Influence FunctionsSEMINARGROOT
Pang Wei Koh and Percy Liang
"Understanding Black-Box prediction via influence functions" ICML 2017 Best paper
References:
https://youtu.be/0w9fLX_T6tY
https://arxiv.org/abs/1703.04730
Main obstacles of Bayesian statistics or Bayesian machine learning is computing posterior distribution. In many contexts, computing posterior distribution is intractable. Today, there are two main stream to detour directly computing posterior distribution. One is using sampling method(ex. MCMC) and another is Variational inference. Compared to Variational inference, MCMC takes more time and vulnerable to high-dimensional parameters. However, MCMC has strength in simplicity and guarantees of convergence. I'll briefly introduce several methods people using in application.
Numerical solution of fuzzy differential equations by Milne’s predictor-corre...mathsjournal
The study of this paper suggests on dependency problem in fuzzy computational method by using the numerical solution of Fuzzy differential equations(FDEs) in Milne’s predictor-corrector method. This method is adopted to solve the dependency problem in fuzzy computation. We solve some fuzzy initial value problems to illustrate the theory.
A machine learning method for efficient design optimization in nano-optics JCMwave
The slideshow contains a brief explanation of Gaussian process regression and Bayesian optimization. For two optimization problems, benchmarks against other local gradient-based and global heuristic optimization methods are included. They show, that Bayesian optimization can identify better designs in exceptionally short computation times.
A machine learning method for efficient design optimization in nano-opticsJCMwave
Explanation of Gaussian process regression and Bayesian optimization. For two optimization problems, benchmarks against other local gradiant-based and global heuristic optimization methods are included. They show, that Bayesian optimization can identify better designs in exceptionally short computation times.
In this work, we study H∞ control wind turbine fuzzy model for finite frequency(FF) interval. Less conservative results are obtained by using Finsler’s lemma technique, generalized Kalman Yakubovich Popov (gKYP), linear matrix inequality (LMI) approach and added several separate parameters, these conditions are given in terms of LMI which can be efficiently solved numerically for the problem that such fuzzy systems are admissible with H∞ disturbance attenuation level. The FF H∞ performance approach allows the state feedback command in a specific interval, the simulation example is given to validate our results.
International Journal of Modern Engineering Research (IJMER) is Peer reviewed, online Journal. It serves as an international archival forum of scholarly research related to engineering and science education.
International Journal of Modern Engineering Research (IJMER) covers all the fields of engineering and science: Electrical Engineering, Mechanical Engineering, Civil Engineering, Chemical Engineering, Computer Engineering, Agricultural Engineering, Aerospace Engineering, Thermodynamics, Structural Engineering, Control Engineering, Robotics, Mechatronics, Fluid Mechanics, Nanotechnology, Simulators, Web-based Learning, Remote Laboratories, Engineering Design Methods, Education Research, Students' Satisfaction and Motivation, Global Projects, and Assessment…. And many more.
A general frame for building optimal multiple SVM kernelsinfopapers
Dana Simian, Florin Stoica, A General Frame for Building Optimal Multiple SVM Kernels, Large-Scale Scientific Computing, Lecture Notes in Computer Science, 2012, Volume 7116/2012, 256-263, DOI: 10.1007/978-3-642-29843-1_29
A new Reinforcement Scheme for Stochastic Learning Automatainfopapers
F. Stoica, E. M. Popa, I. Pah, A new reinforcement scheme for stochastic learning automata – Application to Automatic Control, Proceedings of the International Conference on e-Business, Porto, Portugal, ISBN 978-989-8111-58-6, pp. 45-50, July 2008
Laura Florentina Stoica, Florian Mircea Boian, Florin Stoica, A Distributed CTL Model Checker, Proceeding of 10th International Conference on e-Business, ICE-B 2013, Reykjavik Iceland, paper 33, 29-31 July, pp. 379-386, ISBN: 978-989-8565-72-3, 2013
Algebraic Approach to Implementing an ATL Model Checkerinfopapers
Laura Florentina Stoica, Florian Mircea Boian, Algebraic Approach to Implementing an ATL Model Checker, STUDIA Univ. Babes Bolyai, INFORMATICA, Volume LVII, Number 2, 2012, pp. 73-82
Understanding Blackbox Prediction via Influence FunctionsSEMINARGROOT
Pang Wei Koh and Percy Liang
"Understanding Black-Box prediction via influence functions" ICML 2017 Best paper
References:
https://youtu.be/0w9fLX_T6tY
https://arxiv.org/abs/1703.04730
Main obstacles of Bayesian statistics or Bayesian machine learning is computing posterior distribution. In many contexts, computing posterior distribution is intractable. Today, there are two main stream to detour directly computing posterior distribution. One is using sampling method(ex. MCMC) and another is Variational inference. Compared to Variational inference, MCMC takes more time and vulnerable to high-dimensional parameters. However, MCMC has strength in simplicity and guarantees of convergence. I'll briefly introduce several methods people using in application.
Numerical solution of fuzzy differential equations by Milne’s predictor-corre...mathsjournal
The study of this paper suggests on dependency problem in fuzzy computational method by using the numerical solution of Fuzzy differential equations(FDEs) in Milne’s predictor-corrector method. This method is adopted to solve the dependency problem in fuzzy computation. We solve some fuzzy initial value problems to illustrate the theory.
A machine learning method for efficient design optimization in nano-optics JCMwave
The slideshow contains a brief explanation of Gaussian process regression and Bayesian optimization. For two optimization problems, benchmarks against other local gradient-based and global heuristic optimization methods are included. They show, that Bayesian optimization can identify better designs in exceptionally short computation times.
A machine learning method for efficient design optimization in nano-opticsJCMwave
Explanation of Gaussian process regression and Bayesian optimization. For two optimization problems, benchmarks against other local gradiant-based and global heuristic optimization methods are included. They show, that Bayesian optimization can identify better designs in exceptionally short computation times.
In this work, we study H∞ control wind turbine fuzzy model for finite frequency(FF) interval. Less conservative results are obtained by using Finsler’s lemma technique, generalized Kalman Yakubovich Popov (gKYP), linear matrix inequality (LMI) approach and added several separate parameters, these conditions are given in terms of LMI which can be efficiently solved numerically for the problem that such fuzzy systems are admissible with H∞ disturbance attenuation level. The FF H∞ performance approach allows the state feedback command in a specific interval, the simulation example is given to validate our results.
International Journal of Modern Engineering Research (IJMER) is Peer reviewed, online Journal. It serves as an international archival forum of scholarly research related to engineering and science education.
International Journal of Modern Engineering Research (IJMER) covers all the fields of engineering and science: Electrical Engineering, Mechanical Engineering, Civil Engineering, Chemical Engineering, Computer Engineering, Agricultural Engineering, Aerospace Engineering, Thermodynamics, Structural Engineering, Control Engineering, Robotics, Mechatronics, Fluid Mechanics, Nanotechnology, Simulators, Web-based Learning, Remote Laboratories, Engineering Design Methods, Education Research, Students' Satisfaction and Motivation, Global Projects, and Assessment…. And many more.
A general frame for building optimal multiple SVM kernelsinfopapers
Dana Simian, Florin Stoica, A General Frame for Building Optimal Multiple SVM Kernels, Large-Scale Scientific Computing, Lecture Notes in Computer Science, 2012, Volume 7116/2012, 256-263, DOI: 10.1007/978-3-642-29843-1_29
A new Reinforcement Scheme for Stochastic Learning Automatainfopapers
F. Stoica, E. M. Popa, I. Pah, A new reinforcement scheme for stochastic learning automata – Application to Automatic Control, Proceedings of the International Conference on e-Business, Porto, Portugal, ISBN 978-989-8111-58-6, pp. 45-50, July 2008
Laura Florentina Stoica, Florian Mircea Boian, Florin Stoica, A Distributed CTL Model Checker, Proceeding of 10th International Conference on e-Business, ICE-B 2013, Reykjavik Iceland, paper 33, 29-31 July, pp. 379-386, ISBN: 978-989-8565-72-3, 2013
Algebraic Approach to Implementing an ATL Model Checkerinfopapers
Laura Florentina Stoica, Florian Mircea Boian, Algebraic Approach to Implementing an ATL Model Checker, STUDIA Univ. Babes Bolyai, INFORMATICA, Volume LVII, Number 2, 2012, pp. 73-82
Using genetic algorithms and simulation as decision support in marketing stra...infopapers
F.Stoica, L.F.Cacovean, Using genetic algorithms and simulation as decision support in marketing strategies and long-term production planning, Proceedings of the 9th WSEAS International Conference on SIMULATION, MODELLING AND OPTIMIZATION (SMO ‘09), Budapest Tech, Hungary, September 3-5, ISSN: 1790-2769 ISBN:978-960-474-113-7, pp. 435-439, 2009
An Executable Actor Model in Abstract State Machine Languageinfopapers
F. Stoica, An executable Actor model in Abstract State Machine Language, The Proceedings of the International Conference on Computers and Communications, Oradea, ISBN 973-613-542-X, pp. 388-393, 2004
Modeling the Broker Behavior Using a BDI Agentinfopapers
Laura Florentina Cacovean, Florin Stoica, Modeling the Broker Behavior Using a BDI Agent, Proceedings of the 14th WSEAS International Conference on Computers (CSCC), 22-25 July, 2010, Corfu, Greece, ISSN: 1792-4391, ISBN: 978-960-474-206-6, pp. 699-703
F. Stoica, D. Simian, C. Simian, A new co-mutation genetic operator, Proceedings of the 9th WSEAS International Conference on Evolutionary Computing, Sofia, Bulgaria, ISBN 978-960-6766-58-9, ISSN 1790-5109, pp. 76-81, May 2008
Using the Breeder GA to Optimize a Multiple Regression Analysis Modelinfopapers
Florin Stoica, Cornel Gheorghe Boitor, Using the Breeder genetic algorithm to optimize a multiple regression analysis model used in prediction of the mesiodistal width of unerupted teeth, International Journal of Computers, Communications & Control, Vol 9, No 1, pp. 62-70, ISSN 1841-9836, february 2014
Intelligent agents in ontology-based applicationsinfopapers
F. Stoica, I. Pah, Intelligent agents in ontology-based applications, Proceedings of the 12th WSEAS International Conference on COMPUTERS, Heraklion, Greece, July 23-25, ISBN: 978-960-6766-85-5, ISSN: 1790-5109, pp. 274-279, 2008
Implementing an ATL Model Checker tool using Relational Algebra conceptsinfopapers
Florin Stoica, Laura Florentina Stoica, Implementing an ATL Model Checker tool using Relational Algebra concepts, Proceeding The 22th International Conference on Software, Telecommunications and Computer Networks (SoftCOM), Split-Primosten, Croatia, 2014
Optimization of Complex SVM Kernels Using a Hybrid Algorithm Based on Wasp Be...infopapers
Dana Simian, Florin Stoica, Corina Simian, Optimization of Complex SVM Kernels Using a Hybrid Algorithm Based on Wasp Behaviour, Lecture Notes in Computer Science, LNCS 5910 (2010), I. Lirkov, S. Margenov, and J. Wasniewski (Eds.), Springer-Verlag Berlin Heidelberg, pp. 361-368
A new Evolutionary Reinforcement Scheme for Stochastic Learning Automatainfopapers
F. Stoica, E. M. Popa, A new Evolutionary Reinforcement Scheme for Stochastic Learning Automata, Proceedings of the 12th WSEAS International Conference on COMPUTERS, Heraklion, Greece, July 23-25, ISBN: 978-960-6766-85-5, ISSN: 1790-5109, pp. 268-273, 2008
An AsmL model for an Intelligent Vehicle Control Systeminfopapers
F. Stoica, An AsmL model for an Intelligent Vehicle Control System, Proceedings of the 11th WSEAS Int. Conf. on COMPUTERS: Computer Science and Technology, vol. 4, Crete Island, Greece, ISBN: 978-960-8457-92-8, pp. 323-328, July 2007
Building a new CTL model checker using Web Servicesinfopapers
Florin Stoica, Laura Stoica, Building a new CTL model checker using Web Services, Proceeding The 21th International Conference on Software, Telecommunications and Computer Networks (SoftCOM 2013), At Split-Primosten, Croatia, 18-20 September, pp. 285-289, 2013
DOI=10.1109/SoftCOM.2013.6671858 http://dx.doi.org/10.1109/SoftCOM.2013.6671858
Deliver Dynamic and Interactive Web Content in J2EE Applicationsinfopapers
F. Stoica, Deliver dynamic and interactive Web content in J2EE applications, Proceedings of the Central and East European Conference in Business Information Systems, Cluj-Napoca, Romania, ISBN 973-656-648-X, pp. 780-789, 2004
ACTIVE CONTROLLER DESIGN FOR THE HYBRID SYNCHRONIZATION OF HYPERCHAOTIC ZHEN...ijscai
This paper deals with a new research problem in the chaos literature, viz. hybrid synchronization of a
pair of chaotic systems called the master and slave systems. In the hybrid synchronization design of
master and slave systems, one part of the systems, viz. their odd states, are completely synchronized (CS),
while the other part, viz. their even states, are completely anti-synchronized (AS) so that CS and AS coexist in the process of synchronization. This research work deals with the hybrid synchronization of
hyperchaotic Zheng systems (2010) and hyperchaotic Yu systems (2012). The main results of this hybrid
synchronization research work have been proved using Lyapunov stability theory. Numerical examples of
the hybrid synchronization results are shown along with MATLAB simulations for the hyperchaotic
Zheng and hyperchaotic Yu systems.
ADAPTIVESYNCHRONIZER DESIGN FOR THE HYBRID SYNCHRONIZATION OF HYPERCHAOTIC ZH...ijitcs
This paper derives new adaptive synchronizers for the hybrid synchronization of hyperchaotic Zheng
systems (2010) and hyperchaotic Yu systems (2012). In the hybrid synchronization design of master and
slave systems, one part of the systems, viz. their odd states, are completely synchronized (CS), while the
other part, viz. their even states, are completely anti-synchronized (AS) so that CS and AS co-exist in the
process of synchronization. The research problem gets even more complicated, when the parameters of the
hyperchaotic systems are not known and we handle this complicate problem using adaptive control. The
main results of this research work are established via adaptive control theory andLyapunov stability
theory. MATLAB plotsusing classical fourth-order Runge-Kutta method have been depictedfor the new
adaptive hybrid synchronization results for the hyperchaotic Zheng and hyperchaotic Yu systems.
Automatic control based on Wasp Behavioral Model and Stochastic Learning Auto...infopapers
F. Stoica, D. Simian, Automatic control based on Wasp Behavioral Model and Stochastic Learning Automata, Proceedings of the 10th International Conference on Mathematical Methods, Computational Techniques & Intelligent Systems, Corfu Island, Greece, ISBN 978-960-474-012-3, pp. 289-294, October 2008
International Journal of Instrumentation and Control Systems (IJICS)ijcisjournal
International Journal of Instrumentation and Control Systems (IJICS) is a Quarterly open access peer-reviewed journal that publishes articles which contribute new results in all areas of Instrumentation Engineering and Control Systems. The journal focuses on all technical and practical aspects of Instrumentation Engineering and Control Systems. The goal of this journal is to bring together researchers and practitioners from academia and industry to focus on advanced instrumentation engineering, control systems and automation concepts and establishing new collaborations in these areas.
Authors are solicited to contribute to this journal by submitting articles that illustrate research results, projects, surveying works and industrial
experiences that describe significant advances in the Instrumentation Engineering
ADAPTIVE CONTROLLER DESIGN FOR THE ANTI-SYNCHRONIZATION OF HYPERCHAOTIC YANG ...ijics
In the anti-synchronization of chaotic systems, a pair of chaotic systems called drive and responsesystems
are considered, and the design goal is to drive the sum of their respective states to zero asymptotically. This
paper derives new results for the anti-synchronization of hyperchaotic Yang system (2009) and
hyperchaotic Pang system (2011) with uncertain parameters via adaptive control. Hyperchaotic systems
are nonlinear chaotic systems withtwo or more positive Lyapunov exponents and they have applications in
areas like neural networks, encryption, secure data transmission and communication. The main results
derived in this paper are illustrated with MATLAB simulations.
Projective and hybrid projective synchronization of 4-D hyperchaotic system v...TELKOMNIKA JOURNAL
Nonlinear control strategy was established to realize the Projective Synchronization (PS) and Hybrid Projective Synchronization (HPS) for 4-D hyperchaotic system at different scaling matrices. This strategy, which is able to achieve projective and hybrid projective synchronization by more precise and adaptable method to provide a novel control scheme. On First stage, three scaling matrices were given in order to achieving various projective synchronization phenomena. While the HPS was implemented at specific scaling matrix in the second stage. Ultimately, the precision of controllers were compared and analyzed theoretically and numerically. The long-range precision of the proposed controllers are confirmed by third stage.
Adaptive Controller and Synchronizer Design for Hyperchaotic Zhou System with...Zac Darcy
In this paper, we establish new results for the adaptive controller and synchronizer design for the
hyperchaotic Zhou system (2009), when the parameters of the system are unknown. Using adaptive control theory and Lyapunov stability theory, we first design an adaptive controller to stabilize the hyperchaotic Zhou system to its unstable equilibrium at the origin. Next, using adaptive control theory and Lyapunov stability theory, we design an adaptive controller to achieve global chaos synchronization
of the identical hyperchaotic Zhou systems with unknown parameters. Simulations have been provided for adaptive controller and synchronizer designs to validate and illustrate the effectiveness of the schemes.
GLOBAL CHAOS SYNCHRONIZATION OF UNCERTAIN LORENZ-STENFLO AND QI 4-D CHAOTIC S...ijistjournal
In this paper, we apply adaptive control method to derive new results for the global chaos synchronization of 4-D chaotic systems, viz. identical Lorenz-Stenflo(LS) systems (Stenflo, 2001), identical Qi systems (Qi, Chen and Du, 2005) and non-identical LS and Qi systems. In this paper, we shall assume that the parameters of both master and slave systems are unknown and we devise adaptive control schemes for synchronization using the estimates of parameters for both master and slave systems. Our adaptive synchronization schemes derived in this paper are established using Lyapunov stability theory. Since the Lyapunov exponents are not required for these calculations, the adaptive control method is very effective and convenient to synchronize identical and non-identical LS and Qi systems.
Numerical simulations are shown to demonstrate the effectiveness of the proposed adaptive synchronization schemes for the identical and non-identical, uncertain LS and Qi 4-D chaotic systems.
GLOBAL CHAOS SYNCHRONIZATION OF UNCERTAIN LORENZ-STENFLO AND QI 4-D CHAOTIC S...ijistjournal
In this paper, we apply adaptive control method to derive new results for the global chaos synchronization of 4-D chaotic systems, viz. identical Lorenz-Stenflo(LS) systems (Stenflo, 2001), identical Qi systems (Qi, Chen and Du, 2005) and non-identical LS and Qi systems. In this paper, we shall assume that the parameters of both master and slave systems are unknown and we devise adaptive control schemes for synchronization using the estimates of parameters for both master and slave systems. Our adaptive synchronization schemes derived in this paper are established using Lyapunov stability theory. Since the Lyapunov exponents are not required for these calculations, the adaptive control method is very effective and convenient to synchronize identical and non-identical LS and Qi systems. Numerical simulations are shown to demonstrate the effectiveness of the proposed adaptive synchronization schemes for the identical and non-identical, uncertain LS and Qi 4-D chaotic systems.
ADAPTIVE CONTROLLER DESIGN FOR THE HYBRID SYNCHRONIZATION OF HYPERCHAOTIC XU ...ijait
This paper derives new adaptive results for the hybrid synchronization of hyperchaotic Xi systems (2009)
and hyperchaotic Li systems (2005). In the hybrid synchronization design of master and slave systems, one
part of the systems, viz. their odd states, are completely synchronized (CS), while the other part, viz. their
even states, are completely anti-synchronized (AS) so that CS and AS co-exist in the process of
synchronization. The research problem gets even more complicated, when the parameters of the
hyperchaotic systems are unknown and we tackle this problem using adaptive control. The main results of
this research work are proved using adaptive control theory and Lyapunov stability theory. MATLAB
simulations using classical fourth-order Runge-Kutta method are shown for the new adaptive hybrid
synchronization results for the hyperchaotic Xu and hyperchaotic Li systems.
ACTIVE CONTROLLER DESIGN FOR THE HYBRID SYNCHRONIZATION OF HYPERCHAOTIC XU AN...Zac Darcy
The synchronization of chaotic systems treats a pair of chaotic systems, which are usually called as master
and slave systems. In the chaos synchronization problem, the goal of the design is to synchronize the states
of master and slave systems asymptotically. In the hybrid synchronization design of master and slave
systems, one part of the systems, viz. their odd states, are completely synchronized (CS), while the other
part, viz. their even states, are completely anti-synchronized (AS) so that CS and AS co-exist in the process
of synchronization. This research work deals with the hybrid synchronization of hyperchaotic Xi systems
(2009) and hyperchaotic Li systems (2005). The main results of this hybrid research work are established
with Lyapunov stability theory. MATLAB simulations of the hybrid synchronization results are shown for
the hyperchaotic Xu and Li systems.
ADAPTIVE STABILIZATION AND SYNCHRONIZATION OF HYPERCHAOTIC QI SYSTEM cseij
The hyperchaotic Qi system (Chen, Yang, Qi and Yuan, 2007) is one of the important models of fourdimensional hyperchaotic systems. This paper investigates the adaptive stabilization and synchronization of hyperchaotic Qi system with unknown parameters. First, adaptive control laws are designed to stabilize the hyperchaotic Qi system to its equilibrium point at the origin based on the adaptive control theory and Lyapunov stability theory. Then adaptive control laws are derived to achieve global chaos synchronization of identical hyperchaotic Qi systems with unknown parameters. Numerical simulations are shown to demonstrate the effectiveness of the proposed adaptive stabilization and synchronization schemes.
Adaptive Stabilization and Synchronization of Hyperchaotic QI SystemCSEIJJournal
The hyperchaotic Qi system (Chen, Yang, Qi and Yuan, 2007) is one of the important models of four-
dimensional hyperchaotic systems. This paper investigates the adaptive stabilization and synchronization
of hyperchaotic Qi system with unknown parameters. First, adaptive control laws are designed to
stabilize the hyperchaotic Qi system to its equilibrium point at the origin based on the adaptive control
theory and Lyapunov stability theory. Then adaptive control laws are derived to achieve global chaos
synchronization of identical hyperchaotic Qi systems with unknown parameters. Numerical simulations
are shown to demonstrate the effectiveness of the proposed adaptive stabilization and synchronization
schemes.
International Journal of Mathematics and Statistics Invention (IJMSI) is an international journal intended for professionals and researchers in all fields of computer science and electronics. IJMSI publishes research articles and reviews within the whole field Mathematics and Statistics, new teaching methods, assessment, validation and the impact of new technologies and it will continue to provide information on the latest trends and developments in this ever-expanding subject. The publications of papers are selected through double peer reviewed to ensure originality, relevance, and readability. The articles published in our journal can be accessed online.
In this chapter, the authors extend the theory of
the generalized difference Operator ∆L to the generalized
difference operator of the 풏
풕풉kind denoted by ∆L Where L
=푳 = {풍ퟏ,풍ퟐ,….풍풏} of positive reals풍ퟏ,풍ퟐ,….풍풏and obtain some
interesting results on the relation between the generalized
polynomial factorial of the first kind, 풏
풕풉kind and algebraic
polynomials. Also formulae for the sum of the general
partial sums of products of several powers of consecutive
terms of an Arithmetic progression in number theory are
derived.
International Journal of Computer Science, Engineering and Information Techno...ijcseit
In chaos theory, the problem anti-synchronization of chaotic systems deals with a pair of chaotic systems
called drive and response systems. In this problem, the design goal is to drive the sum of their respective
states to zero asymptotically. This problem gets even more complicated and requires special attention when
the parameters of the drive and response systems are unknown. This paper uses adaptive control theory
and Lyapunov stability theory to derive new results for the anti-synchronization of hyperchaotic Wang
system (2008) and hyperchaotic Li system (2005) with uncertain parameters. Hyperchaotic systems are
nonlinear dynamical systems exhibiting chaotic behaviour with two or more positive Lyapunov exponents.
The hyperchaotic systems have applications in areas like oscillators, lasers, neural networks, encryption,
secure transmission and secure communication. The main results derived in this paper are validated and
demonstrated with MATLAB simulations.
Similar to Generic Reinforcement Schemes and Their Optimization (20)
Laura F. Cacovean, Florin Stoica, Dana Simian, A New Model Checking Tool, Proceedings of the 5th European Computing Conference (ECC ’11), Paris, France, pp. 358-363, April 28-30, 2011
CTL Model Update Implementation Using ANTLR Toolsinfopapers
L. Cacovean, F. Stoica, CTL Model Update Implementation Using ANTLR Tools, Proceedings of the 13th WSEAS International Conference on COMPUTERS, Rodos, Greece, July 23-25, 2009, ISSN: 1790-5109, ISBN: 978-960-474-099-4
Generating JADE agents from SDL specificationsinfopapers
F. Stoica, Generating JADE agents from SDL specifications, International Journal of Computers, Communications & Control, Supplementary Issue, Volume I, ISSN 1841-9836, pp. 429-438, 2006
An evolutionary method for constructing complex SVM kernelsinfopapers
D. Simian, F. Stoica, An Evolutionary Method for Constructing Complex SVM Kernels, Recent Advances in Mathematics and Computers in Biology and Chemistry, Proceedings of the 10th International Conference on Mathematics and Computers in Biology and Chemistry, MCBC’09, Prague, Chech Republic, WSEAS Press, ISBN 978-960-474-062-8, ISSN 1790-5125, pp.172-178, 2009
Evaluation of a hybrid method for constructing multiple SVM kernelsinfopapers
Dana Simian, Florin Stoica, Evaluation of a hybrid method for constructing multiple SVM kernels, Recent Advances in Computers, Proceedings of the 13th WSEAS International Conference on Computers, Recent Advances in Computer Engineering Series, WSEAS Press, Rodos, Greece, July 23-25, 2009, ISSN: 1790-5109, ISBN: 978-960-474-099-4, pp. 619-623
Interoperability issues in accessing databases through Web Servicesinfopapers
Florin Stoica, Laura Florentina Cacovean, Interoperability Issues in Accessing Databases through Web Services, Proceedings of the Recent Advances in Neural Networks, Fuzzy Systems & Evolutionary Computing, 13-15 June 2010, Iaşi, Romania, ISSN: 1790-2769, ISBN: 978-960-474-194-6, pp. 279-284
Using Ontology in Electronic Evaluation for Personalization of eLearning Systemsinfopapers
I. Pah, F. Stoica, L. F. Cacovean, E. M. Popa, Using Ontology in Electronic Evaluation for Personalization of eLearning Systems, Proceedings of the 8th WSEAS International Conference on APPLIED INFORMATICS and COMMUNICATIONS (AIC’08), Rhodes, Greece, August 20-22, ISSN: 1790-5109, ISBN: 978-960-6766-94-7, pp. 332-337, 2008
Models for a Multi-Agent System Based on Wasp-Like Behaviour for Distributed ...infopapers
D. Simian, F. Stoica, C. Simian, Models for a Multi-Agent System Based on Wasp-like Behaviour for Distributed Patients Repartition, Proceedings of the 9th WSEAS International Conference on Evolutionary Computing, Sofia, Bulgaria, ISBN 978-960-6766-58-9, ISSN 1790-5109, pp. 82-86, May 2008
This pdf is about the Schizophrenia.
For more details visit on YouTube; @SELF-EXPLANATORY;
https://www.youtube.com/channel/UCAiarMZDNhe1A3Rnpr_WkzA/videos
Thanks...!
What is greenhouse gasses and how many gasses are there to affect the Earth.moosaasad1975
What are greenhouse gasses how they affect the earth and its environment what is the future of the environment and earth how the weather and the climate effects.
Slide 1: Title Slide
Extrachromosomal Inheritance
Slide 2: Introduction to Extrachromosomal Inheritance
Definition: Extrachromosomal inheritance refers to the transmission of genetic material that is not found within the nucleus.
Key Components: Involves genes located in mitochondria, chloroplasts, and plasmids.
Slide 3: Mitochondrial Inheritance
Mitochondria: Organelles responsible for energy production.
Mitochondrial DNA (mtDNA): Circular DNA molecule found in mitochondria.
Inheritance Pattern: Maternally inherited, meaning it is passed from mothers to all their offspring.
Diseases: Examples include Leber’s hereditary optic neuropathy (LHON) and mitochondrial myopathy.
Slide 4: Chloroplast Inheritance
Chloroplasts: Organelles responsible for photosynthesis in plants.
Chloroplast DNA (cpDNA): Circular DNA molecule found in chloroplasts.
Inheritance Pattern: Often maternally inherited in most plants, but can vary in some species.
Examples: Variegation in plants, where leaf color patterns are determined by chloroplast DNA.
Slide 5: Plasmid Inheritance
Plasmids: Small, circular DNA molecules found in bacteria and some eukaryotes.
Features: Can carry antibiotic resistance genes and can be transferred between cells through processes like conjugation.
Significance: Important in biotechnology for gene cloning and genetic engineering.
Slide 6: Mechanisms of Extrachromosomal Inheritance
Non-Mendelian Patterns: Do not follow Mendel’s laws of inheritance.
Cytoplasmic Segregation: During cell division, organelles like mitochondria and chloroplasts are randomly distributed to daughter cells.
Heteroplasmy: Presence of more than one type of organellar genome within a cell, leading to variation in expression.
Slide 7: Examples of Extrachromosomal Inheritance
Four O’clock Plant (Mirabilis jalapa): Shows variegated leaves due to different cpDNA in leaf cells.
Petite Mutants in Yeast: Result from mutations in mitochondrial DNA affecting respiration.
Slide 8: Importance of Extrachromosomal Inheritance
Evolution: Provides insight into the evolution of eukaryotic cells.
Medicine: Understanding mitochondrial inheritance helps in diagnosing and treating mitochondrial diseases.
Agriculture: Chloroplast inheritance can be used in plant breeding and genetic modification.
Slide 9: Recent Research and Advances
Gene Editing: Techniques like CRISPR-Cas9 are being used to edit mitochondrial and chloroplast DNA.
Therapies: Development of mitochondrial replacement therapy (MRT) for preventing mitochondrial diseases.
Slide 10: Conclusion
Summary: Extrachromosomal inheritance involves the transmission of genetic material outside the nucleus and plays a crucial role in genetics, medicine, and biotechnology.
Future Directions: Continued research and technological advancements hold promise for new treatments and applications.
Slide 11: Questions and Discussion
Invite Audience: Open the floor for any questions or further discussion on the topic.
Comparing Evolved Extractive Text Summary Scores of Bidirectional Encoder Rep...University of Maribor
Slides from:
11th International Conference on Electrical, Electronics and Computer Engineering (IcETRAN), Niš, 3-6 June 2024
Track: Artificial Intelligence
https://www.etran.rs/2024/en/home-english/
Nutraceutical market, scope and growth: Herbal drug technologyLokesh Patil
As consumer awareness of health and wellness rises, the nutraceutical market—which includes goods like functional meals, drinks, and dietary supplements that provide health advantages beyond basic nutrition—is growing significantly. As healthcare expenses rise, the population ages, and people want natural and preventative health solutions more and more, this industry is increasing quickly. Further driving market expansion are product formulation innovations and the use of cutting-edge technology for customized nutrition. With its worldwide reach, the nutraceutical industry is expected to keep growing and provide significant chances for research and investment in a number of categories, including vitamins, minerals, probiotics, and herbal supplements.
(May 29th, 2024) Advancements in Intravital Microscopy- Insights for Preclini...Scintica Instrumentation
Intravital microscopy (IVM) is a powerful tool utilized to study cellular behavior over time and space in vivo. Much of our understanding of cell biology has been accomplished using various in vitro and ex vivo methods; however, these studies do not necessarily reflect the natural dynamics of biological processes. Unlike traditional cell culture or fixed tissue imaging, IVM allows for the ultra-fast high-resolution imaging of cellular processes over time and space and were studied in its natural environment. Real-time visualization of biological processes in the context of an intact organism helps maintain physiological relevance and provide insights into the progression of disease, response to treatments or developmental processes.
In this webinar we give an overview of advanced applications of the IVM system in preclinical research. IVIM technology is a provider of all-in-one intravital microscopy systems and solutions optimized for in vivo imaging of live animal models at sub-micron resolution. The system’s unique features and user-friendly software enables researchers to probe fast dynamic biological processes such as immune cell tracking, cell-cell interaction as well as vascularization and tumor metastasis with exceptional detail. This webinar will also give an overview of IVM being utilized in drug development, offering a view into the intricate interaction between drugs/nanoparticles and tissues in vivo and allows for the evaluation of therapeutic intervention in a variety of tissues and organs. This interdisciplinary collaboration continues to drive the advancements of novel therapeutic strategies.
Generic Reinforcement Schemes and Their Optimization
1. Proceedings of the European Computing Conference
Generic Reinforcement Schemes and Their Optimization
DANA SIMIAN, FLORIN STOICA
Department of Informatics
“Lucian Blaga” University of Sibiu
Str. Dr. Ion Ratiu 5-7, 550012, Sibiu
ROMANIA
dana.simian@ulbsibiu.ro, florin.stoica@ulbsibiu.ro
Abstract: - The aim of this paper is to introduce a generic two-parameters dependent absolutely expedient
reinforcement scheme and to present a method for learning parameters optimization. We optimize, using a Breeder
genetic algorithm, many schemes derived from our generic one, in order to reach the best performance. Furthermore,
we compare our results in terms of speed and efficiency.
Key-Words: - Reinforcement Learning, Breeder genetic algorithm, Optimization
1 Introduction
Reinforcement schemes represent algorithms which
realize the learning process for stochastic learning
automata. Stochastic learning automata adapt to changes
in their environment as results of a reinforcement
learning process. Given a set of possible actions, a
stochastic learning automaton must choose the optimal
one, based on the environment response and the past
actions. Initially equal probabilities are associated to all
possible actions, one action is selected at random and the
actions probabilities are updated based on the
environment response. A detailed characterization of
reinforcement learning can be found in [14]. In [7] is
underlined that the major advantage of reinforcement
learning is that it needs information about the
environment only for the reinforcement signal.
Reinforcement learning has several applications in
autonomic robotics, designing multi-agent systems,
intelligent vehicles control, etc. ([2], [3], [5], [15], [16]).
In [11] we designed a simulator of an intelligent vehicle
control system. The system was based on two learning
automata.
In other articles ([9], [13]), we defined new
reinforcement schemes in order to reach a best
performance of our system. Usually, reinforcement
schemes depends on many parameters, as we can see in
section 3. An important problem is to choose the optimal
scheme’s parameters.
The aim of this paper is to introduce a generic
reinforcement scheme from which many other
reinforcement schems can be obtained and to present an
optimization method of these schemes with respect to
learning parameters. We also optimize this new scheme
and those that we introduced in [9], [12]. We evaluate
and compare our schemes using two criteria: the speed
of the optimization process and the efficiency of the
optimized schema.
The remainder of this paper is organized as follows. In
section 2 we briefly present the mathematical
backgrounds of stochastic learning automata with
variable structure. In section 3 is presented our generic
absolutely expedient reinforcement scheme, together
with other particular schemes derived from it. In section
3 we present our optimization method for reinforcement
schemes learning parameters and analyze the provided
results. Conclusions and further directions of study are
presented in section 5.
2 Mathematical backgrounds of stochastic
automata
A stochastic automaton supposes the existence of a set of
actions, which define the input of the environment and a
response set. The range of the response values depends
on the model we chose. There are three different models
for representation of the response values: P-model, S-model
and Q-model. The P-model uses a set of binary
values, 0 or 1. In the S-model the response values are
continuous in the range (0, 1). In the Q-model the
response set is a finite set of discrete values in the range
(0, 1). In this paper we use the P-model for our
reinforcement schemes.
A stochastic automaton selects one action at random,
observes the response from the environment and updates
the action probabilities based on that response. An action
can be rewarded or punished using a set of penalties
probabilities.
Mathematical model of a stochastic automaton is defined
by a triple {α , c,β } corresponding to the elements
presented before:
a) α ={α1 ,α 2 ,...,α r } - the input actions of the
environment
b) β - the response set.
In the case of P-model, { 1 , 2} β = β β is a binary set:
ISBN: 978-960-474-297-4 332
2. Proceedings of the European Computing Conference
β = 0 is a favourable outcome and β =1 is an
unfavourable outcome.
To reefer the time instant is used the notationα (n) ,
β (n) .
c) c ={c1 , c2 ,..., cr } - the set of penalty probabilities.
The element i c is the probability that action i α
will
result in an unfavourable response:
ci = P(β (n) =1|α (n) =α i ) i =1, 2, ..., r
The evolution in time of penalty probabilities defines
two types of environments: stationary (the penalty
probabilities are constant over time) and nonstationary
(the penalties change over time).
In the following we consider only stationary random
environments.
The action probabilities vector at time moment n+1 is
updated using a mapping T and the current probabilities
pi (n) = P(α (n) =α i ), i =1, r :
p(n +1) = T[ p(n),α (n),β (n)]
Reinforcement schemes are named linear if p(n +1) is
a linear function of p(n) , and nonlinear otherwise.
The evaluation of performances of a learning automaton
is made using a quantitative norm of behavior ([17])
represented by the average penalty for a given action
probability vector, M(n).
M n P n p n
( ) ( ( ) 1| ( ))
= β
= =
r r
=Σ = = ∗ = =Σ
P n n P n cpn
( β ( ) 1| α ( ) α ) ( α ( ) α
) ( )
i i i i
i i
1 1
= =
The only class of reinforcement schemes for which
necessary and sufficient conditions of design are
available is represented by absolutely expedient learning
schemes, defined in [7]. An automaton is absolutely
expedient if M(n +1) < M(n) for all n ([7]).
The general solution for absolutely expedient schemes
was found by Lakshmivarahan and Thathachar in [4].
Other studies about expedient learning algorithms can be
found in [8].
In [17] is presented a nonlinear absolutely expedient
reinforcement scheme, for a stationary N-teacher P-model
environment. In the case of N-teacher model, if
the automaton produced the action i α
and the responses
from environments (or “teachers”) are denoted by
j j N
β i =1,..., , then the updating rules are:
⎤
( 1) ( ) 1 β φ ( ( ))
Σ Σ
≠ =
=
− ∗ ⎥⎦
⎡
⎢⎣
+ = +
r
j
j i
j
N
k
k
i i i p n
N
p n p n
1 1
⎤
1 1 β ψ ( ( )) (1)
Σ Σ
≠ =
=
∗ ⎥⎦
⎡
− −
⎢⎣
r
j
j i
j
N
k
k
i p n
N 1 1
( 1) ( ) 1 ( ( ))
+ = − ⎡ ⎤ ∗ + ⎢⎣ ⎥⎦
p n p n p n
1
1 1 ( ( )), .
⎡ ⎤
+ ⎢ − ⎥ ∗ ∀ ≠ ⎣ 1
⎦
N
k
j j i j
k
N
k
i j
k
N
p n j i
N
β φ
β ψ
=
=
Σ
Σ
(2)
i φ
and i ψ
satisfy the following conditions:
p n
p n
r λ
1 = = = p n ≤
( ( )) 0
( ( ))
( )
...
( ( ))
1
( )
p n
p n
r
φ φ
(3)
p n
p n
r μ
1 = = = p n ≤
( ( )) 0
( ( ))
( )
...
( ( ))
1
( )
p n
p n
r
ψ ψ
(4)
r
Σ
pi n j p n
( ) + φ ( ( )) >
0 (5)
i j j
1
≠ =
r
Σ
pi n j p n
( ) − ψ ( ( )) <
1 (6)
i j j
1
≠ =
p j (n) +ψ j ( p(n)) > 0 (7)
p j (n) −φ j ( p(n)) <1 (8)
for all j∈{1,..., r} {i}
In [1] and [15] is proved that the automaton with the
reinforcement scheme given in (1)-(2) is absolutely
expedient in a stationary environment if the functions
λ ( p(n)) and μ ( p(n)) satisfy the following conditions:
λ ( p(n)) ≤ 0
μ ( p(n)) ≤ 0 (9)
λ ( p(n)) + μ ( p(n)) < 0
3 Generic absolutely expedient
reinforcement scheme
In the following we present a generic two-parameter
dependent reinforcement schemes and prove that this
scheme is absolutely expedient in a stationary
environment. We start from the scheme given in (1) –
(2). This scheme is also valid for a single-teacher model.
In this case we will define a single environment response
denoted by f .
Thus, the updating rules become:
pn pn f Hn pn
( 1) ( ) ( ( )) [1 ( )]
(1 ) ( ) [1 ( )]
+ = + ∗ − ∗ ∗ − −
i i 1
i
f pn
2
i
γ
γ
− − ∗ − ∗ −
pn pn f Hn
p n f p n
( 1) ( ) ( ( ))
( ) (1 ) ( ) ( )
+ = − ∗ − γ
∗ ∗
1
∗ + − ∗ − ∗
γ
2
j j
j j
(10)
for all j ≠ i , i.e.:
2 ( ( )) ( ) k k ψ p n = −γ ∗ p n
1 ( ( )) ( ) ( ) k k φ p n = −γ ∗H n ∗ p n
ISBN: 978-960-474-297-4 333
3. Proceedings of the European Computing Conference
where learning parameters 1 γ and 2 γ are real values
γ 1,γ 2 ∈(0,1) (11)
The function H is defined as:
⎧
( ) min 1; max min ( ) ,
{ {
= ⎨ i
− ⎩ ∗ (1 −
( ))
1
i
H n p n
p n
ε
γ
}}
⎛ ⎫ 1 − ()
⎞ ⎪ ⎜⎜ − ∗ ⎟⎟ ⎬ ⎝ 1 ⎠ 1,
⎪⎭
;0
( )
j
j j r
j i
p n
p n
ε
γ =≠
Parameter ε is an arbitrarily small positive real number.
Our reinforcement scheme differs from schemes given in
[15]-[17], by the definition of H and φ k .
We will show that are satisfied all the conditions of the
reinforcement scheme (1) - (2).
From (3), (4) we have:
p n H n p n Hn pn
p n p n
( ( )) ( ) ( ) ( ) ( ( ))
( ) ( )
φ γ
− ∗ ∗
= 1
=− ∗ = 1
(3’)
k k
k k
γ λ
p n p n p n
p n p n
( ( )) ( ) ( ( ))
( ) ( )
ψ γ
− ∗
= 2
=− = (4’)
2
k k
k k
γ μ
The conditions (5) – (8) become:
p n H n p n
H n p n
( ) ( ) (1 ( )) 0
( ) ( )
− ∗ ∗ − > ⇔
<
i 1
i
1
i
p n
(1 ( ))
i
γ
γ
∗ −
(5’)
Condition (5’) is satisfied by the definition of the
function H(n) .
2 ( ) (1 ( )) 1 i i p n +γ ∗ − p n < (6’)
But 2 ( ) (1 ( )) ( ) 1 ( ) 1 i i i i p n +γ ∗ − p n < p n + − p n =
since 2 0 <γ <1
2 ( ) ( ) 0, {1,..., }{ } j j p n −γ ∗ p n > ∀j∈ r i (7’)
But 2 2 ( ) ( ) ( ) (1 ) 0 j j j p n −γ ∗ p n = p n ∗ −γ >
since 2 0 <γ <1 and 0 < p j (n) <1 for all
j∈{1,..., r}{i}
1
p n
1 −
()
1
( ) ( ) ( ) 1 ( )
( )
j
j j
j
p n H n p n H n
p n
γ
γ
+ ∗ ∗ < ⇔ <
∗
(8’)
∀j∈{1,..., r}{i} .
This condition is satisfied by the definition of the
function H(n) .
Therefore our reinforcement scheme is a candidate for
absolute expediency.
Furthermore, the functions λ and μ for our nonlinear
scheme satisfy:
1 λ ( p(n)) = −γ ∗H(n) ≤ 0
2 μ ( p(n)) = −γ ≤ 0
1 2 λ ( p(n)) +μ ( p(n)) = −γ ∗H(n) −γ < 0
In conclusion, the algorithm given in equations (10) is
absolutely expedient in a stationary environment. This
algorithm defines a two-parameter dependent generic
absolutely expedient reinforcement scheme. We will
denote this scheme by 2
1 Rγ
γ . Choosing different
expressions for the parameters, such that (11) holds, we
obtained several absolutely expedient reinforcement
schemes.
In [9] we introduced and studied the scheme *(1 )
(1 )* Rθ δ
−
− θ δ
,
with 0 <θ <1 and 0 <δ <1. Obviously
0 <θ *(1−δ ) <1 and 0 < (1−θ )*δ <1, therefore this
is a absolutely expedient reinforcement scheme.
In [12] we introduced the scheme θ
Rθ *δ , with 0 <θ <1
and 0 <θ ∗δ <1.
4. Optimization of two-parameters
reinforcement schemes
A very important problem is to find the optimal values
of learning parameters in the scheme 2
1 Rγ
γ in order to
reach the best performance. In [13], we introduced first
the idea of learning parameters optimization in a
reinforcement scheme using genetic algorithms. We
develop here this idea and use a Breeder genetic
algorithm, for providing the optimal learning parameters
for the generic scheme 2
1 Rγ
γ . We also apply the method
for the particular schemes presented in section 3.
Furthermore, we compare our results in terms of speed
and efficiency. For the simplicity of notations, we
consider, in our comparisons, the scheme Rθ
δ , with
1 Rγ
δ ,θ ∈(0,1) instead of 2
γ .
The aim is to find optimal values for the learning
parameters δ and θ in the schemes: Rθ
δ , *(1 )
(1 )* Rθ δ
−
and
− θ δ
θ
Rθ *δ .
Because parameters are real values, we use the Breeder
genetic algorithm, proposed by Mühlenbein and
Schlierkamp-Voosen in [6], which represents solutions
(chromosomes) as vectors of real numbers. This
algorithm is closer to the reality than normal genetic
algorithms which use discrete representation of
solutions. The skeleton of the Breeder genetic algorithm
can be found in [13]. The selection is achieved randomly
from the T% best elements of current population, where
T is a constant of the algorithm (usually, T = 40 provide
best results). Thus, within each generation, two elements
selected from the T% best chromosomes are subject to
crossover operation. On the new child obtained from the
mate of the parents is applied the mutation operator. The
ISBN: 978-960-474-297-4 334
4. Proceedings of the European Computing Conference
process is repeated until are obtained N-1 new
individuals, where N represents the size of the initial
population. The best chromosome (evaluated through
fitness function) is inserted in the new population (1-
elitism). Thus, the new population will have also N
elements.
Let be 1,..., { }i i n x x = = and 1,..., { }i i n y y = = two
chromosomes. The Breeder crossover operator gives a
new chromosome z, whose genes are represented by
( ) i i i i i z x y x α = + − , i=1,…,n, with i α
a random
variable uniformly distributed between [−ε ,1+ε ], ε
depends on the problem to be solved and typically is in
the interval [0,0.5] .
The mutation operator gives i i i i i x = x + s ⋅ r ⋅a , i=1,…n
with { 1,1} i s ∈ − uniform at random,
i xi r = r ⋅ domain ,
r∈[0.1, 0.5] (typically 0.1) , 2 k
i a = − ⋅α withα ∈[0,1]
uniform at random and k is the number of bytes used to
represent a number in the machine within is executed the
Breeder algorithm (mutation precision).
The probability of mutation is typically choosed as 1/ n .
In order to find the best values for learning parameters
δ and θ of our reinforcement schemes and to compare
the results, we consider the same example we used in
[9], [13]. We used our reinforcement schemes for robot
navigation in the grid world presented in Fig. 1. The
current position of the robot is marked by a circle.
Navigation is done using four actionsα ={N, S, E,W} ,
corresponding to the four possible movements along the
coordinate directions.
Fig. 1. Grid world for robot navigation
We have a single optimal action (movement to S). In the
learning process, only this action receives reward.
Initially, we choose for the optimal action a small
probability value (0.0005). We stop the execution when
the probability of the optimal action, popt, reaches a
certain value (popt=0.9999).
We make the performance evaluation of our schemes
using the “number of steps” of the learning algorithm
until the stop condition is achieved.
Using the Breeder genetic algorithm, we can provide the
optimal learning parameters for our schemes, in order to
reach the best performance.
Each chromosome contains two genes, representing the
real values δ and θ . The fitness function for
chromosomes evaluation is represented by the number of
steps necessary by the learning process to reach the
value 0.9999 for the probability of the optimal action.
The parameters of Breeder algorithm are assigned with
following values: δ = 0 , r = 0.1, k = 8 . The initial
population has 400 chromosomes and algorithm is
stopped after 1000 generations.
The results provided by the Breeder genetic algorithm
are presented in Table 1.
Optimal values for learning parameters
provided by the Breeder algorithm
4 actions with
(0) 0.0005,
(0) =
0.9995 / 3
=
opt
p
i≠opt
p
Scheme 4.1
θ
Rδ
Scheme 4.2
θ
Rθ *δ
Scheme 4.3
*(1 )
(1 )*
θ −
δ
R −
θ δ
δ 0.5866 0.7036 0.5741
θ 0.9469 0.8983 0.3640
Average
number of
steps to reach
16.95 16.98 43.70
popt=0.9999
Table 1. Optimal values for learning parameters
provided by the Breeder genetic algorithm
Fig. 2. Schema optimization vs. time passed
In figure 2 is presented the optimization process for
reinforcement schemes analyzed in Table 1, using two
dimensions of data: the time passed vs. the performance
evaluation of optimized scheme (number of steps
necessary to reach the stop conditions of the learning
process).
In figure 3 is presented the optimization process using as
dimensions of data the number of generations in the
Breeder algorithm vs. the performance evaluation of
optimized scheme.
ISBN: 978-960-474-297-4 335
5. Proceedings of the European Computing Conference
Fig. 3. Schema optimization vs. number of generations
in Breeder algorithm
With results obtained in Table 1, we can conclude that
Breeder genetic algorithm is capable to provide the best
values for learning parameters, and thus our schemes
were optimized for best performance. The results
obtained by our nonlinear optimized schemes are
significant better than those obtained in [10], [12], [17].
5 Conclusions
Using a Breeder genetic algorithm, we found
automatically the optimal values for the learning
parameters of many reinforcement schemes, in order to
reach the best performance, measured in number of
iterations in learning process (“number of steps”).
From graphical results of optimization process showed
in Fig.2 and Fig. 3, we can conclude that scheme 4.3,
R θ *(1 −
δ
)
(1 )*
, − θ δ
is more adequate for applications with less time
allocated for schema optimization, and scheme 4.2,
Rθ
, θ *δ is very efficient if we allocate for optimization
enough time. However, the new generic scheme θ
Rδ ,
introduced in section 3, outperforms the other schemes
in terms of speed and qualitative results in the learning
process.
There are many possibilities for choosing the form of
parameters in generic scheme 2
1 Rγ
γ such that the
conditions (11) are satisfied. Breeder genetic algorithm,
presented in section 4, can be used for optimization of
parameters values regardless of choice of 1 2 γ ,γ .
The graphical results obtained suggest than
1 2 γ =δ ,γ =θ , with 0 <δ ,θ <1 give better results
than other more complicated choices. As further
directions of study we want to rigorous prove or to
invalidate this conjecture.
References:
[1] N. Baba, New Topics in Learning Automata: Theory
and Applications, Lecture Notes in Control and
Information Sciences, Berlin, Germany: Springer-
Verlag, pp.750-758, 1984.
[2] O. Buffet, A. Dutech, and F. Charpillet, Incremental
reinforcement learning for designing multi-agent
systems, In J. P. Müller, E. Andre, S. Sen, and C.
Frasson, editors, Proceedings of the Fifth International
Conference onAutonomous Agents, Montreal, Canada,
ACM Press, pp. 31–38, 2001.
[3] M. Dorigo, Introduction to the Special Issue on
Learning Autonomous Robots, IEEE Trans. on Systems,
Man and Cybernetics - part B, Vol. 26, No. 3, pp. 361-
364, 1996.
[4] S. Lakshmivarahan, M.A.L. Thathachar, Absolutely
Expedient Learning Algorithms for Stochastic
Automata, IEEE Transactions on Systems, Man and
Cybernetics, vol. SMC-6, pp. 281-286, 1973.
[5] J. Moody, Y. Liu, M. Saffell, and K. Youn.
Stochastic direct reinforcement: Application to simple
games with recurrence, In Proceedings of Artificial
Multiagent Learning. Papers from the 2004 AAAI Fall
Symposium, Technical Report FS-04-02.
[6] H. Mühlenbein, D. Schlierkamp-Voosen, The science
of breeding and its application to the breeder genetic
algorithm, Evolutionary Computation, vol. 1, pp. 335-
360, 1994.
[7] K. S. Narendra, M. A. L. Thathachar, Learning
Automata: an introduction, Prentice-Hall, 1989.
[8] C. Rivero, Characterization of the absolutely
expedient learning algorithms for stochastic automata in
a non-discrete space of actions, ESANN'2003
proceedings - European Symposium on Artificial Neural
Networks Bruges (Belgium), pp. 307-312, 2003.
[9] D. Simian, F. Stoica, A New Nonlinear
Reinforcement Scheme for Stochastic Learning
Automata, Proceedings of the 12th WSEAS International
Conference on Automatic Control, Modelling &
Simulation, Catania, Italy, pp. 450-454, 2010.
[10] F. Stoica, E. M. Popa, An Absolutely Expedient
Learning Algorithm for Stochastic Automata, WSEAS
Transactions on Computers, Issue 2, Volume 6, pp. 229-
235, 2007.
[11] F. Stoica, D. Simian, Automatic control based on
Wasp Behavioral Model and Stochastic Learning
Automata. Mathematics and Computers in Science and
Engineering Series, Proceedings of 10th WSEAS
Conference on Mathematical Methods, Computational
ISBN: 978-960-474-297-4 336
6. Proceedings of the European Computing Conference
Techniques and Intelligent Systems (MAMECTIS '08),
Corfu 2008, WSEAS Press, pp. 289-295, 2008.
[12] F. Stoica, E. M. Popa, I. Pah, A new reinforcement
scheme for stochastic learning automata – Application to
Automatic Control, Proceedings of the International
Conference on e-Business, Porto, Portugal, pp. 45-50,
2008.
[13] F. Stoica, D. Simian, Optimizing a New Nonlinear
Reinforcement Scheme with Breeder genetic algorithm,
Proceedings of the 11th WSEAS International
Conference on Evolutionary Computing (EC'10), Iaşi,
Romania, pp. 273-278, 2010.
[14] R. Sutton, A. Barto, Reinforcement learning: An
introduction, MIT-press, Cambridge, MA, 1998.
[15] C. Ünsal, P. Kachroo, J. S. Bay, Simulation Study
of Learning Automata Games in Automated Highway
Systems, 1st IEEE Conference on Intelligent
Transportation Systems (ITSC’97), Boston,
Massachusetts, 1997
[16] C. Ünsal, P. Kachroo, J. S. Bay, Simulation Study
of Multiple Intelligent Vehicle Control using Stochastic
Learning Automata, IEEE Transactions on Systems,
Man and Cybernetics – Part A, Systems and Human,
pp.1-42, 1997.
[17] C. Ünsal, Intelligent Navigation of Autonomous
Vehicles in an Automated Highway System: Learning
Methods and Interacting Vehicles Approach, dissertation
thesis, Pittsburg University, Virginia, USA, 1997.
ISBN: 978-960-474-297-4 337