This document provides an introduction to deep learning and fuzzy logic. It discusses neural networks, fuzzy sets, membership functions, fuzzy rules and inference. For fuzzy logic, it covers fuzzification, rule evaluation, aggregation, and defuzzification. It provides examples of fuzzy systems for modeling health based on height and weight. For deep learning, it describes the multilayer perceptron model and how it can be used for classification and regression with supervised learning.
Artificial Intelligence - Problems, State Space Search & Heuristic Search Techniques - Defining the Problems as a State Space Search
Production Systems
Production Characteristics
Production System Characteristics
Issues in the design of Search Programs
Particle swarm optimization (PSO) is an evolutionary computation technique for optimizing problems. It initializes a population of random solutions and searches for optima by updating generations. Each potential solution, called a particle, tracks its best solution and the overall best solution to change its velocity and position in search of better solutions. The algorithm involves initializing particles with random positions and velocities, then updating velocities and positions iteratively based on the particles' local best solution and the global best solution until termination criteria are met. PSO has advantages of being simple, quick, and effective at locating good solutions.
At the beginning, the number of elements in a set of numbers to be stored in a computer system used to be not so large or having a wide range. Then, using a
simple table T [0, 1, ..., m − 1]called, direct-address table, could be used to store those numbers. As the situation became more and more complex, and a new idea came to be:
Definition
An associative array, map, symbol table, or dictionary is an abstract data type composed of a collection of tuples {(key, value)}
This can bee seen in the example of dictionaries in any spoken language. The problem became more complex when the range of the possible values for the
keys at the tuples became unbounded. Therefore a new type of data structure is needed to avoid the sparsity problem in the data, the hash table.
This document discusses real-time operating systems (RTOS). It begins by defining an RTOS and distinguishing it from traditional operating systems by its ability to respond to external events in a timely manner. It describes the different types of RTOS based on timing constraints. It then covers key RTOS concepts like preemptive priority scheduling, multitasking, inter-task communication, priority inheritance, and memory management. The document also discusses the Nucleus RTOS and whether RTOS will replace traditional operating systems.
Knapsack problem ==>>
Given some items, pack the knapsack to get
the maximum total value. Each item has some
weight and some value. Total weight that we can
carry is no more than some fixed number W.
So we must consider weights of items as well as
their values.
Artificial Intelligence - Problems, State Space Search & Heuristic Search Techniques - Defining the Problems as a State Space Search
Production Systems
Production Characteristics
Production System Characteristics
Issues in the design of Search Programs
Particle swarm optimization (PSO) is an evolutionary computation technique for optimizing problems. It initializes a population of random solutions and searches for optima by updating generations. Each potential solution, called a particle, tracks its best solution and the overall best solution to change its velocity and position in search of better solutions. The algorithm involves initializing particles with random positions and velocities, then updating velocities and positions iteratively based on the particles' local best solution and the global best solution until termination criteria are met. PSO has advantages of being simple, quick, and effective at locating good solutions.
At the beginning, the number of elements in a set of numbers to be stored in a computer system used to be not so large or having a wide range. Then, using a
simple table T [0, 1, ..., m − 1]called, direct-address table, could be used to store those numbers. As the situation became more and more complex, and a new idea came to be:
Definition
An associative array, map, symbol table, or dictionary is an abstract data type composed of a collection of tuples {(key, value)}
This can bee seen in the example of dictionaries in any spoken language. The problem became more complex when the range of the possible values for the
keys at the tuples became unbounded. Therefore a new type of data structure is needed to avoid the sparsity problem in the data, the hash table.
This document discusses real-time operating systems (RTOS). It begins by defining an RTOS and distinguishing it from traditional operating systems by its ability to respond to external events in a timely manner. It describes the different types of RTOS based on timing constraints. It then covers key RTOS concepts like preemptive priority scheduling, multitasking, inter-task communication, priority inheritance, and memory management. The document also discusses the Nucleus RTOS and whether RTOS will replace traditional operating systems.
Knapsack problem ==>>
Given some items, pack the knapsack to get
the maximum total value. Each item has some
weight and some value. Total weight that we can
carry is no more than some fixed number W.
So we must consider weights of items as well as
their values.
This presentation talks about Real Time Operating Systems (RTOS). Starting with fundamental concepts of OS, this presentation deep dives into Embedded, Real Time and related aspects of an OS. Appropriate examples are referred with Linux as a case-study. Ideal for a beginner to build understanding about RTOS.
The document discusses problem-solving agents and uninformed search strategies. It introduces problem-solving agents as goal-based agents that try to find sequences of actions that lead to desirable goal states. It then discusses formulating problems by defining the initial state, actions, goal test, and cost function. Several examples of problems are provided, like the Romania tour problem. Uninformed search strategies like breadth-first search, uniform-cost search, and depth-first search are then introduced as strategies that use only the problem definition, not heuristics. Breadth-first search expands nodes in order of shallowest depth first, while depth-first search expands the deepest node in the frontier first.
The document discusses various backtracking techniques including bounding functions, promising functions, and pruning to avoid exploring unnecessary paths. It provides examples of problems that can be solved using backtracking including n-queens, graph coloring, Hamiltonian circuits, sum-of-subsets, 0-1 knapsack. Search techniques for backtracking problems include depth-first search (DFS), breadth-first search (BFS), and best-first search combined with branch-and-bound pruning.
Kevin Knight, Elaine Rich, B. Nair - Artificial Intelligence (2010, Tata McGr...JayaramB11
This document discusses the history of chocolate production. It details how cocoa beans are harvested from cocoa trees and then fermented, dried, roasted, and ground into chocolate liquor. The liquor is then further processed through conching and tempering to produce chocolate in its familiar solid form.
The document discusses local search algorithms, including gradient descent, the Metropolis algorithm, simulated annealing, and Hopfield neural networks. It provides details on how each algorithm works, such as gradient descent taking steps proportional to the negative gradient of a function to find a local minimum. The algorithms are compared, with some having similarities in their methods, like maximum cut problem and Hopfield neural networks using state flipping algorithms, and Metropolis and gradient descent using simulated annealing. Advantages and disadvantages of local search algorithms are presented.
A recurrent neural network (RNN) is one of the two broad types of artificial neural network, characterized by direction of the flow of information between its layers. In contrast to the uni-directional feedforward neural network, it is a bi-directional artificial neural network, meaning that it allows the output from some nodes to affect subsequent input to the same nodes. Their ability to use internal state (memory) to process arbitrary sequences of inputs makes them applicable to tasks such as unsegmented, connected handwriting recognition[4] or speech recognition. The term "recurrent neural network" is used to refer to the class of networks with an infinite impulse response, whereas "convolutional neural network" refers to the class of finite impulse response. Both classes of networks exhibit temporal dynamic behavior. A finite impulse recurrent network is a directed acyclic graph that can be unrolled and replaced with a strictly feedforward neural network, while an infinite impulse recurrent network is a directed cyclic graph that can not be unrolled.
Additional stored states and the storage under direct control by the network can be added to both infinite-impulse and finite-impulse networks. The storage can also be replaced by another network or graph if that incorporates time delays or has feedback loops. Such controlled states are referred to as gated state or gated memory, and are part of long short-term memory networks (LSTMs) and gated recurrent units. This is also called Feedforward Neural Network (FNN). Recurrent neural networks are theoretically Turing complete and can run arbitrary programs to process arbitrary sequences of inputs.
A process represents a program in execution and goes through various states like new, ready, run, and terminate. It has a minimum of 4 states. A thread is a path of execution within a process and provides parallelism by dividing a process into multiple threads. Threads share resources like memory and code with peer threads but have their own program counters and stacks. Threads provide improved performance over processes as they have lower overhead and faster context switching.
Managing the memory hierarchy
Static and dynamic memory allocations
Memory allocation to a process
Reuse of memory
Contiguous and non contiguous memory allocation
Paging
Segmentation
Segmentation with paging
The document discusses classical AI planning and different planning approaches. It introduces state-space planning which searches for a sequence of state transformations, and plan-space planning which searches for a plan satisfying certain conditions. It also discusses hierarchical planning which decomposes tasks into simpler subtasks, and universal classical planning which uses different refinement techniques including state-space and plan-space refinements. Classical planning makes simplifying assumptions but its principles can still be applied to games with some workarounds.
The document discusses greedy algorithms for optimization problems. It provides examples of greedy algorithms for counting money, interval scheduling, and minimizing lateness. For interval scheduling, the greedy algorithm of scheduling jobs in order of earliest finish time is proven to be optimal. For minimizing lateness, the greedy algorithm of scheduling jobs in order of earliest deadline is shown to produce a schedule with no idle time and no inversions.
An adaptative nature inspired algorithm explained, concretely implemented, and applied to routing protocols in wired and wireless networks. The document discusses how ant colony optimization algorithms can be applied to routing by simulating how ants leave pheromone trails to find the shortest path between their nest and food sources. It provides examples of how ant colony algorithms have been implemented in routing protocols like ABC for wired networks, AntNet for MANETs, and ARA and AntHocNet for wireless ad hoc networks. Evaluation results show these ant-inspired routing protocols can find paths more efficiently than traditional routing protocols like OSPF and perform better than protocols like AODV for packet delivery in mobile wireless networks.
This document discusses particle swarm optimization (PSO), which is an optimization technique inspired by swarm intelligence and the social behavior of bird flocking or fish schooling. PSO uses a population of candidate solutions called particles that fly through the problem hyperspace, with each particle adjusting its position based on its own experience and the experience of neighboring particles. The algorithm iteratively improves the particles' positions to locate the best solution based on fitness evaluations.
The document discusses different types of adversarial search algorithms. It describes min-max algorithm and alpha-beta pruning. Min-max algorithm searches through the game tree recursively to find the optimal move assuming the opponent plays optimally. Alpha-beta pruning improves on min-max by pruning parts of the tree that cannot contain better moves based on the alpha and beta values being passed down the tree.
Best-first search is a heuristic search algorithm that expands the most promising node first. It uses an evaluation function f(n) that estimates the cost to reach the goal from each node n. Nodes are ordered in the fringe by increasing f(n). A* search is a special case of best-first search that uses an admissible heuristic function h(n) and is guaranteed to find the optimal solution.
This document discusses various approaches to real-time scheduling such as clock-driven, weighted round-robin, and priority-driven approaches. It also covers topics like dynamic versus static systems, effective release times and deadlines, optimality and non-optimality of algorithms, challenges in validating timing constraints, and differences between offline and online scheduling.
This document contains the student's responses to 7 questions about operating system concepts related to process synchronization and concurrency control. The questions cover topics like the critical section problem, Peterson's solution, semaphores, monitors, and classical synchronization problems like the bounded buffer problem and readers-writers problem. The student provides definitions and explanations of the key concepts and how they can be implemented using constructs like mutexes, condition variables, load-locked and store-conditional instructions. Specific examples of how synchronization applies in areas like process management and bounded buffers are also discussed.
Report about Software Architecture for Robotics, for class of Introduction to Robotics of Prof. Sukhan Lee, of department of computer engineering of Sungkyunkwan University.
Student: Lorran Pegoretti.
Suwon, South Korea, December 2013
CNIT 127 Ch 5: Introduction to heap overflowsSam Bowne
Slides for a college course at City College San Francisco. Based on "The Shellcoder's Handbook: Discovering and Exploiting Security Holes ", by Chris Anley, John Heasman, Felix Lindner, Gerardo Richarte; ASIN: B004P5O38Q.
Instructor: Sam Bowne
Class website: https://samsclass.info/127/127_S17.shtml
This document provides an introduction to fuzzy logic and fuzzy systems. It discusses classical set theory versus fuzzy set theory and membership functions. Types of fuzzy membership functions like triangular, trapezoidal, and Gaussian are shown. The key components of a fuzzy logic controller including fuzzification, fuzzy inference system, and defuzzification are described. Several defuzzification methods such as mean of maxima, centroid, and approximate centroid are explained. Examples of fuzzy applications in areas like washing machines and autonomous vehicles are presented. The document also discusses building fuzzy systems using MATLAB/Simulink and at the command line. Finally, it briefly introduces PID fuzzy controllers.
The document provides an overview of fuzzy logic and approximate reasoning. It discusses fuzzy sets and membership functions, including different types of membership functions like triangular, trapezoidal, and Gaussian. It also covers fuzzy set operations like union, intersection, and complement. T-norm operators for fuzzy intersection are defined. The document serves as an introduction to key concepts in fuzzy logic.
This presentation talks about Real Time Operating Systems (RTOS). Starting with fundamental concepts of OS, this presentation deep dives into Embedded, Real Time and related aspects of an OS. Appropriate examples are referred with Linux as a case-study. Ideal for a beginner to build understanding about RTOS.
The document discusses problem-solving agents and uninformed search strategies. It introduces problem-solving agents as goal-based agents that try to find sequences of actions that lead to desirable goal states. It then discusses formulating problems by defining the initial state, actions, goal test, and cost function. Several examples of problems are provided, like the Romania tour problem. Uninformed search strategies like breadth-first search, uniform-cost search, and depth-first search are then introduced as strategies that use only the problem definition, not heuristics. Breadth-first search expands nodes in order of shallowest depth first, while depth-first search expands the deepest node in the frontier first.
The document discusses various backtracking techniques including bounding functions, promising functions, and pruning to avoid exploring unnecessary paths. It provides examples of problems that can be solved using backtracking including n-queens, graph coloring, Hamiltonian circuits, sum-of-subsets, 0-1 knapsack. Search techniques for backtracking problems include depth-first search (DFS), breadth-first search (BFS), and best-first search combined with branch-and-bound pruning.
Kevin Knight, Elaine Rich, B. Nair - Artificial Intelligence (2010, Tata McGr...JayaramB11
This document discusses the history of chocolate production. It details how cocoa beans are harvested from cocoa trees and then fermented, dried, roasted, and ground into chocolate liquor. The liquor is then further processed through conching and tempering to produce chocolate in its familiar solid form.
The document discusses local search algorithms, including gradient descent, the Metropolis algorithm, simulated annealing, and Hopfield neural networks. It provides details on how each algorithm works, such as gradient descent taking steps proportional to the negative gradient of a function to find a local minimum. The algorithms are compared, with some having similarities in their methods, like maximum cut problem and Hopfield neural networks using state flipping algorithms, and Metropolis and gradient descent using simulated annealing. Advantages and disadvantages of local search algorithms are presented.
A recurrent neural network (RNN) is one of the two broad types of artificial neural network, characterized by direction of the flow of information between its layers. In contrast to the uni-directional feedforward neural network, it is a bi-directional artificial neural network, meaning that it allows the output from some nodes to affect subsequent input to the same nodes. Their ability to use internal state (memory) to process arbitrary sequences of inputs makes them applicable to tasks such as unsegmented, connected handwriting recognition[4] or speech recognition. The term "recurrent neural network" is used to refer to the class of networks with an infinite impulse response, whereas "convolutional neural network" refers to the class of finite impulse response. Both classes of networks exhibit temporal dynamic behavior. A finite impulse recurrent network is a directed acyclic graph that can be unrolled and replaced with a strictly feedforward neural network, while an infinite impulse recurrent network is a directed cyclic graph that can not be unrolled.
Additional stored states and the storage under direct control by the network can be added to both infinite-impulse and finite-impulse networks. The storage can also be replaced by another network or graph if that incorporates time delays or has feedback loops. Such controlled states are referred to as gated state or gated memory, and are part of long short-term memory networks (LSTMs) and gated recurrent units. This is also called Feedforward Neural Network (FNN). Recurrent neural networks are theoretically Turing complete and can run arbitrary programs to process arbitrary sequences of inputs.
A process represents a program in execution and goes through various states like new, ready, run, and terminate. It has a minimum of 4 states. A thread is a path of execution within a process and provides parallelism by dividing a process into multiple threads. Threads share resources like memory and code with peer threads but have their own program counters and stacks. Threads provide improved performance over processes as they have lower overhead and faster context switching.
Managing the memory hierarchy
Static and dynamic memory allocations
Memory allocation to a process
Reuse of memory
Contiguous and non contiguous memory allocation
Paging
Segmentation
Segmentation with paging
The document discusses classical AI planning and different planning approaches. It introduces state-space planning which searches for a sequence of state transformations, and plan-space planning which searches for a plan satisfying certain conditions. It also discusses hierarchical planning which decomposes tasks into simpler subtasks, and universal classical planning which uses different refinement techniques including state-space and plan-space refinements. Classical planning makes simplifying assumptions but its principles can still be applied to games with some workarounds.
The document discusses greedy algorithms for optimization problems. It provides examples of greedy algorithms for counting money, interval scheduling, and minimizing lateness. For interval scheduling, the greedy algorithm of scheduling jobs in order of earliest finish time is proven to be optimal. For minimizing lateness, the greedy algorithm of scheduling jobs in order of earliest deadline is shown to produce a schedule with no idle time and no inversions.
An adaptative nature inspired algorithm explained, concretely implemented, and applied to routing protocols in wired and wireless networks. The document discusses how ant colony optimization algorithms can be applied to routing by simulating how ants leave pheromone trails to find the shortest path between their nest and food sources. It provides examples of how ant colony algorithms have been implemented in routing protocols like ABC for wired networks, AntNet for MANETs, and ARA and AntHocNet for wireless ad hoc networks. Evaluation results show these ant-inspired routing protocols can find paths more efficiently than traditional routing protocols like OSPF and perform better than protocols like AODV for packet delivery in mobile wireless networks.
This document discusses particle swarm optimization (PSO), which is an optimization technique inspired by swarm intelligence and the social behavior of bird flocking or fish schooling. PSO uses a population of candidate solutions called particles that fly through the problem hyperspace, with each particle adjusting its position based on its own experience and the experience of neighboring particles. The algorithm iteratively improves the particles' positions to locate the best solution based on fitness evaluations.
The document discusses different types of adversarial search algorithms. It describes min-max algorithm and alpha-beta pruning. Min-max algorithm searches through the game tree recursively to find the optimal move assuming the opponent plays optimally. Alpha-beta pruning improves on min-max by pruning parts of the tree that cannot contain better moves based on the alpha and beta values being passed down the tree.
Best-first search is a heuristic search algorithm that expands the most promising node first. It uses an evaluation function f(n) that estimates the cost to reach the goal from each node n. Nodes are ordered in the fringe by increasing f(n). A* search is a special case of best-first search that uses an admissible heuristic function h(n) and is guaranteed to find the optimal solution.
This document discusses various approaches to real-time scheduling such as clock-driven, weighted round-robin, and priority-driven approaches. It also covers topics like dynamic versus static systems, effective release times and deadlines, optimality and non-optimality of algorithms, challenges in validating timing constraints, and differences between offline and online scheduling.
This document contains the student's responses to 7 questions about operating system concepts related to process synchronization and concurrency control. The questions cover topics like the critical section problem, Peterson's solution, semaphores, monitors, and classical synchronization problems like the bounded buffer problem and readers-writers problem. The student provides definitions and explanations of the key concepts and how they can be implemented using constructs like mutexes, condition variables, load-locked and store-conditional instructions. Specific examples of how synchronization applies in areas like process management and bounded buffers are also discussed.
Report about Software Architecture for Robotics, for class of Introduction to Robotics of Prof. Sukhan Lee, of department of computer engineering of Sungkyunkwan University.
Student: Lorran Pegoretti.
Suwon, South Korea, December 2013
CNIT 127 Ch 5: Introduction to heap overflowsSam Bowne
Slides for a college course at City College San Francisco. Based on "The Shellcoder's Handbook: Discovering and Exploiting Security Holes ", by Chris Anley, John Heasman, Felix Lindner, Gerardo Richarte; ASIN: B004P5O38Q.
Instructor: Sam Bowne
Class website: https://samsclass.info/127/127_S17.shtml
This document provides an introduction to fuzzy logic and fuzzy systems. It discusses classical set theory versus fuzzy set theory and membership functions. Types of fuzzy membership functions like triangular, trapezoidal, and Gaussian are shown. The key components of a fuzzy logic controller including fuzzification, fuzzy inference system, and defuzzification are described. Several defuzzification methods such as mean of maxima, centroid, and approximate centroid are explained. Examples of fuzzy applications in areas like washing machines and autonomous vehicles are presented. The document also discusses building fuzzy systems using MATLAB/Simulink and at the command line. Finally, it briefly introduces PID fuzzy controllers.
The document provides an overview of fuzzy logic and approximate reasoning. It discusses fuzzy sets and membership functions, including different types of membership functions like triangular, trapezoidal, and Gaussian. It also covers fuzzy set operations like union, intersection, and complement. T-norm operators for fuzzy intersection are defined. The document serves as an introduction to key concepts in fuzzy logic.
In the last decades, a new model of computation based on quantum mechanics has gained attention in the computer science community. We give an introduction to this model starting from the basics, with no prerequisites. Then, with the help of some simple examples, we see why quantum computers outperform standard ones in certain tasks. We then move to the topic of quantum entanglement and show how sharing quantum information can create a strong provable correlation among distant parties. With this basic understanding of quantum computation and quantum entanglement, we can already illustrate two interesting cryptographic protocols: quantum key distribution and position verification. Both perform classically impossible tasks: the first allows to detect an intruder intercepting a secret communication, while the second allows certifying somebody's GPS location.
This document provides an overview of fuzzy inference systems and fuzzy logic modeling. It discusses Mamdani and Sugeno fuzzy models. Mamdani models use fuzzy rules with fuzzy outputs while Sugeno models have fuzzy antecedents and crisp polynomial consequent parts. The document also describes fuzzy rule bases, fuzzification, inference engines, and defuzzification methods like centroid of area and bisector of area. Examples of Sugeno models with one and two inputs are presented.
This document provides an overview of fuzzy logic, including its origins, key concepts, and applications. It discusses how fuzzy logic allows for approximate reasoning and decision making under uncertainty by using linguistic variables and fuzzy set theory. Membership functions are used to characterize fuzzy sets and assign degrees of truth between 0 and 1 rather than binary true/false values. Common fuzzy logic operations like intersection, union, and complement are also covered. Finally, some examples of fuzzy logic control systems are presented, including how they are designed using fuzzy rule bases and inference methods like Mamdani and Sugeno.
The document provides an overview of fuzzy logic and fuzzy sets. It discusses how fuzzy logic can handle imprecise data unlike classical binary sets. Membership functions assign degrees of membership values between 0 and 1. Fuzzy logic systems use if-then rules and linguistic variables. An example shows how fuzzy logic is used to estimate project risk levels based on funding and staffing levels. Fuzzy logic has been applied in various domains due to its ability to model human reasoning.
Brief Introduction About Topological Interference Management (TIM)Pei-Che Chang
This document discusses topological interference management (TIM) techniques for interference channels. TIM exploits interference alignment principles under realistic channel state information assumptions. The key ideas are:
- Focus on canceling strong interference links based on knowledge of the interference pattern
- There is a connection between TIM and the index coding problem
- The goal of TIM is to maximize degrees of freedom (DoF) based on network topology information
- Examples show how transmitting signals over multiple channel uses and exploiting the interference pattern can achieve different DoF values through interference alignment
Waveguides and its Types Field view and structuresAvishishtKumar
The document discusses rectangular waveguides, including their modes of propagation, fields inside, cutoff frequency, and propagation constant. It explains the transverse electric (TE) and transverse magnetic (TM) modes, showing the electric and magnetic field configurations and equations for each. Examples of electric and magnetic field views are also shown for different TE modes inside the rectangular waveguide.
Fuzzy logic provides a method to formalize reasoning with vague terms by allowing membership functions and degrees of truth rather than binary true/false values. It can be used to model problems involving linguistic variables like "poor", "good", and "excellent".
The document discusses a tipping example to demonstrate fuzzy logic. It defines fuzzy rules for tip amounts based on the quality of service and food. For example, one rule is that if service is poor or food is rancid, the tip should be cheap. Membership functions are then used to evaluate the fuzzy rules and determine appropriate tip amounts based on varying degrees of service and food quality.
Fuzzy logic provides a more intuitive way to model problems involving vague
This document discusses the process of backpropagation in neural networks. It begins with an example of forward propagation through a neural network with an input, hidden and output layer. It then introduces backpropagation, which uses the calculation of errors at the output to calculate gradients and update weights in order to minimize the overall error. The key steps are outlined, including calculating the error derivatives, weight updates proportional to the local gradient, and backpropagating error signals from the output through the hidden layers. Formulas for calculating each step of backpropagation are provided.
The document discusses fuzzy logic and artificial neural networks. It provides an overview of fuzzy logic, including fuzzy sets, membership functions, fuzzy linguistic variables, fuzzy rules and fuzzy control. It also covers artificial neural networks, including the biological inspiration from the human brain, basic neuron models, multi-layer feedforward networks, training algorithms like gradient descent, and examples of neural networks solving problems like XOR classification. Hardware implementations on systems like DSpace and Opal RT are also briefly mentioned.
This document discusses various numerical methods for solving systems of ordinary differential equations (ODEs), including:
- Explicit methods like Euler's method can be directly applied to systems of ODEs but may require a very small time-step for stability. Implicit methods require solving a nonlinear system at each step.
- Predictor-corrector methods like Heun's method or Adams-Bashforth/Adams-Moulton methods combine explicit and implicit steps to gain accuracy while maintaining stability.
- Higher order ODEs can be converted to a system of first order ODEs to apply the same methods, with initial value problems (IVPs) readily solved this way but boundary value problems (
Clara_de_Paiva_Master_thesis_presentation_June2016Clara de Paiva
1. The document discusses hierarchical associative memories and sparse codes in neural networks. It describes how hierarchical memories can be organized into multiple correlation matrices to improve network performance.
2. Training involves applying Hebbian learning rules at each layer to update the correlation matrices. Retrieval uses a thresholded decision based on the weighted input to determine neuron outputs.
3. A hierarchical associative memory is proposed with correlation matrices of decreasing size at each layer. Learning proceeds from the top layer down, compressing outputs from the previous layer using an aggregation window to reduce computational requirements.
Most modern devices are made from billions of on /off switches called transistors
We will build a processor in this course!
Transistors made from semiconductor materials:
MOSFET – Metal Oxide Semiconductor Field Effect Transistor
NMOS, PMOS – Negative MOS and Positive MOS
CMOS – Complimentary MOS made from PMOS and NMOS transistors
Transistors used to make logic gates and logic circuits
We can now implement any logic circuit
Can do it efficiently, using Karnaugh maps to find the minimal terms required
Can use either NAND or NOR gates to implement the logic circuit
Can use P- and N-transistors to implement NAND or NOR gates
This document discusses optimization techniques and provides examples to illustrate key concepts in optimization problems. It defines optimization as finding extreme states like minimum/maximum and discusses how it is applied in various fields. It then covers basic definitions like design variables, objective functions, constraints, convexity, local vs global optima. Examples are given to show unconstrained vs constrained problems and illustrate active, inactive and violated constraints. Optimization techniques largely depend on calculus concepts like derivatives and hessian matrix.
Raimundo Soto - Catholic University of Chile
ERF Training on Advanced Panel Data Techniques Applied to Economic Modelling
29 -31 October, 2018
Cairo, Egypt
The document provides an overview of artificial neural networks (ANNs) and the perceptron learning algorithm. It discusses how biological neurons inspire ANNs and how a basic perceptron works using a simple example with inputs, weights, and outputs. The perceptron learning algorithm is then explained, which updates weights based on whether the perceptron's prediction was correct or incorrect on each training example. Finally, the document introduces multilayer perceptrons which can solve non-linearly separable problems by connecting multiple perceptron layers together through a process called backpropagation.
Recursive State Estimation AI for Robotics.pdff20220630
1) Recursive state estimation uses probabilistic methods like Bayes filters to estimate states of a dynamic system from sensor measurements over time. Bayes filters involve prediction of state based on motion model and correction of prediction based on sensor observations using Bayes' rule.
2) An example of applying a Bayes filter to estimate the state of a door being open or closed is given. The robot's belief is updated as it takes actions like pushing the door and receives sensor feedback.
3) Key concepts discussed include belief distributions, probabilistic generative models relating state transitions and measurements, and the Bayes filter algorithm involving prediction and correction steps.
This document discusses deconvolution methods and their applications in analyzing gamma-ray spectra. It provides background on several deconvolution algorithms, including Tikhonov-Miller regularization, Van Cittert, Janson, Gold, Richardson-Lucy, and Muller algorithms. These algorithms aim to improve spectral resolution by mathematically removing instrument smearing effects. They are based on solving systems of linear equations using direct or iterative methods, with regularization techniques to produce stable solutions. Examples are given to illustrate the sensitivity of non-regularized solutions to noise and the need for regularization.
Similar to An Introduction to Fuzzy Logic and Neural Networks (20)
Understanding Inductive Bias in Machine LearningSUTEJAS
This presentation explores the concept of inductive bias in machine learning. It explains how algorithms come with built-in assumptions and preferences that guide the learning process. You'll learn about the different types of inductive bias and how they can impact the performance and generalizability of machine learning models.
The presentation also covers the positive and negative aspects of inductive bias, along with strategies for mitigating potential drawbacks. We'll explore examples of how bias manifests in algorithms like neural networks and decision trees.
By understanding inductive bias, you can gain valuable insights into how machine learning models work and make informed decisions when building and deploying them.
Hierarchical Digital Twin of a Naval Power SystemKerry Sado
A hierarchical digital twin of a Naval DC power system has been developed and experimentally verified. Similar to other state-of-the-art digital twins, this technology creates a digital replica of the physical system executed in real-time or faster, which can modify hardware controls. However, its advantage stems from distributing computational efforts by utilizing a hierarchical structure composed of lower-level digital twin blocks and a higher-level system digital twin. Each digital twin block is associated with a physical subsystem of the hardware and communicates with a singular system digital twin, which creates a system-level response. By extracting information from each level of the hierarchy, power system controls of the hardware were reconfigured autonomously. This hierarchical digital twin development offers several advantages over other digital twins, particularly in the field of naval power systems. The hierarchical structure allows for greater computational efficiency and scalability while the ability to autonomously reconfigure hardware controls offers increased flexibility and responsiveness. The hierarchical decomposition and models utilized were well aligned with the physical twin, as indicated by the maximum deviations between the developed digital twin hierarchy and the hardware.
Low power architecture of logic gates using adiabatic techniquesnooriasukmaningtyas
The growing significance of portable systems to limit power consumption in ultra-large-scale-integration chips of very high density, has recently led to rapid and inventive progresses in low-power design. The most effective technique is adiabatic logic circuit design in energy-efficient hardware. This paper presents two adiabatic approaches for the design of low power circuits, modified positive feedback adiabatic logic (modified PFAL) and the other is direct current diode based positive feedback adiabatic logic (DC-DB PFAL). Logic gates are the preliminary components in any digital circuit design. By improving the performance of basic gates, one can improvise the whole system performance. In this paper proposed circuit design of the low power architecture of OR/NOR, AND/NAND, and XOR/XNOR gates are presented using the said approaches and their results are analyzed for powerdissipation, delay, power-delay-product and rise time and compared with the other adiabatic techniques along with the conventional complementary metal oxide semiconductor (CMOS) designs reported in the literature. It has been found that the designs with DC-DB PFAL technique outperform with the percentage improvement of 65% for NOR gate and 7% for NAND gate and 34% for XNOR gate over the modified PFAL techniques at 10 MHz respectively.
Using recycled concrete aggregates (RCA) for pavements is crucial to achieving sustainability. Implementing RCA for new pavement can minimize carbon footprint, conserve natural resources, reduce harmful emissions, and lower life cycle costs. Compared to natural aggregate (NA), RCA pavement has fewer comprehensive studies and sustainability assessments.
Harnessing WebAssembly for Real-time Stateless Streaming PipelinesChristina Lin
Traditionally, dealing with real-time data pipelines has involved significant overhead, even for straightforward tasks like data transformation or masking. However, in this talk, we’ll venture into the dynamic realm of WebAssembly (WASM) and discover how it can revolutionize the creation of stateless streaming pipelines within a Kafka (Redpanda) broker. These pipelines are adept at managing low-latency, high-data-volume scenarios.
An Introduction to Fuzzy Logic and Neural Networks
1. An Introduction to
Deep Learning
Mehrnaz Faraz
Faculty of Electrical Engineering
K. N. Toosi University of Technology
1
In the name of God
Milad Abbasi
Faculty of Electrical Engineering
Sharif University of Technology
3. Introduction to Fuzzy
• Fuzzy: “Difficult to perceive; indistinct or vague”
– Simplicity and flexibility
– Can handle problems with imprecise data
– More readily customizable in natural language terms
3
4. Introduction to Fuzzy
4
Slowest, 𝐴1 Slow, 𝐴2 Fast, 𝐴3 Fastest, 𝐴4
Subset
0
1
Velocity
𝜇 𝐴
Membership Function
𝜇 𝐴 𝑖
𝑥 : 𝑥 → 0, 1 , i = 1, … , 4
Slowest FastestSlow Fast
5. Introduction to Fuzzy
• Example: Automatic Braking System
5
Close?
Is car close? 0.2 (Not very close)
Brakes: 0.2 (Slight pressure)
Is car close? 0.8 (Pretty close)
Brakes: 0.8 (Fairly heavy pressure)
6. Introduction to Fuzzy
6
• Well-known Membership Functions:
trimf trapmf
gaussmf sigmf
a b c a b c d
sig
c
7. Introduction to Fuzzy
7
• Linguistic variables:
• Weather is quite cold.
• Height is almost tall.
• Speed is very high.
• Weather, Height and Speed are linguistic variables.
• Cold, Tall and high are linguistic value.
8. Introduction to Fuzzy
• Operations with fuzzy sets:
– Complement operation:
– Fuzzy union operation or fuzzy OR:
– Fuzzy intersection operation or fuzzy AND:
8
)(1)( xx AA
A( ) s[ ( ), ( )]A B Bx x x
A( ) t [ ( ), ( )]A B Bx x x
9. 1
1 2 30 4
1
1 2 30 4
1 2 30 4
1
( )A
x
( )B
x
( )A B
x
Introduction to Fuzzy
• Fuzzy union operation (s-norm):
9
Max is a s-norm operator
x
x
x
10. 1
1 2 30 4
1
1 2 30 4
1 2 30 4
1
( )A
x
( )B
x
,
( )A B
x
Introduction to Fuzzy
10
• Fuzzy intersection operation (t-norm):
Min is a t-norm operator
Product is another t-norm operator
(Mamdani Implication)
x
x
x
11. Introduction to Fuzzy
• Other complement operations:
– Sugeno Class:
– Yager Class:
11
1
1
a
c a
a
1
1 w w
wc a a
Aa x
Where:
12. Introduction to Fuzzy
• Other union operation:
– Yager class:
– Drastic sum:
– Algebraic sum:
12
1
, min 1, w w w
ws a b a b
0,w
, 0
, , 0
1 . .
ds
a b
s a b b a
o w
,ass a b a b ab
Aa x Bb xWhere:
13. Introduction to Fuzzy
• Other intersection operation:
– Yager class:
– Drastic product:
– Algebraic product:
13
1
, 1 min 1, 1 1
w w w
wt a b a b
, 1
, , 1
0 . .
dp
a b
t a b b a
o w
,apt a b ab
14. Introduction to Fuzzy
• Example: Assume that we want to evaluate the health of a
person based on his height and weight.
• The input variables are the crisp numbers of the person’s
height and weight.
• Output is percentage of healthiness.
14
Input Output
Fazzifier Defazzifier
Rule Base
Data Base
Inference
Engine
Crisp Number Crisp Number
15. Introduction to Fuzzy
• Step 1: Fuzzification
15
SlimVery Slim Medium Heavy Very Heavy
50 Kg 75 Kg 100 Kg 125 Kg
Weight
𝜇
1
Very Short Short Medium Tall Very Tall
Height
140 cm 160 cm 180 cm 200 cm
𝜇
1
Input Membership Functions:
16. Introduction to Fuzzy
• Step 2: Rules
• Rules reflect experts decisions
• Rules can be redundant
• Rules can be adjusted to match desired
• Rules are tabulated as fuzzy words
• If x is A then y is B
• if 𝑥1is 𝐴1 and/or 𝑥2 is 𝐴2 … and/or 𝑥 𝑛 is 𝐴 𝑛 then y is B
16
17. Introduction to Fuzzy
• Implications:
– Mamdani implication:
17
1 2
1 2
FP FP
IF FP THEN FP
1 2
, min ,QMM FP FPx y x y
1 2
, ,QMP FP FPx y prod x y
18. Introduction to Fuzzy
– Godel implication:
– Zadeh implication:
18
1 2
2
1 ,
. .
FP FP
QG
FP
x y
y o w
1 2 1
, max min , , 1QZ FP FP FPx y x y x
19. Introduction to Fuzzy
• Inference Rules:
– Modus Ponens:
– Modus Tollens:
– Hypothetical Syllogism:
19
X is A
Y is B
If X is A then Y is B
Y is not B
X is not A
If X is A then Y is B
If X is A then Y is B
If X is A then Z is C
If Y is B then Z is C
20. Introduction to Fuzzy
• Rules are tabulated as fuzzy words
– Healthy (H)
– Somewhat healthy (SH)
– Less Healthy (LH)
– Unhealthy (U)
– Rule function f = {U, LH, SH, H}
20
U LH SH H
0.2 0.4 0.6 0.8 1
f
1
Decision
Output Membership Function:
21. Introduction to Fuzzy
21
Very
Slim
slim Medium Heavy
Very
Heavy
Very Short H SH LH U U
Short SH H SH LH U
Medium LH H H LH U
Tall U SH H SH U
Very Tall U LH H SH LH
WeightHeight
• Fuzzy Rule Table:
22. Introduction to Fuzzy
• Step 3: Calculation
• Assume that height = 187 cm and weight = 49 kg
22
SlimVery Slim Medium Heavy Very Heavy
50 Kg 75 Kg 100 Kg 125 Kg
Weight
𝜇
1
Very Short Short Medium Tall Very Tall
Height
140 cm 160 cm 180 cm 200 cm
𝜇
1
0.3
0.7
0.2
0.8
23. Introduction to Fuzzy
23
0.7 0.3 Medium Heavy
Very
Heavy
Very Short H SH LH U U
Short SH H SH LH U
0.8 LH H H LH U
0.2 U SH H SH U
Very Tall U LH H SH LH
WeightHeight
• Fuzzy Rule Table:
24. Introduction to Fuzzy
24
0.7 0.3 Medium Heavy
Very
Heavy
Very
Short
H SH LH U U
Short SH H SH LH U
0.8 0.7 0.3 H LH U
0.2 0.2 0.2 H SH U
Very
Tall
U LH H SH LH
Weight
Height
0.7, 0.3, 0, 0, 0Weight VS S M H VH
0, 0, 0.8, 0.2, 0Height VS S M T VT
25. Introduction to Fuzzy
• Scaled Fuzzified Decision:
25
, , , 0.2, 0.7, 0.2, 0.3f U LH SH H
U
LH
SH H
0.2 0.4 0.6 0.8 1
f
𝟎. 𝟕
𝟎. 𝟑
𝟎. 𝟐
Decision
26. Introduction to Fuzzy
• Defuzzification Methods:
– Centroid:
– Bisector:
26
0
0
( ) ( )
x
A Ax
x dx x dx
0
( )
( )
i A i
A i
x x
x
x
27. Introduction to Fuzzy
– Middle, Smallest, and Largest of Maximum:
27
( )A
x
x
SOM LOMMOM
• Defuzzification Methods:
28. Introduction to Fuzzy
• Centroid Method:
28
*
l l
l
y w
y
w
0.2 0.2 0.7 0.4 0.2 0.6 0.3 0.8
0.4857
0.2 0.7 0.2 0.3
FD
FD= Final Decision
D: Decision
U
LH
SH H
0.2 0.4 0.6 0.8 1
f
𝟎. 𝟕
𝟎. 𝟑
𝟎. 𝟐
Decision
, 1,...,4
l
y l
lw
29. Introduction to Fuzzy
• Step 4: Final Decision
• Assume that crisp decision index (D) is centroid:
D=0.4857
29
U LH SH H
0.2 0.4 0.6 0.8 1
f
Decision
1
0.75
0.25
0.4857
25% in SH group and 75% in LH group
30. Introduction to Fuzzy
• Fuzzy Extension Principle:
– How far is it from Zanjan to Urmia?
30
Tabriz Shahrekord
Zanjan 0.3 0.9
Shahrekord 1 0
Esfahan 0.95 0.1
x
y
Urmia Ahvaz
Tabriz 0.95 0.1
Shahrekord 0.1 0.9
y
z
( , ) ( , ), ( , )POQ y P Qx z S t x y y z
Scaled Distance Among Cities:
31. Introduction to Fuzzy
• Assume that t-norm is product, and s-norm is max
31
0.3 0.95
( , ), ( , ) 0.3 0.95 0.285P Qprod Zanjan Tabriz Tabriz Urmia
0.9 0.1
( , ), ( , ) 0.9 0.1 0.09P Qprod Zanjan Shahrekord Shahrekord Urmia
max 0.285,0.09 0.285
Zanjan is close to Urmia
32. Introduction to Fuzzy
• TSK Fuzzy System:
32
1 1 0 1 1,..., ...l l l l l l
n n n nIf x is C x is C then y c c x c x
:l
iC Fuzzy sets
:l
ic Constants
l l
l
y w
f x
w
1,2,...,l M
1
l
i
n
l
iC
i
w x
Output:
33. Introduction to Neural Network
33
Input
Weight
Σ f
Neuron
Output
Activation Function
• Neural Network:
34. Introduction to Neural Network
• Multilayer Perceptron:
34
InputSignal
OutputSignal
Input Layer
First
Hidden
Layer
Second
Hidden
Layer
Output Layer⋮
⋮
⋮⋮
Supervised
Learning
Random
Initialization
Deep NN
35. Introduction to Neural Network
• MLP:
– Fully connected Overfitting
– Suitable for:
• Classification prediction problems
• Regression prediction problems
• Tabular datasets
– Contain data in a columnar format, each column (field)
must have a name and may only contain data of one type
– Try MLPs on:
• Image data (e.g. the pixels of an image can be reduced down
to one long row of data and fed into an MLP)
• Text data
• Time series data
• Other types of data
35
Supervised
Learning
36. Introduction to Neural Network
• Feed Forward:
36
InputSignal
Output Signal
Input Layer
First
Hidden
Layer
Second
Hidden
Layer
Output Layer
Te
⋮⋮ ⋮
⋮ ⋮
37. Introduction to Neural Network
• Back Propagate:
37
InputSignal
Output Signal
Input Layer
First
Hidden
Layer
Second
Hidden
Layer
Output Layer
Te
Training
⋮ ⋮ ⋮
⋮ ⋮
40. Introduction to Neural Network
40
𝑥1
𝑥2
𝑥 𝑛0
𝑁𝑒𝑢𝑟𝑜𝑛1
1
𝑁𝑒𝑢𝑟𝑜𝑛2
1
𝑁𝑒𝑢𝑟𝑜𝑛1
2
1
1
Σ
Σ
Σ
𝑤10
1
𝒘 𝟏𝟏
𝟐
𝒘 𝟏𝟎
𝟐
𝑤22
1
𝑤21
1
𝑤20
1
𝑤1𝑛0
1
𝑤11
1
𝑤12
1
𝒘 𝟏𝟐
𝟐
𝑤2𝑛0
1
f
f
f
𝒐 𝟏
𝟐
𝑜2
1
𝑜1
1
𝑛𝑒𝑡1
1
𝒏𝒆𝒕 𝟏
𝟐
𝑛𝑒𝑡2
1
…
…
T
e
Back Propagate
'2 1
2 2
1 1
2 2 2 2
1 1 1 1
1e
f o
E E e o net
w e o net w
⋮
41. Introduction to Neural Network
41
𝑥1
𝑥2
𝑥 𝑛0
𝑁𝑒𝑢𝑟𝑜𝑛1
1
𝑁𝑒𝑢𝑟𝑜𝑛2
1
𝑁𝑒𝑢𝑟𝑜𝑛1
2
1
1
Σ
Σ
Σ
𝑤10
1
𝒘 𝟏𝟏
𝟐
𝒘 𝟏𝟎
𝟐
𝑤22
1
𝑤21
1
𝑤20
1
𝑤1𝑛0
1
𝑤11
1
𝑤12
1
𝒘 𝟏𝟐
𝟐
𝑤2𝑛0
1
f
f
f
𝒐 𝟏
𝟐
𝑜2
1
𝑜1
1
𝑛𝑒𝑡1
1
𝒏𝒆𝒕 𝟏
𝟐
𝑛𝑒𝑡2
1
…
…
T
e
Back Propagate
'2 2 '1
11
2 2 1 1
1 1 1 1
1 2 2 1 1 1
1 1 1 1 1 1
1e xf w f
E E e o net o net
w e o net o net w
⋮
42. Introduction to Neural Network
42
21
2
i
i
E k e
1
E k
w k w k
w k
Gradient Descent
Exercise: Rewrite the back-propagation equations for a
neural network with 2 outputs.
43. Introduction to Neural Network
• Popular Activation Functions:
– Sigmoid (Logistic):
• Sigmoids saturate and tend to vanish gradient
• Exp() is a bit compute expensive
• Sigmoid outputs are not zero-centered
43
1
1 x
x
e
0,1x
44. Introduction to Neural Network
– Tanh:
• Zero centered
• Tanh saturate and tend to vanish gradient
• Tanh() is a bit compute expensive
44
2
2
1
1
x
x
e
f x
e
1,1f x
45. Introduction to Neural Network
– ReLU:
• Rectified Linear Unit
• Does not saturate (in range +)
• Very computationally efficient
• Converges faster than sigmoid/tanh
• Not zero-centered output
• Saturate (in range -)
45
max 0,f x x
0,f x
46. Introduction to Neural Network
– LReLU and PReLU:
• Does not saturate
• Computationally efficient
• Converges much faster
• Zero-centered output
46
max 0.01 ,f x x x
max ,f x x x
LReLU:
PReLU:
47. Introduction to Neural Network
– ELU:
• Exponential Linear Unit
• Zero-centered output
• Closer to zero mean output
• Robustness to noise compared with LReLU
47
0
exp 1 0
x x
f x
x x
48. Introduction to Neural Network
• Properties of Activation Functions:
– Nonlinear
– Continuously differentiable
– Range
– Monotonic
– Smooth
• Bipolar and Unipolar:
– Unipolar Sigmoid
– Bipolar Sigmoid
48
f(net)=
𝟏
𝟏+𝒆−𝒈.𝒏𝒆𝒕
f(net)=
𝟏−𝒆−𝒈.𝒏𝒆𝒕
𝟏+𝒆−𝒈.𝒏𝒆𝒕
49. Introduction to Neural Network
• Flexible Neural Network:
49
.
1
,
1
s
j
s
j
s
j g k
s s
j j net k
f gnet k k
e
1s s s
g s
E k
g k g k
g k
.
.
1
,
1
s
j
s
j
s
j
s
j
g k
s
j g k
net k
s s
j j net k
e
f n g ket k
e
Unipolar Sigmoid:
Bipolar Sigmoid:
Training:
50. Introduction to Neural Network
50
𝑥1
𝑥2
𝑥 𝑛0
𝑁𝑒𝑢𝑟𝑜𝑛1
1
𝑁𝑒𝑢𝑟𝑜𝑛2
1
𝑁𝑒𝑢𝑟𝑜𝑛1
2
1
1
Σ
Σ
Σ
𝑤10
1
𝒘 𝟏𝟏
𝟐
𝒘 𝟏𝟎
𝟐
𝑤22
1
𝑤21
1
𝑤20
1
𝑤1𝑛0
1
𝑤11
1
𝑤12
1
𝒘 𝟏𝟐
𝟐
𝑤2𝑛0
1
f
f
f
𝒐 𝟏
𝟐
𝑜2
1
𝑜1
1
𝑛𝑒𝑡1
1
𝑛𝑒𝑡1
2
𝑛𝑒𝑡2
1
…
…
T
e
Back Propagate
*2
2
1
2 2 2
1
1e
f
E E e o
g e o g
⋮
51. Introduction to Neural Network
51
Unipolar Sigmoid:
Bipolar Sigmoid:
Training:
.
2
,
1
ss
j j
s
js
j a k
s s
j j net k
f
a
net k
k
a k
e
.
.
1 1
,
1
s
j
s
j j
s
j
s
n
kn
a k
s
et k
s s
j j s et k
j
j a
e
f net k
a k e
a k
1s s s
a s
E k
a k a k
a k
• Flexible Neural Network:
1s
jg k
52. Introduction to Neural Network
52
𝑥1
𝑥2
𝑥 𝑛0
𝑁𝑒𝑢𝑟𝑜𝑛1
1
𝑁𝑒𝑢𝑟𝑜𝑛2
1
𝑁𝑒𝑢𝑟𝑜𝑛1
2
1
1
Σ
Σ
Σ
𝑤10
1
𝒘 𝟏𝟏
𝟐
𝒘 𝟏𝟎
𝟐
𝑤22
1
𝑤21
1
𝑤20
1
𝑤1𝑛0
1
𝑤11
1
𝑤12
1
𝒘 𝟏𝟐
𝟐
𝑤2𝑛0
1
f
f
f
𝒐 𝟏
𝟐
𝑜2
1
𝑜1
1
𝑛𝑒𝑡1
1
𝒏𝒆𝒕 𝟏
𝟐
𝑛𝑒𝑡2
1
…
…
T
e
Back Propagate
*1
'2 2
11
2 2 1
1 1 1
1 2 2 1 1
1 1 1
1e ff w
E E e o net o
a e o net o a
⋮
53. Introduction to Neural Network
• Radial Basis Function (RBF):
– Similarity between input signal and prototype
53
⋮
𝑥1
𝑥n
𝑥2
𝑤 y
⋮
Gaussian Activation Function
⋮ ⋮
56. Introduction to Neural Network
• Training of RBF Networks:
–
–
56
𝑐𝑗 𝑘 + 1 = 𝑐𝑗 𝑘 − 𝜂
𝜕𝐸
𝜕𝑐𝑗
𝑘
1
22
1
1
1
j j
j
j
j
j j j
e
w k x k c k
o k
k
oE E e y
k k
c e y o c
𝜎𝑗 𝑘 + 1 = 𝜎𝑗 𝑘 − 𝜂
𝜕𝐸
𝜕𝜎𝑗
𝑘
2
1
3
1
1
1
j
j
j
j
j j j
e
w k net
o k
k
oE E e y
k k
e y o
𝒄𝒋 :
𝝈𝒋 :
58. Introduction to Neural Network
• Feedback Types in Recurrent Neural Network:
– Local (A)
– Inter-layer (B)
– Global (C)
58
X(k) y(k)
A
B
C
59. 𝑤𝑥X(k) y(k)
𝑤 𝑟,1
𝑤 𝑟,2
𝑤 𝑟,𝑛1
Introduction to Neural Network
• Local feedback activation:
59
Feed Forward
1
x r
i
net k w x k w net k i
60. Introduction to Neural Network
• Local output feedback:
60
𝑤𝑥X(k) y(k)
𝑤 𝑟,1
𝑤 𝑟,2
𝑤 𝑟,𝑛1
Feed Forward
0 1
x r
j i
net k w x k j w y k i
y k f net k
61. Introduction to Neural Network
• The Vanishing Gradient Problem:
– Causes small gradients
– Prevents the weights from updating
61
0.29
0.28999
InputSignal
OutputSignal
62. Introduction to Neural Network
• The Exploding Gradient Problem:
– Causes large gradients
– The weights get away from their optimum value
62
0.29
1.872351
InputSignal
OutputSignal
64. Introduction to Neural Network
• Elman Neural Network:
64
Feed Forward
1 1 1
1 1
1 1
1 1 1 1x c
o k f net k
f w k x k w k o k
1
1cx o k
2 2 2 2 1
yo K f net k f w o k
65. Introduction to Neural Network
• Elman Neural Network:
65
Back Propagate
2
2
1
1
2
n
j
j
E k e k
'2
1
2 2
2 2
1
y y
e f
o
E E e o net
w e o net w
1y y
y
E k
w k w k
w k
In the same way for 𝑤 𝑥