Newton.ppt

CSE 245: Computer Aided Circuit
Simulation and Verification
Fall 2004, Nov
Nonlinear Equation

October 21, 2023 2 courtesy Alessandra Nardi UCB
Outline
 Nonlinear problems
 Iterative Methods
 Newton’s Method
 Derivation of Newton
 Quadratic Convergence
 Examples
 Convergence Testing
 Multidimensonal Newton Method
 Basic Algorithm
 Quadratic convergence
 Application to circuits
 Improve Convergence
 Limiting Schemes
 Direction Corrupting
 Non corrupting (Damped Newton)
 Continuation Schemes
 Source stepping

Need to Solve
( 1) 0
d
t
V
V
d s
I I e
  
Nonlinear Problems - Example
0
1
Ir
I1 Id
0
)
1
(
1
0
1
1
1
1







I
e
I
e
R
I
I
I
t
V
e
s
d
r
1
1)
( I
e
g 

Nonlinear Equations
 Given g(V)=I
 It can be expressed as: f(V)=g(V)-I
 Solve g(V)=I equivalent to solve
f(V)=0
Hard to find analytical solution for f(x)=0
Solve iteratively

Nonlinear Equations – Iterative Methods
 Start from an initial value x0
 Generate a sequence of iterate xn-1,
xn, xn+1 which hopefully converges to
the solution x*
 Iterates are generated according to
an iteration function F: xn+1=F(xn)
Ask
• When does it converge to correct solution ?
• What is the convergence rate ?

Newton-Raphson (NR) Method
Consists of linearizing the system.
Want to solve f(x)=0  Replace f(x) with its
linearized version and solve.
Note: at each step need to evaluate f and f’
function
Iteration
x
f
dx
x
df
x
x
x
x
dx
x
df
x
f
x
f
ies
Taylor Ser
x
x
dx
x
df
x
f
x
f
k
k
k
k
k
k
k
k
k
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
1
1
1
1
*
*
*




















Newton-Raphson Method – Graphical View

Newton-Raphson Method – Algorithm
Define iteration
Do k = 0 to ….
until convergence
 How about convergence?
 An iteration {x(k)} is said to converge with order q if
there exists a vector norm such that for each k  N:
q
k
k
x
x
x
x ˆ
ˆ
1




)
(
)
(
1
1 k
k
k
k
x
f
dx
x
df
x
x









 )
(
)
(
1
1 k
k
k
k
x
f
dx
x
df
x
x









 )
(
)
(
1
1 k
k
k
k
x
f
dx
x
df
x
x











Mean Value theorem
truncates Taylor series
But
by Newton
definition
Newton-Raphson Method – Convergence
2
*
2
2
*
*
)
(
)
~
(
)
(
)
(
)
(
)
(
0 k
k
k
k
x
x
dx
x
f
d
x
x
dx
x
df
x
f
x
f 





)
(
)
(
)
(
0 1 k
k
k
k
x
x
dx
x
df
x
f 

 

Subtracting
2
1 * 1 * 2
2
( ) [ ( )] ( )( )
k k k
df d f
x x x x x x
dx d x
 
  
Convergence is quadratic
Dividing through
2
1
2
2
1 * *
Let [ ( )] ( )
then
k k
k k k
df d f
x x K
dx d x
x x K x x



  
2
*
2
2
*
1
)
(
)
~
(
)
(
)
(
x
x
dx
x
f
d
x
x
dx
x
df k
k
k





Local Convergence Theorem
If
Then Newton’s method converges given a
sufficiently close initial guess (and
convergence is quadratic)
bounded
is
K
bounded
dx
f
d
b
ro
ay from ze
bounded aw
dx
df
a
)
)
2
2








 
 
   
 
2
1
2 2
1 * * *
* 2
2
1 *
*
( ) 2
( ) 1 0,
2 ( ) 1
2 ( ) 2 ( )
1
( ) (
( 1
)
2
)
k k
k k k k
k k k k k
k k
k
df
x x
dx
x x x x
x x x x x x x
f x x fin
x
d x x
or x x x x
x




   
    




 
 
Convergence is quadratic
Example 1

 
1 2
1 *
* *
1
2 *
( ) 2
2 ( 0) ( 0)
1
0 0
( ) 0,
for 0
2
1
( ) ( )
2
0
k k
k k k
k k k
k k
df
x x
dx
x x x
x x x x
or x x x
f x
x
x x




   
    
  
  
Convergence is linear
Note : not bounded
away from zero
1
df
dx

 
 
 
Example 2

Example 1,2

0
Initial Guess,
= 0
x k 
Repeat {
 
   
1
k
k k k
f x
x x f x
x


  

} Until ?
 
1 1
? ?
k k k
f x threshold
x x threshold 


 
1
k k
 

Need a "delta-x" check to avoid false convergence
X
f(x)
1
k
x  k
x *
x
 
1
a
k
f
f x 


1 1
a r
k k k
x x
x x x
 
 
  
Convergence Check

 
Also need an " " check to avoid false convergence
f x
X
f(x)
1
k
x  k
x
*
x
 
1
a
k
f
f x 


1 1
a r
k k k
x x
x x x
 
 
  
Convergence Check

demo2

X
f(x)
Convergence Depends on a Good Initial Guess
0
x
1
x
2
x 0
x
1
x
Local Convergence

Convergence Depends on a Good Initial Guess
Local Convergence

Nodal Analysis
1 2
At Node 1: 0
i i
 
1
b
v
+
-
1
i
2
i 2
b
v
3
i
+ -
3
b
v
+
-
Nonlinear
Resistors
 
i g v

1
v 2
v
   
1 1 2 0
g v g v v
   
3 2
At Node 2: 0
i i
 
   
3 1 2 0
g v g v v
   
Two coupled
nonlinear equations
in two unknowns
Nonlinear Problems – Multidimensional Example

Outline
 Examples
 Basic Algorithm
 Source stepping

 
* *
Problem: Find such tha 0
t
x F x 
* N N N
and :
x F
 
Multidimensional Newton Method
function
Iteration
x
F
x
J
x
x
atrix
Jacobian M
x
x
F
x
x
F
x
x
F
x
x
F
x
J
ies
Taylor Ser
x
x
x
J
x
F
x
F
k
k
k
k
N
N
N
N
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
)(
(
)
(
)
(
1
1
1
1
1
1
*
*
*





































)
(
)
(
)
(
:
solve
Instead
sparse).
not
is
(it
)
(
compute
not
Do
)
(
)
(
:
1
1
1
1
k
k
k
k
k
k
k
k
k
x
F
x
x
x
J
x
J
x
F
x
J
x
x
Iteration









Each iteration requires:
1. Evaluation of F(xk)
2. Computation of J(xk)
3. Solution of a linear system of algebraic
equations whose coefficient matrix is J(xk)
and whose RHS is -F(xk)
Computational Aspects

0
Initial Guess,
= 0
x k 
Repeat {
    
1 1
Solve for
k k k k k
F
J x x x F x x
 
  
} Until  
1
1
, small en ug
o h
k
k k
x x f x 


1
k k
 
   
Compute ,
k k
F
F x J x
Algorithm

If
   
1
) Inverse is bounded
k
F
a J x 


     
) Derivative is Lipschitz Cont
F F
b J x J y x y
  
Then Newton’s method converges given a
sufficiently close initial guess (and
convergence is quadratic)
Local Convergence Theorem
Convergence

Application of NR to Circuit Equations
 Applying NR to the system of equations we find
that at iteration k+1:
 all the coefficients of KCL, KVL and of BCE of the
linear elements remain unchanged with respect to
iteration k
 Nonlinear elements are represented by a linearization
of BCE around iteration k
 This system of equations can be interpreted as the
STA of a linear circuit (companion network) whose
elements are specified by the linearized BCE.
Companion Network

 General procedure: the NR method applied
to a nonlinear circuit whose eqns are
formulated in the STA form produces at
each iteration the STA eqns of a linear
resistive circuit obtained by linearizing the
BCE of the nonlinear elements and leaving
all the other BCE unmodified
 After the linear circuit is produced, there is
no need to stick to STA, but other methods
(such as MNA) may be used to assemble
the circuit eqns
Companion Network

Note: G0 and Id depend on the iteration count k
 G0=G0(k) and Id=Id(k)
Companion Network – MNA templates

Companion Network – MNA templates

Modeling a MOSFET (MOS Level 1, linear regime)
d

Modeling a MOSFET (MOS Level 1, linear regime)

DC Analysis Flow Diagram
For each state variable in the system

Implications
 Device model equations must be continuous
with continuous derivatives (not all models do
this - - be sure models are decent - beware of
user-supplied models)
 Watch out for floating nodes (If a node
becomes disconnected, then J(x) is singular)
 Give good initial guess for x(0)
 Most model computations produce errors in
function values and derivatives. Want to have
convergence criteria || x(k+1) - x(k) || <  such
that  > than model errors.

Outline
 Examples
 Basic Algorithm
 Source stepping

Improving convergence
 Improve Models (80% of problems)
 Improve Algorithms (20% of
problems)
Focus on new algorithms:
Limiting Schemes
Continuations Schemes

Improve Convergence
 Limiting Schemes
 Direction Corrupting
 Non corrupting (Damped Newton)
 Globally Convergent if Jacobian is
Nonsingular
 Difficulty with Singular Jacobians
 Continuation Schemes
 Source stepping

At a local minimum, 0
f
x



Local
Minimum
 
Multidimensional Case: is singular
F
J x
Convergence Problems – Local Minimum

f(x)
X
0
x
1
x
Must Somehow Limit the changes in X
Convergence Problems – Nearly singular

f(x)
X
0
x 1
x
Must Somehow Limit the changes in X
Convergence Problems - Overflow

0
Initial Guess,
= 0
x k 
Repeat {
   
1 1
Solve for
k k k k
F
J x x F x x
 
   
} Until  
1 1
, small enough
k
k
x F x 


1
k k
 
   
Compute ,
k k
F
F x J x
 
Newton Algorithm for Solving 0
F x 
 
1 1
limited
k k k
x x x
 
  
Newton Method with Limiting

 
1
limited k
i
x 
 
• NonCorrupting
1 1
if
k k
i i
x x 
 
  
 
1
otherwise
k
i
sign x
 
 
1
k
x 

 
1
limited k
x 

• Direction Corrupting
 
1 1
limited k k
x x

 
  
1
min 1, k
x

 
 
 
  

 
 

1
k
x 

 
1
limited k
x 

Heuristics, No Guarantee of Global Convergence
Limiting Methods

General Damping Scheme
Key Idea: Line Search
 
2
1
2
Pick to minimize
k k k k
F x x
  
 
Method Performs a one-dimensional search in
Newton Direction
     
2
1 1 1
2
T
k k k k k k k k k
F x x F x x F x x
  
  
      
1 1
k k k k
x x x

 
  
   
1 1
Solve for
k k k k
F
J x x F x x
 
   
Damped Newton Scheme

If
   
1
) Inverse is bounded
k
F
a J x 


     
) Derivative is Lipschitz Cont
F F
b J x J y x y
  
Then
 
There exists a set of ' 0,1 such that
k
s
 
     
1 1
with <1
k k k k k
F x F x x F x
  
 
   
Every Step reduces F-- Global Convergence!
Damped Newton – Convergence Theorem

0
Initial Guess,
= 0
x k 
Repeat {
   
1 1
Solve for
k k k k
F
J x x F x x
 
   
} Until  
1 1
, small enough
k
k
x F x 


1
k k
 
   
Compute ,
k k
F
F x J x
   
1
Find 0,1 such that is minimi e
z d
k k k
k
F x x
  
 

1 1
k k k k
x x x

 
  
Damped Newton – Nested Iteration

X
0
x
1
x
2
x
1
D
x
Damped Newton Methods “push” iterates to local minimums
Finds the points where Jacobian is Singular
Damped Newton – Singular Jacobian Problem

 
 
Solve , 0 where:
F x   
 
 
a) 0 ,0 0 is easy to solve
F x 
 
   
b) 1 ,1
F x F x

 
c) is sufficiently smooth
x 
 Starts the continuation
Ends the continuation
Hard to insure!
Newton with Continuation schemes
Newton converges given a close initial guess
 Idea: Generate a sequence of problems,
s.t. a problem is a good initial guess for the
following one

 0
)
(x
F
Basic Concepts - General setting

 
     
Solve 0 ,0 , 0
prev
F x x x
 
While 1 {
 
}
=0.01,
  

   
0
prev
x x
 

 
 
Try to Solve , 0 with Newton
F x   
If Newton Converged
    , 2
,
prev
x x
    
  
  

Else
,
1
2
prev

  
 
 

Basic Concepts – Template Algorithm

Basic Concepts – Source Stepping Example

+
-
Vs
R
Diode
 
     
1
, 0
diode s
f v i v v V
R
  
   
v
   
, 1
Not dependent!
diode
f v i v
v v R


 
  
 
Source Stepping Does Not Alter Jacobian
Basic Concepts – Source Stepping Example

Transient Analysis Flow Diagram
Predict values of variables at tl
Replace C and L with resistive elements via integration formula
Replace nonlinear elements with G and indep. sources via NR
Assemble linear circuit equations
Solve linear circuit equations
Did NR converge?
Test solution accuracy
Save solution if acceptable
Select new t and compute new integration formula coeff.
Done?
YES
NO
NO

Newton.ppt

Recommended

Recommended

More Related Content

Similar to Newton.ppt

Similar to Newton.ppt (20)

More from Godspower Bruno, GMNSE

More from Godspower Bruno, GMNSE (20)

Recently uploaded

Recently uploaded (20)

Newton.ppt