Upcoming SlideShare
×

Like this presentation? Why not share!

# Presentation F Sharp

## on May 06, 2010

• 1,455 views

### Views

Total Views
1,455
Views on SlideShare
1,298
Embed Views
157

Likes
0
17
0

### 2 Embeds157

 http://skillsmatter.com 119 http://smash 38

### Report content

• Comment goes here.
Are you sure you want to

## Presentation F Sharp Presentation Transcript

• From the math to an F# interface Implementing the D.E. algorithm A numerical optimization library in F# N. El Bachir1 1 FinBoost Limited 29 April 2010 N. El Bachir F# Solvers
• From the math to an F# interface Implementing the D.E. algorithm Contents 1 From the math to an F# interface The mathematical problem Mapping the problem into an F# interface 2 Implementing the D.E. algorithm Diﬀerential Evolution algorithm F# implementation N. El Bachir F# Solvers
• From the math to an F# interface The mathematical problem Implementing the D.E. algorithm Mapping the problem into an F# interface Optimization under constraints Find x ∗ ∈ A sucht that x ∗ = arg min f (x), (1) x s.t. g (x) (2) Objective function f deﬁned on domain A (A ⊆ Rn ) : f : A→R (3) Constraints g (x) an arbitrary Boolean combination of : equations g (x) = 0, weak inequalities g (x) ≥ 0, and weak inequalities g (x) > 0. N. El Bachir F# Solvers
• From the math to an F# interface The mathematical problem Implementing the D.E. algorithm Mapping the problem into an F# interface Some examples Very simple : f (x) = x 2 , A = R and no constraint. Solution : x∗ = 0 Another example : f (x, y ) = x − y ,A = R2 s.t. −3x 2 + 2xy − y 2 ≥ −1. Solution : x ∗ = 0, y ∗ = 1 and f (0, 1) = −1. Third example : f (x, y ) = x 2 + (y − 1 )2 , A = R2 s.t. y ≥ 0 and 2 y ≥ x + 1.Solution : x ∗ = −0.25, y ∗ = 0.75 and f (−0.25, 0.75) = 0.125. We are interested in numerical algorithms to solve more complex problems. N. El Bachir F# Solvers
• From the math to an F# interface The mathematical problem Implementing the D.E. algorithm Mapping the problem into an F# interface Numerical optimization Iterative procedure : start with initial guess(es) x 0 , keep updating x j → x j+1 = F (x j ) according to some transformation rule F , stop when some criteria are met # of iterations, convergence in x, convergence in objective function. To handle constraints : Transform into unconstrained optimization drop non-active inequalities. search value of the variable that dominates the constraint. Lagrange multipliers. use of merit/penalty functions. Feasible directions method. Repairing/rejecting unfeasible solutions. N. El Bachir F# Solvers
• From the math to an F# interface The mathematical problem Implementing the D.E. algorithm Mapping the problem into an F# interface Solve let Solve (problem) (strategy) (stopWhen): Solution = let rec helper (pblm) (stp): Solution = let solution = strategy.oneStepSolve problem st let isFinished,stopper = solution.stop.Finished solution match isFinished with | true -> solution | _ -> let solution = { best=solution.best bestValue= solution.bestValue stop = stopper } helper pblm solution.stop helper problem stop N. El Bachir F# Solvers
• From the math to an F# interface The mathematical problem Implementing the D.E. algorithm Mapping the problem into an F# interface Main abstractions type Problem = abstract Domain : Domain with get abstract Objective : RnOnDomain -> real abstract Constraint : Constraint with get and Constraint = abstract isSatisfied : RnOnDomain -> bool and Strategy = abstract oneStepSolve : Problem -> StopWhen -> Solution and StopWhen = abstract Finished : Solution -> bool*StopWhen and Solution = { best : RnOnDomain bestValue : real stop : StopWhen } N. El Bachir F# Solvers
• From the math to an F# interface The mathematical problem Implementing the D.E. algorithm Mapping the problem into an F# interface Domains on Rn type RnOnDomain = val rn : Rn val dim: int new(x:Rn,domain:Domain) = let rn = if domain.contains(x) then x else failwith ("Not in the domain") let dim = match x with | Real1 x -> 1 | RealN x -> Array.length x { rn = rn dim = dim } N. El Bachir F# Solvers
• From the math to an F# interface The mathematical problem Implementing the D.E. algorithm Mapping the problem into an F# interface More on domains type Domain = val lowBound : BoundVal[] val upBound : BoundVal[] new(bound:Bounds)= let low,up = let low_,up_= match bound with | Real theBound -> [|theBound.lower|],[|theBound.upper|] | Realn theBound -> theBound.lower,theBound.upper if low_.Length = up_.Length then low_,up_ else failwith ("Invalid domain: " + "# of upper bounds != # of lower bounds.") { lowBound = low upBound = up } N. El Bachir F# Solvers
• From the math to an F# interface The mathematical problem Implementing the D.E. algorithm Mapping the problem into an F# interface More on domains member this.contains(x) : bool = let X = match x with | Real1 x -> [| x |] | RealN x -> x if not( X.Length = this.lowBound.Length ) then failwith ("Input has different dimensions than domain") else let above_lower = [| for k in 0..(this.lowBound.Length - 1) -> let x = X.[k] match z with | Inclusive y -> x >= y | Exclusive y -> x > y |] //let below_upper = ... Array.forall2 (fun x y -> x && y) above_lower below_upper member this.newRn(x:Rn) = new RnOnDomain(x,this) N. El Bachir F# Solvers
• From the math to an F# interface The mathematical problem Implementing the D.E. algorithm Mapping the problem into an F# interface Reals and intervals type BoundVal = | Inclusive of float | Exclusive of float type Range = { lower : BoundVal upper : BoundVal } type Range_nd = { lower : BoundVal[] upper : BoundVal[] } type Bounds = | Real of Range | Realn of Range_nd type Rn = | Real1 of float | RealN of float[] N. El Bachir F# Solvers
• From the math to an F# interface The mathematical problem Implementing the D.E. algorithm Mapping the problem into an F# interface Simple example let bisectPbm = { new Problem with member this.Domain = domain1D member this.Objective (x:RnOnDomain) : float= objFunc member this.Constraint = ( new PosConst domain1D ):> Constraint } let solution = ( Solve (bisectProblem) ( new BisectionMethod()) oneStep ).best.Rn N. El Bachir F# Solvers
• From the math to an F# interface Diﬀerential Evolution algorithm Implementing the D.E. algorithm F# implementation Quick intro to D.E. Stochastic global optimization approach which mimicks evolutionary concepts. Initialization consists in picking an initial population randomly Updating procedure (generation evolution) given by Mutation, Recombination, Selection. N. El Bachir F# Solvers
• From the math to an F# interface Diﬀerential Evolution algorithm Implementing the D.E. algorithm F# implementation Initial population D parameters. Initial population of size N, will evolve in each generation G with selection x i,G = [x1,i,G , . . . , xD,i,G ] (4) Bounds for each parameter : xjL ≤ xj,i,1 ≤ xjU (5) Initial population randomly selected on the intervals xjL , xjU (6) N. El Bachir F# Solvers
• From the math to an F# interface Diﬀerential Evolution algorithm Implementing the D.E. algorithm F# implementation A mutation strategy For each member i of the population, a mutant is generated by randomly picking distinct indices k1 , k2 , and k3 and combining them. For example, using a mutation factor F ∈ [0, 2] : v i,G +1 = x k1 ,G + F × x k2 ,G − x k3 ,G (7) Recombination step will then create a trial member from a mutant and a member from previous generation. N. El Bachir F# Solvers
• From the math to an F# interface Diﬀerential Evolution algorithm Implementing the D.E. algorithm F# implementation Recombination/Crossover Given a crossover probability CR, randj,i U[0, 1] for i = 1, . . . , N and j = 1, . . . , D V a random integer from 1, . . . , D Trial members for the next generation given by v j,i,G +1 if randj,i ≤ CR or j = V u j,i,G +1 = (8) x j,i,G +1 otherwise N. El Bachir F# Solvers
• From the math to an F# interface Diﬀerential Evolution algorithm Implementing the D.E. algorithm F# implementation Selection of the ﬁttest Each trial member is compared to its previous generation and the ﬁttest is selected for each pair (x i,G , u i,G +1 ) : u i,G +1 if f (u i,G +1 ) ≤ f (x i,G ) x i,G +1 = (9) x i,G otherwise The process of mutation, recombination and selection continues until either a maximum number of iterations is reached or some convergence criteria is attained. N. El Bachir F# Solvers
• From the math to an F# interface Diﬀerential Evolution algorithm Implementing the D.E. algorithm F# implementation Some snippets Initialization of the population : initpop = [| for j in 0 .. N-1 -> randomize domain |] // randomize checks the bounds for each dim // returns an array of rands, 1 per dim within the bounds Mutation and crossover : for n = 0 to N-1 do for d = 0 to D-1 do let ui= if unitRand() < CR then bm.[n].[d] + F*(pm1.[n].[d] - pm2.[n].[d]) else pop.[n].[d] trial.[n].[d] <- ui N. El Bachir F# Solvers
• From the math to an F# interface Diﬀerential Evolution algorithm Implementing the D.E. algorithm F# implementation More snippets Convergence criteria : type MaxItOrConv (iter:int, tol:float) = interface StopWhen with member this.Finished solution = let run_iter = if iter = 0 then true else false let distance = solution.bestValue < tol run_iter || distance, solution.stop N. El Bachir F# Solvers