<content> Alternate Nelder Mead </content>

83bcd2f5-5fbd-4b35-b84c-b3d7c1334b64 36 4 5 false false 0 decimal

&[DATE] &[TIME] - &[FILENAME]

Alternative Nelder Mead Algorithm

Code based on "Evolving a Nelder–Mead Algorithm for Optimization with Genetic Programming", by Iztok Fajfar, Janez Puhan & Árpád Bűrmen

https://direct.mit.edu/evco/article/25/3/351/1046/Evolving-a-Nelder-Mead-Algorithm-for-Optimization

<content> Alternate Nelder Mead </content>

NelderMead Alternative code 1 f B O NMA S B 1 el B 3 el 21 mat B 1 el B 4 el 21 mat B 2 el B 3 el 21 mat B 2 el B 4 el 21 mat 14 mat : κ 1 1.375 2 0.625 14 mat : εy εx h MI 14 mat O : 12 mat n S length : r 1 n range : x# C 1 n / x# sum (* : 13 mat x S r el : F r el x r el f : X 0 : 13 mat iter 1 MI range Fx F x augment 1 csort : F Fx 1 col : x Fx 2 col : 12 mat F F C - normi εy < (x 1 el x n el - normi εx < (& break xc x C : xc κ 1 el xc x n el - (* + f F n el < xc κ 3 el xc x n el - (* + f xc f < xn xc κ 2 el xc x n el - (* + : xn xc κ 1 el xc x n el - (* + : if 11 line xn xc κ 4 el xc x n el - (* - : if x n el xn : F n el xn f : 12 mat 31 line if 31 line for x 1 el 61 line :

Using 4 starting points instead the classical 3 just because it's easy relate the simplex with a box for plots.

Pseudocode from the above reference

<content> Plot Utilities </content> <content> FillContour </content>

f.feval# x.feval# y.feval# feval f.feval# num2str (x.feval# num2str, y.feval# num2str) concat str2num 11 line :

n a pCmapJet r 1 n range : m 0.25 n * trunc : M 0 : 13 mat v z v rows 1 matrix : o 255 m 1 matrix + : 12 mat x 2551 m range m / * trunc vectorize : y 255 m / m 2 / m range (* trunc vectorize : 12 mat R y z o z x o y reverse stack : G y z x o x reverse y z stack : B y o x reverse o z y z stack : k R length n - : M r el B r k + el 256 G r k + el * + 2562^R r k + el * + 2563^255 a * trunc * + : M r el M r el 232^2 M r el * 232^1 - ≥ (* - eval : 91 line :

a b n rng a b a - n / 0 n range * + 11 line :

f B N pGrid r 1 N 1 el range : c 1 N 2 el range : Z 0 : 13 mat X B 1 el B 2 el N 1 el 1 - rng : Y B 3 el B 4 el N 2 el 1 - rng : Z r c el f X r el Y c el feval : 13 mat 21 line :

Filled Contour G g pFillContour X Y Z 13 mat G : n.1 X length 1 - : n.2 Y length 1 - : S 0 : 14 mat r 1 n.1 range : c 1 n.2 range : n 1 n.1 n.2 * (range : 13 mat a Z min eval : d g length 1 - Z max a - / eval : 12 mat Z Z a - (d * trunc 1 + vectorize eval : S r c el polygon X r el Y c el X r el Y c 1 + el X r 1 + el Y c 1 + el X r 1 + el Y c el X r el Y c el 52 mat 11 mat g Z r c el el solid 1 g Z r c el el stack : S n el 11 mat 61 line :

<content> Contours </content>

¦AuxIPE 0931061551212515610390116 mat :

¦AuxIPΔ 0110 0.5 1100 0.5 25 mat :

¦AuxIPBin 124841 mat :

¦AuxIPT 0112211331122111042434344323424000000200003000000000030000400000416 mat :

¦AuxIPB 0110011001100110001111000011110000001111111100000101010110101010416 mat :

Contour Plot G NL pContour X Y Z 13 mat G eval : h 14 range : CL 0 : 13 mat L G 3 el max G 3 el min NL 2 + rng 2 NL 1 + (range el eval : nl 1 NL range ZL Z L nl el - eval : M 0 : m 0 : v 0 : P 0 : z 0 : 16 mat r 1 X length 2 - (range c 1 Y length 2 - (range Δx 022014 mat r + : Δy 220014 mat c + : z h el ZL Δx h el Δy h el el : 13 mat n 1 z 0 < vectorize ¦AuxIPBin * + eval : ¦AuxIPE n el 0 ≡ continue ZL r 1 + c 1 + el 0 < (n 11 ≡ (n 6 ≡ (| (& n n 11 ≡ (6 * n 6 ≡ (11 * + eval : 0 if x X Δx el : y Y Δy el : 12 mat k 1312 mat ¦AuxIPT k n el 0 ≠ s ¦AuxIPT k n el ¦AuxIPT k 1 + n el 12 mat : v s el ¦AuxIPB s n el 0 ≠ j 1 s 4 mod + : x s el y s el 12 mat x j el x s el - y j el y s el - 12 mat z j el z s el - / z s el * - eval 21 line m trace if : M m m 1 + : el v s 1 el el v s 2 el el stack eval : 31 line continue if for 31 line if 31 line for for CL nl el M : 31 line for CL mat2sys.1 41 line :

<content> Examples </content>

Examples

B x1 x2 y1 y2 22 mat ≡

Box for plots and guess values

NP 808021 mat :

Points for contours

NP nx ny 21 mat ≡

NSol 1015 -^109 -^020014 mat :

Options for NM Methods

O εy εx h MI 14 mat ≡

g 200 0.9 pCmapJet :

Colormap

zo

Exact solution for the example

Shorthands

f zo za zc Plot f B NP pGrid g pFillContour za 1 el za 2 el x 8 yellow augment zc 1 el zc 2 el x 8 white augment zo 1 el zo 2 el o 8 red augment 41 sys 11 line :

x φ x 1 el x 2 el f 11 line :

1 φ B O NMC x# φ 12 / B 1 el B 2 el + B 3 el B 4 el + 21 mat * eval O 1 el NelderMead 11 line :

Classical NelderMead Method from Nonlinear Solvers plugin using the center of the box as guess instead a simplex.

For an actual comparation between codes they must to start from the same simplex, but it isn't the purpose here, it just to introduce the new algorithm for the method and see if it "works".

x y f 4 x 5 - (2^* y 6 - (2^+ : B 5 - 105 - 1022 mat : zo 5621 mat : 12 mat za x φ B NSol NMA : zc x φ B NSol NMC : 41 line

f zo za zc Plot 1 1 line

zo φ 0

za φ 0.103281213545949

zc φ 0.015543958361777

z.1 13 - x + 2 y * - 5 y 2^* + y 3^- : z.2 29 - x + 14 y * - y 2^+ y 3^+ : x y f z.1 2^z.2 2^+ : B 363622 mat : zo 5421 mat : 12 mat za x φ B NSol NMA : zc x φ B NSol NMC : 61 line

f zo za zc Plot 1 1 line

zo φ 0

za φ 0.000000088906783

zc φ 0.543022665741093

z.1 1.5 x 1 y - (* - (2^: z.2 2.25 x 1 y 2^- (* - (2^2.625 x 1 y 3^- (* - (2^+ : x y f z.1 z.2 + : B 040422 mat : zo 3 0.5 21 mat : 12 mat za x φ B NSol NMA : zc x φ B NSol NMC : 61 line

f zo za zc Plot 1 1 line

zo φ 0

za φ 0.000480493753523

zc φ 0.020060444564884

x y f x y 2^- (2^1001 x - (2^* + : B 030322 mat : zo 1121 mat : 12 mat za x φ B NSol NMA : zc x φ B NSol NMC : 41 line

f zo za zc Plot 1 1 line

zo φ 0

za φ 1.61304251090827 1015 -^*

zc φ 5.65334548004554

Alvaro