NonlinearSolvers plugin

NonlinearSolvers plugin - BDQRF, Bisection, Brent's, Broyden's, Newton-Raphson, Ridder's, Secant, Homotopy - Сообщения

#21 Опубликовано: 07.09.2012 15:20:39
kilele

kilele

133 сообщений из 397 понравились пользователям.

Группа: User

hey w3b5urf3e, I see on the pic that you want to implement Brent, what about the BDQRF method mentioned on your Bisection testings file? they state that BDQRF is better than Brent
#22 Опубликовано: 07.09.2012 17:12:51
kilele

kilele

133 сообщений из 397 понравились пользователям.

Группа: User

Wrote


I also agree about the mentioned observation on the 20-page of your reference. This is up to the user. A simple advice is to rearrange the function f(x)= 0 by avoiding division of terms when denominator can be close to zero.



On that paper (20-page and following) the author also mentions "homotopy continuation methods" to get a good initial guess value. Since initial guess values on multidimensional non-linear systems can be difficult to find and a bad initial guess may cause divergence, maybe it'd be useful to make some initialization function for this golden plugin that w3b5urf3r is preparing
Some simple continuation method would suffice for a good initial value.
#23 Опубликовано: 08.09.2012 10:28:39
Radovan Omorjan

Radovan Omorjan

325 сообщений из 2052 понравились пользователям.

Группа: Moderator

Hello w3b5urf3r
Wrote

...about eval(), you cannot use it where there are undefined variables because the function "force" the numeric evalutation.


I am trying to figure out this regarding eval(). I do not get it. It is quite not logical to me. For instance, we can make a function in this way

[MATH=eng]f(x):line(a:x,y:eval(a^2+5*a-3),2,1)[/MATH]

We can call it and get the result

[MATH=eng]f(-2)=-9[/MATH] [MATH=eng]f(2)=11[/MATH]

But can not use it here

[MATH=eng]Bisection(f(x#),-9,0,10^{-5},50)=#@#[/MATH]

If we remove eval(), everything is fine

[MATH=eng]f(x):line(a:x,y:a^2+5*a-3,2,1)[/MATH]

[MATH=eng]Bisection(f(x#),-9,0,10^{-5},50)=-5.5414[/MATH]

There are functions like Bisection() that actually need the function value regardless the expression behind that function. Moreover, eval() has proven to be sometimes very useful in order to increase the calculation speed dramatically. I think this would be quite a restriction if we could not use it in the situation like these. I suppose that is due to the SMath design and its symbolic engine. I think this is quite a disadvantage regarding the strictly numerical computations and just hope that something could be done about it.

Regards,
Radovan
When Sisyphus climbed to the top of a hill, they said: "Wrong boulder!"
#24 Опубликовано: 08.09.2012 12:54:27
Davide Carpi

Davide Carpi

1416 сообщений из 2873 понравились пользователям.

Группа: Moderator

Plugin updated,

Added new functions (Secant/Brent/Ridder)

The issue of the last post was in the evaluation of the number of functions, now is fixed.

Wrote

Hello w3b5urf3r

Wrote

...about eval(), you cannot use it where there are undefined variables because the function "force" the numeric evaluation.


I am trying to figure out this regarding eval(). I do not get it. It is quite not logical to me. For instance, we can make a function in this way

[MATH=eng]f(x):line(a:x,y:eval(a^2+5*a-3),2,1)[/MATH]

We can call it and get the result

[MATH=eng]f(-2)=-9[/MATH] [MATH=eng]f(2)=11[/MATH]

But can not use it here

[MATH=eng]Bisection(f(x#),-9,0,10^{-5},50)=#@#[/MATH]

If we remove eval(), everything is fine

[MATH=eng]f(x):line(a:x,y:a^2+5*a-3,2,1)[/MATH]

[MATH=eng]Bisection(f(x#),-9,0,10^{-5},50)=-5.5414[/MATH]

There are functions like Bisection() that actually need the function value regardless the expression behind that function. Moreover, eval() has proven to be sometimes very useful in order to increase the calculation speed dramatically. I think this would be quite a restriction if we could not use it in the situation like these. I suppose that is due to the SMath design and its symbolic engine. I think this is quite a disadvantage regarding the strictly numerical computations and just hope that something could be done about it.

Regards,
Radovan



doing a few tests seem that the eval() function force a "local" numeric evaluation... when a 1st function call a 2nd nested function that contains the eval(), SMath first attempt to evaluate the eval() inside the 2nd function, or something similar... f.e. look the attachment eval.sm, f1a() can't be evaluated simbolically.

As for Primer65aa, the Broyden function is not bugged... the f() function contains logharitms and the Broyden function shoot solutions in the undefined domain (see the attachments).

Wrote

hey w3b5urf3e, I see on the pic that you want to implement Brent, what about the BDQRF method mentioned on your Bisection testings file? they state that BDQRF is better than Brent



I've seen the BDQRF pseudocode, I think that could be implemented in the next plugin relase, as well as the Zhengqiu Zhang Improvement to the Brent’s Method



regards,

w3b5urf3r
eval.sm (5 КиБ) скачан 70 раз(а).
If you like my plugins please consider to support the program buying a license; for personal contributions to me: paypal.me/dcprojects
1 пользователям понравился этот пост
Radovan Omorjan 08.09.2012 14:28:00
#25 Опубликовано: 08.09.2012 14:56:32
Radovan Omorjan

Radovan Omorjan

325 сообщений из 2052 понравились пользователям.

Группа: Moderator

Hello w3b5urf3r,

Thank you for your explanation and the updated plugin

Best Wishes,
Radovan
When Sisyphus climbed to the top of a hill, they said: "Wrong boulder!"
#26 Опубликовано: 09.09.2012 08:30:33
Radovan Omorjan

Radovan Omorjan

325 сообщений из 2052 понравились пользователям.

Группа: Moderator

Hello,

Here is another file attached. I was struggling with it and it suits me quite well for my classes. As this plugin is about nonlinear solvers, I used in my file a simple Levenberg-Marqurdt based procedure to obtain curve fitting nonlinear in the unknown parameters. It is quite unsophisticated, I have to play with dumping parameters - but it is solving my problem.

As one of my files motivated this plugin, I really hope that this one will do the same as well
I would not mind some derivation free optimization (minimization) functions in the plugin like BFGS or Nelder-Mead :d .

Regards,
Radovan
NLMinimization-3.sm (79 КиБ) скачан 102 раз(а).
When Sisyphus climbed to the top of a hill, they said: "Wrong boulder!"
#27 Опубликовано: 10.09.2012 14:15:57
Davide Carpi

Davide Carpi

1416 сообщений из 2873 понравились пользователям.

Группа: Moderator

Plugin updated.

Added the Newton-Raphson method for Nonlinear systems of equations and the Bisected Direct Quadratic Regula Falsi method.


regards,

w3b5urf3r



If you like my plugins please consider to support the program buying a license; for personal contributions to me: paypal.me/dcprojects
1 пользователям понравился этот пост
Radovan Omorjan 10.09.2012 14:31:00
#28 Опубликовано: 10.09.2012 15:52:45
kilele

kilele

133 сообщений из 397 понравились пользователям.

Группа: User

Wrote


I would not mind some derivation free optimization (minimization) functions in the plugin like BFGS or Nelder-Mead :d .


There you have more matlab code by Kelley for Nelder-Mead and BFGS
( see bfgswopt.m ; nelder.m ; simpgrad.m )

As for "homotopy continuation methods" I've found this mathematica code.
However I don't know if this method is interesting or too difficult to code, I read that works very well for polynomial systems as well.

EDIT:

More fortran and pseudocode about continuation methods on this pdf
2 пользователям понравился этот пост
Radovan Omorjan 10.09.2012 15:55:00, Davide Carpi 10.09.2012 17:32:00
#29 Опубликовано: 10.09.2012 16:05:09
Radovan Omorjan

Radovan Omorjan

325 сообщений из 2052 понравились пользователям.

Группа: Moderator

Hello w3b5urf3r,

Do not have anything to say, but keep up this way

I hope you would not mind if I suggest you to change your convergence criterion in the root finding functions.
The convergence criterion at the moment is that the absolute function value in some iteration to be smaller than epsilon


[MATH=eng]abs(f(P3))
This is the simplest convergence criterion. Sometimes it is not good enough. To make that stronger you could add, for instance, that the absolute relative difference of two iteration be smaller than epsilon as well.


[MATH=eng](abs(f(P3))
The epsilon in the denominator is in the case the solution is zero.

Regards,
Radovan
When Sisyphus climbed to the top of a hill, they said: "Wrong boulder!"
#30 Опубликовано: 10.09.2012 16:08:45
Radovan Omorjan

Radovan Omorjan

325 сообщений из 2052 понравились пользователям.

Группа: Moderator

Wrote

Wrote


I would not mind some derivation free optimization (minimization) functions in the plugin like BFGS or Nelder-Mead :d .


There you have more matlab code by Kelley for Nelder-Mead and BFGS
( see bfgswopt.m ; nelder.m ; simpgrad.m )

As for "homotopy continuation methods" I've found this mathematica code.
However I don't know if this method is interesting or too difficult to code, I read that works very well for polynomial systems as well.


Thank you, but the second link is broken (mathematica code). Could you check it out.

Regards,
Radovan
When Sisyphus climbed to the top of a hill, they said: "Wrong boulder!"
#31 Опубликовано: 10.09.2012 17:24:26
Davide Carpi

Davide Carpi

1416 сообщений из 2873 понравились пользователям.

Группа: Moderator

Wrote

Plugin updated.

Added the Newton-Raphson method for Nonlinear systems of equations and the Bisected Direct Quadratic Regula Falsi method.

regards,

w3b5urf3r



I've forgot to swap the indices of the jacobian derivatives after the debugging, plugin reupdated...


Wrote

Hello w3b5urf3r,

Do not have anything to say, but keep up this way

I hope you would not mind if I suggest you to change your convergence criterion in the root finding functions.
The convergence criterion at the moment is that the absolute function value in some iteration to be smaller than epsilon

[MATH=eng]abs(f(P3))
This is the simplest convergence criterion. Sometimes it is not good enough. To make that stronger you could add, for instance, that the absolute relative difference of two iteration be smaller than epsilon as well.

[MATH=eng](abs(f(P3))
The epsilon in the denominator is in the case the solution is zero.

Regards,
Radovan



I know, but I'm trying to pay attention to the management of functions with unit of measurement... although I have not advertised this behavior, actually all solvers shown below works also with units. Introducing new covengence criteria preserving this aspect implies a more advanced error handling (and developping time)...


regards,

w3b5urf3r
units.PNG
If you like my plugins please consider to support the program buying a license; for personal contributions to me: paypal.me/dcprojects
#32 Опубликовано: 10.09.2012 18:15:46
kilele

kilele

133 сообщений из 397 понравились пользователям.

Группа: User

Wrote


the second link is broken (mathematica code). Could you check it out.



Link corrected and added another pdf resource
#33 Опубликовано: 11.09.2012 01:19:16
Radovan Omorjan

Radovan Omorjan

325 сообщений из 2052 понравились пользователям.

Группа: Moderator

Wrote

I know, but I'm trying to pay attention to the management of functions with unit of measurement... although I have not advertised this behavior, actually all solvers shown below works also with units. Introducing new covengence criteria preserving this aspect implies a more advanced error handling (and developping time)...


You are right and I respect that

Regards,
Radovan
When Sisyphus climbed to the top of a hill, they said: "Wrong boulder!"
#34 Опубликовано: 12.09.2012 18:35:32
Davide Carpi

Davide Carpi

1416 сообщений из 2873 понравились пользователям.

Группа: Moderator

Wrote

Wrote



As for "homotopy continuation methods" I've found this mathematica code.
However I don't know if this method is interesting or too difficult to code, I read that works very well for polynomial systems as well.

EDIT:

More fortran and pseudocode about continuation methods on this pdf




little experiments in SMath with Homotopy... it's a very interesting and powerful tool b)


regards,

w3b5urf3r


homotopy - systems.png
If you like my plugins please consider to support the program buying a license; for personal contributions to me: paypal.me/dcprojects
1 пользователям понравился этот пост
Radovan Omorjan 13.09.2012 00:55:00
#35 Опубликовано: 13.09.2012 08:51:55
Radovan Omorjan

Radovan Omorjan

325 сообщений из 2052 понравились пользователям.

Группа: Moderator

Hello,

Some more testings and surprises. I attached one of my files and tried all the nonlinear solvers including the SMath solve, roots() and all of them made by w3b5urf3r up to now

I attached a part of my file (primer63w.sm). Here is another situation regarding SMath optimization. The first picture is when the Symbolic Optimization of the function is applied. Everything seems to be fine (although two functions failed - roots(),NewtonRaphosn()). Strangely, when the Optimization is none, numeric - all of them failed and just look at the second picture - now the result of f(1000) is a vector and not scalar as before. This is due to the situation where we have the product of two vector - the result should be a scalar (scalar product of two vectors). When we change some Optimization in the file we can even get the result as in the third picture - solve() gave the result but the other two solvers have different results than solve(). Something must be strange here because other solvers failed. The main problem here is that SMath sometimes returns scalar and sometimes vector depending on optimization applied (fourth and fifth picture on the next post). I think I made some mistakes but can not figure out what happened here.

Could someone take a look at this please.

Regards,
Radovan
Primer63w-3.png
When Sisyphus climbed to the top of a hill, they said: "Wrong boulder!"
#36 Опубликовано: 13.09.2012 08:57:18
Radovan Omorjan

Radovan Omorjan

325 сообщений из 2052 понравились пользователям.

Группа: Moderator

Continuing from the post above with two pictures - the result is scalar or vector of scalar product of two vectors - depending on optimization. Do not know why?.
Primer63w-5.png
When Sisyphus climbed to the top of a hill, they said: "Wrong boulder!"
#37 Опубликовано: 13.09.2012 14:12:44
Davide Carpi

Davide Carpi

1416 сообщений из 2873 понравились пользователям.

Группа: Moderator

Hi omorr

seems there are many little issues (see the attachment)...

if you use a line in the definition of funh(T), the behavior changes further


regards,

w3b5urf3r
Primer63w_debug.sm (102 КиБ) скачан 69 раз(а).
If you like my plugins please consider to support the program buying a license; for personal contributions to me: paypal.me/dcprojects
#38 Опубликовано: 13.09.2012 14:42:58
Radovan Omorjan

Radovan Omorjan

325 сообщений из 2052 понравились пользователям.

Группа: Moderator

Hello w3b5urf3r,

Thank you for checking this out . I can join you with "don't know why" statements about this. I am not sure but I think that has something to this Transpose-functions-bug

I hope that Andrey will have some time to check this out as well.

Regards,
Radovan

EDIT: I might be completely wrong here but I think if eval() could be used in funh():=eval(...) many of the numerical problems might be overcome.
When Sisyphus climbed to the top of a hill, they said: "Wrong boulder!"
#39 Опубликовано: 14.09.2012 06:25:16
Davide Carpi

Davide Carpi

1416 сообщений из 2873 понравились пользователям.

Группа: Moderator

Plugin updated.

added HRE(...) and HRE.B(...) (Homotopy root estimation using NewtonRaphson and Broyden, respectively);

fixed issues of internal variable parser with differentiations.

EDIT: Plugin reuploaded with support for unassigned variables named with apostrophe (f.e. x')


regards,

w3bsurf3r
If you like my plugins please consider to support the program buying a license; for personal contributions to me: paypal.me/dcprojects
1 пользователям понравился этот пост
Radovan Omorjan 14.09.2012 07:57:00
#40 Опубликовано: 14.09.2012 16:07:35
Radovan Omorjan

Radovan Omorjan

325 сообщений из 2052 понравились пользователям.

Группа: Moderator

Hello w3bsurf3r,

Let's go again with testing

I attached a file where all solvers failed. Do not understand why. Could you take a look at it please.

Regards,
Radovan

Primer43w.png
When Sisyphus climbed to the top of a hill, they said: "Wrong boulder!"
  • Новые сообщения Новые сообщения
  • Нет новых сообщений Нет новых сообщений