For start-up projects, I need a non-linear optimization C++ library, especially Levenberg Marquardt algorithm. I found two candidates: levmar and Eigen::LevenbergMarquardt.

In terms of examples, levmar provides good examples homest while Eigen::LevenbergMarquardt does poor one in their package /eigen/unsupported/test/NonLinearOptimization.cpp.

I tried both of them. First impression is levmar is better. Once I understand the structure of the Eigen::LevenbergMarquardt, I’m feeling Eigen::LevenbergMarquardt is more easy to customize.

### Like this:

Like Loading...

*Related*

Thanks a lot for the information!

Is there any useful tutorial or example, using LMA of Eigen? I understand barely the examples of eigen.

I would suggest using ceres-solver [1], which provides a general interface using a set of existing linear/non-linear least square solvers. Its advantages is

(a) we can easily define our original cost function and can switch different solvers easily [2],

(b) official website gives friendly tutorial [3] containing simple curve-fitting as well as typical computer vision problems such as bundle adjustment,

[1] http://ceres-solver.org/

[2] http://ceres-solver.org/features.html

[3] http://ceres-solver.org/tutorial.html

As far as I know, Eigen does not provide sparse non-linear least squares. So, if you are interested in solving some sparse non-linear systems, Eigen is not the right choice.

sba [1] and sparseLM [2] might be another choice.

[1] http://users.ics.forth.gr/~lourakis/sba/

[2] http://users.ics.forth.gr/~lourakis/sparseLM/

Actually I have used the Eigen library in my program, and I dont want to use another third party package. The decision was not on me.

I have a 3D field, as:

y = f(x_1)*f(x_2),

and I need to curve a surface to it. But unfortunately the built-in LM of Eigen can not solve more than 2D. Do you have any idea, how can I do that?

thanks for helping 🙂 😉

Hi Hamed,

I’m wondering if you give me more detail of your problem (even fake is fine) so that I might be able to give you some idea.

I wrote an optimization code for camera calibration, which finds 5 DoF camera intrinsic parameters + 6xN camera motion parameters, where N denotes the number of images (5+6xN unknown parameters in total). Although the unknown parameters is greater than 2, I could solve the problem. So, I don’t think LM of Eigen solver is limited to solve 2D problem. Or I misunderstand what you mean by “LM of Eigen can not solve more than 2D.”

Cheers,

I have a set of data, which are function of two independent vaiables “phi” and “theta”. I want to fit a surface, which is a simple multiplication of two Gaussian function:

psi = [1/(S_phi*sqrt(2*pi))*exp(-0.5*((phi-phi_m)/S_phi)^2)] * [1/(S_theta*sqrt(2*pi))*exp(-0.5*((theta-theta_m)/S_theta)^2)].

The unknown parameters are “S_phi” and “S_theta”.

Eigen’s LM is not limited for a 2D parameter estimation as I mentioned. See a sample code on github [1], which solves y=ax+b given a set of observation (x,y). What you should change is the definition of struct MyFunctor. In your case,

– set phi and theta, which you know,

– set data_phi[i] = phi[i] and data_theta[i] = theta[i],

– set x[0] = S_phi and x[1] = S_theta,

– fvec[i] = f(phi, x[0], data_phi[i])*f(theta, x[1], data_theta[i]), where f(mu,sigma,x) computes the value of gaussian function given mean value mu, standard deviation sigma, and data x.

I tested a simple problem, which is similar to yours, and it worked. The code is available from [2].

[1] https://github.com/daviddoria/Examples/blob/master/c++/Eigen/LevenbergMarquardt/CurveFitting.cpp

[2] goo.gl/01IdPN

I realllllllllllllllllllllllllllllllly thank you.

just a small qustion: whre should I define psi, which are my experimental data?

Maybe I havnt clearly explained. As a small example, consider:

theta = [1,2,3]

phi = [-1,0,1]

psi = [1,2,3

4,5,6

7,8,9]

It is my problem. Or maybe they are the same, and I connot get it. Waht do you think?

(Thanks again)

In my code, observation denotes our observation, which is the product of two Gaussian functions. So, psi = observation if I understand correctly. Note that the optimization doesn’t work with few data.

You can actually use Gaussian Mixture Model (GMM) with 2 Gaussians. Then, you can estimate the parameters of each Gaussian by EM algorithm. See [1,2].

[1] http://docs.opencv.org/modules/ml/doc/expectation_maximization.html

[2] http://www.juergenwiki.de/work/wiki/doku.php?id=public:gaussian_mixture_models_em_algorithm_-_demo

Hi,

Its me again. It was my exams phase.

1. So regarding to out last conversation, I do have still that problem. The Functor needs a vector functon, whereas I have a MATRIX, as I mentioned it in my last comment. It can not be converted to a vector.

2. And also the definition of Gaussian function is different. It is not the function of experimental data, which are in your code:

observation(dimInput,n) *= gaussian(mean[m], sigma[m], observation(m,n));.

It should be a function of independent parameters (phi and Theta).

3. Something very important: what is minimized in your code? LM, which is based on the LS, minimizes the summation of the difference between the experimental data and the related fitting function. But I think in your code, there is no difference.

Thanks a lot.