Levenberg Marquardt algorithm in C++

For start-up projects, I need a non-linear optimization C++ library, especially Levenberg Marquardt algorithm. I found two candidates: levmar and Eigen::LevenbergMarquardt.

In terms of examples, levmar provides good examples homest while Eigen::LevenbergMarquardt does poor one in their package /eigen/unsupported/test/NonLinearOptimization.cpp.

I tried both of them. First impression is levmar is better. Once I understand the structure of the Eigen::LevenbergMarquardt, I’m feeling Eigen::LevenbergMarquardt is more easy to customize.

Advertisements

11 thoughts on “Levenberg Marquardt algorithm in C++

  1. Is there any useful tutorial or example, using LMA of Eigen? I understand barely the examples of eigen.

    • I would suggest using ceres-solver [1], which provides a general interface using a set of existing linear/non-linear least square solvers. Its advantages is
      (a) we can easily define our original cost function and can switch different solvers easily [2],
      (b) official website gives friendly tutorial [3] containing simple curve-fitting as well as typical computer vision problems such as bundle adjustment,

      [1] http://ceres-solver.org/
      [2] http://ceres-solver.org/features.html
      [3] http://ceres-solver.org/tutorial.html

      • Actually I have used the Eigen library in my program, and I dont want to use another third party package. The decision was not on me.

        I have a 3D field, as:
        y = f(x_1)*f(x_2),
        and I need to curve a surface to it. But unfortunately the built-in LM of Eigen can not solve more than 2D. Do you have any idea, how can I do that?

        thanks for helping 🙂 😉

      • Hi Hamed,
        I’m wondering if you give me more detail of your problem (even fake is fine) so that I might be able to give you some idea.

        I wrote an optimization code for camera calibration, which finds 5 DoF camera intrinsic parameters + 6xN camera motion parameters, where N denotes the number of images (5+6xN unknown parameters in total). Although the unknown parameters is greater than 2, I could solve the problem. So, I don’t think LM of Eigen solver is limited to solve 2D problem. Or I misunderstand what you mean by “LM of Eigen can not solve more than 2D.”

        Cheers,

  2. I have a set of data, which are function of two independent vaiables “phi” and “theta”. I want to fit a surface, which is a simple multiplication of two Gaussian function:
    psi = [1/(S_phi*sqrt(2*pi))*exp(-0.5*((phi-phi_m)/S_phi)^2)] * [1/(S_theta*sqrt(2*pi))*exp(-0.5*((theta-theta_m)/S_theta)^2)].
    The unknown parameters are “S_phi” and “S_theta”.

    • Eigen’s LM is not limited for a 2D parameter estimation as I mentioned. See a sample code on github [1], which solves y=ax+b given a set of observation (x,y). What you should change is the definition of struct MyFunctor. In your case,
      – set phi and theta, which you know,
      – set data_phi[i] = phi[i] and data_theta[i] = theta[i],
      – set x[0] = S_phi and x[1] = S_theta,
      – fvec[i] = f(phi, x[0], data_phi[i])*f(theta, x[1], data_theta[i]), where f(mu,sigma,x) computes the value of gaussian function given mean value mu, standard deviation sigma, and data x.

      I tested a simple problem, which is similar to yours, and it worked. The code is available from [2].
      [1] https://github.com/daviddoria/Examples/blob/master/c++/Eigen/LevenbergMarquardt/CurveFitting.cpp
      [2] goo.gl/01IdPN

  3. I realllllllllllllllllllllllllllllllly thank you.
    just a small qustion: whre should I define psi, which are my experimental data?
    Maybe I havnt clearly explained. As a small example, consider:
    theta = [1,2,3]
    phi = [-1,0,1]
    psi = [1,2,3
    4,5,6
    7,8,9]
    It is my problem. Or maybe they are the same, and I connot get it. Waht do you think?

    (Thanks again)

  4. Hi,
    Its me again. It was my exams phase.

    1. So regarding to out last conversation, I do have still that problem. The Functor needs a vector functon, whereas I have a MATRIX, as I mentioned it in my last comment. It can not be converted to a vector.

    2. And also the definition of Gaussian function is different. It is not the function of experimental data, which are in your code:

    observation(dimInput,n) *= gaussian(mean[m], sigma[m], observation(m,n));.

    It should be a function of independent parameters (phi and Theta).

    3. Something very important: what is minimized in your code? LM, which is based on the LS, minimizes the summation of the difference between the experimental data and the related fitting function. But I think in your code, there is no difference.

    Thanks a lot.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s