Friday, March 14, 2014

CP Optimizer, Java and NetBeans

After years of coding CPLEX applications in Java, I've just started working with CP Optimizer (the IBM/ILOG constraint programming solver) ... and it did not take me long to run into problems.

As with CPLEX, you access CP Optimizer from Java through the Concert API. As always, I am using the NetBeans IDE to do the coding. With CPLEX, this required two steps: add a library to the project, containing the cplex.jar file (for compilation); and add an option of the form -Djava.library.path=<path to CPLEX>/cplex/bin/x86-64_linux to Project Properties > Run > VM Options (for execution). Your path will differ depending on which version of CPLEX you have, where it is installed, and what operating system you are using.

So I naively expected to have to do the same two things to use CP Optimizer, albeit with different paths and files. I was half right (which, like being half alive, is not all that useful).

The good news was that for compilation, all that was required was adding a library to the project, this time pointing to ILOG.CP.jar. The bad news was that putting the corresponding path to the binary libraries in java.library.path most emphatically did not work. I got a ton of linking errors.

I won't bore you with the details of the errors, nor my fruitless pounding on Google to find me an answer. The CPLEX Optimization Studio distribution comes with Java examples and a make file for them. I ran the make file (outside NetBeans), and everything worked. That gave me an idea of what I needed to do. Apparently, the path to the binaries cannot go in java.library.path. I'm not sure why, but I think it has something to do with the way the binaries, which are C/C++ code wrapped for use by Java, are linked. Instead, on my Linux system they have to be specified in the LD_LIBRARY_PATH environment variable.

Thus, per this earlier post, I hacked the <NetBeans directory>/etc/netbeans.conf file and added the line

export LD_LIBRARY_PATH=<path to CPLEX Studio 12.6>/cplex/bin/x86-64_linux:<path to CPLEX Studio 12.6>/cpoptimizer/bin/x86-64_linux

Note that if you have other goodies already on your library path that your NetBeans programs might need, you'll need to append them to the path. Also, I'm not sure what the equivalent of LD_LIBRARY_PATH is in the Windows world, let alone on a Mac ... but I suspect that something similar will be required.

I also added the same LD_LIBRARY_PATH export to ~/.bashrc so that, should I run one of my programs from a terminal, the correct paths will be in place.

A couple of warnings:
  • Obviously this hack will need to be updated whenever you upgrade to a newer version of CPLEX Optimization Studio.
  • This will complicate the testing of different versions of CPLEX or CP Optimizer in the same program (other than by editing netbeans.conf for each version). With CPLEX and my previous configuration, I could just switch the CPLEX library version and alter the java.library.path string in the NetBeans project properties for that project. So much for the good old days.

6 comments:

  1. Paul,

    LD_LIBRARY_PATH is an environment variable that states where shared objects can be found at runtime. (On Windows, PATH shoud be sufficient.) So, maybe Netbeans provides a way to set environment variables.

    Instead of .bashrc, it might be better to use .profile (might be depending on the window manager you use, etc.) to set it globally, then you should not need to set it in Netbeans (after reboot/relogin).

    Best regards,
    Thomas

    ReplyDelete
    Replies
    1. Thomas,

      Thanks for the tip. I tried moving the lines to .profile (and commenting them out of both .bashrc and netbeans.conf), but after a reboot neither NetBeans nor a terminal could find LD_LIBRARY_PATH. I've run into this before: a few years ago, when IBM was still using license files for the academic version of CPLEX, I put the path to the license file in .profile, but also had to put it in netbeans.conf.

      One thing I do not understand is why LD_LIBRARY_PATH in .profile did not show up when I ran printenv in a terminal. At any rate, I'm back to using both .bashrc and netbeans.conf for now, but it would have been preferable to have a single file (.profile) to edit whenever the library path changes.

      Cheers,
      Paul

      Delete
    2. Paul,

      hmm, this might have different reasons, depending on the distribution, window manager, etc.

      If you have a .bash_profile, .profile is not executed if I remember correctly.

      Maybe, your window manager does not execute .profile at all. One other hint might be /etc/profile or /etc/profile.d/, where you can put system-wide environment variables (this might also be distribution-dependant).

      Best regards,
      Thomas

      Delete
    3. Thomas,

      I think you are correct about .bash_profile, but I don't have one (just .bashrc). I'm pretty sure .profile is executed, because it adds things to MANPATH, INFOPATH and PATH that I find when I printenv in a shell. So I'm not sure why LD_LIBRARY_PATH didn't work there.

      Thanks,
      Paul

      Delete
  2. Paul,

    do you start Netbeans from a shell or from some menu?

    If you start it from a shell, this is strange. You could try to set some variable FOO=BAR in .profile and check whether it actually exists in the shell.

    Best regards,
    Thomas

    ReplyDelete
  3. Thanks for this really great post
    http://www.imeshlab.com/Core-Java.IM

    ReplyDelete

If this is your first time commenting on the blog, please read the Ground Rules for Comments. In particular, if you want to ask an operations research-related question not relevant to this post, consider asking it on OR-Exchange.