I'm not a potential customer for this, but i have worked on a few commercial projects involving combinatorial optimisation.
Misc thoughts:
- I'm not familiar with the LABS problem, but the LABS benchmark page is interesting & compares against Gurobi. I'd be curious to see how an existing commercial non-mip approximate solver such as Hexaly (formerly LocalSolver) compares here.
- the other two benchmarks aren't very convincing as they don't compare against other methods or show running times
- the front page mentions peer reviewed methodology - consider linking to the publications
- good idea to have case studies of applications. I was a bit confused to see this listed under 'References' but on comparison the Gurobi & Hexaly marketing websites also do this (references -> case studies & references -> customer stories, respectively)
- re the client API, you may want to make the server URL have a default, so your trial users / customers don't have to specify it. It may be easier for you to roll out changes to your server URL in future if you can do it by changing the default server URL in a new version of your client library rather than requiring your customers to update their source code.
Really not trying to be cheeky... but why? Who is the audience here? I can see maybe academics with small grants and want to do the absolute minimum spend on compute... But that is an audience you will have to fight for every cent.
This doesn't solve or provide guidance for the subtle problems in these otherwise opensource solvers... The first example requires the client to manually disambiguate equivalent variables to get a stable solution... Sure that's a pretty common problem everyone working with optimizers should be familiar with but they're also one of the hardest things to track down in a complex derived model.
This may be useful for small demos. For large scale MIP with millions of variables, one needs to have the solver at hand to support custom algos with techniques such as column generation, etc. to achieve time to solution and economics of compute resources. A remote API will not fit.
It seems to just be a wrapper over or-tools and other solvers from their landing page, with the difference being it run on their servers versus your hardware. Their website does not mention what hardware is allocated per model (which determine speed of solving) nor any limit on model size.
I personally disagree with "no free lunch"; (for the uninitiated, "no free lunch" refer to the fact for any deterministic algorithm, there exist a problem that will force the algorithm to go through the entire solution space to find the optimal solution, with every single other possible algorithm beating it (https://en.wikipedia.org/wiki/No_free_lunch_theorem)). For many planning problems, finding a good enough solution is sufficient, and there are many optimization algorithms that work for a wide variety of problems and provide a good enough solution in reasonable time. Different algorithms are better for different problems (ex: Metaheuristic (ex: Late Acceptance) Solvers beats MIP Solvers on vehicle routing, whereas MIP Solvers beat Metaheuristic Solvers on Employee Scheduling and Bin Packing. But both Metaheuristic and MIP Solvers provider good enough solutions for both vehicle routing and bin packing.
No free lunch theorem has nothing to say about approximate solutions, so I'm really not sure what you're going on about.
OR-tools is almost exclusively linear programming which according to its strict assumptions converges more or less trivially, assuming a correctly composed program.
Which means if you're paying for it "as a service" you all but deserve to lose that money.
> Different algorithms are better for different problems
So... why does your rhetorical style have such oppositional tone if you're just going to reaffirm the no free lunch theorem?
I'm not a potential customer for this, but i have worked on a few commercial projects involving combinatorial optimisation.
Misc thoughts:
- I'm not familiar with the LABS problem, but the LABS benchmark page is interesting & compares against Gurobi. I'd be curious to see how an existing commercial non-mip approximate solver such as Hexaly (formerly LocalSolver) compares here.
- the other two benchmarks aren't very convincing as they don't compare against other methods or show running times
- the front page mentions peer reviewed methodology - consider linking to the publications
- good idea to have case studies of applications. I was a bit confused to see this listed under 'References' but on comparison the Gurobi & Hexaly marketing websites also do this (references -> case studies & references -> customer stories, respectively)
- re the client API, you may want to make the server URL have a default, so your trial users / customers don't have to specify it. It may be easier for you to roll out changes to your server URL in future if you can do it by changing the default server URL in a new version of your client library rather than requiring your customers to update their source code.
All the best!
Really not trying to be cheeky... but why? Who is the audience here? I can see maybe academics with small grants and want to do the absolute minimum spend on compute... But that is an audience you will have to fight for every cent.
This doesn't solve or provide guidance for the subtle problems in these otherwise opensource solvers... The first example requires the client to manually disambiguate equivalent variables to get a stable solution... Sure that's a pretty common problem everyone working with optimizers should be familiar with but they're also one of the hardest things to track down in a complex derived model.
This may be useful for small demos. For large scale MIP with millions of variables, one needs to have the solver at hand to support custom algos with techniques such as column generation, etc. to achieve time to solution and economics of compute resources. A remote API will not fit.
looks interesting, how large problem can it solve?
It seems to just be a wrapper over or-tools and other solvers from their landing page, with the difference being it run on their servers versus your hardware. Their website does not mention what hardware is allocated per model (which determine speed of solving) nor any limit on model size.
Keep it simple, just one call to solve every model.
*sigh* We really need to teach this new crop the term "no free lunch". Again.
I personally disagree with "no free lunch"; (for the uninitiated, "no free lunch" refer to the fact for any deterministic algorithm, there exist a problem that will force the algorithm to go through the entire solution space to find the optimal solution, with every single other possible algorithm beating it (https://en.wikipedia.org/wiki/No_free_lunch_theorem)). For many planning problems, finding a good enough solution is sufficient, and there are many optimization algorithms that work for a wide variety of problems and provide a good enough solution in reasonable time. Different algorithms are better for different problems (ex: Metaheuristic (ex: Late Acceptance) Solvers beats MIP Solvers on vehicle routing, whereas MIP Solvers beat Metaheuristic Solvers on Employee Scheduling and Bin Packing. But both Metaheuristic and MIP Solvers provider good enough solutions for both vehicle routing and bin packing.
No free lunch theorem has nothing to say about approximate solutions, so I'm really not sure what you're going on about.
OR-tools is almost exclusively linear programming which according to its strict assumptions converges more or less trivially, assuming a correctly composed program.
Which means if you're paying for it "as a service" you all but deserve to lose that money.
> Different algorithms are better for different problems
So... why does your rhetorical style have such oppositional tone if you're just going to reaffirm the no free lunch theorem?