Top hackers, we need you for Operations Research

Bold statement: A top 1% Operations Research practitioner will lose against a top 10% programmer in a hackathon. Unfortunately, I don’t have any emperical evidence to back this up, but my own humble hunch.

Nothing to be ashamed of, since an OR degree is multi-disciplinary, hence diluted across the many facets of the field — CS included. It’d be good for the OR industry if we all really learn to code.

I have a MSc in OR, but my programming skills were nowhere near the average CS grad, if it weren’t for my sabbatical studying Scheme, Common Lisp and Ruby on Rails to fill a void in my skill set. We were simply not taught (enough) real programming at school beyond the absolute minimum skills required to prototype your algorithm/theory — and churn out your school paper.

Ironically, we do learn about mathematical programming, linear programming, quadratic programming, etc.

Operations Research is a fascinating field of applying Mathematics, Statistics, Machine Learning, Artificial Intelligence, Simulations, etc. to solve real-life business problems. Awesome stuff, but how are you going to apply OR without programming?

Pretty tough. Unless you consider Excel’s solver to be Operations Research. Either that, or one of the plethora of modelling tools that enable ORites to circumvent coding to quite an extent, but which come with a price tag (open-source packages like R aside), but which might not fit the need of the problem you’re tackling.

Quite often, modelling approaches (I consider Matlab and R modelling) won’t suffice to address the complexity of a problem/approach and would require a custom implementation. The only language I could code in at the time was Java, which I didn’t like, so I decided to learn Lisp.

“Lisp is worth learning for the profound enlightenment experience you will have when you finally get it; that experience will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot.” – Eric Raymond in “How to Become a Hacker.” Note that any language of your choice would do. I opted for Lisp for the above reason — and the below.

I blogged about the (Un)common Lisp Approach to Operations Research, where I made a point why I deem Lisp utterly suitable for rapid prototyping in OR.

Sim Cheng Hwee, president of IDSC — an OR consultancy in Singapore replied with: “I’ve not tried Lisp, but [while] it is possible to program a solution to solve a problem fast, […] one gets caught up with the database links, working the GUI, etc since users must be able to view, adjust their inputs and understand a solution before they can accept it and act on it.”

He made a good point. For OR to really make an impact (i.e. for an organization to adopt), we need to build a beautiful and easy to use GUI, link it robustly with scalable databases and these days deploy it to the cloud — while maintaining an elegant code base. In other words, we need some real hackers in the field.

It seem that in most cases, OR projects end up requiring 80% programming skills and 20% OR skills; the latter of which I’ll vaguely describe as anything OR related that CS grads don’t know, but nonetheless can pick up easily and much faster than ORites learn to code — elegantly anyway.

The Hacker vs the ORite

For now, let’s zoom in to the 20% with an example of one of my favourite puzzles in OR: Solving the Vehicle Routing Problem. Put simple, how do you minimize the total distance, while visiting all the clients within the time and capacity constraints?

Now, having an understanding of the problem at hand, the programmer would go and implement a library/platform to start and try solving the problem as good as possible. The ORite will put the rather simple problem into a mathematical framework and start “coding” in Latex. It’d kinda look like this:

Note that this is yet the simple VRP problem, without capacity nor time-window constraints, which gets even more convoluted — see chapter 2 of Joubert 2007

Impressive, at least that’s what I always thought being a smug student back in the days, proudly turning in fancy papers with equations so beautiful it “makes you want to lick it”. In retrospect, those equations were completely unnecessary to the end product. I did not look back at these equations once, nor could I really use them in my coding endeavours, i.e. when it really mattered.

The end product was a model and algorithm written in Java to test the hypotheses. The final code base was not something I was proud of. Frankly, it was barfed up Spaghetti code. It was a quick and inelegant prototype, because I didn’t know better. A year later, I could not even get it to work (and I couldn’t stand Eclipse after getting used to Emacs).

Shame on you, hackers would say. Yes, shame on me, but I wasn’t trained to be a programmer. I’m still just an aspiring computer nerd, but I decided to do it over, this time as an Open Source VRP modelling framework written in Common Lisp — in an attempt to catalyze the research and application of routing solutions. (Feedback from real hackers would be highly appreciated.)

But the point is, I still prefer the easy-to-digest variant of the problem statement: “How do you minimize the total distance, while visiting all the clients within the time and capacity constraints?” That’s enough for me to start tackling it.

Solving the VRP

There are so many different solution approaches to the VRP, from Genetic Algo’s to Ant Colonies and from Simulated Annealing to Hybrid Algorithms — it’s not even funny anymore — but let’s not get into that. For the sake of illustration, let’s take arguably one of the most popular/powerful algorithms called Tabu Search. It is a very simple local search algorithm that I could attempt to teach a 12-year old. The pseudo-code is on the wiki page. Now let’s face it: who would do a better job implementing the Tabu Search algorithm — or any other (meta)heuristic for that matter — the hacker or the ORite?

Not to turn this into a Socratic Dialogue — which I’m doing anyway — but what do you think is the most important part of an OR publication, the mathematical formulation or the actual coding of the algorithm/approach?

Which one do you think decides whether or not your algorithm will outperform other papers? (The question answers itself, it appears).

Which one do you think will make an real tangible impact?

Remember, I’m now talking about the 20% OR part of the project, but as corroborated by Sim Cheng Hwee, 80% of the actual programming still needs to happen before the powerful ideas of OR are actually being applied.

And ultimately: Who would you hire for an OR project, the hacker with some OR knowledge or the ORite with some programming knowledge?

That’s my point. ORites should embrace more real programmers and programmers should try some OR. Some cross-pollination between CS and OR would be pretty explosive. But it needs to come from both ways. Top hackers in dire need of ideas, why don’t you have a look at the fascinating field of OR? There’s quite an opportunity there. You’ve got all the skills it takes. And we need you.

Discussion on Hacker News, OR-Exchange and a great one on Google+.

18 thoughts on “Top hackers, we need you for Operations Research

  1. If some one is getting his\her Ph.D in O.R and doesn’t know how to code. Shame on him\her. One can easily learn a language in 4 month. If he\she pushes himself to the limit. OOP is critical in OR. I mean critical here. Any complex algorithm requires OOP and Functional programming structure assembled together. For instance, ACO is a combination of fast heuristic (usually Greedy), Local search (usually 2-opt or 3-opt) Tabu search together. That’s why it is called a “meta heuristic”. When you encounter OR problems, the first thing you are going to do is, evaluate the problem of its computational complexity. You should be able to tell immediately that whether this problem is NP-Hard or not. Then you are going to tell very fast whether this problem could be solved by an commercial IP solver (Fico Xpress, IBM Cplex, etc) or not. If yes, use the IP solver, If not, use heuristics.

    Talking about heuristics, if you don’t know how to code, how could you use heuristic s? If you can not implement heuristics in solving OR problems, how can you claim yourself an OR practitioner?

  2. Great post, thank you. My question is, which programming language do you suggest ORites should learn (besides Lisp). I have experience with Maple and am learning R now. I would appreciate everyone’s advice.

    • I’m sure there is nothing wrong with any of the above (disclaimer, I’ve only experienced Matlab and R). They are very powerful tools for purposes that they are designed for — it just depends on your goal.

      My goal was to rapidly test a custom algorithm, therefore required custom coding. I’ve actually written a custom hybrid genetic algorithm in Matlab before, only to conclude never again. Why? It is *very* slow and octave is not your typical language you’d resort to for coding.

      So it depends on what paradigm you are working, whether it’s Programming or Modelling:

  3. I think that coding is something young OR practitioners must pick up but as they get more senior, the soft skills such as political awareness & ability to persuade becomes more important than the quality of code produced. If coding prowess is all it takes, you would have seen OR applications everywhere.

    • That’s a good point, Mr. Sim. The importance of soft-skills are definitely the most important factor in the real-world in terms of making a difference. Being in consulting myself, the biggest challenge is indeed to engage senior management in some sort of discussion, let alone convey the value of OR.

      Perhaps the value to good coding may be that it reduces the required time of a project, reduce maintenance requirements afterward and hence the total cost of an engagement, which may lead to a lower threshold for potential clients to engage with OR consultants?

      I suppose costs are no determinant if there is no conversation..

  4. Is anyone in the OR-World using F# in combination with the “easier” languages within the .NET universe?

    So far I don’t know much about F#, but as a functional language, it should be pretty easy to implement mathematical algorithms with it and since it uses the huge .NET library, there should be lot’s and lot’s of specialized packages one could use to fasten the algorithm. And the main advantage is that it can be easily combined with programs written in other .NET languages like C# which are better for designing and implementing interfaces, ui’s, etc.

  5. I totally agree with this article: an OR guy without programming skills is pretty much useless 😦

    I am working on an open source OR framework in Java, called Drools Planner, and the fact is that most code contributions come from enterprise programmers with little OR training. Most of the real OR guys can barely clone the git repository and build it from source with Maven (a Java build tool), let alone contribute code that can withstand the peer review to be accepted in the project. Then again, every real OR guy seems to suffer from NIH syndrome and prefers writing his own OR framework (abandoning it after 2 years when the research funding stops).

    Nice work with the vehicle routing problem. Here’s my vehicle routing example in the cloud (on a very slow free server):

  6. The optimal one is, ORites who knows much about programming or hacrkers with sufficient OR knowledge 🙂 I totally agree your point here, it was my motivation to focus on computer while in undergraduate years. Realizing that, all OR problems needs CS knowledge to solve (on computer), I saw that GUIs are really bad, and I thought the last person who have adequate knowledge about both should be very old. Just by looking interfaces was really sufficient to understand the situation.

    I hope to become a MSc graduate within one month, and I think my programming knowledge is above an average OR student/assistant. We should not restrict ourselves to programming languages, even web languages or script languages (javascript, ajax and at its nirvana, jquery) are incredibly useful. I am very happy to read your post, it could be a good starting point for convincing ORites to learn some programming.

    Nice post, thank you.

    • Thanks — That’s great Sertalp, you seem to be gearing up quite well. Knowledge of web languages/technologies are certainly useful/necessary, especially since anything that’s not running as a service on the cloud these days is frowned upon as legacy.

      Good luck with your graduation!

      • As you talk about the problem, perhaps you think about potential solutions for ORites to learn code faster/easier. Do you have any plans for writing some posts about it for OR people? (Tricks or guidelines for coding, documenting etc.)

What say you?

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s