Should You Trust Your Analytics Model?

The short answer is "no". At least, don't put too much faith in them.

"Remember that all models are wrong; the practical question is how wrong do they have to be to not be useful."  (George E. P. Box)

It's also useful to remember that your model, no matter how sophisticated or how well supported by theory, is calibrated using data that might be a tad suspect.

"The government are very keen on amassing statistics. They collect them, add them, raise them to the nth power, take the cube root and prepare wonderful diagrams. But you must never forget that every one of these figures comes in the first instance from the chowky dar (village watchman in India), who just puts down what he damn pleases."  (Stamp's Law)

All that said, there was a very nice entry recently in the HBR Blog Network, "How to Tell If You Should Trust Your Statistical Models", that is well worth reading. The author is Theodoros Evgeniou, Professor of Decision Sciences and Technology Management at INSEAD. (Tip of the hat to +Aleksandr Blekh for providing the link on Google+.)

The Business of Moving Stuff

In my last post, I linked some articles about the role of operations research in planning/scheduling the movement of people. It occurred to me that the movement of packages and goods provides some more examples of what we OR types are up to. Fortuitously, I came across a link to a relevant article published earlier this summer in Fortune: "The shortest distance between two points? At UPS, it's complicated". The article discusses, in somewhat general terms, the use of OR models by UPS to route delivery vehicles. It contains an assessment of the impact of Orion, a routing/scheduling system developed by UPS:

By the end of 2013—after being applied to just 10,000 routes so far—Orion had already saved 1.5 million gallons of fuel and 14,000 metric tonnes of CO2 emissions.

The article omits any mention of the related problem of loading the vehicles. The order that packages are loaded (and unloaded) makes a difference in the time a driver spends at a delivery location, and of course time is money.

A related (but possibly even more complex) application is the operation of container ships and container terminals (ports). Deciding the order in which containers are loaded on a container ship (which in turn dictates the order in which they are unloaded) is complicated enough, since it impacts the sequencing of trucks making drop-offs or pickups if the containers move directly between truck and ship, and impacts the management of piles of containers in the yard when the containers do not move directly between truck and ship. Unfortunately, I was unable to turn up a "popular press" article describing the use of planning tools (OR) in container ship/terminal operations, although I found no shortage of technical papers, book chapters and promotional materials (by companies selling software or consulting services for port management). The best I could come up with is a Wikipedia entry which states that

In recent years methodological advances regarding container terminal operations have considerably improved. For a detailed description and a comprehensive list of references see, e.g., the operations research literature.

(Take a bow, OR people.) It then provides a couple of links to literature reviews.