Tuesday, February 21, 2012

OR and Base Voters: Common Pitfalls

My adopted state of Michigan is currently afflicted with the Republican presidential primary. (Symptoms include repetitious attack ads on television, robocalls to one's house, and the general malaise associated with staring at any crop of candidates for political office.) Primaries tend to draw out "base" voters (those committed to one party or the other); we swing voters just stay at home, hiding under the covers until it is over.

Last night the local TV news included a sound bite from a generic Republican voter, an apparently intelligent and articulate woman (to the extent one can judge these attributes from a two sentence interview) who said she was still undecided because she wanted to vote for the "most conservative" candidate. The logic, or lack of logic, behind that statement caused me to take notice of the similarities between how some "base" voters think and common errors in operations research.

A single criterion is easy, but multiple criteria may be correct. There are quite a few pressing issues these days, ranging from foreign policy to budget deficits to global warming to unemployment to ... (I'll stop there; I'm starting to depress myself). Our base voter, henceforth Mme. X, has apparently condensed these criteria down to a single value, on a scale from hard core liberal (arbitrarily 0) to hard core conservative (arbitrarily 1). What is not apparent is how the multiple dimensions were collapsed to a single one. OR people know that multiple criterion optimization is hard, more from a conceptual standpoint than from a computational one. Using a single composite criterion (weighted sum of criteria, distance from a Utopia point in some arbitrary metric, ...) makes the computational part easier, but there are consequences (frequently hidden) to the choice of the single criterion. Goal programming has its own somewhat arbitrary choices (aspiration levels, priorities) which again can have surprising consequences. Picking the "most conservative" candidate simplifies the cognitive process but may lead to buyer's remorse. Similarly, arbitrarily collapsing multiple objectives into a single objective may simplify modeling, but may produce solutions that do not leave the client happy.

Averages can be deceptive. Point estimates also make modeling and decision making easier, but they can mask important things. (A colleague has a favorite, if politically incorrect, quotation: "Statistics are like bikinis. What they reveal is interesting, but what they conceal is critical.")

Suppose that Mme. X has narrowed her choices down to two candidates, and that they have both weighed in on five important issues (A through E). If candidate 1 is consistently to the right of candidate 2 on all issues, we have a dominated solution: Mme. X can eliminate candidate 2 and vote for candidate 1. On the other hand, consider the following scenario, where each candidate's position is rated on a scale from 0 (liberal) to 1 (conservative).
Candidate 1 is more conservative than candidate 2 in both mean (0.780 versus 0.756) and median (0.80 versus 0.75); yet candidate 2 is to the right of candidate 1 on two of five issues (A and B), and close to a wash on a third (C).  So if Mme. X truly wants a conservative candidate, it is not all that clear which she should prefer. Likewise, OR models that consider only point estimates without taking dispersion into account can result in solutions that should do well "on average" but sometimes do quite poorly.

A solution that goes unimplemented is not a solution. Missing in Mme. X's search for the most conservative candidate is the quality referred to by pundits as "electability". Neither major political party claims a majority of registered voters in the U.S., so to win a general election, a candidate must capture a significant number of moderates and independents. The most ideologically pure candidate (for either party) may not be able to do so. This is a bit of a paradox in recent elections, where candidates find that they must appeal to "base" voters at one end of the political spectrum to get the nomination, then appeal to voters in the middle of the spectrum to win the election. Ideological "base" voters may not grasp this particular reality; they expect the "correctness" of their candidate's views (which are also their views) to triumph. [This may be at least partly explained by the false consensus fallacy.]

OR modelers sometimes have a similar blind spot. We can pursue perfection at the expense of good answers. We can opt for the approach that uses the most sophisticated or "elegant" mathematics or the most high-powered solution technique available. We may try for more scope or more scale in a project than what we can accomplish in a reasonable time frame (or what users can realistically cope with, in terms of data requirements and solution complexity). Professional journals often encourage this trend by requiring "novel" solution methods in order to publish a paper. The end result can be a really impressive solution that sits on a shelf because the client is unwilling or unable to implement it, or because it is too complex for the client to understand and trust.

Garbage in, garbage out. OR models rely on data, as inputs to the decision process or to calibrate parameters of the model. Feed bad data to an otherwise correct model and no good will come of it.  I have seen estimates that as much as 60% of the time in an OR project can be spent cleaning the data.

Meanwhile, Mme. X has to rely on a variety of unreliable sources to gauge how conservative each candidate may be. Candidates famously say things they may not entirely believe, or express intentions they may not carry out, either in an overt effort to curry voters or because their views change between campaigning and governing. Historical data may be faked or misreported, and sometimes facts may not be what they seem. For instance, a generally pro-military candidate might vote against a military appropriation bill because there is a rider on it that would fund an inordinately wasteful project, or something unpalatable to the candidate and/or the candidate's constituents. Opponents will characterize this as an anti-military stance. Budget projections, and indeed any sort of projections, are subject to forecast errors, so a candidate's magical plan to fix deficits/unemployments/Mme. X's dripping kitchen faucet may turn out not to be so magical after all. Unfortunately for Mme. X, she probably has less ability to filter and correct bad data than an OR analyst typically does.

So, in conclusion, voters and OR analysts face similar challenges ... but OR analysts do not have to cope with a glut of robocalls.

Monday, February 13, 2012

Java Plugin for Firefox 10

Today marked yet another upgrade in Firefox, from version 10.0 to 10.0.1.  Whether it was this upgrade or the previous one I cannot be sure, but I discovered (the hard way) that one of them broke the use of Java applets on my office PC.  Specifically, the Java plugin (which I use infrequently, but which had worked a while back) was simply gone.

Well, the plugin itself (which comes from the portion of Oracle that once was Sun) couldn't be gone -- Firefox can't delete chunks of my JRE.  Only the link to it was gone.  There is a handy Synaptic meta-package (sun-java6-plugin) in one of the Ubuntu repositories, intended (I think) to make the connection between plugin and browser.  I reinstalled it, to no avail. I'll skip the full litany of web search results (many out of date) and failed attempts, and cut to the chase scene.  Besides having a recent Oracle/Sun Java JRE installed (and, of course, Firefox), you need to do something to tell Firefox where to find the Java browser plugin.  The rest of this is specific to Linux; Windows and Mac users will have a somewhat different task.
  1. You need to find a Firefox plugins directory, in which to place a symbolic link to the plugin.  There are multiple choices, and more than one of them may be suitable. Some documents refer to a plugins folder under ~/.mozilla (although I suspect this now should be ~/.mozilla/firefox/<your configuration>).  I didn't have such a folder, so I created one (although I ended up not using it). Another possibility is /usr/lib/firefox/plugins. I'm not sure if that one is scanned, or if it's just vestigial from some earlier version. I ended up using /usr/lib/firefox-10.0.1/plugins ... which I may regret if I have to go through this again with the next upgrade.
  2. You need to find the right plugin file. This is where I wasted most of my time. Older documentation refers to libjavaplugin_oji.so, which I found in /usr/lib/jvm/java-6-sun/jre/plugin/i386/ns7. It's in a folder named "plugin", so it must be the right file, mustn't it? Oops, no. It turns out there's a "next generation" plugin named libnpjp2.so, and apparently FF 10 will only recognize the new plugin. Oracle's installation docs refer to it, but I could not find it in the plugin directory. That's because it's not in the plugin directory. I guess they thought that would be too obvious. It's in /usr/lib/jvm/java-6-sun/jre/lib/i386.
  3. Once the targets are in sight, shut down Firefox, go to the directory you selected in step 1 (in a terminal), delete any symlinks to older plugins, then  execute sudo ln -s /usr/lib/jvm/java-6-sun/jre/lib/i386libnpjp2.so, adjusting the path as needed if your JVM is installed elsewhere. (The sudo is unnecessary if you chose a folder in your home partition in step 1.) Restart Firefox and check the plugins tab (or enter about:plugins in the URL bar) and make sure Firefox acknowledges the plugin.
Update: I just checked my home PC (same configuration as the office PC, other than being 64 bit rather than 32 bit). There's no problem with the Java plugin at home. So perhaps I was wrong in blaming a FF update at the office for zapping the connection to the plugin ... but if not an update, then what?

Saturday, February 11, 2012

The Case of the Dormant Function Keys

There's an intermittent problem on my home PC (Linux Mint) that's been driving me slightly buggy (okay, slightly buggier).  My mail client is Thunderbird, and when composing messages I frequently hit the F9 key, which should open the "Contacts Sidebar", from which I can select recipients.  Usually it does.  On my office PC and laptop it always does.  At home, though, it sometimes doesn't -- the F9 key goes mysteriously dead in T-bird.  I can work around it by opening the contacts sidebar from a menu, and eventually the problem disappears (for a while).

Today, though, it happened again and I decided to try to sort it out.  Testing with other programs showed that all the function keys, not just F9, were dead, and in all applications.  I checked system settings in the control panel, and could find nothing that would be remapping the F keys.  A bit of work with Google, though, turned up the culprit: my keyboard.

I have a Microsoft ergonomic keyboard at home (and only on the home PC). Microsoft decided to assign alternate functions to all the F keys, so that, for example, F9 does double duty as the "Send" key (with a different key code).  At the end of the row of function keys is an "F Lock" key.  (For reasons that will become clear, I cannot get out of my head another name for it, also starting with "F" and ending with "ck".)  The F Lock key toggles function keys between their normal key codes and the alternate codes.  It apparently cannot be controlled by the operating system -- or at least nobody has figured out how.  Microsoft being Microsoft, they decided that the F Lock key has to be on for function keys to act as function keys (which seems to suggest that the being a function key is not the normal role of the function key).  Whatever.

The problem, as reported by various people, is that the F Lock apparently toggles randomly, without the key being struck.  In my case, this may mean that the state of the function keys at startup is random -- the keyboard sometimes wakes with F Lock on and sometimes with F Lock off.  With the culprit identified, there are two possible resolutions.  For now, I'll try the lazy approach: do nothing special, but keep an eye on the F Lock status light.  If it becomes too hard for me to remember this (the problem is, after all, intermittent), the more involved solution is to use the xev utility (which displays keyboard events, including key codes) to make a list of the key codes of the 12 function keys, both with and without F Lock engaged, and then use the xmodmap utility to map the key codes for the non-standard functions (undo, redo, reply, send, ...) back to the same virtual keys as the function keys (so that F9 looks like F9 to the system, regardless of the status of F Lock).

Saturday, February 4, 2012

Slaying a Zombie Browser Extension

I just finished a marathon session (on Linux Mint Katya) trying to uninstall Stylish, a zombie extension for the Firefox web browser. I have nothing against Stylish, and I don't think it was the culprit in this story, but I'm recording my methods here both to preserve them for myself and in case anyone else is locked in this battle.

Somewhere along the line, I must have installed a version of Stylish, although I don't recall when or why.  As Firefox went to its current whirlwind upgrade cycle, the installed version of Stylish (1.0.7) became incompatible.  With each upgrade, I would instruct Firefox to look for a compatible version, and it would either report failure or find one, attempt to install it, and report that the upgrade attempt had failed (with no explanation why).  Either way, the browser would open with Stylish 1.0.7 installed and disabled.

This dance got boring after a while, and I could not think of a reason for keeping Stylish, so when Firefox upgraded itself to version 10.0 today I decided to uninstall Stylish ... which is when I discovered it was now a zombie.  If you know anything about zombies, you know that they are difficult to kill.  The instructions for uninstalling an extension are simple: open the extensions tab, find the one you no longer want, and click the uninstall link ... if there is an uninstall link ... and with Stylish, there was not.  Further searching revealed the helpful suggestion that one boot Firefox in safe mode and then click the previously disabled uninstall link ... except there was no uninstall link, disabled or otherwise (nor did one magically appear when I switched to safe mode).

One can invoke Help > Troubleshooting Information in Firefox and find the file or directory name of the extension, then track the file or directory down in the Firefox extensions folder and nuke it manually.  I was able to find the folder name this way (a hex string inside braces), but no folder or file of that name existed in my profile's extensions folder.  This may mean it is installed globally, and sure enough, I found a folder of the correct name in /opt/firefox/extensions.  I nuked it (as root) and restarted Firefox.  The zombie still lived (still listed as disabled).

Upon further searching, I found an SQLite database belonging to Stylish:  /opt/firefox/defaults/profile/stylish.sqlite.  I nuked it (as root) and restarted Firefox.  The zombie persisted.

Well, perhaps I was overthinking this.  With Firefox in safe mode, I installed the latest version of Stylish (1.2.5), then restarted Firefox.  Although Firefox had failed to do the upgrade on its own, this worked.  The new version was installed and not disabled ... which meant it had a functioning uninstall link ... and there was no mention of version 1.0.7.  So I clicked the uninstall link.  After restarting Firefox, I found that Stylish 1.2.5 had successfully uninstalled ... and the 1.0.7 zombie had reappeared.

So I closed Firefox and opened the extensions.sqlite database (in my Firefox profile folder) using the SQLite database browser.  Here I was able to find an entry for Stylish 1.0.7.  It pointed to a directory (with the same hex string name as the one I'd nuked from /opt/firefox/extensions), this time located in /usr/lib/firefox-10.0/extensions.  Nothing I'd turned up in any of my Google searches pointed to /usr/lib (other than perhaps references to "globally installed" extensions, with no indication of where that would be -- hence my earlier search of /opt/firefox).  I deleted the newly found subdirectory (as root), restarted Firefox, and woo-hoo!  No sign of the zombie.