Philosophy and Qualiteers Code Inspection

Here's an excerpt from the book "Darwin's Dangerous Idea - Evolution and the Meanings of Life" by philosopher Daniel C.Dennett

"2. Judging the Competition
Your Philosophy Department has been chosen to administer a munificent bequest: a twelve-year fellowship to be awarded in open competition to the most promising graduate student in philosophy in the country. You duly announce the award and its conditions in the Journal of Philosophy, and then, to your dismay, you receive, by the deadline, 250,000 legal entries, complete with lengthy dossiers, samples of written work, and testimonials. A quick calculation convinces you that living up to your obligation to evaluate all the material of all the candidates by the deadline for announcing the award would not only prevent the department from performing its primary teaching mission, but - given the costs of administration and hiring additional qualified evaluators - bankrupt the award fund itself, so that all the labor of evaluation would be wasted; no one would gain.

What to do? If only you had anticipated the demand, you could have imposed a tighter eligibility conditions, but it is too late for that: every one of the 250,000 candidates has, we will suppose, a right to equal consideration, and in agreeing to administer the competition you have undertaken the obligation to select the best candidate. (I don't mean to beg any questions with this formulation in terms of rights and obligations. If it makes a difference to you, recast the setting of the problem in terms of the overall disutility of violating the conditions set forth in your announcement of the competition. My point is that you would find yourself in a bind, whatever your ethical persuasion.) Before reading on, please spend a little time, as much as you think it takes, to plot your own solution to the problem (no fantasies about technological fixes, please).

When I have put this problem to colleagues and students, I find that, after a brief exploratory period, they tend to home in on one version or another of a mixed strategy, such as:

(1) choose a small number of easily checked and not entirely unsymptomatic criteria of excellence - such as Grade Point Average, number of philosophy courses completed, weight of the dossier (eliminating the too-light and the too-heavy) - and use this to make a first cut;
(2) conduct a lottery with the remaining candidates, cutting the pool down randomly to some manageably small number of finalists - say fifty or a hundred -
(3) whose dossiers will be carefully screened by a committee, which will then vote on the winner.

There is no doubt that this procedure is very unlikely to find the best candidate. Odds are, in fact, that more than a few of the losers, if given a day in court, could convince a jury that they were obviously superior to the elected winner. But, you might want to retort, that's just tough; you did the best you could. It is quite possible, of course, that you would lose the lawsuit, but you might still feel, rightly, that you could have arrived at no better decisions at the time.

My example is meant to illustrate, enlarged and in slow motion, the ubiquitous features of real-time decision making. First, there is the simple physical impossibility of "considering all things" in the allotted time. Note that "all things" doesn't have to mean everything or even everybody in the world, but just everything in 250,000 readily available dossiers. You have all the information you need "at your fingertips"; there need be no talk of conducting further investigations. Second, there is the ruthless and peremptory use of some distinctly second-rate cut rules. No one thinks Grade Point Average is a remotely foolproof indicator of promise, though it is probably somewhat superior to weight of dossier, and clearly superior to number of letters in surname. There is something os a trade-off between ease of application and reliability, and if no one can quickly think of any easily applied criteria that one can have some faith in, it would be better to eliminate step (1) and proceed straight to the lottery for all candidates. Third, the lottery illustrates a partial abdication of control, giving up on a part of the task and letting something else - nature or chance - take over for a while, yet still assuming responsibility for the result. (That is the scary part.) Fourth, there is the phase where you try to salvage something presentable from the output of that wild process; having oversimplified your task, you count on a meta-level process of self-monitoring to correct or renormalize or improve your final product to some degree. Fifth, there is the endless vulnerability to second-guessing and hindsight wisdom about what you should have done - but done is done. You let the result stand, and go on to other things. Life is short.
(Dennett 1995, ch. 17, emphasis author's)"

It made me think how in my version of code inspection, there are clear and simple cut rules to decide, which pieces of code to check more carefully. Unlike in static analysis tools, like lint, which suffer from too many false positives, the human aspect in Qualiteers Code Inspection helps to limit the findings to the most important ones. There is also the right of the author to ignore any of the findings, which makes him an arbitrator, a randomiser of sorts. And then the author decides how to change something and fixes it in one way. The way he fixes it might not be the best, and the place he changes might not be the place that would most need to be changed and improved, but he is the expert. That is how life is. You always have to make decisions under time pressure. Life truly is short.

If you have good rules and good people, you have a good chance of making the best of it.