Better Software - November/December 2009 - (Page 14)
Test Connection Constructing the Quality Story by Michael Bolton What we know about a product’s quality isn’t inherent in the product; our knowledge is constructed by us. In today’s world, we often construct knowledge by means of experiments that we call “tests”; yet, at one point in history, the experimental approach was both new and controversial. That controversy is outlined in a book, Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life by Steven Shapin and Simon Schaffer. The story begins in 1659, when Robert Boyle and his colleague Robert Hooke finished building a version of the most sophisticated and complex scientific instrument of its time, the air pump. While trying to learn about air, Boyle realized something that we now regard as commonplace: If you want to understand how a system is affected by something, get rid of that thing. In 1660, at roughly the same time as the Royal Society was established, Boyle published the results of several experiments in which he removed the air from the “receiver”—a chamber in the air pump—and observed the effects on animals and objects that he had placed inside. More significantly, his writings discussed how a community could arrive at a matter of fact—something upon which everyone could agree without dispute. Boyle proposed three main points. First, the advance of scientific knowledge would depend upon instruments that would simultaneously extend human observation and remove human subjectivity. Second, experiments should be performed in front of groups of people who could observe the proceedings and the experimenter, and bear witness to accounts of what had happened. Third, a style of recording and writing should be adopted so that anyone else (with sufficient skill and funding, presumably) could reproduce the experiment. Thomas Hobbes, a prestigious natural and political philosopher of the day, had serious objections to the ex14 BETTER SOFTWARE perimental approach. He didn’t object to experiments per se, but at the same time, he didn’t believe that they presented conclusive evidence of anything. For one thing, air pumps were very unreliable. Although Hooke and Boyle spent years trying to improve theirs, it leaked persistently. Hobbes also dimissed the idea that an experiment proved anything definitively—other than the fact that the air pump didn’t work well. That was easy to demonstrate because of the leaks and the fact that different experimenters obtained different results. Hobbes also believed that the air pump couldn’t work to produce universal knowledge. Even in the event that the air pump did work here and now, how do you know that it will work somewhere else later? You can’t, unless or until you actually try it, said Hobbes, so the “knowledge” that you create isn’t forever and always. Experimental knowledge is always provisional. Hobbes also pointed out that the air pump experiments and the theories that they were intended to prove had a kind of circularity to them. The air pump had its effects because Boyle’s theories were true, and Boyle’s theories were true because the air pump proved them. Hobbes may not have been the first philosopher to notice this loop in science (which, at the time, wasn’t yet called science), but he was vocal about it, and he certainly www.StickyMinds.com wasn’t the last to raise the problem. Hobbes’s principal issue was that people would not agree on the evidence of experiments if their interests were at odds. His own objection alone was proof of that. Hobbes believed that true knowledge should be based upon axioms and reasoned analysis that is derived from them—the foundation of his philosophy was geometry. He had alternative and (at the time) plausible theories to explain Boyle’s observations. Hobbes had a stake in maintaining his beliefs and wasn’t going to give them up based on the seventeenth-century equivalent of a product demo. It didn’t help Hobbes’s argument that his attacks were exceptionally nasty and personal. Boyle had widespread support from his colleagues in the Royal Society, and both Hobbes and his point of view were marginalized. The experimental approach, though imperfect, proved sufficiently useful to help generate new explanations for the way the world works, and the objections of Hobbes and several other critics were largely forgotten by history. Yet, Hobbes should be credited for sharpening the scientific method. Boyle and his colleagues were compelled to acknowledge and deal with Hobbes’s criticisms, which led to refinement of the equipment on the one hand, and more explicit and more guarded truth claims about experiments on the other. NOVEMBER/DECEMBER 2009 ISTOCKPHOTO
For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.