Text:The Logic of Scientific Fallibility

Paper presented at the 8th Biennial Conference, Mental Research 113 Institute San Francisco, California, August 1st, 1987. Published in Spanish as: La lógica de la falibilidad científica. Psicoterapia y Familia, 1 (2), 41–48, 1988.

The Logic of Scientific Fallibility

As you know, ever since Galileo there have been fierce debates about whether or not certain areas are to be considered “scientific.” Even today, you can find people arguing that creationism is as scientific as the theory of evolution, or that astrology should count as a science.

One of the roots of this controversy is both parties’ claim that they have access to some kind of incontrovertible Truth. The ones derive their certainty from scripture and revelation, the others from a no less dogmatic view of science. To a dispassionate witness it may seem odd that those who want to preserve the purity of science revert, under pressure, to arguments that are indistinguishable from the arguments creationists bring up to defend their religious faith. The defenders of science will be inclined to say that the knowledge they attain is objective, whereas their opponents’ enterprises are grounded in myth.

This may have sounded plausible in the past, but recent developments in the history and philosophy of science have generated serious doubts concerning that objectivity. Nevertheless, the debate goes on, and frequently degenerates to an emotional clash of unexamined preconceptions.

From my perspective, there need not be a contest between religious faith and any scientific model—including the model of biological evolution that is perhaps the most comprehensive and successful one science has come up with. Any religious faith is a metaphysical attempt to pacify the soul. What science produces, in contrast, are rational constructs whose value resides in their applicability and usefulness. Believers may cling to what they are told or what they want to believe; scientific knowledge is of a different kind because it is tied to a method that involves experiential corroboration. Much of the fuel that keeps the controversy alive springs from the fact that many proponents of science cling to a conception of the scientific method that no longer seems tenable today.

To approach this point, I should like to tell a story. It’s a very simple little story. It reads like a fairy tale, but it is not. In fact, it is a rather serious story. I quote it from Science, the journal of the American Association for the Advancement of Science. It appeared in the issue of June 26, 1987:

In 1959, a badger broke through the security lines here at the world’s first plutonium factory (the Department of Energy facility at Hanford, in the State of Washington). The badger ignored all the warnings and dug a hole in one of the waste pits. After he left, rabbits began to stop by for an occasional lick of salt, but it was no ordinary salt they found. Before long, they scattered 200 curies of radioactive droppings over 2500 acres of the Hanford Reserve. The rabbit mess ... created one of the largest contaminated areas, one that remains hot today with cesium-137 (half-life of 30 years) and strontium-90 (half-life 28 ys.). Hanford also has trouble with ground squirrels, burrowing owls, pocket mice, insects, and plants like rabbit brush and tumbleweed. With roots that can grow 20 feet, tumbleweeds reach down into waste dumps and take up strontium-90, break off, and blow around the dry land. If the dry weeds build up and there is a brush fire, they may produce airborne contamination...1

Airborne contamination spreads over very much wider areas than even the most energetic rabbits can spread their pellets. The problem, therefore, is not a trivial one.

That badgers and rabbits dig burrows, eat certain things, and drop little turds all over the place—these are observations that our ancestors could make, and probably did make, when they lived in their caves 40 or 50 thousand years ago.

That plants grow roots and absorb chemicals from soil and subsoil has also been known for quite a while. In fact, in Milan, where I lived part of my Italian life, legend has it that Leonardo da Vinci experimented with this idea and dug some sinister substances into the soil around a peach tree, in order to see whether one could grow poisoned peaches.

How strange, you might say, that the scientists who direct the Hanford Reserve did not think of what badgers, rabbits, and tumbleweed do, when they lead their normal and quite well-known lives.—I shall try to show that, given the logic of science, this is not surprising. But first, let me tell another story, a story I am sure you have heard before, but it illustrates a slight variation from the first.

For many thousands of years the river Nile flooded the Egyptian lowlands near the Mediterranean coast at least once a year. Vast amounts of fresh water seeped into the soil, fertilized it, and created a natural pressure against the water of the sea. The floods were a nuisance and, quite apart from this, using the Nile’s water to irrigate parts of the desert up-stream seemed eminently desirable. So the Assuan Dam was built to solve these two problems. The Nile no longer got out of hand and new land up- stream could be irrigated and cultivated. For a little while the dam seemed a wonderful success of science and engineering. Then it became clear that the salt of the Mediterranean was slowly but steadily seeping into and devastating the lowlands along the coast which had fed Egypt for millennia.

I do not know whether, prior to this experience, hydrologists knew much about the balance of pressures on the level of the groundwater. They certainly had the theoretical equipment and the formulas to figure it out. Yet, they apparently did not do so before the Assuan Dam was built. Well, you may say, one can’t think of everything—the next time they’ll do better. I would agree with this. Scientists, engineers—even members of the medical profession— are capable of learning. But that is not the problem I have in mind.

Learning is usually defined as “modifying a behavior or a way of thinking on the basis of experience,” and there is, of course, the implication that the modification is towards effectiveness, efficiency, or, at any rate, something that makes it easier to attain the chosen goal.

I have no doubt that the next time a major dumping ground for radio-active waste is chosen and prepared, someone will think of the fauna and flora and of a way to keep them from spreading the poison. And when big dams were built after Assuan, someone, I’m sure, tried to work out how the water table in down-stream lands would be affected. Science and its professionals can, in fact, see more and further than the lay public—precisely because of the often uncommon experiences they have accumulated. The problem I have in mind is that they often do not look.

I would like to submit that it is, indeed, the logic of science and the scientific method that frequently stops scientists from looking outside a specific domain of possibilities. To show this, we have to agree on what it is that we want to call “scientific method.” The conventional definition of “science” is woolly, to say the least. However, I believe that a more adequate definition has now been produced. This definition of “scientific method” stems from Humberto Maturana. It is not only relatively simple, but it also serves my purpose—which is to show how scientists lead themselves up a garden path.

Maturana divides the scientific procedure into four steps:2

1. Observation. In order to count as “scientific,” an observation must be carried out under certain constraints, and the constraints must be made explicit (so that the observation can be repeated).

2. By relating the observations, a model is inductively derived—usually a model that involves causal connections. (Often an idea of the model precedes the observations of step (1) and to some extent determines their constraints.)

3. By deduction, a prediction is derived from the model, a prediction that concerns an event that has not yet been observed.

4. The scientist then sets out to observe the predicted event, and this observation must again comply with the constraints that governed observation in (1).

I am confident that all who have been trained or engaged in “doing science,” will recognize in this description the famous “hypothetico-deductive Method.” In fact, I have not heard of any scientists, conventional or not, who could not agree with this definition of “science.” Some might want it to include more or to formulate it somewhat differently, but all can accept it as a minimal description of what scientists, by and large, are actually doing.

What is new in Maturana’s break-down is that it illustrates the epistemological implications in a way you will not find in any of the textbooks on “scientific method.” The four steps make clear that what matters is experience. Observing is a way of experiencing and, to be scientific, it must be regulated by certain constraints. The inductively constructed model relates experiences, not “things-in-themselves.” The predictions, too, regard experiences, not events that take place in some “real” world beyond the observer’s experiential interface.

Seen in this way, the scientific method does not refer to, nor does it need, the assumption of an “objective” ontological reality—it concerns exclusively the experiential world of observers.

The constraints placed on “observation” are not intended to imply—as is so often stressed in textbooks—that the scientist should be “objective” in the realist sense. They are stipulated to assure that what the scientist does is described in such a way that another scientist can come and replicate it. If another scientist is able to do this, it obviously corroborates the first scientist’s observations. But any such corroboration in no way demonstrates that what has been observed is an observer-independent reality. Corroboration merely establishes an element of consensus, i.e., the kind of element that helps to create what Maturana has called a consensual domain among interacting organisms.

The conventional approach to science has always maintained that the more people observe a thing, the more “real” that thing must be. Yet, the skeptics, ever since Pyrrho in the 3rd century B.C., have produced quite irrefutable arguments against this view. And, in our time, Paul Feyerabend has spoken against it from a different perspective. In his essay How to be a Good Empiricist, he argues as follows. (I have taken the liberty of leaving out a couple of paragraphs that refer to quantum mechanics and of changing one word— I have substituted the word “model” where he has “theory”):

... assume that the pursuit of a (theoretical) model has led to success and that the model has explained in a satisfactory manner circumstances that had been unintelligible for quite sometime.

This gives empirical support to an idea which to start with seemed to possess only this advantage: It was interesting and intriguing. The concentration upon the model will now be reinforced, the attitude towards alternatives will become less tolerant.

... At the same time it is evident, ... that this appearance of success cannot be regarded as a sign of truth and correspondence with nature. Quite the contrary, the suspicion arises that the absence of major difficulties is a result of the decrease of empirical content brought about by the elimination of alternatives, and of facts that can be discovered with the help of these alternatives only. In other words, the suspicion arises that this alleged success is due to the fact that in the process of application to new domains the model has been turned into a metaphysical system. Such a system will of course be very ‘successful’ not, however, because it agrees so well with the facts, but because no facts have been specified that would constitute a test and because some such facts have even been removed. Its ‘success’ is entirely man-made. It was decided to stick to some ideas and the result was, quite naturally, the survival of these ideas.3

More explicitly than Feyerabend’s approach, Maturana’s break-down further demonstrates—and this is relevant to my proposition—the importance of the observer’s active relating (in the construction of the model) and that this relating is an inductive process. When philosophers discuss induction, they are usually concerned with the question whether or not there is a way of demonstrating that inductive inference can lead to objective ever-lasting “truth.” As a constructivist, I am, to put it mildly, not interested in that sort of “truth.” In contrast, I am very interested in how we construct the kinds of rule we use to organize our experience. In this, I go back to the original Empiricists, to Locke, Berkeley, and Hume. All are no doubt familiar with these authors’ names, but I wonder how many would claim to have actually read them. I say this, because what one reads about them in the textbooks of contemporary American psychology makes nonsense of the most important contribution the original Empiricists have made to Western philosophy. Reading Locke, one soon discovers that, while he does indeed argue against the notion of innate ideas, he also states that the source of all our “complex ideas” is not the material we get from the senses, but the mind’s reflection upon its own operations.4 The behaviorists, who often claim Locke as the father of their naive realist brand of empiricism, never mention this second source of ideas. They consider it heresy to mention the mind. Yet—and this is a point of enormous importance—if there were no mind to relate experiences and to become aware of its own particular ways of relating, there would be no complex ideas, no models and no scientific theories. Hume, then, in spite of some nasty things he said about his predecessor, took Locke’s injunction seriously. “It becomes,” he says, “no inconsiderable Part of Science to know the different Operations of the Mind, to separate them from each other, to class them under their proper Divisions,...”5 And he explicitly says that this can be done when these operations are made the “object of reflection.” Hume admittedly simplifies that task quite drastically. He reduces the relations that the operations of the mind produce to three: Contiguity, Similarity, and Cause/Effect. In the roughly two hundred years since Hume quite a few thinkers have advanced the analysis and classification of mental operations. Two, however, stand out above all others: Immanuel Kant and Jean Piaget. The one I want to draw on here, in the discussion of the operations that constitute the “scientific method,” is Piaget. Although he did not invent the notions of assimilation and accommodation, it was he who refined them and made them generally applicable.6 The basic principle of these operations is this: The cognitive subject organizes experience in terms of conceptual structures. The subject’s attempts at organization are always goal-directed and if one wants to treat these goals, whatever they may be, collectively, one may subsume them by saying that the subject is intent upon maintaining some form of equilibrium. At any given moment, the subject “sees” and categorizes its experience in terms of the conceptual structures that it has available. Hence, the seemingly paradoxical assertion that an observer sees only what he or she already knows. This, in fact, is called “assimilation.” Conventional psychologists have tried to make hay of that apparent paradox— and they have succeeded whenever they were able to obscure the fact that, in Piaget’s theory, assimilation always goes together with accommodation. They frequently announced that if the notion of assimilation were correct, infants could never acquire new behaviors or new ideas. Let me try to correct this misinterpretation as simply as I can. When an infant picks up a spoon and shakes it as though it were a rattle, she assimilates her experience of the spoon—which is still a very limited experience from the adult point of view—to her prior rattle experiences. If, then, she shows disappointment, because in spite of her vigorous shaking there is no rattling noise, and she looks at the spoon in her hand, she opens the door to accommodation. This is so, because this new examination may lead her to notice a visual or tactual difference between the spoon she is holding in her hand and the rattle she thought it was. This perception of a difference may lead to novel experimentation, and before long the infant will develop a new scheme for producing a very satisfactory noise—namely by banging the spoon on the table. Let me insert a parenthesis about satisfactory noises. If you have had kittens in your house you may have seen the following. At a certain stage of their development, the kittens chase one another all over the place, and sooner or later one will land on your coffee table. If there happens to be a match box on the tale, or a packet of cigarettes, the kitten may start playing with it in the way kittens play with such things: it will tap the box lightly so that it slides a little way. Inevitably, the moment comes when the box falls off the table. For the kitten this is a most enthralling event. It will stand at the edge and look down on the box in unmistakable fascination. If you then pick up the box and place it beside the kitten, the procedure is likely to be repeated; the kitten will push the box to the edge and, with wide-eyed enchantment, it will watch it fall. I believe that this is just another demonstration of a basic principle inherent in all cognitive organisms: You deliberately carry out some action and, suddenly, the action produces a seemingly spontaneous and therefore enormously satisfactory input in another sensory modality. You push— and the box falls by itself and makes its own noise as it hits the ground; you flail about with a spoon, and it bangs as it hits the table; you pull a trigger, and some distant item is shattered. All this proves to you that you have control over your experience. As a non-professional observer, it has struck me that many marriages are held together by little more than such routines of apparent control; they seem reduced to appallingly simple procedures aimed at producing some visible effect in the partner. To return to the notion of accommodation, it may take place whenever an act of assimilation does not lead to the expected result or, indeed, produces an unexpected result. In the Piagetian theory it is only when things do not go in the expected way, when there is disappointment, surprise, or, quite generally speaking, when there is a perturbation, that reflection is triggered, and the cognizing organism may be led to accommodate and to try something new. This is a very important point. It is important for what I intend to say about science. Also—although I am not a psychotherapist—I would submit that this aspect of Piagetian theory has its use in the practice of psychotherapy. One more thing has to be said about assimilation: There are two kinds of it. When an infant picks up a spoon as though it were a rattle, the “as though” is only in the observer’s mind. It is only the observer who sees the spoon as a spoon. The infant takes the spoon to be a rattle and, consequently, is disappointed when the spoon does not produce the expected noise. In contrast, when the switch of the reading lamp on my desk refuses to function, and I use the letter opener as a screw-driver to open it—because I am too lazy to go down to the workshop—I am deliberately “assimilating.” I am well aware of the fact that the letter opener is not a screwdriver, and that using it as such may lead to all sorts of disappointments or surprises. If it’s made of silver, I may twist it or break it. But, at the moment, I may be prepared to take that risk. Our adult lives are full of such deliberate assimilations, and we very often “make do” with substitutes that turn out to be inadequate. This, too, may lead to accommodation—but not accommodation concerning the basic categories of the items we deliberately use as substitutes. We may simply learn that letter openers are less reliable screwdrivers than we thought. So far, I have been giving you stories, that is, accounts of things that have happened and accounts of conceptual patterns that have been constructed. I have given you examples of scientists going wrong and a Theory of Learning that shows one way how errors may be corrected—namely by accommodation. At this point, I want to make some conjectures on the basis of those stories. As proponent of a constructivist theory of knowledge, I want to stress that when I speak of “science going wrong” or of “correcting errors,” I do not intend that science can or should discover some absolute truth. “Wrong” and “right,” in this context, always refer to a specific goal; for instance, a problem one wants to solve. The solution one comes up with, either works or it does not. In my way of speaking, the answer to a problem is either a viable solution or it is not. In the case of the rabbit droppings and the tumble weed, the problem of getting rid of radio-active waste was not solved in a viable way. In the case of the Assuan Dam, the problem of bringing water to the desert was solved magnificently—but the solution created a new, unforeseen problem. And this unforeseen problem—the salting of the Egyptian lowlands—seems irrevocable and unsolvable, at least at the moment. In both cases, something was learned for the future, and there is hope that the same mistakes will not be made again. This sounds quite reasonable—and I believe it is. It is reasonable to suppose that we learn from our mistakes, and we can safely assume that scientists do, too. The question I want to raise now is: how is this possible, or more precisely, what assumptions are involved in assuming that we learn from experience? First of all, there is the assumption that things are related, especially the assumption that certain events not only precede but effectively cause others. As you know, David Hume, in the middle of the 18th century, suggested that if event B happens often enough after event A, we tend to think of B as the effect, and of A as the cause. As Hume clearly saw, this involves the belief that the real world is essentially an orderly world in which events do not take place randomly. A hundred years after Hume, this was expressed very beautifully by the German scientist von Helmholtz, when he wrote in 1881: It was only quite late in my life that I realized that the law of causality is nothing but our presupposition that the universe we live in is a lawful universe.7 That is to say, we expect the world we live in to be a world in which there are certain regularities, a world that runs according to certain rules. Since scientists call some of these regularities “Laws of Nature,” it may not be a silly question to ask how we come to know, or how we construct regularities? Let me return to Maturana’s methodology: The second step in his break-down, was relating observations, and relating them in such a way that one comes up with a Model. A scientific model, of course, is no more and no less than the crystallization of specific regularities. In order to speak of specific regularities, one must be fairly precise about the events that are claimed to be regular. That is to say, one must define certain experiences so that one can recognize them when one experiences them again. There can hardly be regularity before one has noticed repetition. Two points are important here: (a) The model connects specific observations, i.e., experiences made during the step that is Maturana’s step 1; (b) Experiences are specified by separating from, or isolating them in, the entirety of one’s experiential field— and this separation is made explicit and repeatable by setting up constraints on the acts of observation. That is why we said earlier that the Model in step 2 tends to determine the constraints set up in step 1. Let us try to get some idea as to what those constraints are and how they are set up. An example that is often used is the initiation of physiologist, say, neurophysiologists, into the working routines of the discipline or, as one might say a little irreverently, into the tricks of the trade. First of all, neurophysiologists must learn to look through a microscope; then they must learn to use various dies in order to make certain things visible; finally, they have to learn to see things in terms of the colors that were used to die the preparation. “Preparation” is a key word. Physicists speak of the “preparation of an experiment.” What they mean is that steps must be taken to make visible whatever it is they intend to observe. Sometimes these preparations include building a cyclotron or an accelerator that may cost many millions of dollars. In the social sciences, “preparation” takes a slightly different path. Experimenters know that, in order to be taken seriously as scientists, they must control for what are considered “interfering variables” in their experiments. That means that they must eliminate unwanted elements from the observations. Just as the physiologists, by their techniques of coloration, divert attention from all sorts of details that could be observed, so the social scientists disregard features of the experimental situation simply because, from the point of view of their explanatory model, they are considered irrelevant. For example, the rats, hamsters, and squirrel monkeys that serve in experiments in my Psychology Department, live on the 6th floor in our building. Thanks to the powerful air-conditioning system, the temperature in their environment never changes, and whatever light they see is produced by neon tubes. There is no dawn or dusk, no sun or moon, no wind or rain, and instead of the sounds and smells of the wilderness, they hear and smell whatever is produced by electronic devices, chemicals, and graduate students—a wilderness of a different kind. In most experiments this is considered totally irrelevant because the models used to explain the animals’ behavior do not encompass these experiential elements. Don’t misunderstand me—I have no specific hunches that, for instance, the repeated experience of sun rises, rain storms, or moonlight has a specific influence on the problem-solving abilities of hamsters; I am merely suggesting that even in these relatively simple forms of scientific research there are dozens of “variables” which are considered irrelevant a priori for no other reason than that, so far, there has been no need to incorporate them into the kinds of model animal psychologists use to explain the behaviors the are studying. Until the more or less accidental discovery of the rabbit disaster at the Hanford Reservation, the scientists who developed models for the safe disposal of radio-active waste had apparently looked at that problem in a way that focused exclusively on a few variables pertaining to experiments that were carried out in wholly artificial circumstances. The thought that they were dealing with a rather ordinary piece of natural landscape in North America did not occur to them until Nature forced it upon them. (As it happens, they apparently did not even anticipate what rain would do to the plutonium dump; it now transpires that radio-active particles have seeped into the ground water and are slowly but irrevocably being transported into the Columbia River.)

Conclusion

I have called this talk “The logic of Scientific Fallibility” and I have attempted to give some examples of how science proceeds. I can summarize this in a few sentences. In order to observe anything, in order to “collect data,” one must have some notion—no matter how primitive and preliminary—of the particular experiences one intends to relate to one another. It is, obviously, these experiences that one will be looking for. In order to find them, one necessarily assimilates and disregards all sorts of differences in individual observations. The longer this goes on successfully and the more often the model one has constructed proves useful, the stronger becomes the belief that one has discovered a real connection, if not a Law of Nature. And once that belief has been established, there is a powerful resistance against any suggestion of change and—as Thomas Kuhn has so nicely shown with examples from the history of science—there will be powerful efforts to sweep any observed irregularity under the rug. Thus, anyone not practiced in scientific tunnel vision, say, a farmer or trapper at Hanford, who would have asked: “What about badgers or rabbits burrowing and tumble weed roots soaking up the poison?” would have had little chance of being taken seriously. Science proceeds by modeling limited experiential situations—by focusing on a few variables, and deliberately disregarding many. It cannot do otherwise, and it should not do otherwise. But scientists must never drift into the misplaced religious belief that they are discovering what the world is really like. They should always remain aware of the fact that what they are dealing with is not the real world but an observer’s experience—and it is not even all an observer could experience, but deliberately constrained experiences or experiments that happen to fit the scientific model the observer is working with. And sometimes these very constraints exclude elements that afterwards rear their ugly head. Footnotes 1. Science, 1987, 236, 1616. 2. Maturana has developed this break-down throughout the last decade and whenever I heard him present it, it had become a little more complete and comprehensive. The way I am reporting it here is not verbatim but rather the way I have come to understand it. 3. Paul Feyerabend, How to be a good empiricist—A plea for tolerance in matters epistemological, in P. H. Nidditch, The philosophy of science. Oxford University Press, 1968. 4. John Locke, An essay concerning human understanding, Book II, chpt.1, p. 4. 5. David Hume, An enquiry concerning human understanding. New York: Wasington Square Press, 1963. p. 17. 6. The first to use the terms “assimilation” and “accommodation” systematically was Mark Baldwin, at the turn of the century. The concepts, however, are older. The German psychologist Steinthal used it in his Einleitung in die Psychologie und Sprachwissenschaft (Berlin, 1871) and Hans Vaihinger, in his Die Philosophie des Als Ob (written during the 1870s, published in Berlin, 1913), formulated it in its most general form: “The psyche is an organic shaping force which modifies, to suit its purposes, whatever it takes in and is also capable of adapting itself to the new.” (p.2) 7. Quoted by Franz Exner in: Vorlesungen über die physikalischen Grundlagen der Naturwissenschaften, p.663 (Vienna: Franz Deuticke, 1919). This paper was downloaded from the Ernst von Glasersfeld Homepage, maintained by Alexander Riegler. It is licensed under the Creative Commons Attribution-NonCommercial-NoDerivs License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/2.0/ or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, CA 94305, USA. Preprint version of 15 Feb 2006