The resident felt for a pulse in the cord, about two inches from the navel, and counted. A helper could hold the cord and tap out each beat with one finger. Was the baby breathing? When a soft rubber tube went into its nose, did it grimace, sneeze or cough? Were its arms and legs bent, or limp? Was it pink all over? Each of the five questions got a zero, a one or a two, and the five were added. Ten was the best a baby could get. The score sheet was five lines long.

The sheet came from Virginia Apgar, who taught anesthesia at Columbia. Before it, doctors described a newborn’s slow, weak start as “mild, moderate and severe depression.” Apgar wrote that those words left “a fair margin for individual interpretation.” Two doctors could look at one baby and choose two different words, and no one could add the words up. A ward full of “moderate” babies could not be compared with the ward across town. Her five lines could be added, and the totals could be compared from one hospital to the next.

At Sloane Hospital for Women, her residents scored 1,021 babies this way. The obstetricians on the ward started to compete. Apgar noted their “competitive spirit” and how they “took great pride in a baby with a high score.” A doctor who had just delivered a baby now waited, at the end of the first minute, for someone else’s number. Then the records showed which of those 1,021 babies lived through their first month.

On a Sunday evening this month, the week’s job postings are open on your phone. One has a clean logo. One is at a company a friend likes. One just feels right, so you apply. By the end of the month you have sent twenty applications and heard back from two. A friend asks every Sunday how the search is going, and you give the same answer as last week. You cannot tell the friend what to change, because you cannot tell yourself. Twenty applications, each sent for a different reason, cannot say which of your reasons is working.

Apgar’s residents were not the most experienced people in the room. They had the least stake in the birth. So why would their five quick marks, made in one minute, say more than the eye of the doctor who delivered the baby? And would twenty job applications say more if each had been asked the same five questions?

Four Ways to Say Framework

Apgar’s whole method fit on one card.

Figure 1. The five signs and what earned a 0, 1 or 2, in Apgar’s words. Each row is one sign, each column one mark. Where she described only some answers, the cell says so. Drawn for this piece, after Apgar (1953), pp. 261-262.

Five questions, three answers each, the same on every baby, whoever held the clock. Many fields have a name for a fixed set of questions like that one. The name is framework, and four fields defined it in four ways.

Johnson and Foote wrote software. A class is a reusable piece of a program, and a framework is a bigger piece, reused whole. It works like the floor plan a builder uses for every house on a street: the paint and the tiles change from house to house, but the plan stays. The key words are “a family of related problems.” The problems differ, and the design that solves them does not.

Grove and Meehl were psychologists, and they set the mechanical method against another one. “The clinical method relies on human judgment that is based on informal contemplation and, sometimes, discussion with others.” A doctor at a case conference uses the clinical method. A doctor adding up five numbers on a card uses the mechanical one.

Ostrom won a Nobel Prize in economics for studying how people share a lake, a forest or a pasture. Her framework is a list of things to look at in every shared resource, from “Wisconsin lakes” to “the planet Earth.” It works like the headings on a form that every study fills in, so two studies can be laid side by side.

GRADE writes the rules many medical panels use to turn studies into advice. Its Evidence to Decision framework helps those panels “use evidence in a structured and transparent way to inform decisions.” The framework has three main parts, one for each step: “formulating the question, making an assessment of the evidence, and drawing conclusions.” When two panels reach different advice, the shared steps show where they parted.

The four do not agree on everything. Johnson and Foote make a framework a design that programs reuse. GRADE makes it steps that a panel runs. Ostrom makes it the shared words that theories and models are built from. Grove and Meehl set it against a person’s judgment. But all four hold one part of the method the same from case to case: every house on the street, every patient, every lake, every panel.

Someone reading job postings on the bus can check the next posting against the last. Are the questions asked of it the same ones asked of the last posting, or new ones? A week of postings, read with new questions each time, leaves nothing to set side by side. The harder part is choosing which questions go on the card.

Five Signs Out of Many

Apgar did not start with five questions. “A list was made of all the objective signs which pertained in any way to the condition of the infant at birth.” From that list she kept five “which could be determined easily and without interfering with the care of the infant.”

Each of the five is a guess about what trouble looks like in a baby’s first minute: a slow heart, no breath, no grimace, a limp body, blue skin. A baby’s weight is not on the card, and neither is the length of the labor. That set of guesses is a mental model, the small copy of a system a person carries in their head. The framework is the part that asks the model’s questions the same way every time.

A model can be wrong about one line, and Apgar’s was. Color “caused the most discussion among the observers.” Babies are born blue, and many kept “cyanotic hands and feet,” blue hands and feet, “for reasons still mysterious to us.” Few babies scored a two for color. When several hundred babies were rated again a few minutes later, almost all of them had turned pink. By the end of her paper, she called color “relatively unimportant” at one minute.

She could say that only because the card had not changed. Every baby got the same five questions, so the weak question showed up as weak.

Someone about to send the next application can write down five things to check in every posting. They might be the pay, the commute, the team, the skills asked for and room to grow. The thing to look at is what the list leaves out, and whether each thing was left out on purpose. A list of five is still only a guess until it has been run on many postings, the way Apgar’s was run on every baby.

The Same Way, Every Baby

A baby who gasped at thirty seconds, then stopped, scored zero for breathing. Apgar gave the reason: “he was apneic at the time decided upon for evaluation.” Apneic means not breathing. The gasp did not count, because the rule asked about the sixtieth second.

Three babies had heart rates over 140, which the card did not cover. “I was puzzled as to the proper way to rate this,” Apgar wrote. She gave them the full two points and kept going.

That stubbornness made the count possible. The paper does not say why 712 other babies born alive in her seven and a half months of records were not scored. The scores fell into three piles: low, middle and high. The low pile scored 2 or less, and the high pile 8 or more. Of the 65 babies in the low pile, 9 died in their first month. That is 14 percent. Of the 182 in the middle pile, 2 died. Of the 774 in the high pile, 1 died. That is 0.13 percent.

Figure 2. Babies who died in their first month, by their score at sixty seconds. The tallest bar belongs to the lowest scores. Drawn for this piece from Apgar (1953), p. 267.

The low pile’s death rate was about a hundred times the high pile’s. That division is this piece’s, not Apgar’s. The twelve babies who died had averaged 2.3 points. The scores came from residents, and Apgar found the resident’s score “sufficiently accurate.” A doctor in training, asking five fixed questions, was enough.

Twelve deaths are too few to give the gap an exact size, and Apgar said as much: the groups were “still too small” to analyze statistically. But “mild depression” could not have made three piles at all. Her stated aim was a basis for “comparison of the results of obstetric practices.”

Someone with a spreadsheet of this month’s applications can score each one on the same five lines. On Sunday evening, when the week’s replies are in, the thing to look at is the replies per ten applications, pile by pile. Ten applications in a pile is a small count, like Apgar’s twelve deaths. The piles still say more than twenty separate reasons can. What about the posting that scores low and still feels right?

Counted Across Cases

By 1996, 136 studies had set an expert’s feel for a case against a formula. Each asked which one better predicted how a person would do. Grove and Meehl gathered them. The formula was “almost invariably equal to or superior to” the expert.

In 2000, Grove and four colleagues pooled the studies in one analysis. On average, the formulas were about 10 percent more accurate. They clearly won in 33 to 47 percent of the studies, depending on the analysis. The experts clearly won in 6 to 16 percent. Experts also fell further behind the formula, not closer, when they had interviewed the person.

The formula did not win because it was wiser. On any single case, it can be wrong, and a gifted judge can beat it. A framework is judged by its rate across all its cases, not by one. That rate exists only because the rule did not change from case to case. Hold the rule still, then count. An expert’s hits can be counted too, as the 136 studies did. But when the reasons shift with each case, a miss cannot be traced to any one of them. Apgar’s three piles work like a report card for her framework: a grade for each pile, earned across 1,021 babies. A statistician reads a test the same way: not as right or wrong on this case, but as a procedure with an error rate over many cases.

Holding still also shows the rule its own misses. Apgar noticed that “in an occasional instance the color was worse at five minutes than at sixty seconds, and these cases were therefore missed with our usual method of evaluation.” A miss like that has a name only because the method had a usual way to work.

In 1996, Grove and Meehl could not say for sure why so many experts trusted the eye over the formula. They offered only “possible causes of widespread resistance.”

Someone about to send an application the five-line score says to skip can still send it. The step is to mark it on the sheet as an override. A week later, the reply or the silence goes beside it. After a month, the two reply rates can be set side by side: the score’s picks against the overrides. The friend who asks about the search every Sunday can keep the tally, and that Sunday question finally gets an answer with numbers in it. Twenty rows of marks on the sheet, and one more column filled in by hand.

A Closing Invitation. The score sheet stood for a few questions asked the same way every time, so their misses can be counted. A job search can ask five of its own, and learn by next month which reasons bring replies.

1. Notice the reasons. Right where you sit, recall why each of the last three applications you sent went out: the logo, a friend’s word, the photos of the office. Was it one reason three times, or three different ones?
2. Write the card. Before Sunday, write five questions on an index card or the back of an envelope: pay, commute, team, skills, room to grow, or your own five. The card takes five minutes. The applications take a month of Sunday evenings. Which question sounds wrong read aloud? Which would you drop first, and why?
3. Mark the override. The next time a posting feels right, a mission you believe in or a city you love, write “override” beside it before you send it. At the end of the month, which pile heard back more, the overrides or the rest? Would the friend who checks in on Sundays count the two piles with you?

An index card of five lines stands propped beside the laptop, one row marked “override” in pen. Down the side runs a column of zeros, ones and twos.

Worth remembering Save as image

A rule does not have to be wiser than an expert to beat one. It has to ask the same questions every time, so its misses can be counted. Try five fixed questions on this week’s job postings.

Where This Came From

Each of the four definitions came out of a different argument. In 1954, Meehl’s Clinical versus Statistical Prediction set an expert’s judgment against a formula, and the 1996 count grew from that book. In 1988, Johnson and Foote wrote down what programmers meant by the word. Ostrom first set framework, theory and model apart in 2005. In 2016, GRADE turned the steps from evidence to advice into one framework.

Apgar never used the word. She called her method “grading,” and she wrote that scoring a patient as “a sum total of several objective findings” was not new. It had been used to judge the treatment of drug addiction, in a 1938 government study by Lawrence Kolb and Clifton Himmelsbach.

Intellectual Honesty Note. Reading Apgar’s mortality table as a framework’s error rate is this piece’s own reading, and so is the hundredfold division. Apgar named no baby, doctor or resident, and none is named here; the delivery room scene follows her description of the method, not one recorded birth. Her paper prints the scoring in prose, with no sheet or table. The score sheet and the card are this piece’s picture of it, and Figure 1 marks the cells she gave no wording for. The job search is a hypothetical. Meehl’s 1954 book, Ostrom’s 2005 book and the 1938 addiction study were not reread for this piece; each is cited through a later source.

References

Alonso-Coello, P., Schünemann, H. J., Moberg, J., Brignardello-Petersen, R., Akl, E. A., Davoli, M., Treweek, S., Mustafa, R. A., Rada, G., Rosenbaum, S., Morelli, A., Guyatt, G. H., Oxman, A. D., & the GRADE Working Group. (2016). GRADE Evidence to Decision (EtD) frameworks: A systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ, 353, i2016. https://doi.org/10.1136/bmj.i2016

Apgar, V. (1953). A proposal for a new method of evaluation of the newborn infant. Current Researches in Anesthesia and Analgesia, 32, 260–267.

Grove, W. M., & Meehl, P. E. (1996). Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical-statistical controversy. Psychology, Public Policy, and Law, 2(2), 293–323. https://doi.org/10.1037/1076-8971.2.2.293

Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12(1), 19–30. https://doi.org/10.1037/1040-3590.12.1.19

Johnson, R. E., & Foote, B. (1988). Designing reusable classes. Journal of Object-Oriented Programming, 1(2), 22–35. https://www.laputan.org/drc.html

Kolb, L., & Himmelsbach, C. K. (1938). Clinical studies of drug addiction III. Public Health Reports, Supplement 128, 23–31. (Cited in Apgar, 1953.)

McGinnis, M. D., & Ostrom, E. (2014). Social-ecological system framework: Initial changes and continuing challenges. Ecology and Society, 19(2), 30. https://doi.org/10.5751/ES-06387-190230

Meehl, P. E. (1954). Clinical versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence. University of Minnesota Press.

Ostrom, E. (2005). Understanding Institutional Diversity. Princeton University Press.

The man asking questions was Willett Kempton, an anthropologist who studied how people run the heat at home. He published Bill’s words in 1986. Bill had a full account of his thermostat: what it does, what it predicts, and when to touch it. He had evidence, too: as a boy, he held his hand near the vent and felt it blow hotter as the number went up. He even had a rule. Turning the thermostat up high to get warm fast felt “sinful” to his “kind of Calvinist or Puritan feeling.” So some days he set it where he wanted the room and put up with the cold.

Still, someone had once given him the expert explanation, and he could not repeat it, so he counted his own account as nothing. And the box on his wall does not work the way his account says. Inside, it holds a switch that is only on or off. A higher setting keeps that switch on longer; the furnace burns no hotter.

Bill was not alone. Three people told Kempton the same story about their spouses. Each spouse turned the thermostat up to heat the house faster. Each partner had explained that the thermostat does not work that way. In Kempton’s words, “their repeated attempts to convince their spouse of this had failed.” His guess at why: each partner argued against one belief, when the spouse held a whole theory. Two people, one hallway, one box on the wall, and two different boxes in their heads.

Nor was Bill’s account rare, or only American. Kempton cited a survey of 38 houses in Wales. There, 62 percent of the residents said a thermostat turned up full would warm a house quicker than a lower setting. Kempton estimated, from his interviews and two studies by others, that a quarter to a half of Americans use at least part of that account.

The first cold evening this month, you come home and your partner, or a housemate, has turned the setting to the top “to warm it faster.” The setting reads 85 degrees, the furnace roars, and the room creeps up from 62. Two hours later the room is too hot, and the setting is still at the top, because no one remembered to turn it back. Those two hours show up on this month’s heating bill, and the argument about them starts again. In a hot place, the air conditioner gets set to its coldest instead. Every evening like that costs two hours at the top and leaves one question open: whose copy of the thermostat is wrong? Until someone answers that question, the argument comes back with the next cold night.

The copy Bill carried in his head has a name in psychology: a mental model. Each arguing spouse carries one too, and so does the engineer who built the box. They cannot all be right about the box. How would any of them ever find out?

A Copy That Runs

A machine of brass wheels and wire stands behind glass in London’s Science Museum. William Thomson, later Lord Kelvin, built it in 1876 to predict the tides. Each wheel turns with one rhythm of the sea, such as the pull of the moon or the sun. A wire over the wheels adds their rhythms into one rise and fall: one harbor’s tide, drawn ahead of time. The machine is a copy of the sea that fits in a room.

Figure 1. Thomson’s tide-predicting machine, 1876, Science Museum, London. Brass wheels, a wire, and not a drop of water, yet it drew a harbor’s tides ahead of time. Photograph by William M. Connolley, Wikimedia Commons, CC BY-SA 3.0.

It looks nothing like the sea, and it holds no water and no moon. Yet it says when the water will be high. In 1943, the Cambridge psychologist Kenneth Craik used Kelvin’s tide predictor to explain what a model is. A model, he wrote, “need not resemble the real object pictorially.” It has to work “in the same way as the process it parallels, in the aspects under consideration.” Then he moved the idea inside the head.

Set side by side, the four agree on three things. A mental model is a copy of a system, in the sense this site’s first essay defines it. It is smaller than the thing it copies. And it runs: it can be pushed forward to say what happens next, as Kelvin’s wheels say when the water will rise. Each stresses something different. Craik stresses the use: trying things in the head before trying them in the world. Forrester stresses what is missing: “only selected concepts and relationships.” Johnson-Laird stresses the cost: the copy is “less rich,” and elsewhere he shows it makes errors of a predictable kind. Doyle and Ford stress the eye of the holder: the copy keeps the structure as its holder sees it, which may not be the structure that is there.

Search for “mental models” today, and most pages offer something else: a list of thinking tools borrowed from fields such as economics and biology. That sense comes from a 1994 talk by the investor Charlie Munger, who told business students to hang what they learn “on a latticework of models in your head.” This piece means the older sense, a copy of one system that each person already carries, chosen or not. Writers in the older sense mostly stop at what a copy is. In 2010, the software engineer Jeremy Merritt defined a mental model in The Systems Thinker as “a model that is constructed and simulated within a conscious mind.” Later that year, he judged a model by “the ability to explain the results.” This piece asks the next question, and uses a harder test: what does a copy predict, and by how much does it miss?

Before picking up the keys at rush hour, a driver runs a copy of the route home: the bridge, the left turn, the school zone. The copy holds the roads this driver always takes. It leaves out the one detour that would save twenty minutes tonight. Which parts does a copy leave out, and who chooses them?

Same House, Two Copies

Twenty-six families in identical New Jersey townhouses chose the numbers on their thermostats, and small machines logged them every hour. The records ran for two years, from a Princeton study that Kempton drew on. The houses were the same. The families were not.

Kempton found two copies of the thermostat in eight detailed interviews with twelve people in Michigan. He called them folk theories: ideas a group shares and learns from daily life, as opposed to the theories specialists learn from books and experiments. The first copy is the feedback copy, the one engineers build. The thermostat senses the room and turns the furnace on below the setting and off above it, a loop that keeps pulling the room back toward one temperature. The second copy is the valve copy, Bill’s. The setting opens the heat wider, like a gas pedal, so a higher number means more heat each hour.

Figure 2. Two copies of one thermostat, and the kind of hourly log each copy leaves. Compare the two logs at the bottom: three steps a day against a step every waking hour. Drawn for this piece, after Kempton (1986). The logs are schematic, not Kempton’s data.

Kempton thought each copy left a trace in the records, and the winter of 1976 shows both. In one house, the setting changed at 8 a.m., at noon, and once in the evening, when the household’s day changed. In a second house, that winter, the number changed between almost every hourly reading, whenever anyone was awake. It stayed still only at night, from about 1 to 7 a.m. Over the whole study, that house’s setting ranged from 61 to 85°F. Three changes a day in one house; a change every waking hour in the other, across 24 degrees. Two hands, running two different copies.

Kempton was careful about those two houses. He never interviewed either family. In a footnote, he wrote that the frequent changes “could be due to other causes such as domestic conflict over desirable setting.” An argument over the thermostat would leave the same jagged line.

Those two copies show up in any hall. Walking into a cold house, or a hot one, a hand reaches for the thermostat or the air-conditioner remote. A hand that turns it past the target, to warm or cool the room faster, is running the valve copy, and the degrees past the target show how far. The valve copy is wrong about the box. Does that make it the worse copy to live with?

Wrong Inside, Right in Use

One woman told Kempton what living with her husband’s copy was like. He would argue with her “long enough,” she said, that turning the heat down at night saves nothing. Turn it down to 55 overnight, and every object in the house drops to 55. In the morning, the furnace must warm every object back up to 65. By her account, that uses “more fuel than if you would have left it at 65.”

The husband held the experts’ copy: the thermostat is a switch. His copy was right about the box and wrong about the bill. It left out one part of the house: a warm house leaks heat through its walls faster than a cool one. Every hour at 55 leaks less than an hour at 65, so the night turned down saves fuel.

Bill’s valve copy, wrong about the box, gets the bill right. It says a higher setting burns more fuel, and that prediction is true. In Kempton’s words, “The prediction is correct, even if the explanation is wrong.” In some of his interviews, people who held the valve copy were more likely to believe that turning the heat down at night saves fuel. His verdict: “A theory that is useful for designing thermostats is not guaranteed to be a good theory for using them.”

The valve copy has its own cost: a hand on the thermostat every hour, and a furnace burning all evening if no one turns the heat back down. Each copy left out a different part, so each was good for a different job.

Two housemates who argue at bedtime about turning the heat down can test their copies against the job. Each says what their copy predicts for the morning, and which part of the house that copy leaves out. Over a month, the bill counts the nights turned down against the nights left up. Both copies are wrong somewhere. How would either housemate find out where?

The Gap, Written Down

Bill nearly found out, from a sound he remembered in his parents’ house. In the night, the furnace there went on and off by itself, “making those irritating, windy sounds.” No hand had touched the thermostat. His valve copy had no room for that sound: in his copy, only a hand moves the heat. Later in the interview, Bill patched his copy. Maybe his parents had a smarter thermostat, one that measured the room and switched itself.

Seven months later, Kempton showed Bill his analysis. Bill said the patch was gone. “I discovered it for the moment, but later forgot it. When I went back to using the thermostat, I probably went back to doing it the same way.” The gap between his copy and the world had shown up as a sound in the night. He heard it, explained it, and let it go. He wrote nothing down, so nothing changed.

That gap has a name in any feedback loop: the error signal. In 1976, the statistician George Box wrote that it is the error signal that “can produce learning.” He gave an example: “the discrepancy between what tentative theory suggests should be so and what practice says is so.” He also wrote, “Since all models are wrong the scientist must be alert to what is importantly wrong.” The useful question is where a copy is wrong, and by how much.

Here is the move that makes a copy better: write the prediction down before the world answers, then measure the gap. The thermostat in the hall runs on that very gap: it compares the setting with the room and acts on the difference. A statistician reads a model built from data in the same way. Each of its predictions misses the measured value by a gap called the residual, and the parts the model left out show up there. A null hypothesis is a claim of no difference, such as the husband’s belief that turning the heat down at night saves nothing. Written down before the data arrive, a null hypothesis is a mental model whose gap can be measured.

Try the move on the thermostat. Before turning the setting up on a cold evening, write on the back of an envelope how many minutes the room will take to feel warm. Then check the clock. A big miss is the valve copy meeting its error signal, in writing. A close guess means the copy did its job that evening.

Psychologists still argue over whether people reason by running copies like these at all. Kempton himself wrote that “further data is needed” on how a theory like Bill’s guides what people do at home. But Bill’s copy did not need a theory of the mind to improve. It needed one number on an envelope, propped against the wall beside the thermostat.

A Closing Invitation. The thermostat in the hall stood for the copy each person carries in their head. A copy is judged by its job, and it gets better when its guess is written down and checked.

1. Find your copy. Now, where you sit, picture the thermostat or air-conditioner remote at home. The room is cold: would your hand go to the top, or to the number you want? Which copy just moved your hand, the switch or the valve, and how long would the setting stay there?
2. Count the high nights. This weekend, guess from last month’s heating or cooling bill how many nights the setting stayed high. What would the person you share the house with, or a friend who pays the same kind of bill, guess? How far apart are you, and what would one night turned back sooner have saved?
3. Guess the minutes. The first cold evening this month, or the first hot afternoon, before you touch the setting, write on a yellow sticky note beside the thermostat how many minutes the room will take to feel right. When your fingers warm up, or the sweat dries, how far off was the note? What did your copy leave out?

The thermostat hangs where it always did, set at the room you want. Beside it, a yellow note in pen says twenty minutes, and the clock across the hall keeps the gap.

Worth remembering Save as image

You can be wrong about the furnace and right about the bill. A belief is judged by what it predicts. So before you turn the dial, write down how long the room will take to warm, then check the clock.

Where This Came From

Craik wrote the “small-scale model” in Cambridge during the Second World War, in his short book The Nature of Explanation. Forrester carried the idea into system dynamics, the field he founded at MIT: by 1961, in Industrial Dynamics, he was telling managers that their mental models of a company “are not the real corporation” (as Doyle and Ford quote him). Johnson-Laird made it a theory of reasoning in his 1983 book Mental Models. By 1997, the term had grown loose enough that Doyle and Ford set out to fix its meaning. They offered their definition as a starting point and wrote, “We make no claim that this conceptual definition is correct or complete.”

Intellectual Honesty Note. Bill’s words come from Kempton’s interview transcript, quoted in short fragments with filler words such as “um” dropped at their edges. Figure 2 is a schematic drawn for this piece, not Kempton’s data. In the Johnson-Laird box, an ellipsis stands where the original has a dash, and the Craik box ends at an ellipsis. Reading Kempton’s two copies through Box’s error signal, and calling a null hypothesis a mental model written down, are this piece’s own readings. The driver, the housemates, the envelope and the sticky note are invented.

References

Box, G. E. P. (1976). Science and statistics. Journal of the American Statistical Association, 71(356), 791-799. https://doi.org/10.1080/01621459.1976.10480949

Connolley, W. M. (2005). Tide predicting machine, by Sir William Thomson in 1876 [Photograph]. Wikimedia Commons. https://commons.wikimedia.org/wiki/File:DSCN1739-thomson-tide-machine.jpg. CC BY-SA 3.0.

Craik, K. J. W. (1943). The nature of explanation. Cambridge University Press.

Doyle, J. K., & Ford, D. N. (1997). Mental models concepts for system dynamics research [Conference paper]. International System Dynamics Conference. https://proceedings.systemdynamics.org/1997/doyle2.htm

Forrester, J. W. (1971). Counterintuitive behavior of social systems. Technology Review, 73(3), 52-68.

Johnson-Laird, P. N. (1983). Mental models. Harvard University Press.

Johnson-Laird, P. N. (2010). Mental models and human reasoning. Proceedings of the National Academy of Sciences, 107(43), 18243-18250. https://doi.org/10.1073/pnas.1012933107

Kempton, W. (1986). Two theories of home heat control. Cognitive Science, 10(1), 75-90. https://doi.org/10.1207/s15516709cog1001_3

Merritt, J. (2010a). What are mental models? The Systems Thinker, 21(2), 8. https://thesystemsthinker.com/what-are-mental-models/

Merritt, J. (2010b). What are “mental models”? Part 2. The Systems Thinker, 21(9), 8-10. https://thesystemsthinker.com/wp-content/uploads/pdfs/210902pk.pdf

Munger, C. T. (1994). A lesson on elementary, worldly wisdom as it relates to investment management and business [Lecture]. USC Business School. Transcript: https://fs.blog/great-talks/a-lesson-on-worldly-wisdom/

▶

Figure 1. Operation Cat Drop as it is usually retold, from Sustainability Illustrated (2014), YouTube embed.

The cats came from Kuching, the capital of Sarawak. The city’s name is often said to mean “cat.” Kuching had been asked for thirty cats and could find twenty-three. Later retellings grew the number to fourteen thousand. The cats were bound for Bario, a village high in the Kelabit Highlands, where rats were eating the rice. The plane made five test runs before the first basket went out. Hundreds of villagers watched the baskets come down into the rice fields. A month later, the village sent a message back to Kuching: “Thanks for the cats – no more trouble with rats and mice.”

The message did not say why the rats had come. They were the last link in a chain that started with a good idea. In the early 1950s, malaria teams sprayed DDT and other insecticides on the walls of the longhouses, the long shared houses where families slept. The spray killed the mosquitoes that carried malaria. It also killed tiny wasps. The wasps laid their eggs inside caterpillars, and that kept the caterpillars from multiplying. The caterpillars lived in the thatch, the dried leaves that made the roofs. The caterpillars could sense the chemical and stayed away from it. The wasps could not, and they died. With the wasps gone, the caterpillars ate the roofs. The village cats died too, and the spray was blamed. With the cats gone, the rats came in.

The villagers complained that the spray was ruining their roofs. It took a team from the World Health Organization to trace the ruined roofs back to a wasp small enough to miss.

Cities have their own version of the spray. Houston spent $2.8 billion widening the Katy Freeway to more than twenty lanes, to end its traffic jams. The new lanes did ease the jam at first. Then the wider road drew more drivers. Between 2011 and 2014, the rush-hour drive along it got 30 percent longer in the morning and 55 percent longer in the afternoon. Across American cities, economists have found that driving grows in step with new lanes.

A cheaper house farther out lowers your mortgage, and it works. It also adds an hour to your drive, each way. Someone else takes over the school drop-off. Dinner moves later, then apart, one plate kept warm in the oven. On a Friday, a friend texts about drinks. You read it in traffic, brake lights red for a mile ahead, forty minutes from home. Behind you, a driver honks, long and flat. You text back no. After a few months, the texts stop coming.

In Sarawak, too, the spray did what it was meant to do. It was a correct answer to a question about one part. The malaria teams asked how to stop the mosquito. A better question was what else the insecticide would kill. For the cheaper house, the hour each way has already cost the Friday drinks, so what else will the house cost? That question splits into three, each worth asking before a fix is made, or before the papers for the cheaper house are signed. What else is connected to this part? What does it keep in check? Who decided what counts, and what did they leave out?

What a System Is

Before a concert, one oboe plays a single note, an A, and the rest of the orchestra tunes to it. If the oboe is a little sharp, every player ends up a little sharp. The three questions ask about parts tied together like that. People have tried to define a system for about ninety years.

Bertalanffy was a biologist. He wanted one language for a cell, an animal, and a society. His next sentence explains what “interacting” means. Parts interact when each one behaves differently depending on what it is connected to. If they behave the same no matter what, “the elements behave independently,” and there is no system. Take a violinist. Alone, a violinist can play at any speed. In an orchestra, the same violinist listens and keeps pace with everyone else. What the violinist plays depends on who else is playing.

Meadows split her definition into three things to look for. The elements are the things in the system. The interconnections are how they affect each other. The function, or purpose, is what the whole does. In an orchestra, the elements are the players and their instruments. The interconnections are the score, the conductor, and the players listening to each other. The purpose is music for the people in the seats.

Figure 2. The three things Meadows says to look for in a system: elements, interconnections, and function or purpose.

Meadows also ranked the three. The elements matter least. Replace every player on a football team, and it is still a football team. The interconnections matter more. Change the rules to basketball’s, and it is a new game (p. 16). The purpose matters most. She called it “often the most crucial determinant of the system’s behavior” (p. 17), and it is the hardest to see. To find it, watch what the system does: “Purposes are deduced from behavior, not from rhetoric or stated goals” (p. 14). An orchestra may say its purpose is great music. If it plays the same safe pieces every season because they sell tickets, its behavior says the purpose is selling tickets.

system, n.definition 3 of 5Wave 2

▶

“A system is a whole that cannot be divided into independent parts.”

Ackoff reached that sentence in three steps. A system has two or more parts. Each part can change how the whole behaves, but never alone. What one part does depends on at least one other part. The third step holds for any group of parts too. Take one coworker off a team of five, and every job left on the team shifts. Ackoff’s word for what Meadows calls purpose is the “essential property”: something the whole can do that no part can do alone. In his example, a car can carry its driver from one place to another, and its engine cannot. An orchestra’s essential property is the symphony. Send each player to a separate room, and each one can still play a part, but no one can play the symphony.

Senge studied system dynamics, the field Meadows came from, at MIT. System dynamics builds computer models of how a company or a city changes over the years. His book put the idea in plain words for managers. For Senge, systems thinking is a skill: the habit of asking what else a change touches, and when. A snapshot of a concert catches one chord. The music is the pattern of chords over time.

Senge also listed laws for what happens without that habit. The first is “Today’s problems come from yesterday’s ‘solutions.’” Another is “Cause and effect are not closely related in time and space.” Houston’s wider freeway fits both.

Co-implying means each needs the other. Nothing is a part unless it belongs to a whole. Nothing is a whole unless it has parts. For the Cabreras, seeing systems is one of four moves a mind makes. The other three are telling things apart, relating them, and taking a point of view. Ask it of the violinist. Is the violinist part of the orchestra, the string section, or a quartet that plays weddings on weekends? Each whole makes the violinist a different part. Who decides which whole counts?

Where They Meet

Sand scattered on a road counts as a pile, not a system. All five definitions would call the orchestra one. They agree on four points, though not every definition makes every point.

1. A system is not a pile. The sand is Meadows’ example. Add sand or take some away, and it is still sand on a road (p. 12). Bertalanffy and Ackoff make the point from the other side. In a pile, the parts act independently.

2. A system is held together by its connections. Bertalanffy says “interacting,” Meadows says “interconnected,” and Senge says “interrelationships rather than things.” Ackoff’s parts depend on other parts. The Cabreras relate one thing to another.

3. A system changes over time. Meadows writes that systems produce “their own pattern of behavior over time” (p. 2). The pattern comes from inside: the parts acting on each other, day after day.

4. A system is for something. Meadows’ system “achieves something.” The malaria program was for killing mosquitoes. The longhouse was for a dry, safe place to sleep.

Grapefruit juice and some cholesterol pills show all four points. The pill and the juice are not a pile: together, they do something neither does alone. They are held together by an enzyme in the gut that normally breaks the drug down. Grapefruit juice blocks it. A statistician has a name for this: an interaction effect. Two things interact when the effect of one depends on the level of the other. Here, the pill’s effect depends on how much juice someone drinks.

The change also builds over time. The juice disables the enzyme, and the gut has to make new enzyme to replace it. A day after one glass, a quarter of the effect is still there. In one study, three glasses a day for a week doubled the effect on a blood pressure drug. Too much of the cholesterol drug stays in the body.

The pair, judged by what it does, is no longer just a cholesterol treatment. It also raises the risk of liver and muscle damage.

The pair answers the first question from the opening. What else is connected to this part? For the cholesterol pill, the answer is a glass of juice. But which glass: the one poured this morning, or the one from yesterday?

How a System Moves

Yesterday’s glass of juice is still at work today. A credit card has a delay too: the spending happens now, and the bill comes weeks later. The bill can also push back on the spending that made it. Meadows calls it feedback when a change comes back around to push on itself.

In Thinking in Systems, Meadows calls the positive kind reinforcing and the negative kind balancing, and one credit card shows both. A reinforcing loop feeds itself. Interest adds to the debt, and the bigger debt earns more interest. Meadows calls this kind “amplifying, reinforcing, self-multiplying, snowballing” (p. 30). On the card, the snowball is interest charged on last month’s interest. A balancing loop pulls a system back toward where it was. The monthly statement arrives with the balance in bold at the top, and the cardholder cuts back. Meadows calls balancing loops “both sources of stability and sources of resistance to change” (p. 30). The statement that keeps the debt steady also pushes back on any month of extra spending.

The card’s delay is a few weeks. In Sarawak, the rats took years to come. Someone who opens a frightening bill cuts spending too hard, then splurges when the next bill looks fine, so the delayed balancing loop overshoots and swings back. A statistician would see that swing in the data as a cycle. Months close together look alike. Months half a cycle apart move in opposite directions.

The card answers the second question from the opening. What does this part keep in check? The answer is usually a balancing loop. Balancing loops are the easiest part of a system to miss, because when they work, nothing happens. Going paperless is a fix too. Swap the paper statement for an app nobody opens for three months, and the interest keeps growing with no balancing loop to stop it.

Why Ask Before the Fix

In Sarawak, every link the spray broke was a balancing loop. Without the wasps, nothing held the caterpillars back, and the roofs sagged. Without the cats, nothing held the rats back, and they reached the rice. The twenty-three cats in wicker baskets were a balancing loop, flown back in by hand.

The malaria program counted mosquitoes. It did not count wasps, roofs, or cats. Even now, no one is sure how the cats died. One account, Tom Harrisson’s, has them eating cockroaches the DDT had killed; others have them licking it off their paws.

For Bertalanffy, a system is out in the world, whether or not anyone maps it. For many of the thinkers after him, a system is also like a map: someone draws it, and that person decides where it ends. Meadows wrote that “boundaries are of our own making” (p. 99). Sarawak shows both. The wasps were real whether anyone counted them or not. Whether they got counted was up to the people drawing the map.

No program can count every insect in Borneo, so something always gets left off. The people who live with what was left off tend to notice first. In Sarawak, those people were the villagers. That is the third question from the opening. Who decided what counts, and what did they leave out?

None of the three questions says to skip the fix. Houston may still have needed its lanes. The cheaper house may still be the right one, for the reader or for a friend about to sign the papers on one. The questions add one step before each fix, asked the same way every time. For the lanes, the step is asking who will move in along the new road. For the house, it is counting the hour each way in the car, and the Friday drinks it will cost.

A Closing Invitation. Every fix in this piece worked the way the spray did. It solved the problem it was aimed at. Its cost came later, and somewhere else. The three questions work on your own fixes too.

1. Notice the sag. Find the fix at home that worked and cost you the most. The cheaper car that kept breaking down. The rule that ended one fight and started a quieter one. The cost could sound like a knock from the engine on cold mornings, or a door shut a little harder. How long before you noticed?
2. Find the wasp. Name one thing, at work or at home, that the old way kept in check without anyone noticing. The coworker who caught mistakes, until the day they left. The weekly call with a friend that kept small worries small. Could one call this week bring back your friend’s laugh, the way the cats were flown in?
3. Walk to the roof before the next spray. Take the decision in front of you now: a job offer, a lease to renew, a trip planned for spring. Who will live with its cost a year from now, somewhere you are not looking? Ask that person, a partner or a roommate, before you decide.

Putting back what the old way kept in check is a small cat drop. Over Bario in 1960, hundreds of villagers watched twenty-three cats come down in baskets under parachutes, sent to answer the question about the spray, years late.

Worth remembering Save as image

Sometimes today’s problems are yesterday’s solutions. A fix can solve the problem it was aimed at. Its cost comes later, and somewhere else.

Where This Came From

Each of the five definitions of a system comes from one of four waves of systems thinking. The first wave, from the 1950s, wanted to predict and steer. It brought Bertalanffy’s general system theory, Norbert Wiener’s cybernetics, and Jay Forrester’s system dynamics. A first-wave team would have modeled the village: so many cats per rat, so many wasps per caterpillar. The second wave, from the 1970s, began when people inside a system wanted different things from it. A second-wave team would have asked whose village it was. The third wave, from the 1980s, asked the third question: who gets left out. A third-wave team would have started with the families under the thatch. The fourth wave, proposed in 2021 by the Cabreras and Gerald Midgley, is still unsettled.

Figure 3. The four waves of systems thinking, after Cabrera, Cabrera and Midgley (2021).

Intellectual Honesty Note. Four readings are this piece’s own. Two are about Sarawak: the malaria teams asking how to stop the mosquito, and the story as broken balancing loops. The other two are the malaria program and the longhouse as two purposes, and the grapefruit pair as an interaction effect. The cheaper house is a hypothetical. The fourteen thousand cats of later retellings do not appear in the 1960 press. The video in Figure 1 tells the popular version, which dates the cat drop to the 1950s and adds plague; this piece follows the 1960 record. Senge’s page number comes from later citations. The wave dates are approximate, and the 2021 chapter went unchecked.

References

Ackoff, R. L. (2015, November 2). Systems thinking speech by Dr. Russell Ackoff [Video]. YouTube. https://www.youtube.com/watch?v=EbLh7rZ3rhU

Bailey, D. G., Dresser, G., & Arnold, J. M. O. (2013). Grapefruit–medication interactions: Forbidden fruit or avoidable consequences? CMAJ, 185(4), 309–316. https://doi.org/10.1503/cmaj.120951

Bertalanffy, L. von. (1968). General System Theory: Foundations, Development, Applications. George Braziller.

Cabrera, D., & Cabrera, L. (2022). DSRP theory: A primer. Systems, 10(2), 26. https://doi.org/10.3390/systems10020026

Cabrera, D., Cabrera, L., & Midgley, G. (2021). The four waves of systems thinking. In D. Cabrera, L. Cabrera, & G. Midgley (Eds.), The Routledge Handbook for Systems Thinking. Routledge.

Cortright, J. (2015, December 16). Reducing congestion: Katy didn’t. City Observatory. https://cityobservatory.org/reducing-congestion-katy-didnt/

Duranton, G., & Turner, M. A. (2011). The fundamental law of road congestion: Evidence from US cities. American Economic Review, 101(6), 2616–2652.

Food and Drug Administration. (2021, July 1). Grapefruit juice and some drugs don’t mix [Consumer update]. https://www.fda.gov/consumers/consumer-updates/grapefruit-juice-and-some-drugs-dont-mix

Forrester, J. W. (1961). Industrial Dynamics. MIT Press.

Ho, S. S. (2024, April 1). It’s a cat’s life: Airborne felines fight in war against rats. BiblioAsia. National Library Board, Singapore. https://biblioasia.nlb.gov.sg/history/2024/4/cats-rats-parachute-malaya/

Magnin, A. (2014, May 6). Systems thinking: A cautionary tale (cats in Borneo) [Video]. Sustainability Illustrated. YouTube. https://www.youtube.com/watch?v=17BP9n6g1F0

Meadows, D. H. (1999). Leverage points: Places to intervene in a system. The Sustainability Institute. https://donellameadows.org/archives/leverage-points-places-to-intervene-in-a-system/

Meadows, D. H. (2008). Thinking in Systems: A Primer (D. Wright, Ed.). Chelsea Green.

O’Shaughnessy, P. T. (2008). Parachuting cats and crushed eggs: The controversy over the use of DDT to control malaria. American Journal of Public Health, 98(11), 1940–1948.

Senge, P. M. (1990). The Fifth Discipline: The Art and Practice of the Learning Organization. Doubleday.

Wiener, N. (1948). Cybernetics: Or Control and Communication in the Animal and the Machine. MIT Press.