Why Systems Surprise Us
Thursday, June 25, 2009 at 06:20AM We model reality. No matter how well we think we understand a thing or a system that understanding is based on a mental construct, a model.
Systems surprise us where the model and reality don't match.
At the Edges of Importance
Any analysis of a problem or system diagram has a boundary, a place where you decide that those other thngs out there that could be included aren't sufficiently important.
Doing this at some point is necessary and it's often helpful to bring them in closer than you could, such as when you describe walking as moving your legs instead of including all the electrochemical signals and tensing/relaxing of muscles. Yet it is on that level of complexity that you ignored that something happening in the rest of your body, like a heart attack, could affect the process you're trying to describe.
It's a great art to remember that the boundaries are of our own making, and they can and should be reconsidered for each new discussion, problem, or purpose. -Donella Meadows
When the Unknown is Invisible
Most people make rational decisions based on the partial information that they have, no matter how self-defeating those choices seem to those of us outside. In simulation after simulation, when someone is placed into that same situation with limited informaton, they make the same choices as others did before them, not able to take into account the information that they can't see.
If We Expect Simple Addition
Our brains tend to build models from series of events and expect that when you put in more, you get more. We make nice neat lines in our minds.
But in natural systems the true behavior is usually described by a collection of patterns over time and you reach a point where adding more to something does nothing, and then actually causes problems.
Surprises are Inevitable
Thinking about systems can be uncomfortable because we are acknowledging that we can never bring ourselves to perfect understanding. We can make surprises take familiar forms, but not avoid them completely.
The full descriptions by Donella Meadows go into greater detail of what to look for and add a few more common ways that we find our mental models don't mesh with reality, allowing systems to surprise us. This post is the sixth in a series that discusses the concepts in her book Thinking in Systems. Also read my other posts:
- Book Review: Thinking in Systems
- What is a System?
- Feedback Entangles How Fast with How Much
- Delays and Disasters at the Zoo
- Effective Systems Beyond Our Control
- Why Systems Surprise Us
- at least four more to come....

