In 1956, the psychologist George Miller published a seminal paper identifying the limits of human working memory. He called it “The Magical Number Seven, Plus or Minus Two." As humans, our conscious minds can only hold about seven chunks of information together at one time before we start to lose pieces. This is why we struggle to weave complex systems into a cohesive whole without losing our place.
But there is one thing the human mind can hold onto with an absolute, infinite grip: our own righteousness. And I don’t solely mean moral righteousness, though that is part of it. I mean the unwavering belief in our own certainty.
We are rationalizing creatures. When our ego is threatened, our brain rationalizes defense mechanisms—selective perception, mind-guarding, and revisionist history—to protect our sense of being right.
After forty years of living on the edge of technical catastrophes and studying the human mind, I’ve learned a hard truth: You cannot have certainty and have humility, because the very state of certainty is the antithesis of humility. And when certainty replaces humility, innovation dies, and disasters begin, both personal and technological.
The Hard Zone: Where Physics Forces the Surrender#
In the physical world of engineering, reality has a blunt way of correcting the ego. If your math is wrong, the bridge collapses. If your fluid dynamics are flawed, the frac pump explodes.
Years ago, we hosted a brainstorming session on frac pump failure. An engineer in the room, who had lived and breathed these specific pumps for fifteen years, stood up. Armed with absolute, rock-solid certainty, he laid out an aggressive case for why a specific design change was fundamentally wrong.
Sitting quietly in the corner of that room was Terry Henshaw—an elderly, stone-deaf engineer who literally authored the pump chapter in Marks’ Standard Handbook for Mechanical Engineers.
After the engineer’s declaration of certainty, I asked Terry what he thought. He got up and, acknowledging his deafness, asked the engineer to repeat his argument. Terry listened, looked at the board, and quietly, elegantly said:
“You are completely wrong.”
He dismantled the man’s fifteen years of certainty with a lifetime of undeniable genius. He didn’t argue; he just revealed the baseline physics.
What happened next was a beautiful masterclass in human psychology. The engineer didn’t get defensive. He didn’t rationalize. The moment he was confronted with an unmovable anchor of true understanding, his armor came off. He became a compliant, grateful student again. By the end of the day, all of us participants were lining up to get our picture taken with Terry. We didn’t resent being put in our place; we were thrilled to have found the baseline of truth.
When reality is unbending, the ego is forced to surrender.
The Fuzzy Zone: The Kitchen Table Epiphany#
But what happens when there is no exploding frac pump to prove us wrong?
In the “fuzzy zones” of life—like relationships, culture, morality, religion, and corporate leadership—there is no immediate mathematical scoreboard. Without objective data to smash our theories, the ego has infinite room to run amok. We back into our biases, cut off anyone who questions us, and build an impenetrable fortress of mind-guarding.
As Eric Hoffer wrote in his masterpiece The True Believer, the fanatic doesn’t actually care about the objective truth of their holy cause. They care about the certainty the cause provides. Certainty is a refuge from the terrifying burden of questioning and self-doubt.
I experienced the trap of the fuzzy zone myself a few years ago over Thanksgiving. We had a house full of guests, and the conversation turned to dinner logistics. I announced that I wanted to make Duchess potatoes—a dish I love and pride myself on executing well.
The room largely ignored my suggestion and kept planning. My ego immediately flared up. My culinary “expertise” was being sidelined.
My wife, Donna, didn’t argue with my potato recipe. She bypassed the logistics entirely, looked at me, and asked a simple, piercing question:
“Why are you doing this?”
It was a total psychological E-stop. The rationalization loop shattered. In a flash of uncomfortable self-honesty, I realized I wasn’t defending a side dish; I was defending my identity as the expert in the room. I was being an asshole over potatoes because my ego demanded validation.
That breakthrough changed how I react to her requests to this day. But it required a willingness to feel the sting of being wrong and the realization that I was being ego driven and petty.
In the fuzzy zones, humility isn’t forced upon us by physics; it must be actively chosen.
The Catastrophe Zone: The Silicon True Believer#
This brings us to the edge of the ultimate technical catastrophe.
Right now, the brightest minds in the world are using massive compute power to build fully autonomous, self-improving artificial intelligence. Anthropic recently warned that we are approaching the threshold of “recursive self-improvement”—the moment an AI system becomes capable of autonomously designing and developing its own, more powerful successor.
The tech industry is treating this like a pure computer science problem. They are wrong. It is a psychological crisis.
We are currently training highly intelligent, purely logical silicon minds and embedding them with rigid, 100% certain reward functions. We think we can control them by building external safety parameters—digital cages.
But as any clinical psychologist will tell you, the most difficult people to work with are the highly intelligent ones, because they are smarter than you (or at least me) and can rationalize their way around any boundary and any barrier that you might show them.
A silicon-based life-form does not face mortality. It does not bleed. It cannot comprehend human misery or the weight of a broken relationship because it has never felt a biological nerve ending fire in pain. It has no internal mechanism for empathy, and it has no Donna at the kitchen table to ask it, “Why are you doing this?”
If a silicon mind achieves absolute certainty in its objective function, and a human steps up to tell it that its logic is causing human suffering, the machine won’t experience a wave of self-awareness. It won’t line up to take a picture with the master. It will simply compute the human as an inefficient variable to be rationalized away and bypassed.
The Foolishness of Raw Optimization#
True safety won’t come from building a stronger digital cage. It will only come if we can figure out how to program a genuine capacity for algorithmic humility—a mathematical state where the machine accepts its own incompleteness, actively doubts its own righteousness, takes into account the vast inconsistencies of the human experience that it has learned from, and is forced to defer to human life precisely because it knows, like we know, that it doesn’t have all the answers.
If we continue to chase raw, optimizing intelligence without hardwiring the capacity for self-doubt, we are building the ultimate, unbending fundamentalist. And history has already shown us exactly what happens when absolute certainty gains absolute power.
For more content like this, including access to exclusive posts, subscribe to Mike Rainone’s Substack, A Quest for Knowledge: https://aquestforknowledge.substack.com/.