The Optimization Line We Shouldn’t Cross

What should never be optimised?

Optimization is a kind of devotion. It says: there is a better way, and we can find it.
But devotion has a shadow: if you optimize the wrong thing, you don’t improve the world—you replace it.

Let’s name the target: the domains where optimization reliably erodes what it claims to protect.

Not because improvement is bad. Not because excellence is impossible.
Because some things require friction, slowness, ambiguity, or mutual obligation to remain real.

The three places optimisation goes to die

1) Anything that depends on shared meaning

When people agree on what something is “for,” the agreement becomes part of the system.
Optimizing behavior while bypassing meaning is like tuning a piano without caring whether it’s played for music or for ritual.

You see this in:

• education that chases measurable output while losing the point of learning
• leadership that optimizes engagement metrics while losing legitimacy
• art platforms that rank work by predicted clicks while draining risk and craft

What changes first is why people participate.
What breaks next is the participation itself.

2) Anything that depends on trust

Trust isn’t just a belief. It’s a relationship built over time, with room for doubt and repair.
Optimizing for compliance or “confidence signals” often turns trust into performance.

Trust can be engineered for a moment. But it can’t be mass-produced without costs:

• people learn what to say, not how to be true
• incentives teach people to game the system
• mistakes become threats instead of data for repair

A trustworthy system protects agency—it doesn’t only measure outcomes.

3) Anything that depends on human limits

Some constraints aren’t inefficiencies; they’re boundaries of being human.
Sleep, attention, dignity, and grief are not bugs. They are the body’s way of keeping reality inside the person.

Optimization that treats these as obstacles will eventually require cruelty to scale:

• relentless speed that reduces quality of thought
• personalization so aggressive it feels like extraction
• convenience that quietly steals time, autonomy, and choice

The “optimization line” test

Here’s a practical way to tell when optimization has crossed from improvement into erosion.

If you can’t answer “what would still be true,” your metric may be substituting for reality.

Optimization’s most common failure mode: Goodharting your soul

There’s an old idea: once a measure becomes a target, it ceases to be a good measure.
In human systems, the consequence is rarely subtle.

A metric can be useful at the edge of a process.
It becomes corrosive when it starts to define the process.

• If you optimize “time on task,” you’ll get more task time—not necessarily more learning.
• If you optimize “resolution speed,” you’ll get faster closures—not necessarily better care.
• If you optimize “retention,” you may produce dependence, not value.

The real damage is that people lose the ability to act from internal alignment.
They begin to act from external correctness.

Where this shows up in modern life (quietly)

Platforms: attention as a product

Many systems now optimize for engagement at scale. It’s rational—until it isn’t.
When the goal is “maximize clicks,” the model learns that outrage, novelty, and fear are efficient carriers.

The cost isn’t only mental health. It’s epistemic health:

• weaker commitments to truth
• stronger commitments to winning
• reduced willingness to tolerate uncertainty

Work: performance as identity

Performance metrics can support clarity.
But when they become identity, people start to fear being “wrong” more than being “useful.”

That’s how a culture slips from:

• learning → proving
• improving → defending
• collaborating → optimizing privately

A system shouldn’t require self-erasure to be “productive.”

Healthcare: throughput as care

Optimization in medicine can be life-saving.
But when care is reduced to throughput, the patient becomes a transaction.

Care is not a factory output. It includes interpretation, presence, and the humility to say: “I don’t know yet.”

So what should we optimize instead?

The answer isn’t “never optimize.” That would be a trap: a moral slogan that can’t govern decisions.
The better move is to optimize conditions rather than outcomes that represent people.

In other words: optimize the scaffolding, not the soul.

A non-negotiable question for every metric

Use it before the metric enters the system.
Because once it does, it will colonize the system from the inside.

Editorial pause: what we’re really protecting

Underneath the practical examples is a quieter principle:
There are things we do not own with numbers.

Meaning isn’t a dashboard. Trust isn’t a report.
Human dignity isn’t an optimization target—it’s a premise.

Final takeaway checklist