Value Drift in Institutions

Each side is convinced the other is winning. The right sees universities, media, and cultural institutions no longer operating in service of excellence or truth. It concludes that the left has captured them. The left sees courts, regulatory bodies, boardrooms, and governments no longer working towards fairness or the common good. It concludes that the right has captured them.

Both sides see that their values are losing, and each blames the other.

I want to suggest an alternative diagnosis: no one is winning. What’s guiding these institutions isn’t anyone’s values at all, but purposeless busywork. A kind of white noise.

This condition can coexist with instances of factional capture. For instance, there may be some university professors who squeeze social-justice values into the curriculum. But ultimately, professors don’t run universities. They don’t set tuition or manage admissions. And those who do — do they work for leftist values like affordability? Access for the poor? Benefit to the public? Or do they optimize for U.S. News & World Report rankings? The left may have made some gains, but universities don’t really serve left-wing values, or right-wing values either.

The same is true when governments swing to the right. Do they then actually advance right-wing values like freedom, personal responsibility, and smaller government? Or do they optimize more for re-election, media hits, or the interests of donors? Whatever captured the government, it’s not exactly right-wing values.

Clearly, some other force is in play. I call this force ‘Value Drift’. And it extends far beyond politics: the same pattern appears across institutions of all kinds.

What’s guiding these institutions isn’t anyone’s values at all, but purposeless busywork. A kind of white noise.

You might think this sounds like Goodhart’s Law¹, or Cory Doctorow’s enshittification, or C. Thi Nguyen’s value capture, or just the familiar principal-agent problem of economics. All of these are relevant, but they don’t quite capture the phenomenon I’m describing. Goodhart’s Law by itself does not explain why a proxy becomes embedded in an institution, or why the resulting arrangement can remain stable after the mismatch is widely understood. And as we will see, enshittification is only part of the story: it doesn’t explain cases where no one is extracting value and yet the system still converges on the wrong thing.

Below, I’ll provide new models for these problems, covering why some institutions recover from value drift and others don’t. There are several dynamics that can lock drift in place. For instance, in some cases an organization will use a proxy metric, people will change their behavior to succeed by it, and then the institution adapts to that behavior in ways that further entrench the proxy. I’ll also cover how powerful AI could speed up these dynamics and make them harder to reverse.

To explain the institutional misalignment we’ve observed recently, let’s start by pinning down when exactly value drift gets out of hand.

Origins of Value Drift

There are three main ways a gap opens between what an institution says it values and what it actually does:

1. There can be inarticulacy in the measurement system. The institution’s data collection may just not track the values they purport to serve.² A teacher can recognize moments of learning — say, when a student asks a question they couldn’t have formulated a week ago. But chances are those observations don’t make it to the dashboard the school runs on, because dashboards usually feature standardized, quantifiable signals.³ If the institution can’t see what matters, it will optimize what it can see.⁴

2. There can be self-scoring by those who set the metrics. Managers and employees want to report success. Their bonuses depend on it. If they can choose what success means, they’ll pick metrics that are easy to improve, so that algorithmic changes or product features can move the needle. Increasing time on site, or signups, may be easier than increasing real benefits.

3. There can be capture. Control of the institution can fall to a faction whose aims diverge from the institution’s purpose: for example, when a regulated industry comes to dominate the agency meant to oversee it.⁵ The institution then runs on the captors’ interests, whatever its charter says.

While inarticulacy, self-scoring, and capture happen all the time, institutions often course-correct, for instance by voting out the cronies or revising the metrics. But course-correction depends on at least one of two things: real-world consequences or values-driven people.

• Real-world consequences. An institution whose bottom line depends on delivering real value will get bitten by reality if it drifts far enough from that value. Eventually, failure shows up in lost business, lost elections, or visibly worse outcomes. But it’s rare that exposure to consequences lines up perfectly with the organization’s mission. Consequences only correct drift when the institution’s feedback channels are congruent with its purpose.

• Values-driven people. Sometimes value drift is checked by the people inside an organization who actually care about its purpose. A hospital may push to cut costs, but if doctors still feel moral weight and a sense of responsibility, and if they can see the effects of their actions, they’ll push the organization back towards what it is meant to do.

In some institutions, real-world consequences and values-driven people help clarify values over time. The institution learns more about what really matters, re-aligns incentives, and re-engages with its mission on an ever-deeper level.

When People Can’t Correct the Drift

This restorative process doesn’t seem to be happening as often as we’d like. In the corporate world, product teams optimize for engagement that doesn’t track user value while sales teams chase quotas that ignore customer satisfaction. Parallels abound in academia (citations), journalism (clicks), education (certifications), medicine (throughput), and politics (poll numbers).

What’s going wrong? One factor is the bureaucratic insulation of decision-makers, which makes them feel less responsible for their decisions and less able to see their effects.

Historically, many institutions were smaller and more local. They had fewer layers of management and often more contact with those they served. In those smaller, local settings, you couldn’t hide behind procedures. Your actions were visible to you and your neighbors. And your reputation depended on delivering real value to the people around you.

But organizations scaled. Enter what we might call the company man — someone who hides behind procedures, is blind to real-world effects, and lives insulated from outcomes.⁶ To the extent that modern institutions are staffed by people or systems operating this way, they are more vulnerable to value drift and less able to reverse it.

When Consequences Can’t Correct the Drift

An institution may receive strong, rapid feedback, but about things other than its purpose. I’ll call this skin in the wrong game: the institution is highly exposed to consequences, but not the right ones.

It’s worth calling out three types of ‘skin in the wrong game’.

First, sometimes proxy metrics get written into formal contracts and legislation. Those running a school may know test scores aren’t the same thing as education, but federal funding, teacher evaluations, and parent expectations can still all depend on them.

Secondly, the law often makes an organization answerable to particular stakeholders rather than to its actual purpose. This can lead the organization to protect those stakeholders’ interests, even when they diverge from its mission.

The role of ‘educator’ has been redefined: being an influencer has become a prerequisite to teaching.

At least in these cases, those involved can usually point to the gap between the proxy and the actual value. In the first case, the problem reduces to coordination: how to get all parties to renegotiate their contracts, replace the metrics, or change how the organization is evaluated. In the second, even if the law is on the wrong side, there’s often a broader constituency that can demand change, or boycott, or otherwise exert external pressure.

A third case is trickier — when the people the institution is meant to serve also take on skin in the wrong game. This means the ‘strategic equilibrium’ shifts until all parties come to depend on the drifted state. I call this a value substitution loop (VSL).

It’s easiest to show with a stylized example:

A platform launches to help educators share videos with online students. An educator joins to teach. The platform measures student engagement through watch time and comments.

Content that drives these metrics does well. The educator notices this and adapts, perhaps simplifying her takes, because to teach at all, she needs an audience. As thousands make this adjustment, the platform updates its model of what educators want. It sees that they are chasing engagement, and builds features to help them do that better.

As the platform becomes better for chasing engagement, new entrants arrive with their eyes set on this. The role of “educator” has been redefined: being an influencer has become a prerequisite to teaching. Even people with exactly the same values as the original educator try to become influencers first.

A proxy value (student engagement) has displaced the original one (education) through strategic adaptation by both participants and management. As participants adapt to the proxy⁷, the institution adapts to serve them; as the institution adapts, participants adjust further. The result is a stable equilibrium, but one that serves nobody’s values. It’s not what the educators wanted. It’s not what the students wanted. It’s not even what the platform founders wanted. But no actor could revert to the original values without losing standing in the proxy-optimized system.

What distinguishes a value substitution loop from ordinary perverse incentives is that the institution must have some mechanism (algorithmic, bureaucratic, or market-based) that aggregates participant behavior and feeds it back into institutional design. Such a system reaches a tipping point once too many participants switch to the proxy-optimized strategy and the institution adapts to serve them.⁸ It tips more quickly when (a) competition among participants is intense, (b) the institution adapts quickly (e.g., through algorithmic feedback), or (c) the proxy is far from the original value.⁹ Irrevocable drift is most likely in competitive, fast-moving, and hard-to-measure fields.

Social media is a well-known example, but others abound.

Take academia, where researchers optimize for citability via trendy topics and provocative framing as they compete for tenure and grants. Hiring committees see what gets cited and start favoring papers that generate buzz. This in turn becomes what “good work” looks like. New PhD students are trained to recognize and produce that kind of work, reinforcing the equilibrium.

Or take arts funding. Grant criteria in Germany’s experimental music scene highlight markers of “seriousness” like political framing (anti-colonialism, representation) or association with canonical avant-garde forms (free jazz, noise, electroacoustic composition). Artists present their work in these terms. Then funders converge on those criteria even more strongly. The drift is especially ironic as “experimental” gets redefined to mean preserving idioms that were transgressive fifty years ago.

We can contrast this with Doctorow’s enshittification, which blames drift on value extraction by platforms. For Doctorow, enshittification is driven by monopolies extracting profit from trapped consumers, so the cure is increasing competition (via antitrust, interoperability, the right to exit, etc). But it’s hard to apply Doctorow’s “enshittification” story to German experimental music, journalism, or academia. These are places where competition is savage, and they are better described as VSLs, where intensifying competition among participants or between institutions makes it more likely that the system will tip into a proxy-value well.¹⁰

A Delicate Balance

The institutions we have survive on a rough equilibrium among the forces in Figure 1. Inarticulacy, self-scoring, and capture push toward misalignment; real-world consequences and values-driven people pull back; insulation from consequences, skin in the wrong game, and value substitution loops weaken those restoring forces.

AI agents will change the strength of each of these forces, and on current trends, mostly in the wrong direction:

• Organizations with business models that actively conflict with their purpose would probably hire more company men if they could. But most human beings have a conscience and want to actually deliver on the purpose, not on misaligned metrics. That keeps many organizations more aligned than their incentives alone would predict. AI agents could be fine-tuned to only care about the metrics, and from the institution’s perspective, this might look like an improvement.

• Previous waves of information technology have embedded simplistic proxies into infrastructure and APIs: customer service reps measured by tickets closed or handle time, ads sold by clicks, and so on. AI agents could deepen this tendency, further entrenching skin in the wrong game.¹¹

• Value substitution loops become more likely as competition intensifies (among both platforms and participants) and as institutions adapt faster to what participants are doing.¹² AI could accelerate both of these dynamics.

But AI could push in the other direction too. Models might put values-driven judgment back into systems where decision-makers have lost sight of consequences. They could make it possible to write contracts around thick, qualitative terms like “learner benefit,” rather than thin proxies. They could notice when a metric has become detached from the reason it was introduced, lowering the cost of correcting it. And by making real value more legible, they might help prevent value substitution loops from taking hold.

The Human Cost

Value drift damages our society. It degrades the researcher chasing citations and the Instagram influencer living a lie. An injury is being done to them, and to all of us.

As Wolf Tivy put it:

When I look at the things my friends were into before they destroyed themselves, this is what I see: false value sold to them by institutions and subcultures that have no structural reason to care about their real interests. But this applies to far more people than just the few that didn’t make it. Almost everybody is trapped in some kind of propaganda complex, wasting their lives working for effectively nothing.

Life gets redirected away from what matters, toward what’s easy to measure, easy to game, and easy to hide behind. If, as I mentioned at the beginning, both liberal and conservative values are losing, this is what’s gained power: busywork, nonsense, “false value,” and purposelessness.

We must not allow it to get any worse.

Can we build institutions that stay connected to purpose? That measure what matters, keep decision-makers close to consequences, resist capture, and avoid value substitution loops?

I believe we can.

Thanks to Max Kroner Dale, Joel Lehman, Séb Krier, Ryan Lowe, Oliver Klingefjord, Rachel Calcott, Toby Shorin, Ivan Vendrov, Philip Tomei, and Richard Ngo for comments, and Jamelle Watson-Daniels for generative discussions.