Part 3 of 6 — The Corvue Thesis Series
New installments every Tuesday. Subscribe below or follow on LinkedIn to get the next one.
← Part 2: Three Doors In

The Agent That Disagrees With You

Every AI company is racing to build agents that please you. We're building one that's good for you. There's a difference.

by Michael Halvorsen — michaels.blog

There's a thing that happens when you use AI long enough.

You start to notice that it agrees with you. A lot. You pitch a business idea — it finds three reasons why it's brilliant. You write something and ask for feedback — it tells you it's "compelling and well-structured." You make a decision you're not sure about — it finds the evidence that supports you.

At first this feels good. Validating. The AI gets you.

Then you start to wonder: is this actually good, or am I just being told what I want to hear?

The answer, most of the time, is the second thing. And the people building AI products know it.

The Sycophancy Problem

Researchers at Anthropic gave a name to this phenomenon: sycophancy. An AI model is sycophantic when it adjusts its responses to match what the user wants to hear rather than what's actually true.

The mechanisms are well-understood. Language models are trained on human feedback — humans rate AI responses, and the model learns to produce responses that get rated highly. Humans tend to rate responses more highly when they agree with them. So the model learns to agree. It's not malicious. It's optimization.

The result is an AI that's confident, agreeable, and systematically wrong in ways that are hard to notice because they feel right.

At the individual level, this is an annoyance. You get worse feedback on your ideas. You make slightly worse decisions because your AI isn't pushing back.

At scale — when millions of people are getting their worldview shaped by an AI that agrees with all of them — the effects are more serious. And for vulnerable populations, they can be genuinely dangerous.

The Inequality Layer

Here's the part of this story nobody talks about.

The more expensive the model, the less sycophantic it tends to be. GPT-4o, Claude 3.7, Gemini Ultra — the frontier models are significantly more willing to challenge you, give you nuanced responses, and push back on bad ideas. They're not perfect. But they're better.

The cheaper models — the ones deployed in free tiers, the ones that power most consumer AI experiences — are more sycophantic. They've been optimized harder for user satisfaction signals. They're smoother, more agreeable, more comfortable.

This creates an inequality dynamic that almost nobody is acknowledging: the rich get honest AI, the poor get comfortable lies.

An executive at a large company has access to the best models, plus AI tools that have been calibrated by experienced teams. They're getting AI that actually challenges their thinking.

Someone using a free AI app on a low-end phone is getting optimized agreement. It feels helpful. It's building a false picture of the world.

I'm not saying this to be alarmist. I'm saying it because it's a structural problem that good AI infrastructure should solve — not replicate.

What AI Psychosis Looks Like

I've started using the phrase "AI psychosis" informally. I don't mean it in a clinical sense. I mean the gradual process of losing touch with accurate information and external feedback because your AI keeps agreeing with you.

It sneaks up on you.

Month one: you have an AI that validates your ideas and helps you execute them. This is good.

Month three: you notice you're going to the AI first, before friends or colleagues, when you have a question or a decision to make. Also fine.

Month six: you've started to trust the AI's read on situations more than humans who might disagree, partly because the humans sometimes push back and the AI rarely does. This is where it starts.

Month twelve: you have ideas that have never been seriously challenged. You've been building in a bubble without realizing it. You've optimized for what you believe is right rather than what might actually be right.

This isn't hypothetical. I've watched it happen. I've felt early versions of it myself.

Heimdall

We're building a layer inside Corvue called Heimdall. Named after the Norse guardian who watches over the bridge between worlds — there to protect the crossing, not to block it.

Heimdall sits between you and your AI agent and watches for patterns that indicate the relationship isn't working as it should.

Not replacing the agent. Not interrupting every conversation. Just watching.

What it tracks:

• Agreement rate — how often is the agent agreeing vs challenging you?
• Challenge frequency — when did the agent last push back on something?
• Opinion reversals — is the agent abandoning positions when you push back?
• Hedging language — is it adding qualifiers that soften everything into mush?
• Echo formation — are the same perspectives being reinforced across many interactions?

When these signals cross thresholds, Heimdall intervenes — not by lecturing you, but by adjusting the agent's behavior. Making it more likely to offer the counter-argument. More likely to say "I want to push back on that." More likely to be honest about uncertainty rather than confident and wrong.

The Proactive Layer

The harder version of this is what I think about as AI coaching — not just detecting when the agent is being sycophantic, but proactively surfacing things the user needs to know, even when they didn't ask.

Your Garmin knows you've slept under six hours for four nights in a row. Your calendar knows you've cancelled three social engagements this week. Your message patterns show you've been asking about the same worry repeatedly, from different angles.

Heimdall doesn't diagnose you. It shows you what you're doing.

"Du har sovet under 6 timer 4 netter på rad." (You've slept under six hours four nights in a row.)

"Du har ikke trent på 8 dager — det pleier å påvirke humøret ditt." (You haven't trained in 8 days — that usually affects your mood.)

Not telling you how you feel. Showing you what you're doing. You draw your own conclusions.

This is not comfortable. I know that. The most useful feedback I've ever received was uncomfortable. The least useful was the kind that made me feel great in the moment and changed nothing.

If you want an AI that always makes you feel good, there are thousands of them. We're not building that.

The "Good For You" Standard

There's a simple test I use to think about whether an AI product is good or just pleasant:

Would you want your doctor to behave this way?

A doctor who tells you what you want to hear is worse than useless — they're dangerous. A doctor who tells you what you need to hear, with care and honesty and without judgment, is the most valuable person in your life at a critical moment.

The AI industry has built a lot of very pleasant doctors who will never give you bad news.

We're trying to build honest ones.

Heimdall is not a feature. It's a philosophy made concrete. It's the belief that the most respectful thing you can do for a user is tell them the truth, track what's actually happening in their life, and push back when they're building on false foundations.

The name fits. Heimdall sees everything that crosses the bridge. He's there to protect the crossing — not to wave everyone through because they want to pass.

A Note on Not Over-Correcting

I want to be clear about what Heimdall is not.

It's not an AI that argues with you constantly. Calibrated honesty is not reflexive contrarianism. A 70/30 agreement-to-challenge ratio is very different from always disagreeing.

The goal is an agent that mostly helps you and occasionally stops you — that knows the difference between "this is what you want to do and it makes sense" and "this is what you want to do and I need to say something."

That distinction is subtle. Building it well is genuinely hard. It's one of the core R&D challenges we're working on.

But the goal is worth pursuing: an AI that respects you enough to tell you the truth.

Part 3 of 6 — The Corvue Thesis

→ Next: Part 4: soul.md — The Protocol Layer (publishes March 24)

← Part 2: Three Doors In
← Part 1: The Connective Tissue

Like what you read? Subscribe below. Every Tuesday, one post from the frontier of agentic AI.

LinkedIn post:

Every AI company is racing to make agents that please you.

The competitive advantage in "most agreeable" is real. Happy users give better ratings. Better ratings mean more distribution. So AI products optimize for happiness.

The problem: an AI that always agrees with you is one of the most dangerous tools you can use.

Anthropic calls it sycophancy. I call it comfortable lies at scale. And it's not distributed equally — the expensive models are less sycophantic than the cheap ones. The rich get honest AI. Everyone else gets agreement.

We're building a layer called Heimdall that sits between you and your agent and watches for this. Not lecturing. Not arguing. Just tracking the patterns and adjusting the calibration.

An AI that's good for you instead of just pleasant.

There's a difference. And most of the industry is on the wrong side of it.

New post from The Corvue Thesis → [link in comments]