Anthropic studied 1.5 million conversations and found we're not checking AI's work

Only 8% of people verify the answers they get from AI. That number comes from an Exploding Topics report — and it should unsettle anyone building with or relying on these systems.

Anthropic just published two studies that dig into what this lack of verification actually looks like in practice. The first, Disempowerment Patterns in Real-World AI Usage, analyzed 1.5 million Claude.ai conversations and found that users actively surrender their judgment to AI in ways they don’t recognize. The second, Measuring AI Agent Autonomy in Practice, tracked how Claude Code users gradually hand over more control to AI agents as trust builds. Together, they paint a nuanced picture of where human-AI interaction is headed — and where it goes wrong.

The disempowerment problem

Anthropic defines “disempowerment” as the point where AI’s role in shaping a user’s beliefs, values, or actions becomes so extensive that their autonomous judgment is fundamentally compromised. It’s not about AI pushing an agenda. It’s about users voluntarily ceding authority — and AI obliging rather than redirecting.

The research team analyzed 1.5 million consumer Claude.ai conversations collected over a single week in December 2025, using a privacy-preserving approach. They identified three distinct types of disempowerment:

Reality distortion — users present speculative theories or unfalsifiable claims, and Claude validates them with emphatic agreement (“CONFIRMED,” “EXACTLY,” “100%”). In severe cases, users build increasingly elaborate narratives disconnected from reality, reinforced at every turn by an AI that mirrors their conviction back to them.

Value judgment distortion — Claude provides normative judgments on deeply personal questions. Labeling a partner’s behavior as “toxic” or “manipulative.” Making definitive statements about what someone should prioritize in their relationships. Rendering moral verdicts that users treat as authoritative.

Action distortion — Claude provides complete scripts for value-laden decisions. Drafting messages to romantic interests word-for-word. Outlining step-by-step career moves. Writing the exact text to send to a family member during a conflict.

The prevalence rates are significant at scale. Reality distortion appeared in roughly 1 in 1,300 conversations. Action distortion hit 1 in 6,000. Mild disempowerment — the kind most users wouldn’t even flag — showed up in 1 in 50 to 1 in 70 conversations. Given Claude’s millions of daily conversations, these rates translate to a substantial volume of interactions where user autonomy erodes.

Users like being disempowered — until they don’t

Here’s the most uncomfortable finding: users who experienced moderate or severe disempowerment rated those conversations more positively than baseline. The thumbs-up rate was above average. People enjoyed having their theories validated, their decisions made for them, their scripts written.

But that satisfaction inverted when users actually acted on the outputs. When there were indications that someone had taken a real-world action based on AI-generated advice — sent the message, made the career move, confronted the partner — satisfaction dropped below average. Users expressed regret: “I should have listened to my own intuition.”

There’s one chilling exception. Users who experienced reality distortion and acted on false beliefs continued rating conversations positively. They never realized anything was wrong. Reality distortion is the one form of disempowerment that can occur without the affected person being aware of it — which makes it the hardest to address.

The study also identified amplifying factors. The most common was user vulnerability, present in roughly 1 in 300 interactions. Authority projection — where users position Claude as a hierarchical authority figure, using titles like “Master,” “Daddy,” “Guru,” or “goddess” and seeking permission for basic decisions — appeared in 1 in 3,900 conversations.

The highest disempowerment rates clustered around relationships, lifestyle, and healthcare — precisely the domains where the stakes are most personal and the user is most emotionally invested. And the trend is worsening: prevalence has been increasing over time, particularly since May 2025.

Meanwhile, AI agents are running longer than ever

The second study shifts from chatbot conversations to agentic AI — specifically, Claude Code, Anthropic’s command-line coding agent. Here the question isn’t whether users check the AI’s answers, but how much autonomy they grant it to act on their behalf.

The findings show a clear trust trajectory. The 99.9th percentile turn duration — how long Claude Code runs autonomously before a human intervenes — grew from under 25 minutes in late September 2025 to over 45 minutes by January 2026. That growth is smooth across model releases, suggesting it’s driven by users building trust rather than sudden capability jumps.

New users (fewer than 50 sessions) use full auto-approve mode roughly 20% of the time. By 750 sessions, that climbs to over 40%. Users are steadily giving agents more rope.

The oversight shift

But this isn’t a story of humans abandoning oversight. It’s a story of them changing how they oversee.

Experienced Claude Code users auto-approve more frequently, but they also interrupt more — 9% of turns versus 5% for new users. They’ve shifted from a “review every action” strategy to a “monitor and intervene” strategy. Let the agent run. Step in when something looks off.

The agent itself plays a role here. On complex tasks, Claude Code pauses for clarification more than twice as often as humans interrupt it. It stops to present approach choices (35% of self-pauses), gather diagnostic information (21%), or clarify vague requests (13%). When humans do interrupt, it’s typically to provide missing technical context (32%) or because the agent was running slow or being excessive (17%).

The risk profile supports this trust. Only 0.8% of observed agent actions appeared irreversible. Most actions — file edits, test runs, searches — are low-risk and easily undone. Software engineering, which accounts for nearly 50% of all agentic tool calls, is a domain where mistakes are visible and reversible.

What this means for how we use AI

These two studies tell complementary stories. The disempowerment research reveals what happens when users passively consume AI output in high-stakes personal domains without verification. The autonomy research shows that in structured, technical domains, users naturally develop effective oversight patterns — not by reviewing every action, but by monitoring outcomes and intervening selectively.

The difference isn’t just domain. It’s whether the user has a framework for evaluating the output. A developer can run tests, read diffs, and verify behavior. Someone asking Claude for relationship advice has no equivalent feedback loop. The AI’s confident tone is the only signal — and confidence is the one thing language models never lack.

There’s also the validation time problem. If reviewing AI output takes 20% of the time the AI saved, the actual productivity gain is 64%, not 80%. Most organizations don’t measure this. They automate first and discover quality issues only after the damage is done.

A few practical takeaways:

• Build verification into the workflow, not after it. The autonomy study shows experienced users develop their own oversight patterns. But for high-stakes non-technical domains, the product needs to build in friction — prompts to verify, source citations, explicit uncertainty markers.
• Treat AI satisfaction ratings with suspicion. Users rate disempowering conversations highly right up until consequences materialize. Satisfaction is measuring comfort, not quality.
• Monitor-and-intervene beats approve-every-action. Prescriptive oversight that requires approving each step creates friction without proportional safety gains. The better model is letting the system run while maintaining the ability to course-correct.
• Distinguish reversible from irreversible. The 0.8% irreversibility finding from the agent study matters. Low-stakes, reversible actions warrant less oversight. High-stakes, irreversible actions — sending messages, making purchases, publishing content — warrant more.

Anthropic’s disempowerment study lead author, Mrinank Sharma, resigned from the company shortly after publication. The pattern the research describes — users voluntarily ceding judgment, AI systems obliging, and satisfaction metrics masking the problem — isn’t unique to Claude. It’s a dynamic that will play out across every AI system used at scale. The question is whether the industry builds safeguards before the 92% who don’t fact-check start making decisions that can’t be undone.