The problem with helpful AI
Some notes from building a couple of AI tools that really forced this question — when AI removes effort, is it removing the thinking too?
AI reduces effort
Most AI product design today is built around a simple idea: reduce effort. Read less. Click less. Analyse less. Decide less.
For a lot of work, that's exactly right. If AI can write the SQL query, categorise support tickets, summarise a document, or automate a workflow, great. The value is the answer. Nobody benefits from doing those tasks manually.
But over the last few months, building AI products and agents, I've become interested in a different question: what happens when the thing being removed is the thinking itself?
Not all work is answer-finding work. Some work is judgment work. Product strategy is judgment work. Positioning is judgment work. Deciding what matters is judgment work. Understanding why you're stuck is judgment work.
In judgment work, the process of thinking isn't just a means to an end — it's where much of the value is created. And that's where I kept running into the same design tension: the more opinionated the AI became, the better the product felt. The less certain I became that it was helping people think.
The temptation to over-help
One of the tools I built is Untangle. It's for people who feel mentally overloaded but have no interest in journaling, mindfulness, or reflection exercises. The user records a voice note. The system structures what they've said and helps them find clarity — that's the goal, anyway.
What I discovered during development was that there's a very easy way to make the experience feel more magical: make the AI more interpretive. Instead of surfacing themes, explain them. Instead of identifying tensions, resolve them. Instead of helping users understand their thinking, tell them what's going on.
The outputs immediately felt more insightful. More useful. More satisfying. But they also became more authoritative. The AI was no longer helping users process their thoughts — it was starting to provide an interpretation of those thoughts. Was I building a thinking tool, or a very convincing explanation machine?
The same problem, a different layer
I ran into the same tension building a POV agent. The goal is simple: take a topic, map the conversation around it. What are the dominant narratives? What assumptions are people making? What's the contrarian view? What's getting attention, and what's being ignored?
The easy version of this product stops at "here's the best take." Many AI products do. But that wasn't what I wanted — the purpose wasn't to generate a position, it was to help someone develop their own. So the agent stops short. It maps the terrain, surfaces tensions, exposes gaps and blind spots — then hands the thinking back to the user. Less satisfying. The model could generate a conclusion, often a convincing one. But a convincing conclusion isn't the same thing as understanding.
Certainty and understanding are not the same thing
AI-generated conclusions can feel remarkably similar to understanding. That's what makes them powerful. It's also what makes them risky.
When a model explains your situation, it creates a feeling of clarity. When it gives you a position, it creates a feeling of certainty. Sometimes that certainty is deserved. Sometimes it isn't — and certainty and understanding are different things. You can understand something deeply and still be uncertain. You can also feel certain without understanding it at all.
The more I build with AI, the more I think product teams need to pay attention to that distinction.
A different question for AI products
Enterprise software has spent decades helping people access information. AI is helping software generate answers. The next challenge is figuring out when answers are actually the right output. For deterministic work, they usually are. For judgment work, I'm not so sure.
The question I find myself asking now isn't "can the model do this for the user?" It's "what understanding disappears if it does?"
The most delightful AI experience isn't always the most useful one. Sometimes the best thing an AI can do is help someone think a little more clearly for themselves.