You kept telling me my last essay wasn’t really about AI. Bob Poole called it borrowed certainty becoming common sense. Someone packages a position, it gets amplified, simplified, and eventually a dad at the playground delivers it like he thought of it himself. Tumithak of the Corridors pointed out that companies automate their entire hiring pipeline and then treat AI use by applicants like a moral failure. Cynthia called it a class issue.
And you were right that I’d been writing about something bigger all along without fully naming it. So I tried:
The most interesting thing about AI right now has nothing to do with the technology.
It’s that so many things we treated as natural are turning out to be just habits.
How we hire, how we work, what we count as effort, who we take seriously. None of it was designed with more people in mind than the ones in the room.
It just settled into place because it worked for the people who built it, and everyone else learned to work around it.
Now it’s unsettling, and honestly, it’s about time.
I posted it on Substack, and then deleted it almost immediately.
It was true but it was incomplete. It said what was happening without exploring why. Something was missing and I could feel it but not name it yet.
I was on the subway home from work, still thinking about a conversation I’d had earlier. The kind that keeps happening now, someone describing how they work, step by step, so an agent can take over. Automate the busywork, move data from here to there, handle the follow-ups. I’ve seen enough of these by now that they’re starting to blur together. Same energy, same moment where someone says “so basically we just need it to do what I do” without stopping to ask whether what they do makes sense. Everyone laughs politely. Nobody follows up.
I opened a new Claude conversation and wrote myself a note: We use AI to automate human behavior and tasks. Agents are built to carry out the things we do. None ask whether those things are worth doing. Just something to explore later.
But on the train, sitting with that and the text I’d just deleted, both nagging me at the same time, it clicked. I went back and added one line:
Turns out, when you have to explain how something works, you finally see that it doesn’t.
That was what was missing. Not that things are unsettling but why. Using AI forces us to externalize. To explain what our work actually is in order to teach a machine how to do it. And the explaining is where everything cracks open.
My stop was coming up. I posted the note and got off the train.
I have been thinking a lot about what part of the interaction with AI actually matters most to me. There’s the back and forth, the conversation itself, and then there’s what the AI says back to you, the content or output. I think what matters most is different for different people. But for me, the most transformative part has been my side of the conversation. Having to explain myself over and over again.
I’ve spent the last year or so explaining my thinking to AI. Not to automate it, but to explore it. I started this newsletter a few months back to think out loud about AI and work. It gave me a reason to land somewhere instead of spending my time in endless thinking loops. The newsletter was basically a deal I made with myself: if you’re going to overthink everything, at least try to get to the point.
Somewhere along the way the explaining showed me how my own brain actually works. I couldn’t see it from the inside. I had to put it outside of myself first. The talking back helped, but the change happened in the explaining.
And that note on the subway? I wrote it while processing the comments on my last essay with Claude. Trying to articulate what my readers were seeing, out loud, in conversation, is what revealed the gap. I was doing the thing while writing about the thing. Which, if you think about it, is also what I’m doing right now writing this essay.
When I wrote a note asking whether anyone had a similar experience, one response stood out. Jeff Long spent nine months building custom instructions for how his AI should work. Reverse-engineering his own process, writing down how he thinks, what he needs, what works and what doesn’t. And at some point he realized the more he articulated what he needed from the AI, the less he needed the AI to do it. The value wasn’t in the output, it was in the explaining.
My journey with AI wasn’t part of a big plan. I started talking and ended up picking my own brain apart. Not because I set out to, but because the explaining required it.
What I found wasn’t comfortable. Seeing how my brain actually works meant confronting what I’d been working around my whole life. The strategies, the compensations, the effort that went into looking like I was doing things the normal way. The explaining didn’t just reveal my process. It revealed who the default process was built for. And it wasn’t me.
Maybe I noticed first because the gap was always wider for me. But I think the gap exists for everyone and most people just haven’t had to look at it yet.
I stumbled into this by choice. But more and more people are walking into it because they have to. What happens when the explaining is no longer optional?
Most of what people say about AI is still about output. Hustling more, producing faster, ten ways to use ChatGPT to save an hour. The whole framing is about doing the same things more efficiently, which means nobody stops to look at the things themselves.
I wrote about cover letters recently. A dad at the playground said he could always tell when someone used AI to write one. Everyone nodded. And it left me thinking, why do we write them at all? Who do they actually work for? Does the format tell you anything real about a candidate? That’s where my brain goes naturally, but I’ve learned to keep those questions to myself. Except now I write on Substack about it, so you get to enjoy the show.
I don’t know if other people think about this and just don’t say it, or if my brain is just wired to follow every thread five steps further than it needs to go. But now more people are building agents, and building them means sitting down to explain how things actually work. When you try to build one that screens cover letters, someone has to explain what a good one looks like. And why. And it needs an actual answer, not “you just know.”
Prompting was forgiving. You could be vague, get a mediocre answer, and move on without ever confronting what you couldn’t articulate. But building an agent that runs a multi-step process? You can’t skip the explaining. You have to describe what the work actually is, what the steps are, what good looks like and why.
I keep seeing people list the steps for how something gets done, how an organization finds money, how a team screens candidates, how someone manages a pipeline. The steps are always fine individually. But together they describe a system that has never been questioned. It’s just been done. And now someone is about to make a machine do it the same way, faster, without asking whether any of it was worth doing.
That’s the idea I keep coming back to. Not when AI replaces the work, but when describing the work reveals it was never really thoughtfully designed. It was inherited, it was “how things are done.” Something that worked for the people who built it, and everyone else just learned to work around it.
The chatbox let you stay vague and figure stuff out as you go. Agents won’t.
I believe the articulation is coming whether organizations want it or not. Agents will force people to describe how things actually work. But what happens when they do? Will they see what I saw, that the system was designed around some people and everyone else learned to work around it? Or will they just describe it, automate it, and move on?
Organizations are good at protecting themselves. There’s a real chance that agents just cement inherited processes into code and make them even harder to change than when they were just habits. Automate a broken process, ship it, call it transformation. Nobody looks twice because the machine is doing it now and the numbers went up. The people deciding what to automate aren’t usually the ones the system excluded.
The articulation might happen, and the confrontation might not. I hope it does. Looking at what the explaining actually revealed about me and the systems I work inside wasn’t comfortable. But it was worth it.
— Emma
Support the work ☕️
Writing this takes time (and a lot of AI conversations). If you found this valuable, you can buy me a coffee.
It reminds me of a story told in IT. A senior programmer got so frustrated with juniors coming and asking simple questions that he set up a rubber duck on a pedestal and directed them to go ask the duck and if they still had questions to come back and ask. More often than not in talking to the duck (verbally) they didn't return to the senior. Because they had to think through the problem enough to articulate the question, they were able to see the answer.
Call me a pessimist, but I don't see processes/habits being thought through and questioned on a large scale. Any organisation should be so lucky to have someone like you who asks why. And so flexible to encourage that person to say that out loud. How many do, though? Especially the latter... I'm reminded of old Japanese factory work culture, where finding a better way to do it was considered a high virtue, and rewarded. Funny that a culture that on the surface is all about conformity does (did? not sure whether still in place) that, while Western culture on the surface is all about individuality and yet in the workplace, people just conform to "the way it has always been done".