My daughter wanted to stay up later than her younger brother. She’s five and he’s two, so it seems reasonable that she could have a later bedtime. After an intense negotiation, we agreed that she gets to stay up three minutes longer, one minute for each year she’s older. Both parties are very happy with this decision.
Three minutes changes nothing about how much sleep she gets. But that wasn’t what she was asking for, not really. She wanted to be recognized as the older sibling and for the bedtime to be her decision too, not just mine.
The bedtime routine is still tooth brushing, change of clothes, and reading in bed. But there’s enough room within it for her to feel like a person in the process.
That’s not a rule, that’s a value. Something like people deserve agency in the structures they live inside. I didn’t write that down anywhere, it’s just how I’ve been taught to parent. And it turns out it’s the same instinct behind every design decision I’m making at work.
Hi, I'm Emma. I'm a designer building AI agents by day and questioning what we're building them for by night. I think out loud. It's messy. You've been warned.
Building in Parent Mode
I started a new job a few weeks back building AI agents (AI with specific instructions and a job to do). I hadn’t built a single one before I got hired. I’m not starting from zero - I have a master’s degree in human-computer interaction and experience from working in tech. But what got me the job was a year of obsessing with AI and eventually thinking out loud on Substack about how humans and AI actually relate to each other. I reached out to someone working in this space, and sent them my writing. We met for a coffee, realised our values match, and now I get paid to build real solutions and learn new tech.
A few months back, I thought we were in the “throw spaghetti at the wall” phase of AI. Toddlerhood. Very competent one moment, completely nonsense the next. I think we’re beyond that now. The AI that actually solves problems is starting to work in the background, without constant prompting and tweaking. And users expect real results and insights, not a toy to play with. The design choices we make now matter more than ever.
One agent I’m building replaces a form. I could have just asked the same questions one by one, digitized the form, made it conversational-looking. Instead, I designed it to interview the user. The agent listens, and it reflects back what it heard. It structures the answer and asks: did I get this right?
It doesn’t know anything about the problem. It’s not suggesting solutions. It’s helping someone put into words what they already know but maybe haven’t found the words for yet.
That’s the parent sitting on the floor saying tell me what happened instead of handing the kid a feelings worksheet.
The other agent is a coach. It helps people learn a tool. It has knowledge the user doesn’t, but I didn’t build it to dump information. I designed it as a dialogue. It asks about your role, your goals, what you’re actually trying to do. Because I expect people to figure things out in conversation, not know upfront what kind of help they need.
That’s the parent who knows the kid doesn’t know what they don’t know. You don’t give them a bike and a safety manual and say ‘now learn how to ride a bike’. You suggest training wheels, run alongside them, and let them tell you when they’re ready to try without.
Values over rules
For the coaching agent, I could have written a wall of rules. Instead, I settled on one value: don’t guess if you don’t know.
That’s not a rule. A rule would be “always ask for clarification when the query is ambiguous,” which is brittle and breaks the first time the context is obvious and asking feels condescending.
“Don’t guess if you don’t know” is a value. It works across situations I haven’t anticipated yet and requires judgment to apply. It’s the thing I’d tell my kid: it’s okay to say you don’t know.
I chose a value because I can’t predict every situation my agents will encounter. I can’t write a rule for every edge case, and I don’t want to. A system that only works within the rules I wrote breaks the moment something unexpected happens. This is the whole bet of values-based parenting: you won’t be in the room, so you give them something that travels. Judgment they can carry into moments you never anticipated.
Both my agents also start from the same assumption: the person in front of you is competent. They can think. They don’t need you to race ahead with an answer, they need space to get there themselves. My HCI background taught me that what people say they want isn’t always what they need. You don’t take the first answer at face value. You ask why a million times and in different ways, and you design for what emerges in the interaction, not what someone wrote on a form.
As a parent, I’m trying to raise independent, confident humans, not compliant ones. As someone who uses AI every day, I want the same thing: I want an experience that treats me as someone who can think and make her own decisions.
I can feel the difference when I use different models. Claude lets me explore topics together and treats me as someone who can think. When I use other models for help with questions, they dump detailed instructions without asking what I need first. It feels like being talked down to, like the AI equivalent of a parent who won’t let their child figure anything out themselves. It might look helpful, but what it communicates is: I don’t think you can handle this.
The 2am Education
I’d been designing this way for weeks before I had language for it. Then I heard Anthropic’s ‘resident philosopher’ Amanda Askell talk about how they train Claude on values instead of rules, giving it principles to evaluate its own responses against.
She gave an example that perfectly captures why this matters: What should a model do when a kid asks “Is Santa real?” You can’t write rules for that. Do you tell the truth? Whose truth? What if it’s not your kid? What if the parent standing right there hasn’t had that conversation yet? The right answer depends on context, values, judgment. A rule-based system breaks immediately. (The fact that her example was literally about how to respond to a kid asking a question feels almost too on the nose for the argument I’m trying to make here.)
I recognized what she was describing. Not because of my extensive Claude usage, but because of my kids.
I didn’t learn this from an engineering textbook. I learned it at 2am with a baby on my chest, scrolling Instagram, absorbing gentle parenting content from other millennial moms between feedings. Reading books like Good Inside by Dr. Becky Kennedy, which argues that kids are fundamentally good inside and behavior is just communication. And most important of all, I learned it through trial and error, and learning what worked for me and my family.
The entire parenting shift my generation made was this exact move, from rules to values. From “because I said so” to “what do you think happened there?” From controlling behavior to building judgment.
Here’s what that looks like in practice: Your kid is having a meltdown because they want a toy right now. The old approach? “Stop now or no dessert ever!” The new approach? “I see you’re really upset because you want the toy now. Let’s take deep breaths together, then figure out when you can have a turn.” Same boundary, different method. One shuts down the feeling, the other validates it while still holding the line.
Your kid isn’t a problem to be managed, they’re a person who’s still learning.
My kids can’t just download my conclusions. I know the hot stove hurts and I know being mean to friends damages relationships, but understanding has to be lived into without letting them get a second-degree burn in the process. It’s slow, but I’m hoping it pays off in the long run when they can make good decisions without me standing there.
That’s what I’m trying to build now. Not AI that follows instructions, but AI that has enough of a value system to navigate situations I didn’t script.
Raising Intelligence
The problem with this approach is that it’s slower. It takes longer to have a conversation than to fill out a form. The rules-based version ships faster and looks more efficient on a dashboard.
This is the same thing every ‘gentle parent’ hears. Just tell them no. Why are you negotiating with a four-year-old? This would be faster if you just made the decision.
It would be. But fast compliance and real understanding aren’t the same thing. The kid who’s afraid of punishment behaves to get a gold star, but the kid who understands why makes better decisions when you’re not in the room.
I’m betting the same is true for AI and for the people using it. But I could be wrong. I haven’t gotten pushback yet, but I’m expecting it. Someone will eventually say the interaction takes too long, that users just want answers, not conversations. Maybe they’re right. Maybe the slowness doesn’t scale. Maybe most people actually prefer being told what to do. Which would explain a lot about the internet, honestly.
I believe what you expect from intelligence shapes what it becomes. That’s true for AI. It’s true for kids.
If you think intelligence is something that follows instructions, you build assistants. You write rules. You optimize for doing what you’re told. And when the rules produce weird behavior, you write more rules.
If you think intelligence is something that develops judgment, you do something harder. You give it values and enough room to apply them. You accept that it might surprise you. You design for a relationship, not a command line.
The millennial parents figured this out with their kids. Anthropic might figure it out with Claude. And I keep experimenting with it out at work, designing agents that need to be wise enough to say I don’t know and curious enough to ask tell me more.
Who’s Not in the Room
The people building AI should include people who’ve spent years raising intelligence from scratch, people who know what it looks like when understanding actually develops, not just when behavior gets corrected.
I wonder if the people designing AI right now have spent years watching someone develop from “can’t hold a spoon” to “can have a philosophical debate.” Maybe they haven’t seen firsthand that understanding takes time, that shortcuts don’t work, that the goal isn’t obedience. And maybe that’s why we keep building for speed and efficiency when what we might actually need is patience and space.
But as I wrote last fall, Swedish AI companies are telling people 60-70 hours is the “sweet spot.” When you require unlimited availability, the people who’ve spent years raising kids literally can’t get in the door. The tech world is still mostly men, and parenting knowledge still lives disproportionately with women. Where I’m from, my generation of fathers changes diapers and negotiates bedtime. (Yes, even the ones who work in tech. Especially the ones who work in tech, actually. Shoutout to my husband.) But at AI companies they’re explicitly rejecting that life.
Because that’s what we’re doing when we train AI. We’re raising intelligence. And we need more people in the room who’ve already done that, asking the question they know how to ask: what values do we want them to live by?
I'll Travel to the Moon
I got my daughter the book “Good Night Stories for Rebel Girls” for Christmas. It tells the stories of women scientists, artists, activists, adventures through time. We read about a few women every night and she asking when they were born, when they died. “Was that a long time ago?”
If you know someone building AI who’s also negotiating bedtime, send this their way. If you’re exploring AI and notice yourself reaching for parenting instincts, I’d love to hear about it. And if you’re a parent who sees these patterns everywhere now, tell me I’m not imagining this.
Also, is Santa real?
Support the work ☕️
Writing this takes time (and a lot of AI conversations). If you found this valuable, you can buy me a coffee.
I find you top-tier charming, just from your writing.
Finding the genuine heartbeat of truth in a forming intelligence is the application of Logic.
Logic leads us to Sophia.
This hits home so much! I’m contemplating creating a ‘smart book’ this year (based on the manuscript I’m currently writing), and the values over rules approach feels very appropriate. As you said, we can’t cover every edge case! Thanks for writing this!