Better AI Tools are Built Together

I’ve spent most of my career working at the intersection of product management and software engineering. Traditionally, much of that work involved bridging a gap: a subject-matter expert understood the problem, but needed a software team to turn that knowledge into something people could use.

The process usually followed a familiar path. A product manager translated the expert’s knowledge into requirements. A designer turned those requirements into workflows. Engineers turned the workflows into code. The expert might review the result along the way, but building the product was largely something the software team did on their behalf.

AI is changing that relationship.

At Praxplay, our co-founder Paul is a mental health expert with a PhD. He can now use AI to explore a clinical framework as software…not just describe it in a document and hand it off. He can test an interaction, change the rules, generate sample clients, and see how an idea behaves without waiting for a product team to write a ticket or an engineer to start a sprint.

That shortens the distance between expertise and a working tool. It also removes some of the translation work that has traditionally belonged to product managers and engineers. A small team, or sometimes one motivated expert, can now build things that recently required several specialists.

Which raises an obvious question about our own company: if Paul has the domain expertise and AI gives him the ability to build, why isn’t Praxplay a one-man show? If AI is taking on the translation, prototyping, and coding work, what are the rest of us still here to do?

The answer isn’t that Paul can only make a toy and needs engineers to build the “real” version. That would preserve the old hierarchy under a new name. He can make meaningful parts of the product, and that should change how the rest of us work.

What he cannot, and should not have to, do alone is supply every form of judgment the product needs. The person defining the clinical truth should not also be the only person testing the implementation, anticipating how unfamiliar users will behave, challenging assumptions, tracing failures, and deciding whether the system is reliable enough to earn trust.

For those of us in product and engineering, that is the shift. Our value can no longer rest solely on turning someone else’s well-formed idea into requirements or code. Increasingly, our most important work begins after an expert gets something working, because “it works” is where product development traditionally starts getting difficult.

The gap between a working tool and a dependable product

At Praxplay, we’re building simulated clients that counseling students can practice with before sitting across from a real person. That gives us very little room to confuse an impressive demonstration with a dependable product.

Paul has written a clinical model spanning more than a hundred pages. It describes how clients present, protect themselves, resist, respond, and change over the course of a counseling session. With AI, he can do far more than hand that model to engineering as a static document. He can explore it, challenge it, generate examples, and participate directly in shaping how it behaves as software.

That is exactly what we want. The closer the person with the domain knowledge is to the thing being built, the less expertise gets lost in translation.

But a tool can produce a convincing response and still be wrong in ways that are difficult to see. It can work beautifully on the example its creator had in mind and fail on the next twenty. It can confuse a plausible answer with a clinically sound one. It can behave differently after a model update. It can be too slow, too expensive, or too inconsistent for repeated use. It can make perfect sense to the expert who built it while leaving a first-time user unsure what to do next.

AI makes it much easier for Paul (and for all of us) to reach the first successful demonstration. It does not remove the distance between that demonstration and a product people can trust.

Closing that distance is not work we take back from Paul. It is work we do with him, bringing different kinds of scrutiny to the same product.

We debug more than code

Traditional software bugs are often concrete. A request fails. A value is calculated incorrectly. A screen doesn’t load. AI systems add less obvious failure modes. The code may be correct while the behavior is wrong. A problem might come from the prompt, the underlying domain model, retrieved context, conflicting instructions, an ambiguous definition, or the model itself. The same input can expose a problem only some of the time.

Debugging that requires a particular kind of stubbornness: reproducing the failure, separating its layers, creating useful tests, and refusing to accept “the AI is unpredictable” as the final explanation.

It also requires close work with the expert. Engineering can determine where a behavior came from, but only the domain expert can tell us whether that behavior is meaningful or merely convincing.

We saw this while implementing a formula in our clinical model. Two concepts appeared to be opposites, but the formula used one where I expected the other. Every engineering instinct I had said it was a bug. It wasn’t. The concepts weren’t mathematical inverses. Their relationship came from the clinical framework, and once Paul explained it, the formula made sense. If I had “fixed” it, the implementation might have looked cleaner and become less clinically correct.

The reverse is also important. Translating the framework into an executable system exposed ambiguity that was easy to pass over in prose. Code forced an exact question. I could locate the inconsistency; he could determine what was true.

AI can help both of us reach that moment faster. Resolving it still requires both perspectives: engineering to make the ambiguity visible and clinical expertise to determine what the model actually means.

We look for the cases the builder isn’t imagining

When someone builds a tool around their own expertise, they naturally test it using the way they understand the problem. Their mental model fills in gaps the interface does not. This is one reason even an expert empowered by AI benefits from a team around them.

Users don’t have that model. They misunderstand labels. They skip setup steps. They enter incomplete or contradictory information. They abandon a workflow halfway through and return three days later. They use a feature for a purpose its creator never considered.

In our case, we also have to ask questions with higher stakes. What happens when a trainee pushes too hard, too early? Can the simulated client become unrealistically cooperative? Does a fluent response hide a contradiction in the client’s history? Could the system inadvertently reward poor counseling techniques? What happens when the clinical model does not define a situation the AI can still confidently improvise around?

This is more than quality assurance. It is the discipline of distrusting the happy path. Subject-matter experts are essential to defining the important edge cases. Product managers, designers, and engineers contribute experience finding categories of failure, turning them into testable scenarios, and designing what the system should do when it cannot proceed safely.

We bring product judgment, not just requirements

Giving an expert the ability to build does not automatically answer what should be built, or how that capability should meet a user who does not share the expert’s knowledge.

An expert tends to see the full complexity of a domain. A user needs the right amount of that complexity at the right moment. Those are not the same thing.

Product work is deciding where a tool fits into someone’s actual day: what they need to understand before they begin, which decisions the system can make for them, where they need control, how they recover from mistakes, and what feedback helps them improve.

UX knowledge matters here. So do interviews, observation, prototypes, usage data, and the willingness to remove a feature that is technically impressive but makes the experience worse.

AI can generate interface options. It can summarize research and critique a flow. It can help an expert create a surprisingly polished application. But it does not sit with a confused user and notice the pause before they click. It does not own the decision about whether that confusion is acceptable. Someone still has to connect capability to behavior.

The product manager’s future is probably less about moving information between people and more about improving the decisions the team makes together.

We make the system accountable

A prototype is allowed to work today. A product has to keep working tomorrow.

That means versioning the domain model so we know which clinical rules produced a session. It means regression tests based on examples Paul has worked by hand. It means logs that help us reconstruct a failure instead of guessing what happened. It means monitoring cost, latency, model changes, privacy, and security. It means designing fallbacks for the moments when an external service fails or the system does not know enough to respond.

We also document the judgment calls made while translating expertise into software. When the source material leaves room for interpretation, I can make a provisional engineering decision…but I write down what was ambiguous, what I chose, and why. Paul can review those decisions in batches and overturn them.

That process lets us move without allowing engineering assumptions to quietly become clinical truth.

None of these practices are as immediately impressive as generating a working application from a conversation. They are what allow us to trust the application after the demo.

Fewer translators, more collaborators

So why isn’t Praxplay a one-man show?

Because a product like ours requires more than one kind of expertise and more than one person’s scrutiny. Paul determines whether the clinical behavior is sound. Product and UX work help us understand whether the experience serves students and educators. Engineering makes the behavior reproducible, testable, observable, and resilient. Each discipline catches problems the others are not trained to see.

I do think AI will reduce the number of product managers and software engineers needed to build many products. One expert working with AI and a small technical team can now cover ground that used to require a much larger organization.

I also think the people who remain will work differently. The product manager cannot survive as a requirements courier. The engineer cannot define their value by how much code they type. Neither role gets to claim ownership simply because software used to be inaccessible to everyone else.

Our value is in helping an expert’s idea survive contact with users, edge cases, ambiguity, scale, and time.

That means asking the irritating question after the exciting demo. Reproducing the failure everyone else saw once. Recognizing when a technically elegant solution creates a confusing experience. Making tradeoffs around speed, cost, and reliability visible. Building tests for domain behavior, not just code paths. Knowing when the system should stop improvising and admit uncertainty.

Most importantly, it means giving up the old handoff. The expert should not be asked to transfer their knowledge to a software team and wait for it to come back as a product. AI now lets them stay inside the building process. Product and engineering should make that participation more powerful, not reclaim control from them.

The expert can build now. Our job is to help make what they build trustworthy.