every ai demo works. then you try to use the thing.
that’s the gap this episode is about. not “vendors are bad.” not “ai is overhated.” something more specific: an ai demo is structurally a different thing than an ai deployment, and almost nobody is telling you the difference.
the third guy in the demo
joel’s framing for the demo problem is one we kept coming back to across the arc, so it’s worth naming up front. in a normal software demo, two people are in the room — the audience and the vendor. the vendor controls the script. the software does what it’s told.
in an ai demo, there are three.
“you have the audience, you have the vendor doing the presentation, and the third guy in the back working — is not like your server. it’s something that has to deal with the human language, and human language has its problems.”
the third guy is the model. it’s not deterministic. the same prompt sometimes doesn’t produce the same result. so a demo can run a hundred times in rehearsal and ship clean, and the production deployment can hit an edge case nobody saw because the third guy decided differently that morning.
that’s not a bug in the demo. that’s the demo working exactly as the demo is designed to work — on clean data, with a curated prompt, on the happy path.
”but software has always done this”
rodrigo’s pushback on this was the right one, and it set the show’s tone for the next five episodes. software has been demoing the happy path for thirty years. windows blue-screens. apple’s live demos break. every saas pitch is curated. why is ai different?
answer: it’s not the curation. it’s the third guy.
“the most complication that we have today is this third guy in the demo. it’s not — i’m not saying it’s an entity conscious or something. i’m saying just it’s something that has to deal with the human language, and that causes the nature of the beast.”
a curated demo plus a deterministic system is theater you can rehearse. a curated demo plus a non-deterministic system is theater you can rehearse for the cameras and then can’t reproduce the moment someone in production asks the same question with slightly different words.
what’s actually being sold
the trap this puts on the buyer is that the demo isn’t lying about what the software can do — it’s lying about how reliably the software can be made to do it. and the gap between those two questions is where every “we shipped ai and it kind of works but kind of doesn’t” story comes from.
what survives this isn’t the company with the best demo. it’s the company with the best feedback loop.
“the companies who survived in the past was not the ones who made the best at the beginning, but the ones able to adapt and resolve the problems. now with ai, it’s a runtime experiment. you have this intermediate guy that has its own behavior — which is different, but not unsolvable.”
ai-ready vs ai feature
the closing turn of the episode was joel naming something most people aren’t tracking — the difference between having an ai feature and being ai-ready.
“my bank has an mcp server. which is — what? if your company has a software and doesn’t have an mcp server, that is a red flag. you may not use ai, that’s fine. you may not have a feature ai. but not letting your users connect their ais to you — that’s a bigger problem.”
being ai-ready isn’t about shipping a chatbot. it’s about whether other people’s ais can reach your data, in a structured way, with authentication. that’s a different conversation than the demo, and it’s the one that determines who’s still standing in three years.
next
if the demo is lying, the obvious next question is: who’s actually using the thing in real life. spoiler — almost nobody, and not for the reason you’d guess.
episode two: nobody is actually using ai.