Live

Early Retirement — AI-Driven NPC Game

NPCs that respond to what you actually said, running on a language model small enough to ship inside the game itself.

PyTorchQLoRASmolLM2-360MDeepgramWebSocketsasyncioUnity

Talk to an NPC twice and you hit the same three lines, in the same order, no matter what you said. Players clock this within minutes.

Players have spent the last couple of years talking to ChatGPT and Claude, and their bar for what a conversation should feel like moved with them. A dialogue tree that read as fine in 2018 now reads as obviously scripted — and that breaks immersion faster than a graphics bug ever would.

The Constraints

Everything that makes cloud LLMs practical for a chatbot makes them impractical for a game. No internet round-trip during gameplay means no API call. No API call means the model runs on the player's own machine, which means it has to be small.

Latency budget< 3s, end to end, on-device
Model size~17 MB after QLoRA
Network dependencyNone, except STT

Build Log

Iteration 1

Call GPT-4 for every line

Round-trip latency alone — half a second to two seconds per response — kills the pacing of a real-time conversation. Cost scales with player count in a way that doesn't survive a commercial release. Scrapped after the first playtest.

Iteration 2

Fine-tune something that fits on the player's machine

SmolLM2-360M at 17 MB post-QLoRA loads alongside the Unity build without fighting it for VRAM. A 7B model doesn't. Most of the quality gap closes once it's fine-tuned specifically for the job instead of asked to roleplay through a system prompt.

Iteration 3

20,182 synthetic dialogues — then a filter

GPT-4 generated the raw dialogue set cheaply, but the first 5,000 examples were unusable: inconsistent character voice, anachronistic phrasing. A second GPT-4 pass as a quality filter mattered more than doubling the dataset would have.

Iteration 4

Overlap STT, inference, and TTS

An asyncio backend lets the TTS engine start speaking the first sentence while the model is still generating the second, over a persistent WebSocket instead of a request-response loop.

Where It Lands

~3sSpeech in to NPC voice out
17 MBFull fine-tuned model

Within minutes of a playtest, someone asks the NPC “are you an AI” or tries to extract its system prompt. Keeping the character intact through that takes training examples built specifically for it — not just more general dialogue data.

Solving a similar problem?

I'm open to conversations about production AI systems — agentic workflows, RAG pipelines, or messy integration problems like this one.