How to Build an AI Companion in Unreal Engine
AI companions — characters that travel with the player, remember shared experiences, and respond intelligently to the game world — have been a dream of game development for decades. LLMs finally make them possible.
What Makes an AI Companion Work
- Persistent personality. The character maintains consistent traits, speaking patterns, and opinions across every interaction. A system prompt defines who they are — their backstory, quirks, knowledge, and limitations.
- Conversation memory. The companion remembers what you've discussed, what you've done together, and how the relationship has evolved. Multi-turn conversation history makes this work.
- World awareness. By injecting game state into the AI context — current location, recent events, inventory, quest progress — the companion comments on what's happening naturally.
- Emotional range. Modern LLMs like Claude Opus and GPT-5 generate genuinely nuanced emotional responses. Companions can be worried, excited, frustrated, or amused in ways that feel authentic.
- Agency. With function calling, companions can take actions — heal the player, point out dangers, suggest routes, or react to enemies they "see" through multimodal vision.
Architecture
- System prompt — define the character's personality, knowledge, and rules
- Context injection — feed current game state into each request
- Conversation history — maintain multi-turn memory
- Function calling — let the companion trigger game actions
- Streaming — deliver responses word-by-word for natural pacing
- Voice — use TTS to give them a unique voice (ElevenLabs, OpenAI TTS)
Implementation Loop (What Actually Works)
- Keep a short memory buffer for live dialogue (last 10-20 turns).
- Store long-term memory as compact facts ("player saved companion in chapter 2").
- Inject only relevant world state each turn (location, nearby actors, active objective).
- Use function calling for actions, not free-form text parsing.
- Stream responses and interrupt cleanly if combat or cutscenes begin.
Companion Guardrails
- Lore drift: add hard constraints in system prompt (factions, timeline, forbidden knowledge).
- Over-talking: enforce max token length and cooldown timers between unsolicited lines.
- Gameplay conflicts: prioritize gameplay events over dialogue generation callbacks.
- Cost spikes: route ambient chatter to smaller/local models and reserve premium models for story beats.
Choosing a Model
- Cloud (best quality): Claude Opus 4.6 for character depth, GPT-5 for speed + quality, Gemini 3.1 for multimodal
- Local (offline/free): Llama 3.1 8B or Mistral 7B via Ollama for companions that work without internet
- Hybrid: Use local models for routine banter, cloud models for important story moments
Tools
GenAI for Unreal provides everything needed: streaming chat, function calling, conversation history, vision, and TTS across all major providers.
GenAI Llama adds offline companion AI with zero cost using local models.