LLM Function Calling in Unreal Engine
Function calling is the bridge between AI conversation and game action. Instead of just generating text, the LLM can decide to open a door, spawn an enemy, change the weather, give the player an item, or trigger a cutscene — all through natural conversation.
How Function Calling Works
You define a set of functions your game exposes — each with a name, description, and parameter schema. When the player talks to an NPC, the LLM can choose to call one of these functions instead of (or alongside) generating text.
Example: a player says "It's getting dark, can you light the torches?" The NPC AI calls setLighting("torches", true) and responds "Of course, let me get those lit for you." The torches actually light up in the game world.
What You Can Build
- Interactive NPCs that affect the world. Shopkeepers that actually hand over items. Guards that open gates. Wizards that cast spells on the environment.
- AI game masters. A dungeon master NPC that spawns encounters, adjusts difficulty, and narrates events — all through natural conversation with the player.
- Voice-controlled gameplay. "Build a wall here" → AI calls
placeStructure("wall", playerLookAt). Natural language becomes a game input method. - Dynamic quest assignment. AI evaluates the player's state and calls
assignQuest(questData)with a procedurally generated quest that fits the player's level and story progress. - Structured data generation. Use JSON schemas to guarantee valid output: inventory items, enemy configurations, dialogue trees, skill trees.
Supported Providers
Function calling is supported by all major providers in GenAI for Unreal:
- OpenAI — GPT-5, GPT-4o, o3, o4-mini (function calling + structured output)
- Anthropic — Claude Opus 4.6, Sonnet 4.6 (tool use)
- Google — Gemini 3.1 (function declarations)
- DeepSeek — V3.1 (function calling)
- XAI — Grok 4.1 (function calling)
All work through the same Blueprint and C++ API — define your functions once, switch providers freely.