A Complete Guide to AI Function Calling (Tool Use) in Unreal Engine 5
For a long time, integrating AI into games meant building a glorified chatbot. An NPC could give you a beautifully written backstory, but if you asked them to unlock a door, they couldn't actually do it. They were trapped inside a text box.
Function Calling (also known as Tool Use in the Anthropic ecosystem) shatters that barrier. It allows Large Language Models (LLMs) to request the execution of predefined functions within your Unreal Engine 5 game. This gives AI true agency to affect the game state.
How the Conversation Loop Works
Function Calling is not a single API request; it is a multi-step conversation loop handled by the developer:
- Define the Tools: You send a system prompt to the AI, along with a list of tools (functions) it is allowed to use. You provide a JSON schema describing the required parameters (e.g., give_item(item_id, quantity)).
- The AI Decides: The player says, "I need a health potion." The AI analyzes the request and, instead of replying with standard text, returns a special JSON payload indicating it wants to invoke give_item("health_potion", 1).
- Local Execution: Your Unreal Engine code intercepts this response. You parse the requested function, validate the arguments, and execute your gameplay logic (spawning the potion in the player's inventory).
- Return the Result: You append a new message to the conversation history containing the result of your function (e.g., {"status": "success", "inventory_full": false}) and send it back to the AI.
- Final Synthesis: The AI reads the result and generates a natural language response for the player: "Here is your health potion, traveler!"
Implementing Tool Use in UE5
The GenAI for Unreal plugin provides a unified set of Structs and Blueprint utility nodes that abstract away the manual JSON parsing for both OpenAI (GPT-5.4 Pro) and Anthropic (Claude 4.6).
1. Defining the Tool (C++)
You use the FGenAIToolDefinition struct to declare a capability to the AI.
FGenAIToolDefinition WeatherTool;
WeatherTool.Name = TEXT("set_weather");
WeatherTool.Description = TEXT("Changes the current weather in the game world.");
WeatherTool.ParametersJson = TEXT(R"({
"type": "object",
"properties": {
"weather_type": {
"type": "string",
"enum": ["clear", "rain", "storm", "snow"],
"description": "The type of weather to set."
}
},
"required": ["weather_type"]
})");
// Add this tool to your Chat Settings
ChatSettings.Tools.Add(WeatherTool);
2. Handling the Tool Call (Blueprint)
When the async request finishes, game designers can use Blueprint utility nodes to check if the AI requested a tool, rather than returning text.
// Pseudocode for Blueprint Tool Parsing
If (Response Has Tool Calls)
{
ToolCallsArray = Parse OpenAI Tool Calls(Response)
ForEach(ToolCall in ToolCallsArray)
{
If (ToolCall.Name == "set_weather")
{
WeatherVal = Get Tool Call Argument(ToolCall, "weather_type")
ExecuteGameWeatherChange(WeatherVal)
// Append Result to History and Re-send Request!
}
}
}
Crucial Best Practices for Game Security
Giving an AI agency comes with severe risks if not managed correctly. Players *will* attempt to "jailbreak" your NPCs to spawn infinite gold or skip quest flags.
- Never trust the AI blindly: Just because the AI calls grant_gold(1000000) does not mean you should execute it. Your C++ or Blueprint function must validate that the NPC actually *has* the authority or inventory to grant that request.
- Use Enums in Schemas: As shown in the weather example above, use JSON enum arrays to force the AI to pick from a predefined list of valid strings. Do not let it hallucinate a weather state called "acid_rain" if your game doesn't support it.
- Strict Schema Adherence: When using OpenAI models, you can set bStrict = true on your Tool Definition. This forces the model to guarantee its output matches your JSON structure perfectly, preventing serialization crashes in Unreal.