Muddy Terrain Games

Products Careers Support

Documentation: GenAI for Unreal

Welcome to the complete guide for the GenAI for Unreal plugin. This document provides all the necessary information to integrate, use, and master the plugin in your Unreal Engine projects.

Table of Contents

1. Quick Start: Your First AI Chat in 5 Minutes

Follow these minimal steps to get a basic chat completion working instantly.

  1. Install Plugin: Add "GenAI for Unreal" from the Fab.com marketplace and enable it in Edit > Plugins.
  2. Add API Key: Go to Project Settings > Plugins > GenAI Plugin and enter your OpenAI API key.
  3. Create Blueprint: Make a new Actor Blueprint. In the Event Graph, add the following nodes:
    4. Quick Start Blueprint Example
    A simple setup to test chat completion.
  4. Node Setup:
    • Connect Event Begin Play to a Request OpenAI Chat Completion node.
    • For the Settings pin, add a Make Gen AI Chat Settings node.
    • In the Messages array, add a Make Gen Chat Message with the role user and content "Tell me a joke."
    • Drag off the OnComplete event and add a Print String node, connecting the Response output to the string input.
  5. Press Play: You will see an AI-generated joke printed to the screen.

2. Initial Project Setup

Follow these steps to get the plugin running in your project.

Adding the Plugin

  1. Acquire the GenAI for Unreal plugin from Fab.com. Once downloaded click on Install to engine from the epic's launcher.
  2. Open your project and enable the "GenAI For Unreal" plugin from the Edit > Plugins menu. Restart the editor if prompted.

C++ Project Configuration (Optional)

To use the plugin's classes in your C++ code, you must add its modules to your project's build file.

  1. Open YourProject.Build.cs located in your project's Source/YourProject directory.
  2. Add "GenAI" to your list of public dependencies. This gives you access to all the core functionalities.
using UnrealBuildTool;

public class YourProject : ModuleRules
{
    public YourProject(ReadOnlyTargetRules Target) : base(Target)
    {
        PCHUsage = PCHUsageMode.UseExplicitOrSharedPCHs;
    
        PublicDependencyModuleNames.AddRange(new string[] { 
            "Core", 
            "CoreUObject", 
            "Engine", 
            "InputCore", 
            "GenAI"  // <-- ADD THIS MODULE
        });
    }
}

3. Getting API Keys

To use this plugin, you need an API key from at least one of the following services. An API key is a unique code that authenticates your requests with the AI provider's servers. You will be billed by the respective provider based on your usage.

  • OpenAI (for GPT models, DALL-E, Whisper, TTS):
    1. Go to the OpenAI Platform and create an account.
    2. Navigate to the "API Keys" section in your user dashboard.
    3. Create a new secret key and copy it immediately. For security, OpenAI will not show you the key again.
    4. You may also need to set up a payment method to access certain models.
  • Anthropic (for Claude models):
    1. Visit the Anthropic Console and sign up.
    2. Go to "Account Settings" and find the "API Keys" section.
    3. Generate and copy your new API key.
  • DeepSeek / XAI / Google Gemini:
    1. Follow the specific sign-up and API key generation process on each provider's respective developer platform.
    2. DeepSeek Platform
    3. XAI (Grok)
    4. Google AI for Developers (Gemini)

Once you have your key(s), proceed to the Authentication & Security section below to learn how to add them to the plugin securely.

4. Authentication & Security

Setting API Keys:

Never hard-code API keys. The plugin provides a secure, centralized settings panel.

  1. In the Unreal Editor, navigate to Edit > Project Settings.
  2. Scroll down to the Plugins section on the left and click on GenAI Plugin.
    3. Settings panel for API keys
    Settings panel for API keys
  3. Enter your API keys from each provider (OpenAI, Anthropic, etc.) into the corresponding fields.
  4. These keys are automatically saved to an encrypted binary file at YourProject/Saved/Config/GenAI/secureconfig.bin.
  5. This file must not be committed to source control. Generally "Saved" folder is already part of .gitignore, if it is not add the following line to it to prevent leaking your keys: 
    /Saved/Config/GenAI/secureconfig.bin

Using a Proxy Server (Optional, For Production):

Why use a proxy?
  • Security: Your API keys never leave your server. The client application makes requests to your server, and your server securely attaches the API key before forwarding the request to the AI provider.
  • Centralized Control: Manage keys, monitor usage, and control access for all users from a single backend.
  • Custom Logic: Add logging, caching, or pre/post-processing logic on your server before returning data to the game client.
Retargetting the plugin to proxy:

Once your own backend server is running:

  1. Go to the GenAI Plugin settings in the Unreal Editor.
    2. Settings panel for API keys
    Retargetting the API requests to proxy server
  2. Under the API Endpoint Management section for each provider, check the "Override" box.
  3. Enter the URL of your proxy endpoint. For example, for OpenAI, you might enter https://api.yourgame.com/openai.

Now, all API calls from the plugin to that provider will be routed through your server instead of directly to the AI provider's endpoint.

5. Core Concepts

Asynchronous by Design

All API calls in this plugin are asynchronous (non-blocking). This is essential to prevent your game from freezing while waiting for a response from the AI server. The workflow is always:

  1. You call a function to send a request (e.g., UGenOAIChat::SendChatRequest).
  2. You provide a callback delegate (in C++) or connect to an event pin (in Blueprints).
  3. Your game continues running without interruption.
  4. When the response arrives, your callback function or event is executed with the result.

Using Custom Models (Not in Enums)

AI providers release new models frequently. You don't have to wait for a plugin update to use them. All settings objects (e.g., FGenOAIChatSettings) allow you to specify a model name as a simple string, which will always override the enum selection.

Blueprint Example:

In the "Make Gen OpenAI Chat Settings" node, simply type the new model's name (e.g., "gpt-5-exclusive-preview") into the Model (String) input field. This will be used instead of the enum dropdown.

C++ Example:
#include "Models/OpenAI/GenOAIChat.h"

void AMyActor::SendCustomModelRequest()
{
    FGenOAIChatSettings ChatSettings;
    ChatSettings.Model = EOpenAIChatModel::Custom; 
    ChatSettings.CustomModelName = TEXT("gpt-5-exclusive-preview"); 
    
    // ... complete the rest of the request
}

Additional Points to Note

  1. HTTP Timeouts: Complex reasoning models (like DeepSeek Reasoner) can take longer to generate a response. If you experience frequent timeout errors, consider increasing the default HTTP timeout in your project's DefaultEngine.ini file: 
        [/Script/Engine.NetworkSettings]
    NetConnectionTimeout=180.0 
    [HTTP]
    HttpConnectionTimeout=180.0 
    HttpReceiveTimeout=180.0
  2. Audio Formats (TTS): The Text-to-Speech feature currently outputs audio in the PCM format. The plugin includes Blueprint utilities to convert this raw data into a playable USoundWave asset at runtime.
  3. API Key Security: The secureconfig.bin file is encrypted and non-portable by design. Do not attempt to copy it between projects or machines. Always add API keys through the Project Settings panel. Never commit this file to source control.
  4. Platform Compatibility: The plugin uses standard engine modules (HTTP, Json) for maximum cross-platform compatibility. However, always test features on your specific target hardware, especially file-system-dependent operations like loading images for vision requests.
  5. Plugin Dependencies: This plugin relies only on core Unreal Engine modules. It does not introduce any external third-party libraries, ensuring a clean and stable integration.

6. Usage Guides & Examples

Finding the Example Assets

  • Blueprint Examples: Example blueprints can be found inside the plugin's content folder at Engine/Plugins/GenAIForUnrealContent/ExampleBlueprints/.
    • Blueprint Example
    Blueprint Examples
  • C++ Examples: The plugin has a seperate module for C++ examples called GenAIExamples.

Blueprint Usage Walkthrough

All Blueprint nodes are asynchronous (latent) and have a similar structure. Here is a walkthrough for a OpenAI multimodal chat request.

Blueprint Node Example

  1. Request OpenAI Chat Completion: This is the main node that initiates the API call.
  2. Settings: Use a "Make" node (e.g., Make Gen OpenAI Chat Settings) to configure the request. Here you set the model, temperature, max tokens, and importantly, the messages.
  3. OnComplete (Event): This event pin fires as soon as a response is received from the server. It provides the final Response string, an Error Message if something went wrong, and a `Success` boolean.

C++ Usage Walkthrough (OpenAI Chat)

The C++ workflow mirrors the Blueprint logic, using settings structs and response delegates.

#include "Data/OpenAI/GenOAIChatStructs.h"
#include "Models/OpenAI/GenOAIChat.h"

void ATOpenAIChatExample::TestTextChat()
{
	UE_LOG(LogGenAI, Log, TEXT("--- Testing OpenAI Text-Only Chat ---"));
	
	FGenOAIChatSettings ChatSettings;
	ChatSettings.Model = EOpenAIChatModel::GPT_4_1_nano; // Use the latest text model
	ChatSettings.Messages.Add(FGenChatMessage(TEXT("user"), TEXT("What is the distance between Earth and the Moon in kilometers?")));
	ChatSettings.MaxTokens = 100;

	UGenOAIChat::SendChatRequest(
	   ChatSettings,
	   FOnChatCompletionResponse::CreateLambda(
		  [this](const FString& Response, const FString& ErrorMessage, bool bSuccess)
		  {
			 if (!UTHelper::IsContextStillValid(this)) return;
			 
			 if (bSuccess)
			 {
				 UE_LOG(LogGenAI, Log, TEXT("OpenAI Text Chat Response: %s"), *Response);
			 }
			 else
			 {
				 UE_LOG(LogGenAI, Error, TEXT("OpenAI Text Chat Error: %s"), *ErrorMessage);
			 }
		  })
	);
}

7. Structured Output

Structured Outputs is a feature that ensures the model will always generate responses that adhere to your supplied JSON Schema, so you don’t need to worry about the model omitting a required key, or hallucinating an invalid enum value.

Some benefits of Structured Outputs include:

  • Reliable type-safety: No need to validate or retry incorrectly formatted responses
  • Explicit refusals: Safety-based model refusals are now programmatically detectable
  • Simpler prompting: No need for strongly worded prompts to achieve consistent formatting

Schema JSON example

{
"type": "object",
"properties": {
    "count": {
        "type": "integer",
        "description": "The total number of users."
    },
    "users": {
        "type": "array",
        "items": {
            "type": "object",
            "properties": {
                "name": { "type": "string", "description": "The user's name." },
                "heading_to": { "type": "string", "description": "The user's destination." }
            },
            "required": ["name", "role", "age", "heading_to"]
        }
    }
},
"required": ["count", "users"]
}

Using structured output in the C++:

FString MySchemaJson = R"(<insert your schema JSON here>)";

UGenAISchemaService::RequestStructuredOutput(
    TEXT("Generate a list of users and their details"),
    MySchemaJson,
    [](const FString& Response, const FString& Error, bool Success) {
       if (Success)
       {
           UE_LOG(LogTemp, Log, TEXT("Structured Output: %s"), *Response);
       }
       else
       {
           UE_LOG(LogTemp, Error, TEXT("Error: %s"), *Error);
       }
    }
);

In Blueprints:

Blueprint Structured Output Example
Blueprint Structured Output Example

8. How to Run Tests

The plugin includes a suite of automation tests to verify that all API integrations are working correctly. To run them:

  1. Ensure you have set valid API keys in the plugin settings for the providers you wish to test.
  2. In the Unreal Editor, go to Window > Session Frontend.
  3. In the "Automation" tab, you will see a list of available tests.
  4. Find the tests prefixed with GenAI. (e.g., GenAI.OpenAI.Chat).
  5. Check the boxes for the tests you want to run, and press the "Start Tests" button.

The tests will run and report their status. This is a great way to confirm your setup is correct or to debug integration issues.

9. Quick Links & API Reference

This documentation is based on the GenAI for Unreal plugin. Specific function names, struct members, and Blueprint node names may vary slightly based on the final plugin version. Always refer to the plugin's source code or example content for the most accurate implementation details.