Open Source LLMs for Unreal Engine - The Complete Guide
You don't need to pay per-request API fees to add AI to your Unreal Engine game. Open-source large language models have reached a point where they deliver genuinely useful results — and they run entirely on local hardware. No cloud, no API keys, no ongoing costs.
This guide covers every major open-source LLM you can run in Unreal Engine today, which ones work best for different game scenarios, and how to integrate them.
The Open Source LLM Landscape
Meta Llama 3
The most popular open-source model family. Llama 3.2 comes in 1B, 3B, 8B, and 70B sizes. The 8B variant is the workhorse — handles dialogue, quest generation, and narrative AI with high quality. Vision variants can process images. Full Llama 3 guide.
Mistral / Mixtral
French AI lab Mistral builds exceptionally efficient models. Mistral 7B rivals much larger models, and Mixtral 8x7B uses mixture-of-experts to deliver near-frontier quality at moderate hardware requirements. Excellent multilingual support. Full Mistral guide.
Microsoft Phi
The smallest capable models available. Phi-3 Mini at 3.8B parameters runs on integrated GPUs and still handles structured game logic well. Perfect for shipping AI that works on the widest range of player hardware. Full Phi guide.
Google Gemma
Built from the same research as Gemini. Gemma 3 1B is one of the fastest models available — sub-second responses on any GPU. Multimodal variants support vision input. Full Gemma guide.
DeepSeek
DeepSeek V3 and R1 are among the strongest open-source models period. R1 is a reasoning model — it "thinks" through problems step by step, making it excellent for puzzle logic, complex quest design, and game master AI. Available locally via Ollama or through cloud APIs. Full DeepSeek guide.
NVIDIA Nemotron
NVIDIA's Nemotron 3 Nano is optimized for NVIDIA GPUs with excellent inference speed. If your players have NVIDIA hardware (most do), Nemotron delivers strong performance.
CodeLlama / DeepSeek Coder / StarCoder
Code-specialized models are useful for procedural content generation with structured output. When you need your AI to output valid JSON for inventory items, quest structures, or game events, code models follow formatting instructions more reliably.
TinyLlama / Qwen 2.5 0.5B
The smallest usable models. Under 1B parameters. Limited capability but can handle simple tasks like generating item names, flavor text, or random NPC barks. Run on anything.
Which Model Should You Use?
| Use Case | Recommended Model | Why |
|---|---|---|
| NPC barks / simple text | Gemma 3 1B or Phi-3 Mini | Ultra-fast, runs on anything |
| Multi-turn dialogue | Llama 3.1 8B or Mistral 7B | Good quality, reasonable hardware |
| Quest / story generation | Llama 3.1 8B or DeepSeek V3 | Creative writing strength |
| Complex reasoning / puzzles | DeepSeek R1 or Mixtral 8x7B | Strong logical reasoning |
| Structured JSON output | CodeLlama or DeepSeek Coder | Reliable formatting |
| Multilingual games | Mistral or Qwen 2.5 | Best multilingual support |
| Vision / image analysis | Llama 3.2 Vision or LLaVA | Multimodal capability |
How to Run Open Source LLMs in Unreal Engine
All of these models work through Ollama, which provides a simple local API. We offer two Unreal Engine plugins:
GenAI Llama (Local AI Plugin)
GenAI Llama connects Unreal Engine to 7 local inference providers (Ollama, LM Studio, llama.cpp server, vLLM, LocalAI, Jan) plus embedded llama.cpp for in-process GGUF inference. Chat completions, streaming, vision support. Available on Fab.
GenAI for Unreal (Full Platform)
GenAI for Unreal supports local models through OpenAI Compatible Mode alongside ChatGPT, Claude, Gemini, Grok, and DeepSeek cloud APIs. Run local models offline and switch to cloud when connected.