Optimizing UI for Streaming LLMs in Unreal Engine 5
When you ask an AI model like ChatGPT or Claude for a long paragraph of text, waiting for the entire generation to finish before showing the user a response can feel sluggish. The solution is Streaming—delivering the text token-by-token (the "Typewriter Effect") via an open HTTP connection.
However, blindly dumping streaming text into an Unreal Engine 5 UMG (Unreal Motion Graphics) UTextBlock can cause severe performance bottlenecks and strange visual bugs if not handled correctly. This guide covers how to optimize streaming UI using the GenAI for Unreal plugin.
The Problem: The Naive Approach
When a streaming request is active, the AI provider sends tiny chunks of text called "deltas" (e.g., "The", " quick", " brown"). The naive approach is to use the OnStreamUpdate event to append the delta to a String, and then immediately call SetText() on your UI widget.
Why this is bad:
- Modern LLMs generate tokens incredibly fast. Your OnStreamUpdate event might fire 30 to 50 times a second.
- Calling SetText() on a complex UMG widget forces Unreal's Slate UI framework to recalculate layout, text wrapping, and rendering bounds. Doing this dozens of times per frame will tank your game's framerate.
The Solution: Delta Buffering
Instead of updating the UI every time a token arrives, you should buffer the text and flush it to the UI on a fixed timer (e.g., every 0.1 seconds).
Blueprint Workflow for Buffering
- Create two String variables in your UI Blueprint: AccumulatedText and DisplayedText.
- When OnStreamUpdate fires, simply append the Delta string to AccumulatedText. Do not touch the UI.
- Use a Set Timer by Event (looping every 0.1s).
- In the Timer event, check if AccumulatedText is different from DisplayedText. If it is, call SetText() on your UI widget using AccumulatedText, and update DisplayedText.
By decoupling the network stream from the UI rendering, you guarantee smooth framerates regardless of how fast the AI generates text.
Handling "Phantom Text" (Cancellation)
Another common bug in streaming AI interfaces is "Phantom Text." This happens when a player asks an NPC a question, the text starts streaming, and the player suddenly hits 'Escape' or walks away, closing the UI.
Because the HTTP stream is still active in the background, the plugin will continue receiving tokens. If you destroy the widget without canceling the request, or if your weak pointers aren't set up correctly, you risk game crashes or memory leaks.
Graceful Cancellation
The GenAI for Unreal plugin's Blueprint nodes are built on UCancellableAsyncAction. This means the node returns a reference to itself. You should promote this reference to a variable (e.g., ActiveStreamingRequest).
When the player closes the UI or walks away from the NPC, call the Cancel function on that variable. This immediately severs the HTTP connection, stops the token generation, and saves you money on API costs by halting the AI mid-sentence.
// Blueprint Pseudocode: On Widget Destruct or Player Walk Away
If (IsValid(ActiveStreamingRequest))
{
ActiveStreamingRequest -> Cancel()
Clear Timer by Handle (UI Update Timer)
}
Implementing delta buffering and proper cancellation ensures your AI-driven game feels polished, performs well, and doesn't waste API credits.