SteonMod
Sign in
SteonMod docs
View as Markdown

0003: The Unreal kit draws with SceneCaptures and copies them on the game's own GPU queue

  • Date: 2026-10-03
  • Status: accepted (step 1.4)

Context

BUILD.md step 1.4 asks for VEIN Demo (Unreal 5.6, D3D12) as a Fusion guest: Fusion's camera drives the game's view, and the picture plus a linear depth reach Fusion through shared textures. The plan offered two paths: a SceneCaptureComponent2D first, or a hook on the swap chain's Present that copies the back buffer and the scene depth.

What we found (experiments in VEIN, step 1.4):

  • A SceneCapture with SCS_FinalColorLDR gives the game's finished picture (tonemapped, gamma-encoded), but far too dark unless the capture keeps its history (bAlwaysPersistRenderingState, which auto exposure and TAA need). With it, the picture matches the game's own view.
  • SCS_SceneDepth into an R32F target gives view depth in cm, aligned with the color.
  • The back buffer would contain the game's HUD and menus, and Unreal 5's scene depth lives in transient, aliased GPU memory, so it can be gone by Present.
  • Normal UE4SS C++ mods need UE4SS's private UEPseudo submodule (Epic-linked GitHub account). UE4SS.dll exports its mod base class and LuaMadeSimple::Lua helpers, but not Lua's C API.

Decision

  • Two SceneCaptures (color, depth), moved to Fusion's camera every frame by the kit's Lua. The game's own view is switched off (show Rendering) once Fusion shows it.
  • The copy happens inside the game, on its own queue. main.dll hooks D3D12 resource creation (to recognise the two render targets by size and format), ResourceBarrier (to know their state, applied in submission order) and ExecuteCommandLists. Right after the game submits the work that finishes both targets, the kit submits its own small command list on the same queue: barriers, CopyResource into a free shared buffer, barriers back, fence signal. The game's render targets never leave the game's process or queue, and Fusion's protocol and Godot side stay the same as for fake-guest (only BGRA color was added).
  • No UE4SS source needed: cpp/ue4ss_abi.hpp re-declares CppUserModBase with the same layout and virtual function order, and imports the few LuaMadeSimple::Lua methods through an import library built from cpp/UE4SS.def. The C++ part only registers FusionKit_* Lua functions; everything Unreal-specific is Lua (UE4SS's Lua 5.4) and everything else is Rust.
  • Router scripts are UE4SS Lua 5.4 for now (routers/<game>/unreal.lua), which answers VISION.md §15's question for v0. Decided for good in ADR 0004 (step 5.2): they stay Lua 5.4, now written against the Router API (routers/<game>/router.lua).

Consequences

  • Works for any Unreal game with UE4SS and D3D12 that follows legacy resource barriers on these targets. Enhanced barriers are detected and logged, not handled yet.
  • The depth capture costs a second scene render (~5 ms at 1280x720 on an RTX 3050 Laptop). Possible later savings: reuse the color capture's own depth buffer, or render depth at a lower resolution.
  • If UE4SS changes CppUserModBase, ue4ss_abi.hpp must be updated to match.
  • D3D11 Unreal games (Hello Neighbor Alpha 2) need the same hooks for D3D11.