Visual Testing¶

SimVX provides headless Vulkan rendering and pixel-level assertions for automated visual testing. Tests run with a hidden window — no display required beyond a Vulkan driver.

Testing Layers¶

SimVX has three testing layers, each suited to different needs:

Layer	Module	GPU Required	What It Tests
Logic	`simvx.core.testing`	No	Game logic, node lifecycle, input actions
UI draw calls	`simvx.core.ui.testing`	No	Widget layout, draw commands, focus/input
Visual rendering	`simvx.graphics.testing`	Yes	Actual GPU output — shaders, z-order, colours

This page covers the visual rendering layer. See Testing for the logic and UI layers.

Quick Start¶

from simvx.core import Node, Camera3D, MeshInstance3D, Material, Mesh, Vec3
from simvx.graphics import App
from simvx.graphics.testing import assert_pixel, assert_not_blank

class RedCube(Node):
    def ready(self):
        cam = Camera3D(position=Vec3(0, 2, 5))
        cam.look_at(Vec3(0, 0, 0))
        self.add_child(cam)

        cube = MeshInstance3D()
        cube.mesh = Mesh.cube(size=2.0)
        cube.material = Material(colour=(1, 0, 0, 1), unlit=True)
        self.add_child(cube)

app = App(width=320, height=240, visible=False)
frames = app.run_headless(RedCube(), frames=3, capture_frames=[2])
frame = frames[0]  # (240, 320, 4) uint8 RGBA

assert_not_blank(frame)
assert_pixel(frame, 160, 120, (238, 0, 0, 255), tolerance=20)  # center is red

Headless Rendering¶

`App.run_headless()`¶

Runs the full engine pipeline (physics, process, draw, render) for a fixed number of frames with a hidden window, returning captured framebuffer contents as numpy arrays.

App.run_headless(
    root_node,
    *,
    frames: int = 1,
    on_frame: Callable[[int, float], bool | None] | None = None,
    capture_frames: list[int] | None = None,
    capture_fn: Callable[[int], bool] | None = None,
) -> list[np.ndarray]

Parameters:

root_node — Scene root (Node subclass). ready() is called before the first frame.
frames — Total number of frames to simulate and render.
on_frame — Called with (frame_index, time) before each frame. Return False to stop early. Use this to inject input between frames via InputSimulator.
capture_frames — Which frame indices to capture (0-based). None captures every frame.
capture_fn — Alternative to capture_frames: a callable (frame_index) -> bool that decides per-frame whether to capture.

Returns a list of (height, width, 4) uint8 numpy arrays in RGBA order.

Each call creates a fresh Vulkan context and tears it down on completion — no state leaks between runs.

Hidden Window¶

App accepts visible=False to create an invisible window:

app = App(width=640, height=480, visible=False)

This uses GLFW_VISIBLE=FALSE under the hood. The window still has a real Vulkan surface and swapchain — rendering is identical to a visible window.

Frame Capture¶

Engine.capture_frame() copies the last-rendered swapchain image to CPU memory. It handles the full Vulkan pipeline: barrier transition, image-to-buffer copy, BGRA-to-RGBA swizzle. The returned array is a detached copy — safe to store, compare, or save.

Window Resize¶

For testing resize behaviour:

app = App(width=320, height=240, visible=False)
# ... render some frames ...
app.set_window_size(640, 480)
# next frame will trigger swapchain recreation

Pixel Assertions¶

All functions in simvx.graphics.testing operate on (H, W, 4) uint8 RGBA numpy arrays. No external dependencies.

`assert_pixel(pixels, x, y, expected_rgba, tolerance=2)`¶

Check a single pixel. Fails if any channel differs by more than tolerance.

assert_pixel(frame, 160, 120, (255, 0, 0, 255), tolerance=10)

`assert_not_blank(pixels)`¶

Fails if every pixel is the same colour. Catches “rendered nothing” bugs.

assert_not_blank(frame)

`assert_colour_ratio(pixels, colour, expected_ratio, tolerance=0.02, colour_tolerance=10)`¶

Assert that approximately expected_ratio of all pixels match colour.

# Red should cover ~25% of the screen
assert_colour_ratio(frame, (255, 0, 0), 0.25, tolerance=0.05, colour_tolerance=30)

`colour_ratio(pixels, colour, tolerance=10) -> float`¶

Returns the fraction of pixels matching colour. Useful for flexible assertions:

ratio = colour_ratio(frame, (255, 0, 0), tolerance=30)
assert ratio > 0.1, f"Not enough red: {ratio:.3f}"

`assert_region_colour(pixels, rect, expected_colour, tolerance=5)`¶

Assert all pixels within a rectangle (x, y, width, height) match a colour.

# Top-left 100x100 should be blue
assert_region_colour(frame, (0, 0, 100, 100), (0, 0, 255, 255), tolerance=15)

`assert_no_colour(pixels, colour, tolerance=5)`¶

Assert a colour is absent from the image.

# Nothing should be bright green
assert_no_colour(frame, (0, 255, 0), tolerance=10)

`save_image(path, pixels)` / `save_diff_image(path, actual, expected)`¶

Debug helpers that write PPM files (no Pillow required). save_diff_image amplifies differences 5x for visibility.

from simvx.graphics.testing import save_image, save_diff_image
save_image("/tmp/actual.ppm", frame)
save_diff_image("/tmp/diff.ppm", frame, expected_frame)

Testing Patterns¶

Z-order verification (no baseline needed)¶

Render overlapping objects with distinct colours and check which colour wins at the overlap point:

class ZOrderScene(Node):
    def ready(self):
        cam = Camera3D(position=Vec3(0, 0, 5))
        cam.look_at(Vec3(0, 0, 0))
        self.add_child(cam)

        # Blue cube at z=0 (far)
        back = MeshInstance3D(position=Vec3(0, 0, 0))
        back.mesh = Mesh.cube(3.0)
        back.material = Material(colour=(0, 0, 1, 1), unlit=True)
        self.add_child(back)

        # Red cube at z=1.5 (near)
        front = MeshInstance3D(position=Vec3(0, 0, 1.5))
        front.mesh = Mesh.cube(2.0)
        front.material = Material(colour=(1, 0, 0, 1), unlit=True)
        self.add_child(front)

frames = app.run_headless(ZOrderScene(), frames=3, capture_frames=[2])
center = frames[0][120, 160]
assert center[0] > 150  # red wins at center
assert center[2] < 50   # not blue

Input injection¶

Use on_frame with InputSimulator to test interactive behaviour:

from simvx.core.testing import InputSimulator
from simvx.core import Key

sim = InputSimulator()

def inject(frame_num, time):
    if frame_num == 2:
        sim.press_key(Key.SPACE)
    elif frame_num == 3:
        sim.release_key(Key.SPACE)

frames = app.run_headless(MyScene(), frames=5, on_frame=inject, capture_frames=[4])

Colour coverage for render-order verification¶

When you don’t have a baseline, verify that each rendered element covers the expected proportion of the screen:

frames = app.run_headless(ThreeColourScene(), frames=3, capture_frames=[2])
frame = frames[0]

red = colour_ratio(frame, (255, 0, 0), tolerance=80)
green = colour_ratio(frame, (0, 255, 0), tolerance=80)
blue = colour_ratio(frame, (0, 0, 255), tolerance=80)

assert red > 0.01, "Red cube missing"
assert green > 0.01, "Green cube missing"
assert blue > 0.01, "Blue cube missing"

Pytest Fixtures¶

The graphics test suite provides fixtures in packages/graphics/tests/conftest.py:

def test_my_scene(capture, px):
    frames = capture(MyScene(), frames=3, capture_frames=[2])
    frame = frames[0]
    px.not_blank(frame)
    px.pixel(frame, 160, 120, (255, 0, 0, 255), tolerance=20)

Fixture	Type	Description
`headless_app`	`App`	Pre-configured `App(320x240, visible=False)`. Skips if no Vulkan.
`capture`	`callable`	Shorthand for `headless_app.run_headless(...)`.
`px`	`PixelCheck`	Namespace with all assertion functions as methods.

The px fixture provides: px.pixel(), px.not_blank(), px.region(), px.ratio(), px.no_colour(), px.get_ratio().

CI Setup¶

Visual tests require a Vulkan driver. For headless CI, use a software renderer:

Linux (GitHub Actions / Docker)¶

- name: Install software Vulkan
  run: sudo apt-get install -y mesa-vulkan-drivers xvfb

- name: Run visual tests
  run: xvfb-run pytest packages/graphics/tests/test_visual.py
  env:
    VK_ICD_FILENAMES: /usr/share/vulkan/icd.d/lvp_icd.x86_64.json

mesa-vulkan-drivers provides lavapipe, a CPU-based Vulkan 1.4 driver. xvfb-run provides a virtual X display for GLFW.

Arch Linux / Manjaro¶

sudo pacman -S vulkan-swrast
xvfb-run pytest packages/graphics/tests/test_visual.py

Running Tests¶

# All visual tests
cd packages/graphics && pytest tests/test_visual.py -v

# Specific test
pytest tests/test_visual.py::TestRedCube::test_center_is_red