Screen-space refraction¶

a glass ball bends the scene behind it.

Tags: 3d

A transparent material that reads the screen draws what is behind it, bent:

glass = Material(colour=(0.75, 0.87, 1.0, 0.55), blend="alpha", needs_scene_colour=True)

needs_scene_colour=True makes the renderer split the frame in two. The opaque objects draw first, the colour they produced is copied into a texture, and then the transparent objects draw, each screen-reading one sampling that copy at its own pixel shifted by its surface normal. Where the normal turns, the sample shifts, so a curved surface bends the image behind it; a flat face seen head on has nothing to bend. The material’s alpha is how much of that bent scene shows through the glass’s own lit colour.

The split costs nothing when unused: a frame with no screen-reading material renders exactly as it would without the feature. WaterSurface3D builds its water shading on the same scene-colour copy (see water.py).

Here a glass ball sits in front of a grid of coloured cubes. Turning refraction off clears needs_scene_colour on the live material, and the ball becomes plain tinted glass that shows the grid straight through. The camera is an OrbitCamera3D: drag to orbit and scroll to zoom, or use the keys.

Controls: Space / button - Toggle refraction A / D / drag - Orbit the camera (arrow keys too) W / S / scroll - Zoom in / out (arrow keys too)

Source¶

# /// simvx
# screenshot_frame = 30
# web = { width = 960, height = 720, responsive = true }
# ///
"""Screen-space refraction: a glass ball bends the scene behind it.

A transparent material that reads the screen draws what is behind it, bent:

    glass = Material(colour=(0.75, 0.87, 1.0, 0.55), blend="alpha", needs_scene_colour=True)

``needs_scene_colour=True`` makes the renderer split the frame in two. The
opaque objects draw first, the colour they produced is copied into a texture,
and then the transparent objects draw, each screen-reading one sampling that
copy at its own pixel shifted by its surface normal. Where the normal turns,
the sample shifts, so a curved surface bends the image behind it; a flat face
seen head on has nothing to bend. The material's alpha is how much of that
bent scene shows through the glass's own lit colour.

The split costs nothing when unused: a frame with no screen-reading material
renders exactly as it would without the feature. ``WaterSurface3D`` builds
its water shading on the same scene-colour copy (see ``water.py``).

Here a glass ball sits in front of a grid of coloured cubes. Turning
refraction off clears ``needs_scene_colour`` on the live material, and the ball
becomes plain tinted glass that shows the grid straight through. The camera is
an ``OrbitCamera3D``: drag to orbit and scroll to zoom, or use the keys.

Controls:
    Space / button   - Toggle refraction
    A / D / drag     - Orbit the camera (arrow keys too)
    W / S / scroll   - Zoom in / out (arrow keys too)
"""

import math

from simvx.core import (
    Anchor,
    Axis1D,
    Button,
    CanvasLayer,
    DirectionalLight3D,
    Input,
    Key,
    Label,
    Material,
    Mesh,
    MeshInstance3D,
    Node3D,
    OrbitCamera3D,
    PointLight3D,
    WorldEnvironment,
    on_input,
)
from simvx.graphics import App

GRID_COLOURS = [
    (0.90, 0.20, 0.20),
    (0.20, 0.85, 0.30),
    (0.20, 0.45, 0.95),
    (0.95, 0.80, 0.20),
    (0.85, 0.30, 0.85),
    (0.25, 0.85, 0.85),
]


class Refraction(Node3D):
    input_context = {
        "refraction": [Key.SPACE],
        "orbit": [Axis1D(low=Key.A, high=Key.D), Axis1D(low=Key.LEFT, high=Key.RIGHT)],
        "zoom": [Axis1D(low=Key.S, high=Key.W), Axis1D(low=Key.DOWN, high=Key.UP)],
    }

    def on_ready(self):
        self.camera = self.add_child(
            OrbitCamera3D(yaw=0.0, pitch=math.radians(-4), distance=7, min_distance=4, max_distance=14, fov=55)
        )
        self.add_child(DirectionalLight3D(colour=(1.0, 0.96, 0.9), intensity=1.6, direction=(-0.4, -0.7, -0.6)))
        self.add_child(PointLight3D(colour=(0.5, 0.6, 0.9), intensity=0.6, range=20.0, position=(-4, 3, 5)))
        self.add_child(WorldEnvironment(bloom_enabled=True))

        # The opaque grid behind the glass: what the refraction reads.
        cube = Mesh.box(1.2)
        grid = [(x, y) for y in (1.4, 0.0, -1.4) for x in (-2.0, 0.0, 2.0)]
        for i, (x, y) in enumerate(grid):
            material = Material(colour=GRID_COLOURS[i % len(GRID_COLOURS)], roughness=0.55)
            self.add_child(MeshInstance3D(mesh=cube, material=material, position=(x, y, -1.2)))

        self.glass = Material(colour=(0.75, 0.87, 1.0, 0.55), blend="alpha", roughness=0.05, needs_scene_colour=True)
        ball = Mesh.sphere(radius=1.5, rings=32, segments=32)
        self.add_child(MeshInstance3D(mesh=ball, material=self.glass, position=(0, 0, 0.8)))

        hud = self.add_child(CanvasLayer())
        self.button = hud.add_child(
            Button("Refraction", action="refraction", checked=True, anchor=Anchor.TOP_LEFT, inset=12)
        )

        # Example help overlay: not part of the technique shown.
        help_text = "Space: refraction   A/D / drag: orbit   W/S / scroll: zoom"
        hud.add_child(Label(help_text, anchor=Anchor.BOTTOM_LEFT, inset=12, font_size=14))

    @on_input("refraction")
    def toggle_refraction(self, _event):
        # The next frame renders with or without the scene-colour copy.
        self.glass.needs_scene_colour = not self.glass.needs_scene_colour
        self.button.checked = self.glass.needs_scene_colour

    def on_update(self, dt):
        self.camera.orbit(Input.value("orbit") * math.radians(45) * dt, 0.0)
        self.camera.zoom(Input.value("zoom") * 8.0 * dt)


if __name__ == "__main__":
    App(title="Screen-Space Refraction", width=960, height=720).run(Refraction())