Music Streaming¶

loops that crossfade and duck under dialogue.

📄 Docs only

Tags: audio music streaming crossfade fades

Music is long, so it streams: an AudioPlayer with stream_mode="streaming" decodes its file a chunk at a time as it plays instead of loading it whole.

theme = AudioPlayer(AudioClip("theme.ogg"), stream_mode="streaming", loop=True, bus="Music")

Only a file can stream. A synthetic clip (AudioClip.tone, AudioClip.from_pcm) cannot: play() refuses it and emits stream_open_failed, so give one stream_mode="memory". This example ships no audio, so it synthesises two short loops with NumPy and writes them as WAV files under $XDG_CACHE_HOME/simvx/music/ (~/.cache by default, or wherever MUSIC_CACHE_DIR points) on every run, so an edit to TRACKS is heard at once. A game points AudioClip at the files it ships and skips that step.

a.crossfade(b, 2.0) hands playback over in one call: b starts from the top of its file and fades in while a fades out and stops. The curve is equal power, so the pair holds a steady loudness through the middle, and calling it again mid-fade retargets both ramps from where they are.

Ducking is fade_to. music.fade_to(0.25, 0.3) drops the music to a quarter of its amplitude (about -12 dB) while a line of dialogue plays, and music.fade_to(1.0, 0.5) brings it back. The gain multiplies the player’s own volume_db, so a duck never loses the volume the player was set to. A player has one fade at a time and a new one replaces it, so a crossfade during a line is followed by fade_to(0.25, 2.0) on the incoming track, and the line ends by restoring whichever track is current.

Fades and the streaming feed both advance in the player’s update, so a player must be in the tree to be heard. get_playback_position() works on a stream too.

Press Crossfade (Space) to swap tracks and Dialogue (D) to duck the music under a spoken line.

Source¶

  1# /// simvx
  2# tags = ["audio", "music", "streaming", "crossfade", "fades"]
  3# [web]
  4# disabled = true
  5# reason = "stream_mode='streaming' needs the native backend's chunk-fed decoder, absent in the browser runtime."
  6# ///
  7"""Music Streaming: loops that crossfade and duck under dialogue.
  8
  9Music is long, so it streams: an `AudioPlayer` with `stream_mode="streaming"`
 10decodes its file a chunk at a time as it plays instead of loading it whole.
 11
 12    theme = AudioPlayer(AudioClip("theme.ogg"), stream_mode="streaming", loop=True, bus="Music")
 13
 14Only a file can stream. A synthetic clip (`AudioClip.tone`, `AudioClip.from_pcm`)
 15cannot: `play()` refuses it and emits `stream_open_failed`, so give one
 16`stream_mode="memory"`. This example ships no audio, so it synthesises two short
 17loops with NumPy and writes them as WAV files under `$XDG_CACHE_HOME/simvx/music/`
 18(`~/.cache` by default, or wherever `MUSIC_CACHE_DIR` points) on every run, so an
 19edit to `TRACKS` is heard at once. A game points `AudioClip` at the files it ships
 20and skips that step.
 21
 22`a.crossfade(b, 2.0)` hands playback over in one call: `b` starts from the top of
 23its file and fades in while `a` fades out and stops. The curve is equal power, so
 24the pair holds a steady loudness through the middle, and calling it again
 25mid-fade retargets both ramps from where they are.
 26
 27Ducking is `fade_to`. `music.fade_to(0.25, 0.3)` drops the music to a quarter of
 28its amplitude (about -12 dB) while a line of dialogue plays, and
 29`music.fade_to(1.0, 0.5)` brings it back. The gain multiplies the player's own
 30`volume_db`, so a duck never loses the volume the player was set to. A player has
 31one fade at a time and a new one replaces it, so a crossfade during a line is
 32followed by `fade_to(0.25, 2.0)` on the incoming track, and the line ends by
 33restoring whichever track is current.
 34
 35Fades and the streaming feed both advance in the player's update, so a player
 36must be in the tree to be heard. `get_playback_position()` works on a stream too.
 37
 38Press **Crossfade** (Space) to swap tracks and **Dialogue** (D) to duck the
 39music under a spoken line.
 40"""
 41
 42import os
 43import wave
 44from pathlib import Path
 45
 46import numpy as np
 47
 48from simvx.core import (
 49    Anchor,
 50    AudioClip,
 51    AudioPlayer,
 52    Button,
 53    CanvasLayer,
 54    HBoxContainer,
 55    Key,
 56    Label,
 57    Node,
 58    VBoxContainer,
 59    on_input,
 60    wait,
 61)
 62from simvx.graphics import App
 63
 64RATE = 44100
 65DUCK = 0.25  # music gain under dialogue, about -12 dB
 66
 67# Both loops last 2.4 s and wrap without a click: every pluck decays inside its
 68# own slot, and each bass drone completes whole cycles over the loop.
 69TRACKS = {
 70    "Warm arpeggio": {
 71        "notes": [220.00, 261.63, 329.63, 392.00, 440.00, 392.00, 329.63, 261.63],
 72        "note_len": 0.3,
 73        "bass": 110.0,
 74        "harmonics": [(1, 1.0), (2, 0.35)],
 75    },
 76    "Bright reed": {
 77        "notes": [329.63, 392.00, 493.88, 587.33, 493.88, 392.00, 329.63, 392.00, 493.88, 659.25, 587.33, 493.88],
 78        "note_len": 0.2,
 79        "bass": 82.5,
 80        "harmonics": [(1, 1.0), (3, 0.33), (5, 0.2)],
 81    },
 82}
 83
 84
 85def pluck(freq, duration, harmonics) -> np.ndarray:
 86    """One exponentially decaying note built from (multiple, amplitude) partials."""
 87    t = np.arange(int(RATE * duration)) / RATE
 88    tone = sum(amplitude * np.sin(2 * np.pi * freq * multiple * t) for multiple, amplitude in harmonics)
 89    return np.exp(-5.0 * t / duration) * tone
 90
 91
 92def render_loop(notes, note_len, bass, harmonics) -> np.ndarray:
 93    """An arpeggio of plucks over a sine bass drone, normalised to [-1, 1]."""
 94    melody = np.concatenate([pluck(freq, note_len, harmonics) for freq in notes])
 95    t = np.arange(len(melody)) / RATE
 96    mix = 0.6 * melody + 0.22 * np.sin(2 * np.pi * bass * t)
 97    return mix / np.abs(mix).max()
 98
 99
100def write_wav(path: Path, samples: np.ndarray) -> Path:
101    """Write mono float samples as a 16-bit WAV file, replacing any earlier one."""
102    path.parent.mkdir(parents=True, exist_ok=True)
103    with wave.open(str(path), "wb") as out:
104        out.setnchannels(1)
105        out.setsampwidth(2)
106        out.setframerate(RATE)
107        out.writeframes((samples * 0.8 * 32767).astype(np.int16).tobytes())
108    return path
109
110
111def cache_dir() -> Path:
112    """Where the generated loops live: `MUSIC_CACHE_DIR`, else the per-user cache."""
113    override = os.environ.get("MUSIC_CACHE_DIR")
114    base = Path(os.environ.get("XDG_CACHE_HOME", Path.home() / ".cache")) / "simvx" / "music"
115    return Path(override) if override else base
116
117
118class MusicDemo(Node):
119    input_context = {"crossfade": [Key.SPACE], "dialogue": [Key.D]}
120
121    def on_ready(self):
122        folder = cache_dir()
123        self.tracks = []
124        for i, (name, spec) in enumerate(TRACKS.items()):
125            path = write_wav(folder / f"track_{i}.wav", render_loop(**spec))
126            self.tracks.append(
127                self.add_child(AudioPlayer(AudioClip(path), name=name, stream_mode="streaming", loop=True, bus="Music"))
128            )
129        self.active = self.tracks[0]
130        self.active.play()
131        self.speaking = False
132
133        hud = self.add_child(CanvasLayer())
134        header = hud.add_child(VBoxContainer(anchor=Anchor.TOP_LEFT, inset=16))
135        header.add_child(Label("Music Streaming", font_size=28))
136        self.status = header.add_child(Label(""))
137        bar = hud.add_child(HBoxContainer(anchor=Anchor.BOTTOM_LEFT, inset=16))
138        # An action button presses its action's key, so a click and the key take the same path.
139        bar.add_child(Button("Crossfade", action="crossfade"))
140        bar.add_child(Button("Dialogue", action="dialogue"))
141
142    @on_input("crossfade")
143    def crossfade(self, _event):
144        incoming = self.tracks[1] if self.active is self.tracks[0] else self.tracks[0]
145        self.active.crossfade(incoming, 2.0)
146        if self.speaking:
147            incoming.fade_to(DUCK, 2.0)  # replaces the crossfade's fade-in, so it arrives ducked
148        self.active = incoming
149
150    @on_input("dialogue")
151    def dialogue(self, _event):
152        if not self.speaking:
153            self.start_coroutine(self.duck_under_line())
154
155    def duck_under_line(self):
156        self.speaking = True
157        self.active.fade_to(DUCK, 0.3)
158        yield from wait(1.8)
159        # Whichever track is current now: a crossfade during the line may have swapped it.
160        self.active.fade_to(1.0, 0.5)
161        self.speaking = False
162
163    def on_update(self, dt):
164        lines = [
165            f"{'>' if track is self.active else '   '} {track.name}: "
166            f"{'playing' if track.is_playing() else 'stopped'} at {track.get_playback_position():.1f} s"
167            for track in self.tracks
168        ]
169        lines.append('Dialogue: "The music dips while I speak."' if self.speaking else "Dialogue: (none)")
170        self.status.text = "\n".join(lines)
171
172
173if __name__ == "__main__":
174    App(title="Music Streaming", width=960, height=540).run(MusicDemo())