AUDIO REACTIVE VISUALS

01 / FFT DISSECTOR

FFT
Dissector

See inside sound for the first time.

TIME DOMAIN — WAVEFORM

FREQUENCY DOMAIN — SPECTRUM

SPECTROGRAM — TIME × FREQUENCY

FFT SIZE

Larger FFT = more frequency resolution, less time resolution.

FREQUENCY BANDS

SUB 20–60Hz BASS 60–250Hz LO-MID 250–500Hz MID 500Hz–2kHz HI-MID 2k–6kHz AIR 6k–20kHz

Click any bar in the spectrum to audition that frequency.

HEISENBERG UNCERTAINTY

Larger window = more frequency precision, but the time of events is blurred.
Smaller window = precise in time, blurred in frequency.
This is not a limitation of the software — it is a mathematical law.

04 / BEAT DETECTION WORKSHOP

Beat Detection
Workshop

Teaching a computer to feel the groove.

SENSITIVITY 60%

HOLD TIME 200ms

RELEASE TIME 100ms

MUSIC STYLE

DETECTED BPM

130

TRUE DETECTIONS

94%

FALSE POSITIVES

BEAT FLASH

Beat detection works by monitoring low-frequency energy. When the energy exceeds a threshold relative to recent average energy, a beat is registered. The threshold determines sensitivity — too low catches non-beat transients; too high misses quieter beats. The hold time prevents one loud event from triggering multiple detections.

06 / THE RYOJI ROOM

The Ryoji
Room

Data as architecture. Binary as beauty.

THRESHOLD 80

Ryoji Ikeda (born 1966, Gifu Prefecture) creates mathematical, systematic audio and visual installations at monumental scales. test pattern (2008–present) converts barcode data into precise sine tones. data.tron (2009) filled the Linz Museum of Modern Art with this visual language. The Barbican, Park Avenue Armory, and the High Line have hosted his work. His process: raw data → binary → barcode → sound.

"The only control is the threshold. Everything else is the music."

08 / AUDIOVISUAL HISTORY

Audiovisual
History

A century of making sound visible.

1877

Thomas Young — first visual representation of sound as waveform drawing using tuning forks on smoked glass

1920s

Oskar Fischinger — abstract films synchronized to classical music; proto-audiovisual animation

1940

Fantasia (Disney) — the original mass-audience audiovisual cinema. Bach's Toccata rendered as animation for millions

1961

John Whitney Sr. — computer graphics + music. Catalog: the first audio-reactive computer animation

1970s

Laser light shows — Pink Floyd's Wish You Were Here Tour uses laser projectors synced to music in arenas

1990s

VJ culture — real-time video mixing synchronized to electronic music in clubs worldwide

1996

Processing (nascent) — creative coding tools enable generative audiovisual art for visual artists

2008

Ryoji Ikeda — test pattern at monumental scale. FFT as installation aesthetic, data as visual material

2024

AI audiovisual generation — real-time neural synthesis connects audio features to latent visual spaces

09 / ARTISTS

Artists

Who made sound visible.

Ryoji Ikeda

Mathematical audiovisual installation

test pattern, data.tron, superposition. The reference for data as visual material. Transforms raw binary into barcode patterns and precise sine tones at architectural scale.

Robert Henke

CBL laser series / Monolake

Precisely controlled laser light in darkness, responding to composed electronic music. Uses custom software to choreograph light with mathematical rigor.

Refik Anadol

Machine Hallucinations

Neural network audiovisual installations at architectural scale. MoMA, Serpentine, and the Sphere in Las Vegas. Data sculptures from millions of images.

Memo Akten

Body of Work

Computational audiovisual art exploring perception and consciousness. Uses deep learning and signal processing to create works that interrogate human experience.

Floating Points

Sam Shepherd — live electronic

Live electronic performances with audio-reactive lighting rigs. Subtle spectral response to temperature and texture, not just beat. Sound-reactive light design as composition.

Bileam Tschepe

Browser-based audiovisual art

Generative systems that respond to music with geometric precision. Works entirely in the browser with WebGL and Web Audio API. Data structures made tangible.

Universal Everything

Studio — computational design

Fluid, organic morphology driven by audio envelopes. Works for Google, Nike, and gallery exhibitions. Biological forms that breathe with music.

Casey Reas

Processing co-creator

Generative art systems where simple rules create complex emergence — audio feeds the rules. Co-created Processing, democratizing computational art for a generation of artists.

10 / GLOSSARY

Glossary

The language of sound made visible.

FFT

Fast Fourier Transform. An algorithm that converts a time-domain signal into its constituent frequencies in O(n log n) time. The foundation of all audio analysis.

Frequency Bin

One output cell of the FFT. Each bin represents a narrow frequency range; its value is the amplitude of that frequency in the current analysis window.

Spectral Centroid

The "center of mass" of the frequency spectrum. High centroid = bright, treble-heavy sound. Low centroid = dark, bass-heavy. Correlates strongly with perceived timbre.

Onset Detection

The identification of the moment a new note or sound event begins. Computed by measuring sudden increases in spectral flux (change in energy between frames).

Beat Detection

Identifying rhythmic pulse in music, typically by monitoring low-frequency energy and detecting when it exceeds a threshold relative to the recent energy average.

RMS

Root Mean Square. The square root of the mean of squared sample values. A perceptually accurate measure of loudness, corresponding closely to how humans perceive volume.

Spectrogram

A 2D visualization showing time on the X axis, frequency on the Y axis, and amplitude encoded as color. Makes the harmonic and temporal structure of sound visible simultaneously.

Waterfall Display

A scrolling spectrogram that continuously moves the historical data to reveal new analysis. Named for its resemblance to falling water when amplitude is high.

Lissajous Figure

A curve traced by plotting two sinusoidal signals against each other on X and Y axes. Used in audio as a stereo vectorscope — the shape reveals phase relationships between channels.

Audio Reactivity

The technique of extracting real-time features from audio signals and using them to drive visual parameters. The core discipline of this entire field.

Parameter Mapping

The deliberate assignment of audio features to visual parameters. The creative act that defines audiovisual work — the "what controls what" decision.

GLSL Shader

OpenGL Shading Language. A C-like language that runs on the GPU, executing in parallel for every pixel simultaneously. Enables real-time complex visual effects.

Uniform Variable

In GLSL, a variable that remains constant for an entire frame but can be updated between frames. Audio features are typically passed to shaders as uniforms.

Mel Scale

A perceptual frequency scale where equal distances correspond to equal perceived pitch differences. More bins at low frequencies where human hearing has higher resolution.

Chromagram

A representation of audio that collapses all octaves into 12 pitch classes (C through B). Reveals harmonic content and chord progressions independent of octave.

Synesthesia

A neurological phenomenon where one sensory pathway triggers automatic experiences in another — sound producing color, for example. Experienced by ~4% of the population.

Envelope Following

Tracking the amplitude contour of a signal over time. The envelope describes attack, decay, sustain, and release — the shape of sound as opposed to its pitch or timbre.

Threshold

A level value that triggers an action when crossed. In beat detection and Ryoji-style visualization, the threshold is the sole creative parameter — everything else is the music.

Lag Filter

A smoothing technique that makes values follow their targets gradually rather than instantly. Prevents visual jitter and creates organic motion: new = old + (target - old) × rate.

Real-Time

Processing that keeps pace with live input — completing each analysis frame faster than the next frame arrives. Typically 60fps (16.7ms per frame) for smooth visual output.

AUDIO REACTIVE
VISUALS

FFT
Dissector

Parameter
Mapping Lab

Frequency
Mandala

Beat Detection
Workshop

GLSL Reactor
Lab

The Ryoji
Room

Synesthetic
Mixer

Audiovisual
History

Artists

Glossary

AUDIO REACTIVEVISUALS

FFTDissector

ParameterMapping Lab

FrequencyMandala

Beat DetectionWorkshop

GLSL ReactorLab

The RyojiRoom

SynestheticMixer

AudiovisualHistory

Artists

Glossary

AUDIO REACTIVE
VISUALS

FFT
Dissector

Parameter
Mapping Lab

Frequency
Mandala

Beat Detection
Workshop

GLSL Reactor
Lab

The Ryoji
Room

Synesthetic
Mixer

Audiovisual
History