Live Performance XR Platform
This is not a second screen experience.
This is an extension of the stage.
00 — Concept Visual
FFT analysis maps bass, mid & treble to visual parameters in real time
MindAR face tracking anchors glitch visuals to the user's face
Gyroscope & accelerometer translate physical motion into distortion
No headset, no download — scan once, experience instantly
01 — Core Premise
Identity continuously transforms, refuses to stabilize into a fixed form.
Glitch as behavioral language — disrupting visual coherence from within.
Expression as interference, not object. Presence as signal, not image.
"The medium is the message" — McLuhan, 1964[1]. Here, the body is the medium.
02 — Personas
Age 18–35 · Attends concerts, raves, and queer arts events · Urban, digitally fluent
"I want to capture the moment but I also want to be in it."
Produces 4–8 events per year · Manages 200–5,000 person events · Seeking novel audience engagement tools
"I need something that feels specific to our event — not a generic app."
03 — Behavioral System
Minimal visual intervention. The system is latent, waiting for signal.
Glitch elements begin to appear. Forms suggest identity before collapsing.
Audio and motion intensify. Fragmentation increases. Signal overwhelms form.
Identity dissolves into abstraction. No stable form is achieved. Only flux.
04 — Experience Storyboard
User scans code at venue. Experience loads instantly in browser — no download required.
Single tap grants mic, camera, and motion access. The system awakens.
Rear camera activates. Ambient visuals overlay the live environment. System is latent.
Front camera activates face tracking. Glitch attaches to the face. Self becomes canvas.
Audio peaks. Identity dissolves into abstraction. No stable form is achieved. Only flux.
05 — XR Differentiation
Device movement alters visual output in real time. Tilt, shake, proximity — physical gestures become expressive data shaping the experience.[3]
Identity-linked visuals via front camera. Self becomes the canvas. Glitch attaches to the face, distorts, and ultimately collapses.[4]
Visuals driven by the live audio environment. The experience cannot be replicated outside its context. It only exists in the moment.[2]
06 — Information Architecture
07 — XR Experience Layers
Each layer feeds into the one above. The physical environment generates audio; audio drives state; state shapes visuals; visuals anchor to the face. The stack is embodied — not abstract.[3]
08 — Technical System
09 — Market Positioning
Scalable QR-based entry. No hardware overhead. Transforms passive crowds into active participants instantly.
Enhancement tool for DJs, producers, and electronic acts seeking deeper audience connection.
Interactive identity exploration for curatorial and gallery contexts. The body as medium.
Culturally resonant tool for queer arts organizers. Visibility that refuses fixed categorization.
10 — Prototype in Action
Hydra generative output responds to live FFT analysis. Bass, mid, and treble each map to distinct visual parameters.
MindAR browser-based face detection. Glitch visuals anchor to facial landmarks. Self becomes the canvas in real time.
Device gyroscope and accelerometer feed directly into visual distortion. Physical gesture shapes the image.
Audio + motion thresholds trigger automated state transitions. The system behaves — it is not controlled.
Three.js 3D model rendered in the scene with audio-driven opacity. Presence linked to sound intensity.
11 — Roadmap
12 — Bibliography
13 — Proof Points
Working prototype with audio-reactive visuals and face tracking
Headsets required. Browser-native XR delivery via QR code.
Scalable across venues, formats, and events worldwide
Single scan to enter the full immersive experience
Between States is an extension of the stage.
Let's build it together.
or find me on Instagram @noir_mak