I develop interactive projection mapping installations that transform physical spaces into dynamic, responsive environments. Using TouchDesigner as my main tool, I design real-time visuals that react to sound, movement, or audience interaction, creating experiences that blend the digital and physical world seamlessly. By aligning projections with architectural features, objects, or custom-built surfaces, I can turn ordinary structures into living canvases. These works often incorporate sensor data and live input, allowing each performance to feel unique and participatory. The result is an immersive layer of storytelling where technology and creativity meet, inviting audiences to not just watch but actively shape the visuals around them.
Hand Gesture Interaction (No 1-3)
These interactive designs use an Xbox Kinect to capture hand gestures in real time, allowing participants to directly influence projected visuals. Movements such as raising a hand, swiping across the air, or holding a position act as inputs that drive animations, particle effects, and dynamic distortions, transforming simple gestures into an expressive visual language.
Index Tracking and Overlays
The next iteration extends beyond broad gestures by tracking skeletal index points of the body. This precision makes it possible to overlay digital effects directly onto specific joints or body parts—such as glowing trails that follow the hands, or animated textures that wrap around the silhouette—blurring the line between the participant’s physical presence and the digital visuals.
Kinect to Unreal via OSC (No 4)
In addition to standalone visuals, I also stream Kinect data into Unreal Engine using OSC (Open Sound Control). This pipeline allows real-time body and hand tracking information to drive Niagara particle systems, environmental effects, or interactive gameplay elements. By bridging sensor data with a game engine, the experience becomes both performative and highly customizable, adaptable to installations, performances, or VR/AR projects.
Full Body Tracking (No 5 – 10)
Building further, full body tracking with the Kinect recognizes the participant’s entire figure as a controller. Every movement, from walking to dancing, becomes a trigger for reactive environments, shifting light compositions, and evolving projections. This approach creates an immersive layer where the body itself becomes the canvas for interaction.
Projector Mapping (No 11)
To present these works, I integrate projector mapping techniques that align visuals with real-world architecture, objects, and surfaces. By mapping the Kinect-driven visuals onto physical structures, participants see their gestures and movements ripple across walls, installations, or custom-built forms, merging digital animation with tangible space.
Contact Form