I got bored of WebGL demos where you click a button and watch something spin. Cool for 5 seconds, then you close the tab. I wanted to build something you could actually mess around with - no mouse, no clicking, just wave your hand and watch thousands of particles react.

What it is

AETHER is a 3D particle system controlled by your hands through your webcam. Wave, pinch, make a fist - the particles respond. 15,000+ of them, running at 60fps, all in your browser.

The problem I was solving

Most WebGL stuff feels disconnected. You're moving a mouse cursor, clicking buttons - there's always something between you and the visuals. I wanted to remove that layer entirely.

Goals:

Hand gestures instead of mouse/keyboard
Smooth performance with tons of particles
No backend, everything runs in the browser
Still works if you don't have a webcam

How I built it

Everything runs client-side. Your webcam feeds into MediaPipe's hand tracking, which spits out 21 landmarks per hand. I translate those into gestures (fist, pinch, peace sign, etc.) and map each gesture to a particle behavior.

For people without webcams or who don't want to use hand tracking, I added mouse/touch controls as a fallback.

Tech choices

Three.js - Handles all the rendering. Writing raw WebGL would've taken forever, and Three.js is fast enough for what I needed.

MediaPipe Hands - Google's hand tracking library. Runs entirely in-browser, no API calls, works surprisingly well.

Custom shaders - Particles are rendered on the GPU. CPU couldn't handle 15k particles with glow effects and trails.

One Euro Filter - Hand tracking data is jittery as hell. This filter smooths it out without adding noticeable lag.

Vanilla JS - No React, no framework. For a visual project like this, every millisecond counts.

What sucked

Hand tracking noise - Raw landmark data jumps around constantly. Had to combine filtering with angle-based gesture detection to make it reliable.

Performance - First version ran at like 20fps. Fixed it by:

Adding automatic quality scaling based on frame rate
Capping pixel ratio on high-DPI screens
Making visual effects optional

Gesture discoverability - Nobody knows what gestures are available just by looking at it. Added an onboarding tutorial, visual feedback, and keyboard shortcuts as backup.

Results

60fps with 15,000 particles (up to 30k on good hardware)
Works without webcam
Accessibility options: reduced motion, high contrast, keyboard nav
Learned a ton about real-time graphics and computer vision

What I took away

GPU-first design is non-negotiable for complex browser visuals. And if you're introducing new interaction patterns, the UX better be solid or people will just leave.

Try it · Code

Blog

Blog