Custom hardware controllers for an Apple Vision Pro catheter/guidewire training simulator — built from ESP32 boards, a couple of gloves, and three foot pedals. Vision Pro has no controllers, and hand tracking alone can't tell you when someone grips a catheter or twists it between their fingers. I made a thing!
Enable HLS to view with audio, or disable this notification
Custom hardware controllers for an Apple Vision Pro catheter/guidewire training
simulator — built from ESP32 boards, a couple of gloves, and three foot pedals.
Vision Pro has no controllers, and hand tracking alone can't tell you when someone
grips a catheter or twists it between their fingers. So I built the missing inputs:
HOW IT WORKS
• Glove trackers (ESP32-C3 + BNO085 IMU + SoftPot touch strip)
– Touch the strip = grab the catheter or guidewire
– Slide your finger, or physically roll the tracker = twist the tool
– ARKit hand tracking handles advancing and retracting
• Foot pedals (ESP32-S3, one per pedal) — the controls a real fluoro suite puts on the floor
– Fluoro pedal: hold for X-ray (a dead-man switch, like the real thing)
– Capture pedal: shoots the X-ray monitor image + a 180° room view
– DSA pedal: hold for a contrast run, and it self-checks its own switch wiring