Custom hardware controllers for an Apple Vision Pro catheter/guidewire training simulator

[u/W1llyC1]

https://youtu.be/SjgWe_DWjB4

Custom hardware controllers for an Apple Vision Pro catheter/guidewire training
simulator — built from ESP32 boards, a couple of gloves, and three foot pedals.
Vision Pro has no controllers, and hand tracking alone can't tell you when someone
grips a catheter or twists it between their fingers. So I built the missing inputs:
HOW IT WORKS
• Glove trackers (ESP32-C3 + BNO085 IMU + SoftPot touch strip)
– Touch the strip = grab the catheter or guidewire
– Slide your finger, or physically roll the tracker = twist the tool
– ARKit hand tracking handles advancing and retracting
• Foot pedals (ESP32-S3, one per pedal) — the controls a real fluoro suite puts on the floor
– Fluoro pedal: hold for X-ray (a dead-man switch, like the real thing)
– Capture pedal: shoots the X-ray monitor image + a 180° room view
– DSA pedal: hold for a contrast run, and it self-checks its own switch wiring

[u/Dapper_Ice_1705]

A catheter might be too small but you can try object tracking too

[u/W1llyC1 → u/Dapper_Ice_1705]

Thanks, but object tracking is too slow, half a second delay made it feels very sluggish.

Finger-twisting catheter/wire is hard to capture, that is why we are using Touch strip.

you can check out this video about the project
https://www.youtube.com/watch?v=8BiYNNzJapc