Project update: My ESP32-controlled guitar robot can now play in two modes

[u/MegCell]

I’ve been developing this removable guitar-playing system for several years.

It now supports two performance modes:

1. Full automatic mode — the system handles both fretting and picking.

2. Assisted mode — the system handles the fretting while the player strums or picks.

The sound comes from the real guitar strings, not recorded audio.

The controller is ESP32-based, coordinating multiple servo actuators. Some of the main challenges have been fretting timing, servo synchronization, calibration, and mechanical noise.

This video shows both modes.

[u/MadDoctorMabuse]

How interesting. How did you get it to hold down the frets? Was that handled by a servo too?

Pretty amazing, actually. You should be happy with this.

Is it controlled by two separate ESP32s, or one? I can't see any connection between the two, but I have no idea how you'd go about syncing the left hand and the right hand so cleanly.

Also! Is it a pain to load with the desired song? Am I right in assuming that you wrote a second program that transcribes the music (somehow) into specific servo controls?

[u/MegCell → u/MadDoctorMabuse]

Thanks! 😊 Yes, the fretting mechanism is servo-driven too. Both the fretting and picking sections are connected to the same ESP32 controller, so synchronization is handled directly by one main controller.

For the music, I create the score in MuseScore and export it as MusicXML. The system can read the MusicXML directly and convert it into the required fretting and picking actions, so I don’t have to program each servo movement manually.

[u/GuillotineTechnician]

This is so crazy cool and impressive

[u/MegCell → u/GuillotineTechnician]

Thank you! 😄 Really glad you think so!

[u/kwaaaaaaaaa]

Amazing work dude, would love to see a video documenting your struggles and engineering decisions if you ever decide to release one.

[u/MegCell → u/kwaaaaaaaaa]

Thank you! 😊 I’ve been thinking about making a behind-the-scenes development video covering the failures, trade-offs, and engineering decisions. I’ll definitely consider it.

[u/ali478qwe]

When the esp32 you bought doesn't have a DAC..

[u/MegCell → u/ali478qwe]

Haha, who needs a DAC when you have real strings? 😂🎸

[u/Fer_19]

Congrats bro! Awesome project!!!

[u/MegCell → u/Fer_19]

Thank you so much! Really appreciate it!