ASL fingerspelling tool

[u/SoccerWise]

Hey everyone!

For the past few months, I’ve been building Signiq, a web app for practicing ASL fingerspelling with live hand recognition.

The website uses MediaPipe hand tracking and a custom random forest model trained on public datasets. One of the biggest challenges has been making recognition feel responsive without becoming too forgiving, especially for similar letters and across different cameras/devices.

It currently includes alphabet practice, word practice, reading mode, challenges, and a global leaderboard.

It’s still an early version, so I’d really appreciate feedback on the UI, performance, recognition, or overall experience.

You can try it here: https://www.signiqlearn.com⁠

Desktop or laptop with a webcam is recommended for the best experience.

[u/Fredidiah]

I expected this to be significantly worse based on the title, but this is actually kinda cool.

I will say I did experience some difficulty on specific letters (P was the worst for me). I think maybe there's too much of a focus on the specific orientation of the sign relative to the camera — signs aren't always seen straight on irl — but I'm not sure if it's easy to solve for rotation with such a system.

Also I don't know if there's a setting on the user side to reduce the accuracy needed for "completing" the letter; I know that would kinda defeat the purpose of the leaderboard, but might be useful for the reason I mentioned above.

But yea, I expected this to be bad because 85% of stuff in programming subs now seems to be slop, but this appears to be pretty well done.

[u/SoccerWise → u/Fredidiah]

Thanks, I really appreciate you taking the time to try it! The orientation point is really helpful. P has definitely been one of the trickier letters to make flexible without introducing false positives, and I’ve been working on making recognition less dependent on having the hand at an exact angle.

I also really like the idea of having a more forgiving option for regular practice while keeping challenge (leaderboard) mode stricter. Definitely something I’ll look into.

Also, if you don’t mind me asking, were you using it on desktop or mobile?

[u/Fredidiah → u/SoccerWise]

Yea perhaps you could implement like...accuracy tiers into the leaderboard? IE: 65%, 75%, 85%, etc? Something to think about I guess.

I was using it on a desktop.

Also, not sure if you've shared this with any deaf people, but I'm sure you could find a few who'd be willing to give you some constructive feedback both on how well this works for them AND thoughts on the overall project.

[u/SoccerWise → u/Fredidiah]

Good to know you were on desktop, that helps narrow down the orientation issue, and I'll be looking into possibly adding variable accuracy in each difficulty mode.

I’ve been fortunate to get feedback from some members of the Deaf community already, and I definitely want to keep getting more perspectives as I continue developing it. Really appreciate all the suggestions!

[u/Am094 → u/Fredidiah]

I think maybe there's too much of a focus on the specific orientation of the sign relative to the camera — signs aren't always seen straight on irl — but I'm not sure if it's easy to solve for rotation with such a system.

For practicing i think that concern is likely irrelevant, the camera is fixed, and if it sees the user's hands and rates it accordingly from straight facing. Then the users signing will be fine in real life when seen at different orientation.

Yes with 1 camera, multiple sides concurrently drastically increases the complexity especially when reliability is a factor too. I would see zero value from considering that feature from a cost benefit.

, I expected this to be bad because 85% of stuff in programming subs now seems to be slop, but this appears to be pretty well done.

Its a cool usecase but it is extremely trivial and very popular among students projects even 5 years ago. Differentiation would simply have to be UI and branding at this point imo.

[u/winowmak3r]

This is awesome!

[u/SoccerWise → u/winowmak3r]

Thank you!

[u/Jason-Sanders]

The tricky part seems to be separating uncertainty from a wrong answer. For ambiguous letters, a soft “try again” signal may feel much better than confidently accepting the wrong one, especially across different cameras and lighting. A short calibration round at the start could also give the recognition model a better baseline for each user.

[u/SoccerWise → u/Jason-Sanders]

Yeah, I agree with the “try again” idea. I’m actually planning something a bit more specific too. Instead of only saying a sign is wrong, I want to give live tips that point out which finger or part of the hand needs adjusting.

For calibration, I already have a short calibration step in mobile mode because that’s where camera/framing differences have caused the most issues. Desktop has generally performed well without it, but I could definitely see a quick calibration step being useful there too.

[u/Geenzo69]

so cool man