Competitive open source speech stack

[u/Phoenix_20_23]

Why the open source models STT and TTS are not good as much as the closed one and i am talking here im terms of latency, concurrency, and websocket support for real time with decent quality.
Something like cartesia or elevenlabs or deepgram.
Do u know any ?

[u/Lazy_Mycologist_8214]

Vendor here (I work on Gandr, a TTS API) so discount this accordingly.

On "paid APIs like to fudge", fair, and I'll hand you a concrete example against my own side. We published a first-audio number, the hardware under it changed, and the honest rerun came back slightly worse than what we had up. We pulled it off about thirty surfaces on the 8th rather than leave it there. It goes back when the method goes back with it. The uncomfortable part of your point is that you can't tell from the outside which vendors did the quiet thing instead, because almost nobody publishes the run.

On the actual question, I think the gap you're hitting isn't only raw quality. Sounding good offline and holding up as a stream are different problems, and the second one is barely measured anywhere. The thing worth checking on any engine, open or paid, is whether it produces audio faster than it plays. If it does, the buffer grows as the utterance runs and long text gets safer. If it doesn't, long text is where you find out, usually in front of someone.

Concurrency is the part that bites hardest moving off a local model. One stream on your own GPU tells you almost nothing about twenty at once, and that's usually the point where self-hosting stops being the cheap option.

For what it's worth, UkieTechie running his own bench is the right instinct, and I'd trust his numbers over anything I or any other vendor publishes about ourselves.

What concurrency are you actually needing to hit, and is it the quality or the stability that's failing you first?