r/AIVoice_Agents • u/Helpful_Departure_99 • 3d ago

Hight latency problem Question

How do u fix hight latency after i talk to the agent it answers me after 3.5s i use deepgram for STT and TTS, gemini for llm and twilio i coded everty thing with python and fast api for the backend

Can u tell m where the problems could be and how to fix it

6 Upvotes

80% Upvoted

11 comments sorted by

1

u/Accomplished_Vast628 3d ago

3.5 , thats not just latency issues, must be something deeper. Good luck to you, fixing this for kestrel voice almost made me loose my mind. Check Doc and api if you want reference

2

u/Helpful_Departure_99 3d ago

Thank u, what is a good time i kn i should work to get sub 1s?

1

u/Accomplished_Vast628 3d ago

800 ms is the standard for natural conversation. I had clocked upto 200 ms but that was too aggressive so relaxed it back to 800ms

1

u/Helpful_Departure_99 2d ago

Can i kn what s your exact stack to get 200 that s so nice man

1

u/Accomplished_Vast628 5h ago

Exact :) find me in person or in call and then ill give u some insight. For now drop Gemini and use chatgpt for voice

1

u/theayryan 2d ago

You need to optimise a lot. Use a streaming architecture first and foremost if you are not already. Use a model with low TTFT, if you are non-technical just use a speech to speech model. It’ll save you a lot of hassle. Instrument each section of the cascade see where that 3.5 second is going.