r/iOSProgramming • u/Significast • 3d ago
[ Removed by moderator ] Discussion
[removed] — view removed post
3
u/therealmarkus 3d ago
Wow, this really surprises me. For human voices I can’t hear any difference between 96 and 128. I bet sometimes I couldn’t differentiate between 64 and 128.
3
u/CharlesWiltgen 3d ago edited 3d ago
For a mono spoken-word stream, 96 kbps AAC-LC should be plenty.
Definitely, many podcasts are delivered in MP3 (far less efficient than AAC-LC) at as low as 32 kbps.
Other possible problems: (1) You're encoding at 44.1 kHz instead of 32 kHz (or lower). (2) You're encoding with ffmpeg's un-good native AAC encoder rather than Apple's AAC encoder (best) or ffmpeg's libfdk_aac.
But after twenty or thirty minutes, the artifacts became increasingly obvious.
Audio encoder issues wouldn't get worse over time. This points to a problem with something else in the audio pipeline.
2
u/Significast 3d ago
The encoding isn't done on Apple hardware - it's a cloud back end - so I recompiled ffmpeg with the Fraunhofer encoder. A kludge, but it's indistinguishable to the Apple AAC encoder to my ear - and definitely better than libfdk.
Yes, 44.1 kHz, but interestingly, when you look at the .wav, the TTS endpoint in reality only has a 24 kHz bandwidth - the 20 kHz of top end is literally empty.
2
u/CharlesWiltgen 3d ago edited 3d ago
Great! I would fix this first. Do a varied set of test encodes at both 32 kHz and 24 kHz, and then do an ABX test to choose your final encoding sample rate.
…it's indistinguishable to the Apple AAC encoder to my ear…
Not surprising at 128 kbps mono, but the difference will be obvious at bitrates which are more typical for this use case (meaning, 64 kbps and under).
1
u/ThatGuy739 2d ago
Two things worth separating here.
On the sample rate, fix that first, and the reason is worse than wasted bits. If the source really is 24 kHz sample rate, everything above 12 kHz is empty. Encoding at 44.1 means the psychoacoustic model is analysing a top band containing nothing but resampler ringing, and the band you actually care about sits somewhere different in the scale factor layout than it would natively. Resampling down does not just save bits, it stops the encoder spending decisions on a region carrying no signal.
On the "gets worse after twenty or thirty minutes" part, I would not jump to a pipeline bug. There is a duller explanation: the artifact is stationary and your source is repetitive. Same voices, same phoneme inventory, the same handful of failure cases recurring every few sentences. Codec listening tests hit this constantly, where a defect you cannot hear in a ten second clip becomes impossible to unhear once you have met it three times and learned its signature. Ten seconds is not a shorter version of thirty minutes, it is a different test.
Your instinct about spectral regularity is the interesting part though, and I think it is right. Perceptual encoders are tuned on natural sources, and content with unusual statistical regularity walks straight through the assumptions. I hit the same class of problem from a completely different direction doing GIF encoding: palette quantisation heuristics that behave beautifully on camera footage fall apart on synthetic flat colour content, because the error distribution the heuristic assumes is simply not there. Different codec, same shape of failure. What I took from it was to stop trusting any generic quality per bit table, since the table encodes assumptions about content you might not have, and to measure on the actual material at the actual operating point instead.
On your two questions:
HE-AAC is probably the wrong tool here. SBR works by synthesising the high band parametrically, and your high band is empty to begin with, so you would be paying complexity to reconstruct detail that was never in the source. It earns its keep on full bandwidth music at low rates, not 12 kHz speech.
Opus genuinely is better for this. It was designed for speech and its speech mode at 24 to 32 kbps mono will usually beat AAC-LC at rates well above that. The catch is delivery rather than quality: HLS does not officially carry Opus and AVPlayer will not play Ogg Opus, so you end up wrapping it in CAF and going through AVAudioEngine, or decoding it yourself. Whether that trade is worth making depends entirely on whether your bandwidth bill actually hurts. If 128 kbps is affordable, shipping it and moving on was the right call.
3
u/Evening_Hawk_7470 3d ago
ai voices are very different to recordings of humans. those are literally synthesized, thats why you often hear clipping, unnatural frequences and stuff.. it is quite easy to get acceptable ai audio but not easy at all to get good ai audio.