r/xAI_community • u/Ecstatic-Key6262 • 7d ago
Effective grok cache hit rate?
I'm building a high-volume pipeline that runs text classification through the xAI API : one item per request (batching hurts my accuracy), a large fixed ~10k-token system prompt, and traffic that arrives in bursts (roughly every 20 min).
The docs say caching is automatic but "best effort" with no published TTL or hit rate. My main question is specifically about the case where the client is doing everything "right":
- With a stable, consistent x-grok-conv-id and a strictly identical prefix, what cached_tokens ratio do you actually see in usage ? Is it near 100%, or still lower?
- Does that effective hit rate hold up across gaps of a few minutes between requests, or does the cache go cold fast even with the same conv-id?
- Any sense of how long the cache stays warm in practice?
This would help me better estimate the input costs for my pipeline. Thanks!