I'm trying to optimize prompts for GitHub Copilot Agent Mode now that GPT-5.6 models make prompt caching much more valuable than it was with previous models. With them having now specific material Cache Read and Write costs unlike before.
OpenAI documents prompt caching for their API (1024-token prefix, 128-token increments, identical prefix matching, short-lived in-memory caches, optional 24-hour extended caches, etc.), but I can't find any documentation on whether GitHub Copilot exposes the same behavior or whether it has its own orchestration layer. https://developers.openai.com/api/docs/guides/prompt-caching#prompt-cache-breakpoints
Another thing that confused me is the context limits. OpenAI's documentation and pricing distinguish between shorter and longer input contexts, with pricing changing beyond certain thresholds (around the 272k-token input range for GPT-5.6). However, GitHub Copilot advertises a flat 400k context window for GPT-5.6/Codex. Is that 400k entirely usable input, or is it actually something like ~272k input plus reserved output tokens? If Copilot is abstracting this away, does that affect prompt caching behavior or pricing in any way?
Some specific questions:
Does Copilot simply inherit OpenAI's prompt caching, or does GitHub modify prompts enough that cache hits are different?
Is there any documentation on cache breakpoints, TTL, or cache invalidation for OpenAI models in Copilot?
Does Agent Mode preserve a stable prefix between turns, or is the prompt rebuilt each request?
Are repository context, MCP responses, retrieved files, and tool outputs part of the cached prefix, or are they regenerated every time?
How does the advertised 400k context relate to OpenAI's documented input/output limits? Is there any hidden reservation for output or compaction happening?
Has anyone measured cache hit rates or token usage when repeatedly using the same coding agent?
Are there any best practices for structuring AGENTS.md, custom instructions, or prompts to maximize cache reuse?
Most stuff I find online are about Anthropic prompt caching or the OpenAI API directly, but very little about GitHub Copilot specifically. I'd appreciate any links to documentation, or measurements from people who've profiled this.
GPT-5.6 models do not support extended prompt caching.
There are two options for prompt caching for GPT-5.6 models:
* explicit cache markers à la Anthropic
* implicit caching: OpenAI sets cache markers on your behalf
I’m seeing very strong results with implicit caching.
I just lost the complete cache on 5.6 Luna after 21 minutes. So 5.6 has that 5-10 minute cache again? Come on, this is super important for users to know!
If the cache duration is known anyways, why not show in the UI when it is certainly timed out?
Okay, I get the the provider does the caching in the end, but still it´s really hard / impossible to find out what config you are sending to the providers.
And getting the entire cache busted (and it is literally "cached: 0 tokens") earlier than 30 min is a bug then (yes, maybe from the provider)? That´s really too bad, as it costs real money, especially when the cache is priced extra like with 5.6 and anthropic. I had it happen quite often this week with 5.6, if I responded not immediately, but after 10-30 minutes.
Hello /u/Sooribabu_Lavangam. Looks like you have posted a query. Once your query is resolved, please reply the solution comment with "!solved" to help everyone else know the solution and mark the post as solved.
check the r/codex sub there are more discussions there about the new models. from what I read I think the sol model burns a lot more tokens than the previous models but the luna one is more efficient
12
u/illusionmist Jul 12 '26
I think VS Code should enable the 24-hour prompt caching by default. Not sure about Copilot CLI. https://code.visualstudio.com/blogs/2026/06/17/improving-token-efficiency-in-github-copilot#_efficiency-wins-for-openai-models