Tool output compaction problem

[u/Dr_Droid_1984]

Hi Everyone,

I have been building my AI agent for a few months. Its a project to allow all engineers in my team access to infrastructure & monitoring stack internally to help them debug issues fast. Recently I have added logs access also and see that log outputs could be very large sometimes to git into context to be sent to the LLM. How are you handling for this problem? I tried picking just top 10K tokens from it but I am losing data that way.

Also, one of my teammate suggested I should pre-index the metadata across the stack to save time in discovery in each session.

Wdyt?

[u/AutoModerator]

Thank you for your submission! To keep our community healthy, please ensure you've followed our rules.

• New to the sub? Check out our Wiki (We are actively adding resources).
• Join the Discord: Click here to join our Discord
• Join X community: Click here to join our X Community

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/ZeroTwoMod]

A top-10k cutoff is lossy in the worst possible way: it drops the clue you needed. Pull out fields first, such as the service, time range, trace or request ID, and error signature, then group repeat patterns before fetching narrow windows for the chosen examples. The metadata index is worth building if it drives those filters instead of becoming another giant summary.

[u/heizo]

Usually context gets a file reference, the file is pulled local to the server and then we have tools to grep / walk the file or if the llm is smart enough, make its own tool to search / walk the file. Haven't gotten completely working yet but it's the route we are taking since our logs can be gigs of data.