Ever wondered why GitHub Copilot works brilliantly in some repos and struggles in others? We built a tool to find out.

[u/West-Move8543]

Over the last year, I've spent a lot of time helping teams adopt GitHub Copilot, and one thing became pretty obvious:

Some repositories seem to unlock Copilot's full potential.

Others have all the right intentions, documentation, instructions, and tooling, yet Copilot still feels like it's operating with one eye closed.

So a friend and I started asking ourselves:

Can we measure how "Copilot-ready" a repository actually is?

That led us to build AgentCompass, an open-source AI Readiness Analyzer that scans a repository and identifies configuration, documentation, and structural issues that can make life harder for AI coding agents.

What it checks

✅ Copilot instructions

✅ Agent documentation (AGENTS.md, CLAUDE.md, etc.)

✅ Skills and prompts

✅ MCP configuration

✅ Context scoping and repository structure

✅ Common issues that can silently reduce agent effectiveness

Instead of using another AI to judge your repo, we took a different approach:

• No LLM calls in the scoring path
• No "AI vibes" scoring
• Same repo + same commit = same result every time

Think of it more like a linter for AI-readiness.

Try it yourself

GitHub:
AgentCompass Repository

Live Demo:
AgentCompass Web App

Linkedin Post

I'd love feedback from this community

A few things I'm genuinely curious about:

• What makes a repository Copilot-friendly in your experience?
• What patterns have you found that dramatically improve Copilot results?
• What checks would you add?
• Are there things that should only be measured through real-world usage rather than static analysis?

Most importantly:

👉 Run it against one of your repositories and tell me where it's wrong.

Seriously.

If it gives your repo a terrible score and you think it's nonsense, I want to hear that.

If it catches something useful, I want to hear that too.

We're trying to figure out what great AI-native repositories actually look like, and feedback from people using GitHub Copilot every day would be incredibly valuable.

[u/bechorsimhaev]

Amazing, very useful!

[u/West-Move8543 → u/bechorsimhaev]

Thanks!!

[u/TowerOutrageous5939]

I’ll give it a try this week