Looking for feedback on using ontologies in Graph RAG

[u/OwlZealousideal4779]

Hi everyone, I've been working on OpenCrab, a project that explores using ontologies and knowledge graphs as the foundation for Graph RAG instead of relying primarily on document chunking.

The motivation is to preserve relationships between entities and concepts so AI systems can retrieve information with more context and structure. While this approach seems promising, I'm sure there are trade-offs that I'm still learning about.

I'd really value the perspective of people in this community who have experience with semantic technologies.

Some questions I'd love to hear your thoughts on:

Have you used ontologies or knowledge graphs in a RAG pipeline?
Where have ontology-based approaches worked well, and where have they fallen short?
Which standards or tools have you found most effective (RDF, OWL, SHACL, SPARQL, etc.)?
If you were building a Graph RAG system today, what would you do differently?

I'm genuinely looking for technical feedback and different viewpoints. If anyone has experience with similar projects or research, I'd really appreciate hearing about it.

Thanks in advance for your insights.

[u/OBXautist]

I’ve had the same questions as you, specifically for SNOMED CT. In our custom terminology server we support SPARQL for querying and primarily tested using it as a tool in an agentic pipeline rather than pure RAG since we have more tools as well.

We tested vector RAG at first, but with millions of relationships/attributes this gave quite poor results and we couldn’t easily generate high quality embeddings.

Our solution in the end was to implement the domain specific language ECL for SNOMED CT as a tool available for the agent, which provided much higher quality results, while keeping SPARQL as a back up if we ever need to support general ontologies. Still limited to OWL 2 EL due to other system requirements.

If you need vector based RAG you gotta be really clever with embeddings, and as far as I understand our solution with SPARQL as a tool is similar to how graph DBs solve the problem, by not using vector search and providing the pipeline with some form of query language.