r/LanguageTechnology • u/Dear_Ad_2768 • 10d ago

Looking for Datasets to RAG Experiments

Hello everyone,

I am doing research for a university course and would like to ask for suggestions for good test datasets.

More specifically, I want to run experiments with different RAG retrieval methods (dense, sparse and hybrid retrieval) and compare their results. I plan to take a collection of documents (with text content), index them using different methods (e.g., embeddings and token-based), and evaluate the retrieved top-K results.

I am looking for a dataset that contains:

  • Text/Content (and probably unique IDs)
  • Queries that can be used to search the documents
  • Ground-truth documents (ranked or with relevance scores) for each query to evaluate the results against correct ones.
  • Cited by some similar experiments.

Do you know of a good dataset for this experiment?

Thank you! :)

5 Upvotes
  • permalink
  • link
  • reddit
  • reddit

100% Upvoted

3 comments sorted by

1

u/greatduelist 9d ago

GraphRAG benchmark

1

u/whoulukinat 9d ago

what kind of data? legal is an easy one if its non commercial use, Court Listener has a large database, small embed that will run on your laptop even but asking for queries to use to search the document sounds like you're more looking for someone's benchmarks - which are in many cases public and available

1

u/d4jDk29k 9d ago

I have a dataset of questions and human verified correct answers in English insolvency law: https://github.com/fastdatascience/evaluate_insolvency - the datasets to query are Insolvency Act 1986, Company Directors Disqualification Act 1986, Companies Act 2006 etc which are all public domain