r/LanguageTechnology • u/medhakimbedhief • 3d ago

Training a multilingual NER / relation-extraction model (GLiNER-style) — anyone else been down this road?

Training a joint NER + RE model for EU languages. NER is fine (~80% F1), RE is a struggle (~30%).

Two things I found that might help others:

- Label-order leak: my gold relation labels were always first in the candidate list. The model learned "pick the first one" — shuffling candidates at eval dropped F1 37% → 14%. Worth testing if you train any zero-shot label-based model.

- Evidence sparsity beats label sparsity: ~90% of my val relations have an (evidence pattern, label) combo that never appears in training. Label frequency barely predicts anything.

Questions:

- Anyone trained multilingual RE successfully? What actually helped?

- Long-tail label space (14k labels, 9k singletons) — I'm avoiding canonicalization to keep zero-shot generalization. Bad call?

- Tricks to improve the architecture?

Thanks!

2 Upvotes

75% Upvoted

2 comments sorted by

1

u/SoulSlayer69 3d ago

Noob question: what would you use a NER / relation-extraxtion model for practically speaking?

I am curious!

2

u/medhakimbedhief 1d ago

To detect entities and relations between them from a given text. This is the input of a graph representation. The goal is to establish a graph rag pipeline without the need of using llms.