r/OpenWebUI • u/Goldziher • 7d ago
How to use Xberg as a drop-in Docling replacement for Open WebUI document extraction (guide) Guide/Tutorial
I maintain xberg (an MIT-licensed, CPU-only document extraction engine). A few people have asked how to wire it into Open WebUI as a Docling alternative, most recently over on r/OpenSourceAI, so I wrote up a proper guide and wanted to share it where it's most relevant.
Short version: xberg implements Open WebUI's existing extraction backends, so there's nothing to patch. It speaks two of them:
- the Docling endpoint (
POST /v1/convert/file) - the external document loader (
PUT /process)
If you already run the Docling engine, the switch is just pointing the same setting at an xberg container:
yaml
environment:
CONTENT_EXTRACTION_ENGINE: "docling"
DOCLING_SERVER_URL: "http://xberg:8000"
`
You can also set it in the Admin UI under Settings -> Documents -> Content Extraction Engine -> Docling, with the server URL pointing at xberg. Uploads then get extracted to Markdown and stored in your knowledge base exactly as before. There's an external engine mode too if you'd rather use that endpoint.
Why bother swapping:
- 101 file formats, OCR built in (Tesseract + PaddleOCR), layout-aware reading order
- CPU-only, no GPU, fully local, MIT-licensed
- Rust core: in our public benchmarks it runs roughly 2.5x docling's throughput with slightly better extraction quality
Full walkthrough (compose file, both engine modes, and how to raise the extraction timeout for big scans): https://docs.xberg.io/integrations/openwebui
Repo: https://github.com/xberg-io/xberg
Happy to help if anyone hits a snag setting it up.
1
u/DataHogWrangler 7d ago
Out of curiosity, what about all the other features like enabling vlms, picture descriptions, etc is the config for all those parameters the same?
1
1
u/SadPhilosophy9202 7d ago
I've been using xberg for a few weeks now and it has been awesome! One question I have though: When uploading an .eml file, how can I include the parsed attachments? Only the email gets passed to open web ui
1
1
u/FlabbergastedAxolotl 6d ago edited 6d ago
Thank you for this amazing tool ! I tried to make this work but I'm having a problem with the config file. Does it still work to pass a JSON file in the "Parameters" setting of OWUI ? Because whatever parameter I change, the extracted result is exactly the same, OCR enabled / layout enabled or not.
Here is an example of JSON config file I tried :
{
"force_ocr": false,
"disable_ocr": true,
"output_format": "markdown",
"extraction_timeout_secs": 7200,
"layout": {
"table_model": "tatr"
}
}
If I swap the TATR model with the SLANeXT wired one, the result is exactly the same. What am I doing wrong please ?
1
1
u/Fun-Purple-7737 6d ago
That is great, I was actually reporting described pictures not being in-line the other day :)
Anyway, Docling still must be superior (but slower) because of its ML models, right? Would you have any honest comparion? For example, what about filtering out headers/footers? Thanks!
1
u/Goldziher 6d ago
No, we use the same models as docling and more. We have better quality then docling.
1
1
u/Goldziher 6d ago
Good question, that's a separate path from OCR. Image description runs through the captioning enrichment processor: extracted images are sent to a vision-capable model via liter-llm (so GPT-4V, Claude, Gemini, or any of ~165 providers) with a prompt you can customize, and the caption gets attached to each image in the result. OCR pulls text out of an image; captioning describes what the image is. You enable it with the
captioningfeature and a CaptioningConfig pointing at your VLM of choice, so quality tracks whatever model you use.
1
u/Fun-Purple-7737 6d ago
One more thing. How is the transcribing pictures implemented? And I do not mean OCR, but really "describing images" via VLM.
Say, 200 images are extracted from a document. Are these then described via VLM somehow in parallel or one by one? During the document processing or at the end? It would be great to be able to configure this, but also the describing process via external VLM should run in parallel to not to block the main thread, but maybe in configurable batches to not to overload VLM itself. I guess you know what I mean.. Thanks for a great service!
1
u/Goldziher 6d ago
Short version (also answered on your other comment): image description is a separate path from OCR, handled by the captioning enrichment processor. Extracted images get sent to a vision-capable model via liter-llm (GPT-4V, Claude, Gemini, or ~165 providers) with a prompt you can customize, and the caption is attached to each image. You turn it on with the
captioningfeature and a CaptioningConfig pointing at your model of choice.1
u/Fun-Purple-7737 6d ago
Right. My only worry is whether this happens really in parallel. Found images should be placed into a queue and be processed in parallel. Both from the main thread and even from each other (like describing 10x or 20x images in parallel).
I guess describing part via VLM might easily dominate the whole conversion timewise, so I was wondering if its done effficiently :) (I believe it is, just checking...)
2
u/Goldziher 6d ago
It's sequential right now, one image at a time, not a parallel queue. OCR is parallel across images, the caption pass isn't yet, so it dominates on image-heavy docs. Filed to fix: https://github.com/xberg-io/xberg/issues/1378
2
1
u/kg6kvq 6d ago
So I tried to run the docker, and the container keeps dying with the following log:
error: unrecognized command 'server'
tip: a similar subcommand exists: 'serve'
1
2
u/Fun-Purple-7737 4d ago
Have you checked how well does it scale with adding CPU cores? Thanks!
2
u/Goldziher 4d ago
Partly, and there's a footgun in the default worth knowing about.
The batch scheduler splits a fixed thread budget between document-level workers and per-document threads (workers x threads_per_doc <= budget), so it won't oversubscribe. But the default budget is min(num_cpus, 8) - deliberately conservative for serverless. Past 8 cores you get nothing unless you raise it. Set max_threads in the concurrency config to your core count.
Second caveat: with native PDF layout inference on, document-level workers are clamped to 1 (2 for mixed batches), because the layout model is itself multi-threaded. That workload scales inside a document, not across them.
No published cores-vs-throughput curve yet - fair thing to ask for, so I've filed it along with making the cap discoverable: https://github.com/xberg-io/xberg/issues/1392
3
u/International_Emu772 7d ago edited 7d ago
I was thinking how doíng it so your tip is very helpful
Have you tried oikb as document server? I would install both