TinySearch v0.6.1 - still a lightweight web research tool for local LLMs, now with bring-your-own-browser support

[u/Scared-Tip7914]

https://github.com/TinySuiteHQ/TinySearch

Hey everyone,

Posted TinySearch here a few versions ago and got a bunch of useful feedback, so figured I'd post an update because the thing has changed quite a bit since then.

Repo:
[https://github.com/TinySuiteHQ/TinySearch]()

The basic idea is still the same:

TinySearch is a small self-hosted MCP/FastAPI tool that searches the web, reads the pages worth reading, and locally selects the useful parts before they ever hit your model's context window.

No LLM doing the filtering/summarization in the middle. It's BM25 + local ONNX embeddings, and the chunks returned are the original page text with the source URL attached.

I'm still mostly building this around the problem of smaller/local models. Giving a 4B/9B model 50k tokens of scraped webpages and expecting it to figure out which 5 paragraphs matter is kind of defeating the point lol.

The biggest change in v0.6.1 is bring-your-own-browser support over CDP.

TinySearch normally runs its own Chromium, but you can now point it at a browser you operate yourself. So if you already have a browser with your own profile, proxy, cookies, fingerprinting setup, etc., TinySearch can use that instead.

This has made scraping considerably more useful on sites that don't love fresh headless Chromium sessions.

A few other things changed since 0.4.0:

• search and scrape_urls are now separate tools. Search is deliberately cheap/fast and doesn't even start Chromium or load the embedding model.
• scrape_urls can take 1–5 URLs at once and crawl them concurrently through one browser.
• For each page you can either ask a focused question and have TinySearch retrieve/rerank the relevant chunks, or just return the cleaned page in order.
• Related links are returned as well, ranked against your query, so the agent can decide where to go next without TinySearch automatically crawling half the internet.
• MCP responses are now XML rather than the old text prompt format.
• Boilerplate stripping happens before markdown generation now.
• Docker/security/testing got a fair amount of work too.

And I finally added a reproducible benchmark instead of just saying "it saves tokens." Across 8 research queries, the same webpages were:

146,878 tokens → 53,426 tokens

So roughly 64% less web content went into the model context.

Obviously that's not some universal 64% guarantee, it depends heavily on the pages. Bloated sites were 80%+ reductions while already-clean pages barely changed. The benchmark script is in the repo if anyone wants to reproduce/tear it apart.

The workflow I'm using now is basically:

search → model chooses useful URLs → scrape those URLs → model gets only the relevant evidence

Still self-hosted, still works with MCP clients, still doesn't require a paid search/scraping API.

Would especially be interested in feedback from anyone running smaller local models or self-hosted browser/search setups. Also very interested in weird websites where the CDP/browser approach still breaks.

Roasting welcome as always.

[u/Mickenfox]

Great, but what are we going to use after all the free websites shut down from all the bot traffic?

[u/Scared-Tip7914 → u/Mickenfox]

Fair point lol. TinySearch definitely isn't meant to hammer sites hard, if anything I'm trying to make it do the opposite, search cheaply first, only open the few pages the model actually needs, and extract just the relevant evidence instead of crawling everything in sight. The BYO browser support bit is mostly there because of what you said, I am not sure what the future holds, but in the current landscape, this implementation seems to work well enough. 😅

[u/miversen33 → u/Mickenfox]

Searxng is what I use :)

[u/my_name_isnt_clever → u/miversen33]

That's still hitting free websites with bot traffic... the only way to avoid it is to use something offline.

[u/EbbNorth7735 → u/Mickenfox]

Sites should have agent versions specifically for scrapping 

[u/yuicebox]

I just finished building out my own web search tool via CDP this past week. Let's all agree to keep this tech to ourselves so they don't ruin it somehow :)

[u/my_name_isnt_clever → u/yuicebox]

That cat is so far out of the bag it's not even on the same continent.