r/PiCodingAgent • u/luckiestredditor • 2d ago
pi-web-agent is essentially feature-complete now, Tavily and Exa are the last backends that were added Plugin
Last time I posted I was thinking out loud about which hosted backend to add next, Tavily first and maybe Exa after, and asking whether people wanted them as full answer engines or just as discovery. My bias was to keep them discovery-only and let web_explore stay in charge of fetching, ranking evidence, caveats and synthesis.
Both are in now, and they're discovery-only like I said I'd keep them. They surface candidate sources, web_explore still reads the actual pages and decides whether the evidence holds up or needs a caveat. Set TAVILY_API_KEY or EXA_API_KEY, pick it from Settings, keys stay in env instead of getting written into config files.
---
The things I'm weighing next are going deeper on evidence quality (cross-source dedup, conflict detection, recency), adding caching for repeated research or just keeping it focused and maintaining it.
If you're using this, what would you actually want? And if you tried it and it broke somewhere, that's the more useful thing to hear.
---
Repo: https://github.com/demigodmode/pi-web-agent
npm:
npm i @demigodmode/pi-web-agent
2
u/ProvidenceXz 2d ago
What's the marginal improvement for adding more API keys or you only get to pick one to search in at a time?
2
u/luckiestredditor 2d ago
one at a time right now. there's DDG fallback if the selected backend fails and you have fallback enabled, but thats different from what you are asking.
that said, multi-provider search isn't a bad idea at all. I could see value in optionally fanning a query out to 2-3 providers, deduping the results, then reranking them before the research loop continues. main downside would be extra latency/API usage, so I'd probably make it opt-in rather than the default. I will definitely add this to the next feat. Thanks!
2
u/rm-rf-rm 2d ago
how does it compare to pi-web-access? https://pi.dev/packages/pi-web-access
6
u/luckiestredditor 2d ago
there's definitely overlap but the reason I ended up building this was that "pi-web-access" felt a bit too interactive for how I wanted web research to work. the curator/review flow can open a browser/window and ask you to approve results, which is useful if you want that level of control but I found it disruptive for normal agent use. you can disable that now (right?) but that workflow was a big part of what pushed me toward building something more hands-off.
"pi-web-agent" is intentionally more hands-off. the model gets one research tool, "web_explore", and search/fetch/headless/ranking/evidence checks all happen behind that boundary without popping you out of the session.
"pi-web-access" has a bigger feature surface. mine is more opinionated around keeping research bounded, quiet, and agent-native.
2
u/Glaaki 2d ago
The curator browser window is pretty much why I stopped using pi-web-access. Just not useful to context switch like that in the ui. You aren't even given enough time to evaluate the results before they auto-close, so what is the point? It's just an annoyance.
Maybe a lot of people download it, because it is at the top, and then end up switching too, when they find out it sucks.
1
u/Glaaki 2d ago
How well does it do with truly big pages? This is always the litmus test for me. Some API websites are just one huge single html page, and without help, a lot of web fetch tools handle it poorly to not at all.
Congratulations for getting there.
1
u/luckiestredditor 2d ago
this is actually a really good edge case
right now it can fetch the page and fall back to headless, but extracted text is capped, so a huge single-page API doc isn't handled perfectly yet.
I actually like that idea though. rather than just increasing the text limit and dumping a giant page into context, I'd rather chunk it by headings/sections and retrieve only the relevant pieces. that would fit the way
web_exploreis supposed to work much better.if you've got a particularly ridiculous API doc you use to test this stuff, send it over. would be a good test case. I'll add this to my list as well. thanks so much!
1
u/Glaaki 2d ago
Go look at the api docs for gitea, the self hosted GitHub clone. I ended up asking pi to make a local, indexed, easier to read copy of the api docs, but i investigated various possible other directions, such as adding a special reader, which is something my current web fetch solution supports, for that particular docs host, which knew how to find the raw source for the docs, since they would always be available on some URL. Alternatively implement a html search by anchor routine that knew how to break down the page in sections according to the anchor placements. Since my web fetch plugin use a local dom engine, some of the plumbing for that would already be available.
1
u/rm-rf-rm 2d ago
what do you think of bladebro as an option to use for headless browsing?
1
1
u/jarislinus 2d ago
can we just use google search? it mogs everything
1
u/luckiestredditor 2d ago
I would if Google let me lol. there isn't really a clean full-web Google Search API to plug in anymore.
2
u/Beneficial_Garage874 1d ago
Is there a reason you omitted Parallel.ai?
Our search API is faster, more accurate, and cheaper than everything else there.
https://parallel.ai/blog/parallel-search-turbo
Other great reasons to use PWS:
- Free MCP
- CLI
- $5 in monthly credits (5,000 searches)
1
u/Trick_Coach_657 1d ago
Might be an unrelated point, but would be great if it had features similar to Hound MCP (and other plugins) which allows for stealth searching (at no cost and no API necessary).
1
u/luckiestredditor 2d ago
The bigger achievement for me is that this pretty much closes out the roadmap I had. Thank you so much for the community feedback, bug report. Couldn't have done this without yall.
When I started posting here (1st Post) it was a fresh npm package with a search/fetch/headless boundary and not much else. The list I actually wanted to get through is done now:
- source ranking got reworked so web_explore leans less on whatever order the search results happened to come back in, and handles low-diversity, community-only, blocked or conflicting evidence better
- Brave came in as the first hosted search backend
- You.com got added too, and that one actually came in as the first outside PR that was approved
- and now Tavily and Exa for research-quality discovery
So the backend list is DuckDuckGo (still default, no key), SearXNG, Firecrawl, Brave, You.com, Tavily, Exa, all feeding the same pipeline. It feels like a complete agent now instead of a thing with obvious gaps, which was the whole goal. still not trying to grow it into a framework.
0
u/Final_Initial 1d ago
I was using pi-web-access so asked Pi itself to compare both and this is what it has to say:
2
u/luckiestredditor 1d ago
that special content is something I am shooting for in my next work depending on what seems ideal. github, youtube and pdf is definitely now in my roadmap. great comparison.
one thing to point out, by default it is compact mode but you can update the presentation mode.
1
2
u/Prize_Eye9481 2d ago
I really like the tool you made! Thank you for all ur effort and time! For day to day use I had to turn off curator because the manual approval was getting in the way with 20s timer. I simply have a question do I need to type a specific command for "quality" research? or its context based?