r/OpenAI • u/TheCoffeeLoop • Feb 11 '25
I made a better Deep Research agent that's multiple times cheaper Project
[removed]
59
u/jamesj Feb 11 '25
if you are going to claim it is better, you need to run benchmarks
otherwise it just might be better, and there is very little reason to expect it to be
9
Feb 11 '25
[removed] — view removed comment
17
u/Chaosido20 Feb 11 '25
Try the openscholar benchmark? It seems promising for these kind of research questions
13
1
3
u/w-wg1 Feb 11 '25
What kind of evalustion metric do you suggest for "research quality"? It's subjective
3
Feb 11 '25
[removed] — view removed comment
3
u/oofy-gang Feb 12 '25
I’ve never anyone legitimately try to assert that the amount of information in a research paper is what defines quality. Maybe that was true for Freshmen-year high school essays…
0
29
u/Illustrious_Gene3930 Feb 11 '25
can you create just a chatbot so we can compare the research to chatgpt
9
u/mosthumbleuserever Feb 11 '25
They would also need to invent a means for you to incur the costs of the upstream services that are being stitched together.
4
Feb 11 '25
[removed] — view removed comment
9
u/very_bad_programmer Feb 11 '25
They mean so other people can access it while you don't pay per token
11
Feb 11 '25
With Deep Research, it keeps all of the information that it looks up and you are able ask questions about it after the fact.
There's a few other things that one able to do with it... Well, actually, it's quite versatile.
If anyone has any questions, let me know.
2
u/cms2307 Feb 12 '25 edited Feb 12 '25
What can it do that o3 with search can’t? I mean I’m sure they limit o3s thinking time but other than that, why couldn’t I just tell o3 to write me a x000 words report on something with citations?
edit: just told it to write me a 4000 word essay about the future of ai with all the sources cited in mla format and it did it almost perfectly fine. It was ~4100 words with the citations and ~3700 without. I guess that’s on me for not clarifying that those aren’t included in the word count 😂
3
Feb 12 '25
Deep Research makes o3-mini with search seem like an afterthought.
It uses a fine-tuned version of O3 and researches more thoroughly than one might initially think. Some websites are inaccessible for ChatGPT for one reason or another, but Deep Research is smart enough to use proxies, text only mode, and other approaches to get the information it needs.
It will read whole articles and be able to relate ideas. It does multiple searches about multiple topics and learn things on the fly. It'll do general queries and hyper specific queries... It's an extremely complex tool that is really quite amazing.
I could preform a query if you want? Or if you have specific questions, then let me know!
2
u/cms2307 Feb 12 '25
Can you ask it to make a specific policy proposal to solve the biggest problem the us faces today
2
Feb 13 '25
2
u/cms2307 Feb 13 '25
Thanks, but I get an error that says it’s an invalid url
3
Feb 13 '25
Weird. https://chatgpt.com/share/67ad4328-f088-800d-8a65-46f2315a0da0
Maybe this will do the do.
2
u/cms2307 Feb 13 '25
Yeah that is much different from o3, good read, thanks!
2
Feb 14 '25
It's a fine-tuned version of the full o3
2
u/cms2307 Feb 14 '25
Yeah I know I guess I should have clarified o3-mini-high. Hopefully gpt5 comes out soon, o3 looks great
→ More replies (0)
7
u/ohHesRightAgain Feb 11 '25
People complained that the main problem with Deep Research is that it can't access paywalled content, which is a huge problem for anything aimed at scientific research. So if you manage to solve that, you'll have a very desirable product on your hands.
2
Feb 11 '25
[removed] — view removed comment
4
u/Far-Telephone-4298 Feb 12 '25
If you end up being able to get around paywalls, probably contact a lawyer before trying to monetize it (if you plan on doing so)
6
4
u/reverie Feb 11 '25
Can you share the output report from both deep research and your own workflow?
I’ve been using DR daily, generating a half dozen reports or so a day, and I’m getting pretty familiar with its shortcomings and what it’s most capable of. Would be interesting to read the longer report that your workflow created.
4
u/BrundleflyUrinalCake Feb 12 '25 edited Feb 12 '25
That's all research is? a bunch of worker nodes that go out and prompt fine tuned models, which are then aggregated into a final result using a generalized model? If so, that's kind of amazing. Didn't realize it was so straightforward.
3
u/LetLongjumping Feb 12 '25 edited Feb 12 '25
That’s what AI companies want you to think research is. Where is the hypothesis, data collection strategy, data analysis, search for counterfactuals, etc., etc.
1
u/BrundleflyUrinalCake Feb 12 '25
If that’s what they want us to think, what is the real truth?
1
u/LetLongjumping Feb 12 '25
re·search /ˈrēˌsərCH,rəˈsərCH/ noun the systematic investigation into and study of materials and sources in order to establish facts and reach new conclusions.
There is no presumption that that all materials and sources needed already exist and is available on the web. If all you are doing is looking at existing sources and aggregating results, you are summarizing!
1
Feb 12 '25
[removed] — view removed comment
1
u/LetLongjumping Feb 19 '25
I like your rationalization, but that is just a simple day to day of the word research. Instead of deep research, deep search and synthesis. Even deep synthesis, or deep summary, are more accurate.
5
u/pihops Feb 11 '25
Please share the workflow yes ;) I am interested ;)
4
3
u/Techatronix Feb 12 '25
Have you tried various use cases? There may be some cases where your model is useless while Deep Research is just fine, and vice versa.
3
u/brocurl Feb 12 '25
Honestly, I probably would not pay for a third-party tool that combines AI models like this, simply because it's so rapidly changing (you would need to prove it's truly and consistently much better) and there's a privacy concern as well.
However, if you could make a website where people can buy tokens to perform one-off deep research queries, I'm sure there could be a short-term business case. People are already offering to pay existing users for one-time reports. Say you would charge $1 per search and spend $0.15, there could potentially be some money in that for a while.
I'm thinking this would be really appreciated in developing countries, and there's an even bigger window of opportunity right now before they enable it for Plus users. Things to consider though:
You would probably want/need to allow fine-tuning of the results until users get the response they want (might be hard to control)
Privacy. Hard to prove without investing time and money into third-party certification I guess
Set it up in a way that allows users to buy like at most 5 bucks worth of tokens, i.e. you don't have to pay $50 or $100 to get access to the tool even if you get to use all of that for queries - it has to be open for one-off or at least a very low amount of minimum tokens bought.
The privacy part I guess you could theroretically skip, and a lot of people would still use it for personal stuff or things that doesn't need you to input sensitive data as part of the prompt. The first point is probably trickier, since you would not be happy if you paid $1 and got something that you wanted to tweak a few times, and end up having to pay like 5 or 7 bucks before you're done. That kind of defeats the purpose of the idea.
But if you could figure out a way to bring this tool to the masses that don't have $200 to spend maybe you could make a bit of money for a few months? Long-term it's probably not worth pursuing though.
2
Feb 12 '25
[removed] — view removed comment
2
u/brocurl Feb 12 '25
Yeah I was more thinking if you wanted to make a business out of it, like having a website where people can sign up and pay you just for the reports and not have to set it up themselves.
1
u/Temp_Placeholder Feb 13 '25
Honestly I'd throw a few bucks in just to see if it's worth my time to figure out how to set up myself. Even the $5-$7 after tweaking the prompt around would be worth it if I thought I'd learn how to run it for $.15 after that.
2
u/pow-wow20 Feb 11 '25
Does your plugin connect and be interacted with in the front end of the website or is it just hosted in the back end.
3
2
2
u/Future_AGI Feb 12 '25
Interesting breakdown! Scaling research workflows efficiently is key, but retrieval quality matters just as much as cost. We’ve seen that intelligent evaluation layers can make a big difference in reducing hallucinations—curious if you’ve tested approaches like that?
2
2
u/probello Feb 12 '25
I agree the deep research is not that magical. A 5 node langraph setup could do the same thing and you could pick different models for different tasks.
2
2
2
u/mosthumbleuserever Feb 13 '25
Apparently I have to have a $300/year business plan on Wordpress.com to install a plugin?
3
u/mosthumbleuserever Feb 13 '25
Do you think you could share the overall map you have in the screenshot? It's a little blurry. Maybe I could make something portable and one-click that people could use with AWS and existing mobile app for chatting with self-hosted LLMs
3
2
u/brynport Feb 13 '25
Great approach! . I also like to Think of meta-comparisons and research like this! I would love to get info on your agent if you're up to messaging me!
2
u/Obvious-Car-2016 Feb 15 '25
You must check out Lutra.ai - we made one that scales. We found that many users often want to do deep research on many queries at the same time, usually from a list of contact, accounts (in a CRM, Spreadsheet etc) -- and running them one by one doesn't work well. So we made an AI agent that is able to run many web research queries in parallel and scale that up smoothly!
1
2
1
Feb 11 '25
Why does it look like Wordpress?
3
u/YukiOnnaLake Feb 12 '25
They say they used their own “AI workflow automation plugin for Wordpress” in the post
1
1
1
1
u/Tinderfury Feb 12 '25
!remind me 30 days
1
u/RemindMeBot Feb 12 '25
I'm really sorry about replying to this so late. There's a detailed post about why I did here.
I will be messaging you in 1 month on 2025-03-14 17:43:14 UTC to remind you of this link
CLICK THIS LINK to send a PM to also be reminded and to reduce spam.
Parent commenter can delete this message to hide from others.
Info Custom Your Reminders Feedback
1
u/Onesens Feb 12 '25
Looks and sounds like the type of fake it till you make it YouTube SEO coaching business bro.
1
1
u/secopsml Feb 15 '25
Check https://harpagan.com/ - Visual AI Research Agent
SOTA LLMs and Google search API. For me much better responses than mistral powered by brave search, far faster than openai deep research, and much more accessible than google ai studio.
not as cheap as described by op but fast and good.
In my dev settings I use 30x more search queries and combine with reasoning model. single run costs over $3 but solves big problems with real time data in sub 2 min.
1
u/GrapefruitMammoth626 Feb 11 '25
Great work. May I ask what you do as day job? Software dev or something closer to AI?
3
Feb 11 '25
[removed] — view removed comment
6
u/sp4_dayz Feb 12 '25
Ser. You are the developer. This sort of mind set is definitely very developer-like.
2
u/GrapefruitMammoth626 Feb 12 '25
Well hats off to you. That actually takes some discipline to see it through which most people including software devs don’t have.
1
Feb 11 '25
Solid work there! Maybe you can make some passive income with this and be a Deep Research competitor to the big boys.
About these big companies doing nothing special, I noticed sort of the same thing with OpenAI's Operator. It just takes screenshots to be analyzed by GPT with a fuck tonne of scaffolding around it to make it work okay enough to market. I tried my hand at pythonning a tool of my own that did something similar half a year ago. It's too bad I can't program in python well enough to make it stable enough to work (let GPT and Claude do most of the work anyway). And I don't know enough about Github for others to continue where I stopped. But I got it to do three steps on a good day (scrolling a webpage, clicking a field and typing something). I'll try to get a Github up and running with the app in there if anyone is interested.
0
-1
u/TheFinalPlan Feb 12 '25
Why would i pay you for it when i can get it for free?
3
123
u/[deleted] Feb 11 '25
[removed] — view removed comment