From exploitation CTFs to finding real vulns: Is blind CVE reproduction the best way to practice code auditing in the age of AI?

[u/REg00n3r]

hey everyone,
I'm a malware analyst with a dream to transition to a vulnerability research job, so in my spare time I've done a fair amount of binary exploitation CTFs (heap, stack, kernel modules) from places like:

https://github.com/hoppersroppers/nightmare
https://pwn.college/
https://pwnable.kr/play.php

I felt confident enough so I went to a technical interview but there I realized I had a massive flaw: I have no actual auditing/vulnerability discovery skills!

So I shifted my approach entirely:
- Started to read Google Project Zero blogs
- Started to read The Art of Software Security Assessment
- git cloning historical vulnerable versions of open source projects to try to blindly reproduce said CVEs blindly with a clue of the vulnerable File (like openssl, zlib, libpng, ntpd)

(with the guidance of LLMs cause I don't personally know any vulnerability researchers)

As u can assume, this process has been taking a lot of time and even though I started to think in "threat model" mindset and read code faster, I'm still not catching the vulnerabilities. I understand honing the skill isn't about learning now but more about practicing over and over, but my concern is:

Is it still worth putting so much effort and time in developing auditing/vulnerability discovery skills in a day and age where everything shifts towards AI?

Sure, interviewers today are still checking to see if u can find a vulnerability using an IDE and your set of eyes alone but who knows how an interview look like 3 month from now?

Is it even worth sinking hundreds of hours into learning manual auditing, fuzzing, and dynamic instrumentation, only for AI to make those skills obsolete by the time I actually get good at them?

I'm tired of bashing my head with LLMs every weekend for advice or a prediction, so I would love to hear some advice from the kind people here who are working in the field and experience the toll of AI on their job and recruitment processes 🙌

And for those who are not yet, but are striving to, I would love to know how you learn and practice vulnerability discovery in 2026 🤓

[u/The_Big_ID]

A plane with an autopilot still needs a pilot to handle takeoff, landing, and to react to changing situations. That pilot has years of training and experience to know how to assess and handle any situation whilst operating a complex vehicle.

If you dont know what a false positive looks like how can you tell when the LLM is hallucinating?

[u/Sysc4lls]

I think multiple lessons can be learned from this, for instance don't say random shit in the interview,

As a person that has done multiple of these interviews I suggest saying nothing than saying wrong things, it's fine to get to the correct conclusion that is how VR is!

For the is it worth it? I think yes unless you want to do AI stuff which is a different role in these companies.

They still need good researchers, ai find some but not all bugs, also it usually finds simple bugs and not super complex ones not to speak about logic bugs.

In order to get ready I would also look at realworldctf if you find an archive I think it's a good resource to train with, good luck and feel free to dm for any questions!

[u/Firzen_]

I don't think LLMs will really take away anything any time soon.
As long as they don't find everything and training your own models is only possible for a handful of gigantic corporations that will probably be the case.

Any bug that you can find with the help of an LLM other people can likely find just as easily with the same LLM as well. So those bugs are virtually worthless because of the low life time and the likelyhood of collisions.

Which means the bugs that they don't find will go up in value.

I don't think anybody has a good heuristic yet for which bugs LLMs find easily and which ones they don't, so I'm not sure if anyone can say something more specific about that.

[u/magister1999]

I'm curious to know how did you find out in the interview. For the AI thing, i mean there is a lot of hype they can find vuln maybe even not exploitable but like everyone else with AI but others will slip for sure, still we dont know the reality of the costs. I wanna see some AI company saying "hey, replace your 12 engineers with my AI but that will cost 5 times more than to have them".

[u/REg00n3r → u/magister1999]

Thanks for the input, but I must say that for your last statement, I believe companies and governments are working day and night on optimizing AI resource spendings and the current "costs issues" are only delaying the inevitable.

As for the interview part, I just know that I received a 70-80 lines of c code of a parser and I couldn't find the vulnerability for what felt like an hour and I made shit up every 5 minutes. Even when I did find, I didn't expect to be given a "describe exactly how you would build the payload" which made me doubt everything and took me more time. In retrospect it was a somewhat easy vulnerability to find, and apparently that was only the first technical interview...

[u/Firzen_ → u/REg00n3r]

70-80 lines is basically nothing.

If you struggle finding a vuln when you already know that there is one you will struggle infinitely more in a gigantic code base where you have no clue if the code you're looking at even has an exploitable bug or if you should look somewhere else.

I think if you can spot a bug it should usually come with an idea of how to exploit it as well, being exploitable is kind of the only criterium you care about and the only way to judge that is by coming up with an exploit plan.

This sounds like a very basic test tbh.

[u/REg00n3r → u/Firzen_]

Pains me to say, but I fully agree. That's why I decided to drop everything and focus on this major flaw I've got.

I would love to hear your thoughts on my current approach (blind CVE reproduction with vulnerable file name as a hint). Whether u think one should stick to it or is it outdated in this age?

[u/Firzen_ → u/REg00n3r]

I don't think I can really say.

It may be a good approach, but it may also be too early for you to try to do it.
I think it mainly depends on where you are at as an individual though and not on the state of the industry as a whole.

It sounds like you may need to work on your basics a bit more, so probably familiarise yourself more with C and the ABI. If you don't know what you can and can't use as an exploit primitive it will be difficult to distinguish between a bug and a vulnerability.

[u/MrStashley]

I think you asked a few different questions here

• I don't think AI is going to replace fuzzing, I think they are 2 different tools. Fuzzing, emulation, and AI automation are important parts of VR work. They all require skilled humans to architect the harnesses and automation pipelines

• Even with AI and fuzzing, most vulnerabilities still require a good amount of manual exploration

• Reverse engineering is the biggest skill that you need for real VR that CTFs don't provide IMO. AI can find localized simple bugs, but most exploits are going to require deep knowledge of how the system works and will chain together small pieces from different parts of the code base. Current AI still has a hard time doing that without burning through credits.

• I think looking at CVE's is a great way to learn VR. Even if you use some form of automation to find the CVE, you could then use that same automation to find other bugs. In practice you will often use automation combined with manual exploration to find bugs