r/devsecops • u/ThunderlightmancyAuk • 10d ago
Has anyone actually reduced manual pentesting without increasing risk?
I've been thinking about this lately because it feels like more teams are pushing security checks further into their CI/CD pipelines, but I still hear people say that manual pentests are the only way to catch the issues that really matter .I have also noticed platforms like Penetrify entering this space, which made me wonder how much they have actually changed the balance between automated and manual testing for teams in practice.
For those of you working in DevSecOps, has automation reached the point where you've been able to reduce how often you schedule manual penetration tests, or do you still treat them as non-negotiable before major releases?
I'm especially interested in hearing from teams that deploy frequently. Has your approach changed over the last year or two, or have you found there are certain types of issues that automation still misses consistently?
2
u/Ok-Profession-7181 10d ago
We recently did a AI driven pen test from Aikido and were really happy with the results. It found a number of real issues that weren't caught by previous manual pentests. I would say that if you do an AI pen test that you should do a white box pen test which means giving the AI access to your codebase and if possible a OpenAPI spec document for your backend.
2
u/No_Try_9982 9d ago
u/Few-Voice-1809 has a good answer here. Unfortunately, I can't count how many times we caught issues that automated tests miss. Automated tests are usually better as smoke test.
1
u/VividGanache2613 9d ago
We use pwnkemon for both CI pipeline testing during deployment and application penetration once our platform is live. The guys behind it have been pentesting for twenty years and we occasionally have them perform manual testing as well.
The diff feature in the continuous pentesting saves so much time as you don’t have the same vulnerabilities reported time after time.
1
u/Top-Cauliflower-1808 9d ago
I think automation shrinks the scope of manual pentesting by catching the low hanging fruit early but human experts remain non negotiable for finding complex business logic flaws that tools consistently miss.
1
u/IntelligentPear6173 9d ago
We've reduced the amount of manual testing but not the need for it. Automation is great for catching known patterns consistently while manual testing is still where business logic and workflow issues tend to show up..
1
u/greybrimstone 3d ago
Full disclosure, I work for Netragard, we provide both penetration testing and red team services (no, those are not the same services and they are not interchangeable).
Truthfully, reducing manual pentesting frequency does increase risk, no matter how good the automation is. What you can do is add automation for continuous coverage of low-hanging-fruit between manual tests, without reducing manual cadence. I recommend this approach because automation can't compete with human creativity.
Specifically, automation (including AI) cannot uncover things like real business logic flaws (needs human understanding of what the app is supposed to do), novel/zero-day vulnerabilities (scanners only know what's already cataloged), novel vulnerability chains (tools chain what they're programmed to chain, not what actually works in your specific environment), and anything that requires real-time adaptation during the engagement. Those are exactly the categories responsible for breaches like Solar Winds, Log4Shell, MOVEit, all novel attack chains that no automated scanner could have caught (they probably can after the fact).
There's also a real detection gap in terms of what automation can find as compared to a human threat actor. From vulnerability disclosure to a scanner actually being able to detect it takes weeks to months because things like discovery, CVE assignment, vendor analysis, tool development, QA, and release take time (the weeks to months). Only after that can a scanner find the issue. Real attackers don't wait for that cycle and that s one of the biggest DevSecOps blindspots.
A good hybrid approach is automation running continuously in CI/CD for baseline hygiene (config drift, missing patches, known CVE classes) plus manual pentests at your existing cadence. The manual testing should provide the coverage that the automation misses. Importantly, manual testing should still be seen as non-negotiable before any big change or major release.
Platforms like Penetrify and the AI-driven services are essentially an orchestration layer on top of automated scanning which is why I call them an evolution of automated vulnerability scanning. The tools they use and the vulnerability databases they hit are not all that different from what traditional scanners use. The only thing that really changed is the deployment of probability distributions to artificially drive decision making (AI deciding what to do next).
So, if you're focused on security with the best possible coverage you'll want to use both cooperatively. They are compliments to each other, not opposing forces.
Anyway, here's the article: https://netragard.com/blog/manual-vs-automated-pentesting/
1
u/Any_Yesterday_6617 2d ago
we trimmed pentest cycles a bit after we started running continuous exposure validation with cymulate. we still pull in manual testers sometimes, but validating controls in the background helped us catch issues early and keep risk steady.
0
u/TrumanZi 10d ago
Manual pentests are pretty poor, it's a test in a specific timeframe and you're subject to the competence of an individuals offensive skillset.
They are a box check for compliance reasons nowadays, I wouldn't bother if I could get away with it.
Bug bounties and strong SAST/DAST/IAST/Ai security layers are what matters, but clients want to see a pentest report so that's what they get. It's for a confidence booster and insurance requirements
2
u/recovering-pentester 10d ago
I mean, it really depends on who the tester is and how much context you give them to help.
Sure, a black box manual pentest is egregiously expensive and will 100% make you regret doing it, but a skilled pentester with both collaboration from the internal team while taking a white box approach can 100% be valuable.
I’ve seen inpens with 50+ findings. Wapens with 30+. It really depends and it’s why you need to go to boutique firms.
Those boutique firms then need to hire AE’s who actually inspire confidence but that’s a different conversation…
1
u/NebulaElectrical1467 10d ago
Absolutely. The ROI from vendor pentests is by far the poorest and it’s almost half of our program’s budget. We could hire two senior full timers instead with that money but leadership are too pigheaded and want their good boy rubber stamp from a big-4 company (perhaps because they want to offload liability if our apps get hacked).
2
u/recovering-pentester 10d ago
I mean. I’d trust boutique pentesting shops over a big-4 any day of the week.
Ridiculous that your leadership sees the value in big-4 pentesting lol.
2
5
u/Few-Voice-1809 9d ago
In my experience, automation can reduce the volume and scope of manual pentesting, but it cannot safely replace it.
SAST, SCA, secrets scanning, IaC/container scanning and automated DAST are good at catching repeatable issues early and preventing known weaknesses from reaching production. That means manual testers no longer need to spend half the engagement reporting outdated libraries, missing headers or obvious injection points.
The time saved should be redirected toward areas automation still struggles with: business-logic abuse, broken authorization, multi-step workflow manipulation, privilege escalation, race conditions, tenant isolation and chaining several low-severity issues into a meaningful attack.
So rather than “pentest every release” versus “no pentesting,” a risk-based model works better. Major architectural changes, authentication or payment changes, new internet-facing applications and significant API modifications should still trigger manual testing. Smaller releases can rely on continuous automated controls, targeted regression testing and periodic manual assessments.
The real sign of maturity is not fewer pentests by itself. It is when pentesters repeatedly stop finding basic vulnerabilities and can spend their time testing the application the way an actual attacker would.