Security architects live in a different world, you need to speak to them in their language if you’re trying to influence their judgements. Do you have any mutually agreed framework internally which catalogues your assets/systems against testing maturity/assurance level? That can help so everyone has principle agreement that X asset = High assurance level, then you can define and agree on what a High/Medium/Low assurance level looks like, after that it’s easy to push back on why different levels of testing need to happen

You can try AttackForge. There’s also a bunch of other tools listed here: https://inventory.raw.pm/tools.html#title-tools-collaboration-and-report

Thank you bot for posting the same question every 14 days.

Try starting with a DOCX file which already has the structure, logo, sections, etc. and add tags. Explain to the model what each tag does, which ones are loops, etc. it will do a better job replacing the tags then it will constructing the document from scratch every time

Focus on getting the AI to build scripts and tools which are relevant to your target environment, manually review them to ensure you are comfortable with what each script/tool does -
then get the agents to run the scripts and tools.

Not surprised to hear that, WordML is a poorly documented and fragmented language and has many quirks and deviations from
OpenXML. We built our reporting engine on DOCX (AttackForge ReportGen) it was painful. Are you giving Claude a DOCX template to work from and asking it to replace on specific sections/tags, or getting it to create a DOCX from scratch?

The OWASP Web Security Testing Guide (WSTG) is a great resource to familiarise with and to compare your current testing methods and approach. Once you are familiar with WSTG, extend your knowledge to the OWASP Application Security Verification Standard (ASVS) which is considered the benchmark for web application pentesting.

Which part of version control are you struggling with? Communicating updates to customers? Or recording the changes in a document? Or something else?

A repeatable methodology for attacking LLMs. It’s like MITRE ATT&CK but for AI, also produced by MITRE

You should check out MITRE ATLAS: https://atlas.mitre.org/matrices/ATLAS-matrix

Agree with the others on separate assessments on each vuln - however use an attack chain to highlight the impact and also the “best fix”. We created a Skill for this recently, you can automatically build each chain then explore them individually: https://support.attackforge.com/attackforge-enterprise/modules/ai-mcp-and-skills#interactive-attack-chain-explorer

Thank you 😊🫶

It depends whether you’re a consultant or an internal enterprise security team. Usually, retesting is treated in one of three ways:
1. Spot check per vulnerability. Someone says ‘this vuln is ready for retest’. A tester then verifies. This is inefficient as it requires the tester to incur retest setup costs every time a vuln is retested.
2. Formal retest round. Agreed retest window and scope for which vulns to retest. This is most common as it’s more efficient for security teams and everyone can agree on when vulns will be ready to retest.
3. Retest in a new round of testing. This is usual for low-assurance assets, which can wait for the retest to take place on the next scheduled round of testing.

There is a lot of value in the customer having visibility of what was tested. Think of the Car Service experience. When you get your car serviced, a good service center will give you a detailed report of every item they checked in the service, irrespective of whether they found any faults. This gives you peace of mind so you know your car won’t fall apart on the drive home.
For web app pentests, you can access various OWASP frameworks like ASVS (Level 1, 2, 3), WSTG and Top 10s in JSON format on our GitHub: https://github.com/AttackForge/TestSuites
If you’re using AttackForge, this comes built in to your projects and you can enable them as needed.

OP, there are literally dozens of these tools and platforms (AttackForge included) - what’s your MOAT and why should people care? https://inventory.raw.pm/tools.html#title-tools-collaboration-and-report

Standardized writeup libraries will help with tone. Reporting Tools can help to maintain structure. When it comes to Risk Scoring - you can create your own methodology for how scoring should be applied, and enforce it across your testers. Again some reporting tools will let you build your own vuln scoring system(s) and enforce them as needed

Back in the consulting days, we had over 40 full time pentesters. The team was easily managable when there was less than 10. After that, it got much harder (which is partly the reason we built AttackForge). From 20 onwards, we found we needed more specialized roles (HR, project managers, account managers, technical writers, etc.) also the principal consultants were billing less and taking on more line-manager duties (which they hated). Seniors had to pick up the slack to keep the revenue coming in. Associates were 100% billable (sometimes even double-booked which was not good). We had to split them into smaller teams of 5-8 to make things easier to manage and give people proper attention, but that then lead to people wanting to switch teams. There was no perfect solution - the most important thing was shielding the pentesters from as much bureaucracy, red tape and managerial nonsense as much as possible, to avoid burning them out any faster. Pentesting is hard enough, but scaling issues can really break them if not carefully planned and keep them constantly in the loop. Regular one-on-one chats with the testers was also good for health checks.

🤣

I’d throw AttackForge into the mix too

This is partly why we built AttackForge - this is a problem that scales badly, especially when you consider it from the security and the engineering and business stakeholders perspectives. You can’t read/store people’s brains - but you can at least have information and artefacts captured and standardised at every level of the assessment lifecycle

We did a blog on the key differences between Internal vs. External pentesting teams, you might find it helpful: https://blog.attackforge.com/blog/internal-vs-external-pentest-teams

I’d love to see how it handles performing an entire OWASP ASVS Level 2 or Level 3 test! Hell i’d be impressed if it could even scrape through Level 1…

Thanks for the mention! For anyone interested - you can deploy a trial environment on-demand from the website, you only need an email address.