Penetration Testing Compliance: What BHP Suppliers Actually Need to Know

What does penetration testing compliance actually require, beyond just booking a test and ticking a box? At its core, penetration testing compliance refers to authorized cyberattacks on systems to meet security mandates.

In practice, that means engaging a qualified provider to manually test your systems for exploitable weaknesses, then receiving a clear, evidence-based report that shows what was tested, what was found, and whether critical issues were retested after remediation – not just running an automated vulnerability scan and calling it a penetration test.

For small to medium-sized businesses, especially Australian organisations and BHP suppliers facing supplier cybersecurity reviews, this is one of the most commonly misunderstood compliance requirements.

A cheap scan and a genuine, manually validated test may both be labelled “penetration testing”, but they satisfy very different levels of assurance, and getting it wrong can mean a failed compliance check while real security gaps remain.

This guide breaks down what penetration testing compliance actually means, how often testing is typically expected, why CREST accreditation matters, what a genuinely compliant test includes, the mistakes that undermine it, and how penetration testing relates to BHP supplier requirements.

Key Takeaways

• Penetration testing compliance depends on the quality of the test, not just whether one happened. An automated vulnerability scan and a genuine, manually validated penetration test are frequently confused, and only the latter satisfies most real compliance expectations.
• No major framework mandates a single universal testing frequency, but annual testing is the consistent baseline recommendation across the ASD Essential Eight and APRA’s CPS 234 standard, with additional testing expected after significant system changes.
• CREST accreditation, through CREST ANZ in Australia, is the clearest independent signal that a provider’s testers and methodology meet a recognised professional standard, worth checking before engaging any provider for compliance-driven testing.
• A genuinely compliant penetration test includes manual validation of findings, a clear report mapped to the relevant framework, and typically a retest to confirm issues were actually fixed, not just identified.
• The most common way penetration testing compliance gets undermined is treating it as a one-off, last-minute purchase rather than a planned, recurring part of an ongoing security program.

Quick Answer

Penetration testing compliance means engaging a qualified provider to manually test your systems because penetration testing is important to identify vulnerabilities that automated scanning alone may miss, and being able to produce a proper report showing what was tested, what was found, and that critical issues were fixed and retested.

A vulnerability assessment can support that process as an automated scan, but it does not replace a manual penetration test. Testers then attempt to exploit vulnerabilities to show whether an attacker could gain access and create real business impact.

Annual testing is the widely accepted baseline frequency across major Australian frameworks, with additional testing expected after significant system changes. Look for CREST-accredited providers or CREST-certified individual testers as a reliable independent signal of genuine capability, since penetration testing compliance is judged on the quality and rigour of the test, not simply whether one took place.

What Penetration Testing Compliance Requirements Actually Mean

The single biggest source of confusion in penetration testing compliance is the difference between a vulnerability scan and a genuine penetration test, as they are distinct security assessment methods used in compliance testing and often get marketed almost interchangeably.

A vulnerability scan uses automated tooling to check systems against a database of known weaknesses and produces a list of findings. It’s useful, relatively inexpensive, and fast, and those automated steps support vulnerability analysis by surfacing known security vulnerabilities, but they don’t demonstrate whether those weaknesses could actually be chained together into a real attack.

A genuine penetration test uses skilled human testers to manually validate findings and attempt to exploit them the way a real attacker would, documenting the actual impact rather than just a theoretical list of issues.

For penetration testing compliance purposes, this distinction matters enormously. A supplier who has only ever run automated scans, even regularly, may not actually satisfy a requirement asking for genuine penetration testing evidence, since the two are different levels of assurance entirely. In practice, penetration testing requirements under common compliance frameworks focus on evidence of real testing rather than scan output alone.

Why Pen Testing Matters for Cyber Security and Data Security

Pen testing matters because it’s the only way to know, with real evidence, whether your cyber security controls would actually hold up against a determined attacker rather than just looking adequate on paper.

For businesses handling customer records, financial data, or supplier credentials, this translates directly into data security. A successful penetration test confirms that access controls, encryption, and monitoring are doing what they’re meant to do, rather than assuming they are because a vendor said so.

That gives the business a realistic picture of its overall security posture rather than a narrow snapshot of a single system.

How Often Does Penetration Testing Compliance Require Testing?

There’s no single universal mandate here, frequency expectations vary by framework and by an organisation’s specific risk profile, but a consistent pattern shows up across the frameworks most relevant to Australian businesses.

Some compliance frameworks set explicit baselines, while others expect testing frequency to follow documented risk assessments and business objectives.

Annual Testing Is the Widely Accepted Baseline

The ASD Essential Eight doesn’t mandate a specific testing cadence outright, but it advises aligning testing frequency with risk level, and most organisations working toward Essential Eight alignment adopt annual penetration testing as the practical baseline.

APRA’s CPS 234 standard mandates penetration testing for financial institutions since July 2019, and — while it’s specifically aimed at regulated financial entities — it’s frequently referenced as a broader Australian benchmark, explicitly expecting a minimum of annual testing for most organisations.

Additional Testing Is Expected After Significant Changes

Beyond the annual baseline, testing is generally expected after any significant system change — a major software upgrade, a new application deployment, or a change to third-party arrangements that could introduce new exposure.

Additional testing in these cases helps ensure compliance and confirms that new security measures and access controls work as intended.

Penetration testing compliance isn’t a once-a-year checkbox so much as an ongoing practice that responds to how the environment actually changes, commonly including changes affecting remote access or third-party connectivity.

Higher-Risk Environments, Such as PCI DSS, Often Need More Frequent Testing

Organisations with higher risk profiles — more sensitive data, greater system integration, or elevated third-party access — are commonly expected to test more frequently than the annual baseline, sometimes semi-annually or quarterly depending on the specific risk.

Higher-risk environments often schedule regular penetration testing as one of their proactive measures against evolving cyber threats.

For PCI DSS specifically, this typically means testing needs to cover the entire cardholder data environment, not just the systems that process transactions directly, and organisations operating critical infrastructure face similar expectations for broader, more frequent coverage.

This is worth confirming directly for any contract carrying elevated access requirements rather than assuming the standard annual cadence applies, as more frequent testing may also be needed where government systems or other sensitive environments impose stricter expectations.

Why CREST Accreditation Matters for Penetration Testing Compliance

CREST is an international not-for-profit accreditation body for penetration testing, and in Australia and New Zealand it operates through CREST ANZ, which certifies both individual testers and the organisations that employ them against defined technical, professional, and trust standards.

For penetration testing compliance specifically, CREST accreditation is one of the clearest independent signals that a provider’s methodology and staff competency meet a recognised professional bar, rather than relying on a provider’s own marketing claims.

Individual testers hold specific CREST certifications, such as CRT or CCT.

Organisations pursuing CREST accreditation undergo assessment of their governance, staff competence, and the processes implemented around quality assurance and reporting, with ongoing periodic review to maintain that status rather than a one-off certification.

It’s a reasonable, concrete question to ask any prospective provider directly: are your individual testers CREST-certified, and is your organisation CREST-accredited, or working toward it.

A provider unable to answer clearly is worth scrutinising further. Providers with offensive security credentials are generally better positioned when an external specialist performs pen tests that need to stand up to compliance scrutiny.

What a Genuinely Compliant Penetration Test Should Include

A well-structured penetration testing program includes scope definition, authorization, and remediation tracking — the pieces that turn a one-off technical exercise into something a compliance reviewer can actually rely on.

Manual Validation and Vulnerability Assessment, Not Just Automated Output

Automated scanner results should be treated as a starting point, not the finished product: automated tools can help identify vulnerabilities, but they are only one step in pen testing.

Skilled testers need to manually validate and attempt to exploit findings, confirming whether weaknesses in security systems can actually be abused, rather than simply reporting raw scan output as the deliverable.

This matters because automated tools frequently flag issues that turn out to be false positives, and just as often miss the kind of chained, business-logic vulnerabilities that only a human tester probing the environment will find.

This is the single biggest quality gap in the market. A report built entirely from automated output can look thorough at a glance, dozens of pages, colour-coded severity ratings, plenty of technical detail, without ever demonstrating that a real person actually tried to break into the environment the way an attacker would.

For penetration testing compliance purposes, that distinction is exactly what a reviewer is trying to verify, because manual testing is what uncovers security gaps that scan-only reporting may miss.

A Clear, Evidence-Based Report

The report should explain what was tested, what was found, and the actual risk of each finding in plain business terms.

Ideally, it should also map findings against the relevant framework your business needs to demonstrate compliance with — whether that’s the Essential Eight, ISO 27001, SOC 2, PCI DSS, or the General Data Protection Regulation where relevant.

Guidance from the National Institute of Standards and Technology, including NIST SP 800-53 and SP 800-115, can also help frame security assessments and penetration testing.

A clear report also documents which technical safeguards were tested — firewalls, access controls, encryption — and how each held up under real attack conditions.

ISO 27001 emphasizes managing vulnerabilities through penetration testing and requires organizations to manage vulnerabilities and conduct testing, while SOC 2 does not explicitly require penetration testing but commonly expects evidence of appropriate controls.

Reporting obligations under the Australian Privacy Act since 2018 make clear, evidence-based reporting especially important after incidents or discovered exposures involving data breaches.

A genuinely useful report reads as evidence a non-technical reviewer, a procurement officer, an auditor, a business owner, can actually understand and act on, not a raw technical dump that only makes sense to another security specialist. If your current or prospective provider can’t produce a sample report that meets that bar, it’s worth asking to see one before committing.

A Remediation Plan, Not Just a List of Problems

A genuinely useful penetration testing compliance engagement includes practical guidance on fixing what was found, not just identifying it, and a remediation plan should help achieve compliance by prioritising fixes to critical vulnerabilities, not just listing issues.

Finding a vulnerability and explaining how to fix it are two different skill sets, and a provider who stops at the first without the second is leaving the harder, more valuable half of the job undone.

This matters especially for smaller businesses without an internal security team to interpret findings independently.

ISO 27001 emphasises managing vulnerabilities through an information security management system, using testing to support risk management and guide remediation in line with security controls and day-to-day security practices.

A remediation plan that prioritises issues by actual risk, rather than presenting an undifferentiated list, is what turns a report into something a business can genuinely act on, with practical guidance that ties recommended actions back to those same measures.

A Retest to Confirm Fixes Actually Worked and Reduce the Risk of Data Breaches

Many providers include a complimentary retest within a defined window, commonly around 90 days, to verify that critical findings and high-risk issues were genuinely resolved, not just marked as addressed without verification, because retesting is often necessary to ensure data security and confirm that fixes truly closed the original weakness.

Without this step, there’s a real gap between “we identified the problem” and “we confirmed it’s actually fixed,” and that gap is exactly where risk quietly persists — and where unresolved findings can eventually turn into actual data breaches — even after a business believes it has dealt with its findings.

A retest also protects against remediation that looks correct on paper but wasn’t implemented properly in practice, which happens more often than most businesses expect, particularly when fixes are rushed under time pressure, while also providing valuable insights into whether remediation was implemented effectively.

Common Mistakes That Undermine Penetration Testing Compliance

Assuming a Cheap Quote Covers the Same Scope as an Expensive One

Given the wide price range in this market, from a few hundred dollars to tens of thousands, a significantly cheaper quote is often a narrower scope or an automated scan dressed up as a penetration test, not the same service at a discount. A reputable provider will usually offer a free consultation to understand your environment and define scope before pricing.

Two quotes covering wildly different depths of testing can look identical on paper if neither one clearly states its methodology, and the scope should also say whether the provider is delivering network penetration testing, application-focused testing, or only a vulnerability assessment.

The safest approach is asking directly what the engagement actually includes, manual testing hours, tester certifications, report format, and retest inclusion, rather than comparing dollar figures alone.

For compliance evidence, independent penetration testing by a third party is generally more credible than an internal-only check.

Leaving It Until Close to a Deadline

Genuine penetration testing providers book out weeks in advance, and last-minute engagements often mean either a rushed, narrower test or an expedited scheduling premium on top of the standard cost.

A business that starts looking for a provider two weeks before a compliance deadline has already limited its own options considerably. That is because the penetration testing process takes time: scoping, scheduling, execution, reporting, and any retest all need to be planned before the deadline.

This is one of the most avoidable mistakes on this list, since penetration testing compliance timelines are almost always known well in advance, a renewal date, a contract milestone, an annual review cycle, yet testing is routinely left until the pressure is already on.

Treating It as a One-Off Purchase Rather Than an Ongoing Practice

Penetration testing compliance is judged on a pattern of regular testing aligned to actual risk and system changes, not a single historical test from several years ago.

A business that tested once, two or three years ago, and hasn’t tested since is in a materially weaker position than one testing annually — even if the original test was thorough at the time.

Effective programs also conduct risk assessments periodically, so retesting is driven by current exposure rather than habit alone.

Systems change, new features get deployed, and new vulnerabilities get discovered constantly, which is exactly why a stale test loses its value as evidence of current security posture, regardless of how well it was conducted originally.

In practice, continuous monitoring can complement regular testing by highlighting new exposures that may justify earlier retesting.

Not Confirming What Happens After the Report Is Delivered

A report full of findings that never get remediated or retested doesn’t actually demonstrate compliance — it demonstrates that problems were identified and then left unresolved. That can be worse than not testing at all, if it later becomes clear the business knew about a vulnerability and didn’t act on it.

Unresolved findings can also leave data security exposed and increase the chance of later data breaches.

Before engaging any provider, it’s worth confirming what happens after the report is delivered.

Ask whether they explain what penetration testing activities were performed, and whether that’s paired with remediation guidance for closing security vulnerabilities, a retest window, or ongoing advisory support — rather than assuming the relationship ends the moment the report is delivered.

How Compliance Penetration Testing Helps Ensure Compliance with BHP Supplier Requirements

Penetration testing shows up repeatedly across BHP supplier compliance requirements, and BHP suppliers may need compliance penetration testing evidence aligned to customer or industry expectations, not just a generic scan.

It’s one of the most commonly missing pieces of evidence in common BHP supplier compliance failures, and it’s consistently the single largest cost variable covered in our guide to the hidden costs of cybersecurity compliance for BHP suppliers.

For a broader look at what BHP’s baseline requirement actually asks for, see our guide to BHP’s minimum requirements for suppliers. Our free BHP Supplier Cyber Security Self-Assessment takes about five minutes and includes a direct question on penetration testing history, a useful starting point before engaging a provider.

Before procurement review, penetration testing services should be scoped around the supplier’s environment and relevant compliance requirements, and, more broadly, around the regulatory compliance obligations that apply to the industry the supplier operates in.

Frequently Asked Questions