Third-party risk management (TPRM) teams rely on assurance reports to help evaluate vendors, suppliers, and business partners. But having an assurance report is not the same as having the information needed to make a risk decision.
That distinction was at the center of The Assurance Gap: What 103 SOC 2 Reports Actually Showed Us, a recent HITRUST webinar featuring AJ Yawn, Author and GRC Engineering Lead at Rippling, and HITRUST's Ryan Patrick. The conversation explored findings from an analysis of more than 100 SOC 2 reports and asked a fundamental question: Are the assurance reports organizations rely on actually providing visibility into the risks that matter most?
That analysis is now detailed in The Assurance Gap, a new HITRUST paper.
The purpose of the analysis is not to suggest that SOC 2 has no value. A SOC 2 Type 2 report can provide detailed information about a service organization, its controls, testing, and identified exceptions. But the flexibility inherent in SOC 2 also means that scope, control selection, testing, and other elements can vary from report to report.
For organizations relying on those reports to make third-party risk decisions, understanding what the report actually provides is critical.
Ransomware recovery requires more than a report label
One of the findings Ryan highlighted during the webinar was particularly stark: 0% of the sampled SOC 2 reports included an organizational control requiring offline or immutable backups.
Offline or immutable backups can play an important role in recovering from ransomware by helping organizations restore data without relying on backups that may also have been compromised.
The issue for a report recipient is visibility. The absence of an offline or immutable backup control from a SOC 2 report does not mean the organization does not have those backups. It means the report does not provide evidence that allows the recipient to confirm the control.
That distinction becomes especially important when an organization is using the report to evaluate a third party. If ransomware affects a critical vendor, the TPRM team needs to understand whether that vendor can respond and recover. If the assurance artifact does not address a control that is material to that decision, additional evidence may be necessary.
During the webinar, Ryan and AJ also connected several of the findings as parts of the same potential attack chain. Phishing may provide an initial entry point. Multi-factor authentication can help prevent compromised credentials from being used. If ransomware is ultimately deployed, recovery capabilities such as offline or immutable backups become critical.
Yet the analysis found gaps in the visibility provided by the sampled reports across each of those areas.
Phishing remains persistent, but assurance coverage was limited
Of the 103 SOC 2 reports analyzed, only 7% included a control for dedicated phishing training or simulations, while only 5% included a prescriptive control requiring an email filtering solution to block suspicious emails and unnecessary file types before they reached employee inboxes.
Only 1% included controls addressing both requirements.
This does not mean the organizations represented in the sample were not conducting general security awareness training. In fact, the analysis found that most included general security awareness training. What was far less common was explicit coverage of dedicated phishing training, simulations, and preventive email filtering controls.
AJ focused on that distinction during the webinar. Phishing is not a new or obscure cyber risk. It is a well-understood threat, which makes the limited visibility in the sampled reports especially important for organizations relying on those reports to make third-party risk decisions.
As AJ put it during the discussion, if organizations are trusting third-party reports to make decisions, those reports should at a minimum be relevant to the existing threat landscape.
That is the broader issue behind the findings. Cyber threats continue to evolve. Assurance must provide enough relevant evidence for recipients to understand whether controls addressing those threats are actually included and validated.
What does the report actually allow you to conclude?
The ransomware and phishing findings point to a larger question: What risk decision is the assurance report actually supporting?
Organizations often need assurance artifacts because evaluating every control at every vendor independently is not scalable. A reusable assurance report can reduce significant duplication for both vendors and TPRM teams.
But reuse only works when the artifact is fit for the decision.
The burden is placed on the person consuming the report. That person must determine what was tested, whether it is relevant, and whether the report provides the information needed to make a risk decision. For teams responsible for hundreds or thousands of third parties, performing that level of interpretation across every report can quickly become difficult to scale.
This is where the assurance gap emerges. A familiar report label can create confidence, but the underlying evidence may not necessarily address the service, threat, dependency, or risk decision that matters to the recipient.
The Assurance Gap paper outlines five questions organizations can use when evaluating any assurance artifact:
- Does the scope cover the provided service?
- Is the assurance responsive to the expected risks and threats?
- Who prepared and quality reviewed the work?
- What service providers are included, and are they assessed?
- Can two reports be measurably compared?
These questions shift the focus from whether a vendor simply has an assurance report to whether the evidence in that report is sufficient for the decision being made.
Move from assurance labels to evidence that supports the decision
SOC 2, HITRUST, ISO 27001, PCI DSS, NIST CSF, questionnaires, and other assurance mechanisms were designed for different purposes. Treating them as interchangeable can create gaps between what an organization believes has been validated and what the underlying evidence actually supports.
The goal is not to collect more assurance artifacts. It is to understand what each artifact proves, what it leaves open, and whether additional evidence is necessary.
The Assurance Gap expands on the findings discussed by AJ and Ryan, including threat-landscape coverage, quality, supply chain coverage, measurable outcomes, and the questions TPRM teams can use to evaluate the evidence they rely on.
The conversation continues
Register to attend SOC 2 vs. HITRUST: A Practitioner Debate on What Third-Party Risk Actually Needs on October 8 at 10:00 AM CT to hear Nick Norton, AJ Yawn, and Ryan Patrick explore what SOC 2 and HITRUST each provide for third-party risk management, where the gaps remain, and what TPRM teams need from assurance today.
