The Assurance Gap: Five Questions to Ask Before Relying on an Assurance Report

A new HITRUST paper examines how third-party risk management teams can determine whether an assurance artifact provides the evidence they need for a risk decision. 

Organizations rely on assurance reports, certifications, , and questionnaires to evaluate vendors and  understand risk. but they do not all provide the same type or level of assurance. 

The assurance gap appears when an artifact does not match the decision that needs to be made. An artifact  may carry a familiar label but cover the wrong service, omit material controls, exclude key service providers, or require extensive interpretation before a team can compare it with other artifacts. 

To examine how this gap can appear in practice, HITRUST analyzed 103 SOC 2 Type 2 reports from 37 audit firms. SOC 2 reports were selected as they are a commonly accepted artifact by TPRM teams, but can be highly variable in scope and quality. A carefully reviewed SOC 2 report can provide value but only when the reader has the expertise and time to review its content. The analysis shows why recipients must understand what each report supports, what it leaves open, and where they may need supplemental evidence.  

The paper organizes that review around five questions.

1. Is the scope of the assessment appropriate? 

Before relying on an artifact, report recipients must confirm that its scope matches the relationship under review, such as the correct legal entity, product, system, environment, data, geography, review period, and dependencies. A report may cover a parent company but not the product your organization uses. It may exclude key locations, environments, or service providers. It may also cover a period that no longer reflects the vendor’s current operations. 

SOC 2 allows management to define the system, establish the system boundaries, select the applicable Trust Services Criteria, and describe the controls. That flexibility allows organizations to tailor their reports, but it also means that two reports with the same label may cover very different services and risks.

2. Does the Report Address the Expected Risks and Threats?

An assurance report can support a risk decision only when it includes and validates the controls relevant to the report recipient.

SOC 2 uses five Trust Services Criteria categories: Security, Availability, Processing Integrity, Confidentiality, and Privacy. Organizations must address the common criteria within the Security category, but they can choose whether to include the other four categories.

Only 4% of the reports in the paper’s sample included all five categories. Even when an organization selected all five, the criteria mapped to 67% of the attack methods within MITRE ATT&CK. Organizations can add controls that increase coverage, but SOC 2 does not require them to do so.

The analysis also examined whether the sampled reports specified controls for several common threat actions:

• 77% of the sampled SOC 2 reports included controls requiring MFA for remote access.

• 30% of the sampled SOC 2 reports included controls requiring MFA for privileged access.

• 7% of the sampled SOC 2 reports included dedicated phishing training or simulations, while 5% included a prescriptive email-filtering control.

• 0% of the sampled SOC 2 reports included an organizational control to maintain offline or immutable backups.

These percentages reflect which controls the reports specified. They do not establish whether organizations implemented controls that their reports omitted. However, when a report omits a relevant control, the recipient cannot use that report to confirm it. The recipient must then request supplemental evidence or use another assurance mechanism that includes and tests the control.

3. Who Prepared and Quality-Reviewed the Work? 

Report recipients should consider the practitioner’s qualifications, independence, experience, licensure, testing approach, peer-review status, and quality record. They should also understand how the assurance program identifies and addresses quality concerns. 

Individual CPA firms produce and issue SOC 2 reports. The AICPA requires those firms to implement Quality Management Standards, but it typically does not review the underlying assessment or the report before issuance. 

The AICPA primarily enforces those standards through peer review. Another CPA firm performs that review after the issuing firm releases its reports, generally once every three years. The issuing firm selects its peer reviewer, and the reviewer examines only a sample of completed assessments. 

This quality model does not make SOC 2 reports inherently unreliable. It does mean that a recipient should evaluate the audit firm, especially when the organization plans to place significant reliance on the report.

4. Which Service Providers does the Report Include, and Does it Assess Them? 

Many vendors rely on cloud platforms and other service providers to deliver critical functions. An assurance report may not include the controls those providers perform. 

SOC 2 offers two methods for addressing subservice organizations. Under the inclusive method, the report includes the relevant controls of the subservice organization. Under the carve-out method, the report excludes those controls and describes how the primary service organization monitors the provider. 

In the paper’s sample, 80.4% of organizations used at least one subservice organization. Every corresponding report used the carve-out method. As a result, those reports did not cover or test the key service providers as part of the examination. 

This finding does not show that the vendors failed to manage their service providers. It shows that report recipients may need to obtain and review separate assurance reports before they can evaluate the outsourced services.

5. Can the Team Compare Two Reports through Measurable Outcomes? 

Third-party risk management teams often need to compare results across many vendors. That comparison becomes difficult when reports use different scopes, control sets, testing approaches, and presentation methods. 

SOC 2 does not use a standardized numeric cybersecurity score across reports. A team can compare two SOC 2 reports, but it must review the details and apply considerable judgment. 

All 103 reports in the paper’s sample received a clean, or unqualified, opinion. However, 40% contained at least one exception that the recipient needed to review. Among the reports with exceptions, each contained an average of 3.6 exceptions. 

A clean opinion does not mean that every test produced no exceptions. It also does not mean that two vendors with clean opinions demonstrate the same security maturity. Recipients must review the tests, results, exceptions, management responses, and available remediation evidence before drawing a conclusion.

Start with the Decision. 

The right assurance mechanism depends on the services and data in scope, the expected threats, the level of independence required, and the amount of reliance the organization intends to place on the result.

A lower-risk vendor may warrant a lighter review. A critical processor of regulated data may require precise scope, explicit controls, independent validation, and ongoing monitoring. In some cases, a reusable report or certification may provide the necessary evidence. In others, the recipient may need targeted supplemental information.

Third-party risk management teams should start by defining the risk decision and the evidence needed to support it. They can then determine whether the available artifact provides sufficient evidence or leaves an assurance gap.

A familiar label may start the review. The evidence should determine the decision.

Read The Assurance Gap for the complete analysis.