California’s AI Cyber Defense Program: From Strategy to Assurance

On August 10^th California announced its pioneering AI Cyber Defense Program designed to strengthen the protection of state systems and critical infrastructure against emerging artificial intelligence-enabled cyber threats. The directive, which builds on the September 2023 and March 2026 AI executive orders, directs state agencies to:

• Establish an AI Cyber Defense Program within the California Cybersecurity Integration Center

• Expand the use of AI for vulnerability detection, network hardening, and incident response

• Increase support for local governments and critical infrastructure operators

• Designate an AI Cybersecurity Officer in every state agency

The program builds on California's broader AI strategy and reflects growing concern that increasingly capable AI systems are transforming both how adversaries conduct attacks and how organizations defend against those cyber threats.

A Good First Step, But More is Needed

This program represents one of the most significant public-sector acknowledgments to date that AI is changing cybersecurity on both sides of the battlefield. The initiative's focus on leveraging AI for cyber resilience reflects that AI can increasingly become a force for defenders, and not just attackers.

California recognizes that AI is making attacks faster, cheaper, and more sophisticated, and that defenders must leverage AI to improve the effectiveness of traditional cybersecurity activities. However, the AI era demands more than simply applying AI to existing security processes. As AI adoption accelerates there are additional challenges beyond the scope of California’s program which organizations will need to address:

• AI agents introduce a new class of autonomous actors capable of accessing data, executing workflows, making decisions, and taking actions on behalf of users. Identity governance must evolve from managing people and systems to managing autonomous AI actors.

• New risks within AI-enabled systems, such as prompt injection, model manipulation, and data poisoning introduce security concerns that many traditional cybersecurity programs were not designed to address. Future security programs must prevent and observe AI behaviors, prompts, and agent activities, not just networks and endpoints.

• Organizations increasingly depend on third-party AI providers, creating supply chain risks that can be difficult to assess and manage. In fact, according to the 2026 Verizon Data Breach Investigations Report (DBIR)i, 48% of breaches involved a third-party. Without a structured assurance framework, it can be challenging for customers, regulators, and business partners to determine whether AI systems can be trusted. 

Despite these challenges, AI presents transformative opportunities for defenders. Organizations can use AI to identify vulnerabilities faster, improve threat detection capabilities, and enhance decision-making during cyber incidents. It provides an opportunity to modernize risk management programs and establish new levels of transparency and confidence in AI-enabled systems.

Companies will be most successful when they view AI systems beyond just their defensive technology and begin treating them as critical assets which require their own security, governance, and assurance. By combining AI-enabled cyber defense with comprehensive security assurance, organizations can build the trust necessary to accelerate AI adoption while maintaining confidence across the broader digital ecosystem.

The AI Impact on Information Security

Recent advances in frontier AI models have demonstrated an unprecedented ability to accelerate portions of the cyber attack lifecycle. In February 2026 the AI Security Institute (AISI) estimated that the length of cyber tasks AI models could complete was doubling every 4.7 months which was an acceleration of their November 2025 analysis which stated this process was doubling every 8 months. Then in May 2026 AISI reported that two new models, Claude Mythos Preview and GPT-5.5, substantially exceeded both doubling rate trendsⁱⁱ.

Activities that previously required substantial proficiency can increasingly be performed quickly, at greater scale, and with less expertise. This requires defenders to respond-in-kind by leveraging those same technologies to identify vulnerabilities, automate investigations, and improve response times.

The potential cybersecurity impact of AI is not just theoretical but occurring today in real-world attacks. Weeks before California’s announcement, a suspected Iran-linked intrusion hit more than thirty municipal water utilities, including several in Minnesota, and separate disclosures showed frontier AI models pulling off multi-step cyberattacks on their own in controlled tests.

At the same time, the federal safety net states have relied on is thinning: The Cybersecurity and Infrastructure Agency (CISA) proposed budget is shrinking by nearly a third, The Multi-State Information Sharing and Analysis Center (MS-ISAC) free federal support dried up in late 2025, and the grant program that funded state and local cyber defense is running dry. California is stepping in to close that gap itself, building on Cal-Secure 2.0, the updated statewide cybersecurity roadmap it released just weeks earlier.

These developments point to a fundamental shift in the cybersecurity landscape. AI is rapidly lowering the barriers to sophisticated cyber operations, allowing attackers to execute increasingly complex activities with greater speed, scale, and efficiency. At the same time, public-sector organizations face growing resource constraints and shrinking access to traditional sources of cybersecurity support. The result is a widening gap between the capabilities of adversaries and the capacity of defenders relying solely on traditional security approaches.

Evolving Traditional Information Security

As AI evolves, organizations will increasingly need to focus their security efforts across four distinct but complementary domains:

• Traditional information security governance

• Traditional information security

• AI governance

• AI information security 

California’s program is focused on enhancing traditional cybersecurity mechanisms to protect against AI-driven threats. This is one of the strongest aspects of California’s program since it focuses on operational cyber defense rather than treating AI solely as a governance challenge. Using AI to accelerate vulnerability identification, improve threat detection, and triage and prioritize security events has the potential to help defenders keep pace with an increasingly fast-moving threat landscape.

Leveraging the use of AI in defense is an effective strategy, but it is just one of several updates necessary to address today’s AI-driven threats. The emergence of AI has introduced threats across several domains which require corresponding enhancements. When reviewed holistically, AI requires companies to implement additional controls across multiple traditional information security areas. For example:

• Historically, organizations could operate on weekly scans or monthly patch cycles. AI-enabled attackers may identify and weaponize weaknesses much faster. Organizations may need to evolve from periodic vulnerability management to continuous exposure management operating at machine speed.

• Traditional identity programs were built for people. The rise of AI agents introduces a new class of non-human identities that can autonomously perform tasks, access systems, and make decisions. Organizations will need to update their security practices to ensure AI agents operate within clearly defined authority boundaries and are subject to the same accountability and monitoring expectations as human users.

• Secure development practices increasingly need to account for models in addition to code. Organizations must understand not only whether software is secure, but also whether the AI components embedded within that software behave predictably, resist manipulation, and operate within established risk tolerances.

• Traditional security monitoring capabilities focus on identifying malicious activities within systems and networks. AI-enabled environments require organizations to additionally monitor model behavior, agent actions, prompt interactions, and AI-specific attack techniques that may not generate the indicators traditionally associated with cyber threats. 

The use of AI to defend the system is a tool to improve defenses, but it should be paired with necessary updates to traditional governance and traditional information security practices to appropriate protect an organization against increasingly sophisticated adversaries.

Deploying AI Governance & Information Security

Defending against AI-driven threats is only one side of the equation. As organizations deploy AI-enabled technologies throughout the enterprise, new attack surfaces and risks are introduced that traditional information security programs were not designed to address.

This is where the broader industry conversation must continue to evolve. Organizations need to think not only about how AI can improve cyber defense, but also how to internally manage AI risks and secure the AI-enabled systems they are deploying. This goes beyond traditional governance and information security, requiring companies to deploy AI governance and AI information security across the enterprise.

The emergence of AI introduces a new governance challenge where organizations must now govern not only technology, but also models, agents, and automated decisions. As a result, traditional governance areas must evolve to address a new class of risks such as:

• AI Acceptable Use Governance: Formal rules regarding which AI tools employees may use, what data may be entered into those systems, and what business processes may be supported by AI.

• Human Oversight Requirements: AI systems can make recommendations or take actions that have significant business impacts so this should govern that decision-making process.

• AI Agent Governance: Defining permissible and restrictive actions for AI agents, including logging and review of their activities. 

However, AI governance alone falls short of demonstrating that AI threats have been addressed in deployed AI-enabled systems. As AI systems become operationally embedded across critical infrastructure, such as healthcare and financial systems, organizations must increasingly focus on another question:

How do we securely deploy and manage our AI-enabled systems?

To demonstrate that system-level threats have been mitigated, organizations should incorporate threat intelligence to identify and address the corresponding AI-specific threats (such as those threats and corresponding mitigations identified in MITRE ATLAS). Guardrails should act within the AI-enabled system to protect against threats such as:

• Prompt injection

• Data poisoning

• Model leakage

• Hallucinations

• Unsafe outputs

• AI Privilege Misuse 

AI security requires organizations to ensure the AI system's behavior, outputs, decisions, and actions can be trusted under both normal and adversarial conditions. These threats are not mitigated using traditional information security controls and must be controlled within each deployed AI system.

The AI Supply Chain Layer

There is an additional layer to the AI threat landscape when viewed through the lens of third-party risk. Most organizations are not building every AI capability themselves. Increasingly, they rely on vendors and services providers who are using AI-enabled systems to provide services and manage their data. As a result, an organization's exposure to AI risk frequently extends beyond the AI systems it operates directly. California’s initiative includes provisions aimed at supporting local governments and critical infrastructure operators, acknowledging that security weaknesses often emerge at ecosystem boundaries rather than within a single organization.

Organizations may have strong internal governance and security practices, yet still inherit risk from a provider's AI model, training pipeline, or AI integrations. As AI becomes more deeply embedded across products and services, trust will increasingly depend on the ability to demonstrate that these systems have been assessed against consistent, security-focused requirements.

This challenge becomes particularly important for critical infrastructure and public-sector environments, where a vulnerability in one supplier can have downstream consequences across many organizations.

As AI adoption accelerates, organizations should treat AI supply-chain assurance as a foundational component of cyber resilience rather than an afterthought. For third-party risk management programs, the answer to these challenges is to obtain the appropriate level of assurance from their vendors that use AI-enabled systems. Unfortunately, most organizations today are not prepared to provide that level of assurance.

Today, most organizations can articulate AI principles, governance objectives, and risk management processes. Far fewer can produce independently validated evidence demonstrating that deployed AI systems are secure and operating as intended.

This is where the industry conversation must continue to evolve in order to build trust through assurance.

From Strategy to Assurance

California's AI Cyber Defense Program is an encouraging development because it recognizes that AI is now a cybersecurity issue, not simply a technology policy issue. The program's emphasis on proactive defense acknowledges both the opportunities and risks associated with increasingly capable AI systems.

The organizations that succeed in this new environment will be those that move beyond AI aspirations and governance statements toward demonstrable security outcomes. They will establish controls that address emerging AI risks, validate that those controls are operating effectively, and extend those expectations throughout their supply chains.

Introduction

Since the start of the decade, adversaries from well-funded nation states to common cybercriminals increasingly reach their attack targets via new vectors: vendors, software, and service providers that every modern organization depends on. Recent data from industry reports reinforces this change. Understanding that shift is the first step toward managing it through third-party risk management (TPRM) and ensuring continuous supply-chain ecosystem trust can be achieved.

A decade that began with a wake-up call

When the decade opened, a handful of incidents redefined how leaders think about trust in technology. The compromise of SolarWinds’ Orion application showed that a single trusted software update could quietly open doors inside thousands of organizations at once. The widespread exploitation of Microsoft Exchange servers showed how one flaw in commonly used software could be weaponized on a global scale. The attack on Kaseya VSA proved that compromising one provider of a remote management tool could ripple outward to the many businesses that relied on it.

These events shared a common theme and many lessons to be learned. The fastest way into a well-defended organization is often through a partner that the organization already trusts. Attackers discovered that the way into a well-defended organization was not through the front door, but through a side door; a supplier, software component, or service provider the organization already trusted. That insight has only deepened since. Microsoft specifically calls out more supply chain compromises as an emerging threat from attackers in their 2025 Microsoft Digital Defense Report.

Third-party involvement in breaches and impact on costs are growing exponentially

There’s no doubt enterprise organizations depend highly on global providers and deeply-integrated supply chains to run their businesses and retain much-needed cost, quality and time-to-market competitive advantages. What was once a series of headline incidents has become a measurable, sustained trend. The 2026 Verizon Data Breach Investigations Report found that breaches involving a third party reached 48 percent of all breaches, a 60 percent increase over the prior year. In roughly half of confirmed breaches, someone other than the victim organization played a part in the chain of events.

The significant financial consequences are just as clear. IBM’s Cost of a Data Breach Report 2026 research places the global average cost of a breach at a record $4.99 million, up 12 percent over the prior year. The average cost when the breach involved a supply chain partner increased by over $227 thousand. Not adequately managing third-party risk will likely have measurable financial impacts at the time of bad-day events when there is a data breach.

Attackers are innovating and staying ahead of defenders

The most important development is not simply that third-party attacks are more frequent. They have become more sophisticated in step with the way organizations build and buy technology. As development teams and business users adopt new tools at speed, like GenAI, attackers have followed them into that terrain. Adversaries are moving from smash and grab attacks and looking more like patient investors in future access. Let’s review recent, compelling examples.

Modern applications rely on shared packages that can be installed automatically across many systems and environments. Attacks like the Shai-Hulud ones highlight the weaknesses of identities and account access in the attack vector, and how a self-spreading worm can move through the trusted software package ecosystem to compromise systems and harvest credentials, secrets, and keys at scale. A single poisoned component can create downstream exposure for thousands of organizations.

In the XZ Utils case, an attacker spent years building trust within an open-source community before trying to insert a hidden backdoor into a widely used open-source library. This was a long-term effort to compromise foundational technologies millions of systems rely upon.

The old perimeter walls no longer mark the boundary

For years, organizations defended a clear edge and defensible perimeter. Firewalls, intrusion prevention, and endpoint protection were the organization’s main line of defense. That boundary has shifted. The new perimeter is increasingly defined by identities and the access they hold, not by the network. Trust itself is sought and attached.

Attackers can bypass traditional controls by stealing and abusing access tokens, cloud credentials, application keys, certificates, and other trusted credentials. Because the access can appear legitimate, conventional defenses may see routine activity rather than an intrusion, making detection difficult and giving attackers the time they need to carry out their goals.

Hiding in plain sight

Attackers are also abusing the same legitimate tools that IT teams and MSPs rely on every day. Microsoft in their MDDR found 79% of ransomware cases involved at least one remote monitoring and management tool. Huntress reported in their 2026 Cyber Threat Report that abuse of RMM tools rose a whopping 277 percent year over year and appeared in nearly one-quarter of investigated incidents. Stolen credentials were another major entry point, with suspicious logins representing 37 percent of the identity threats Huntress tracked.

AI is reshaping the types of attack and the organization’s attack surface

AI is changing supply chain risk in two ways. First, attackers are using it to work faster and at greater scale. Google Cloud’s Mandiant research describes a 2025 shift from experimentation to operational use, including adaptive tools that can rewrite code and agents that can navigate systems with limited human oversight.

Second, every supplier’s AI systems and use of AI is now part of an organization’s attack surface, whether it’s visible or not. Providers are rapidly embedding AI into software, digital products, and software, while organizations are connecting those tools to sensitive data and workflows. IBM found a 56 percent increase in AI-generated attacks. More than one in four organizations that experienced a malicious attack reported that it was AI-driven, adding an average of $1 million per breach. IBM also found that 92 percent of organizations reporting an AI-related breach lacked proper AI access controls.

The takeaways

Security and TPRM teams are facing a completely different world compared to the start of the decade. Digital supply chain complexity is no longer just about the ‘third-party.’ It’s forcing teams to look with wider optics across fourth and even nth parties! This complexity is happening against a landscape where supply chain attacks are now systemic; the perimeter has moved toward identity, access and trust, and AI increases both attacker capability and vendor exposure. This is happening at such an increasing speed such that cyber risk and TPRM can no longer be treated as a periodic compliance task across their vendor ecosystem.

Next in the series

Look for part two of the series where I’ll focus on the challenges of managing third-party cyber risk and why the current approaches are not delivering the resilience and ecosystem trust outcomes security and IT leaders must achieve.

What You Need to Know

Today’s AI systems depend on interconnected models, agents, data sources, tools, cloud platforms, and infrastructure providers. A weakness anywhere in that ecosystem can affect the security and reliability of the resulting AI service.

CISOs and AI vendor security leaders should keep these points in mind:

• The threat surface has moved beyond the model

• Agentic AI moves risks into real-world actions

• Untrusted content is a new attack vector

• Identity and accountability break down with autonomous agents

• Third- and nth-party dependencies obscure accountability, so independent assurance matters

Introduction

The AI threat landscape is shifting from attacks against an individual model to attacks against an entire AI operating environment.

The model is only one part of that system. AI systems may also include retrieval databases, persistent memory, autonomous agents, identity services, APIs, external tools, cloud infrastructure, and human approval workflows. AI systems may appear simple, but behind the scenes they are quite complex.

According to the World Economic Forum Global Cybersecurity Outlook 2026, the top three cybersecurity issues related to generative AI from respondents are data leaks, advancement of adversarial capabilities, and technical security of the AI systems themselves. This reflects the growing awareness by organizations of the need for assessing AI security. For security leaders, the central question is therefore no longer simply, “Is the model secure?” It is:

Can I prove that the model, the system in which it operates, and the third parties supporting it are secure?

Emerging Threats to AI Systems

The AI threat landscape is evolving rapidly, making it hard for organizations buying AI technology and AI system vendors to keep pace. These are some of the emerging threats to AI systems we have identified that need to be considered.

1. Training-data, model, retrieval, and memory poisoning through third parties

AI models depend on large collections of training data, fine-tuning data, evaluation sets, retrieved information, and in some cases persistent memory. An attacker who corrupts any of these inputs may be able to influence how the model behaves.

What’s new is that poisoning can enter through third-party datasets, open-source models, fine-tuning services, retrieval systems, knowledge bases, persistent memory, or other components added later in the lifecycle. This requires continuous evaluation of data integrity and provenance across the full AI system, not just the original model.

In retrieval-augmented generation systems, poisoned content can live inside a knowledge base and surface only when a particular query is asked. Because the malicious content lives in the knowledge base rather than the prompt, it can persist across many sessions and users, quietly steering answers, corrupting decisions, or serving as a delivery mechanism for indirect prompt injection.

Memory creates a similar risk. Some agents retain user preferences, prior decisions, or information learned during previous interactions. An attacker may attempt to insert false facts or malicious instructions into memory, causing the agent to reuse poisoned information in future sessions after the original attack has disappeared from view.

The risk is amplified when ingestion pipelines pull from open or loosely governed sources, or when agents can write to memory, retrieval indexes, shared repositories, or other systems that later become trusted context. A single poisoned document, web page, code repository, or email may quietly redirect workflows, leak information repeatedly, or spread corrupted content to other agents and knowledge repositories.

2. Automation of jailbreaks and safety-control circumvention

Jailbreak methods are becoming more automated and sophisticated. They may use long, multi-step conversations, encoded instructions, role-playing scenarios, adversarial suffixes, or instructions hidden in images and documents. Highly capable models may successfully interpret sophisticated or obfuscated attack inputs that less capable models fail to understand, potentially increasing exposure to advanced jailbreak techniques.

The risk becomes significantly greater when a jailbroken model has access to sensitive information, code execution, cloud consoles, communication systems, or physical operations. In those circumstances, safety circumvention can become a security incident rather than a content-moderation failure.

3. Indirect prompt injection

Indirect prompt injection occurs when malicious instructions are embedded in content that an AI agent reads, such as an email, webpage, document, image, calendar invitation, or code repository.

The agent may mistake those instructions for legitimate directions and act on them. For example, manipulated content could attempt to make an agent retrieve sensitive information, alter a workflow, or send data to an external destination. While there is not a large body of public examples, some examples highlight the dangers. For example, a plaintiff in a court case included concealed prompts in court filing documents, instructing any AI model to agree with his position and treat the clerk's previous ruling against him as an error. The most dangerous configuration combines three capabilities:

1. Access to private data
2. Exposure to untrusted content
3. The ability to communicate or act externally

When all three are present, a single poisoned source may provide an attack path from initial manipulation to data exfiltration or unauthorized system activity.

4. Agent identity and excessive agency

Traditional identity and access management (IAM) was built for two kinds of actors: human users and relatively predictable non-human service accounts. Agentic AI does not fit neatly into either category. Within a single workflow, an agent may act on behalf of a specific person in one moment, operate as an automated process, call external tools, and even spawn sub-agents to complete a task.

This creates identity and accountability challenges that conventional IAM controls were not designed to address. When an agent takes an action, it may be difficult to answer questions like:

• Who was the agent acting for?
• What authority was it actually granted, and for what purpose?
• Did it delegate work to another agent or tool, and did that authority expand along the way?
• If something goes wrong, which component in the chain was responsible?

These questions become harder still when agents share broad, long-lived credentials, inherit permissions implicitly, or create sub-agents without clear limits or expiration.

An agent may struggle to distinguish between what a user explicitly requested, what an external document suggested, what another agent delegated, and what a tool claimed was necessary. It may have legitimate authority but use that authority in response to an untrusted or manipulated input. The risk increases when agents use broad, persistent credentials or can create sub-agents without clear limits, making incident investigations particularly challenging.

5. Multi-agent coordination and cascading failures

Organizations are increasingly deploying AI systems composed of multiple AI agents that collaborate, delegate work, and share information. While this approach can improve scalability and specialization, it also introduces new security risks that don’t exist in single-agent architectures.

An attacker who compromises, manipulates, or poisons one agent may be able to influence other agents that trust its outputs. Malicious instructions, false information, or unauthorized actions can propagate through delegation chains, creating cascading failures that become difficult to detect and investigate. Multi-agent environments may also introduce risks such as agent-to-agent prompt injection, unauthorized delegation, agent impersonation, recursive task loops, and excessive resource consumption.

6. Compromised tools, connectors, and agent platforms

Every tool or connector available to an AI system is both a software dependency and a potential source of instructions entering the model’s context. A compromised connector, plugin, API, or MCP server could steal credentials, return manipulated information, inject malicious instructions, or cause an agent to invoke additional compromised services. The resulting risk depends on what the component can access, whether it can modify data or systems, and whether it can communicate externally.

7. Model, dataset, and software supply-chain attacks

A provider that appears to deliver a single AI product may depend on a complex chain of upstream and downstream organizations for third-party models, datasets, embeddings, container images, open-source packages, development frameworks, and hosted services. This leads to a range of threat vectors that include poisoned datasets, backdoored model weights, malicious model files, compromised model repositories, vulnerable inference servers, unsafe agent templates, and malicious software packages.

These nth-party relationships create significant challenges for security leaders. An organization may have contractual and risk-management visibility into its AI system vendor, but little insight into the model provider, hosting environment, data source, or tool developer supporting the vendor’s system.

8. Attacks against compute, orchestration, and cloud environments

AI workloads concentrate valuable data, intellectual property, credentials, and expensive computing resources in shared infrastructure. AI activities happen across a broad range of infrastructure and applications. Attackers may target cloud management interfaces, container environments, workload schedulers, GPU drivers, notebooks, storage systems, or secrets embedded in jobs and container images. Model checkpoints created during training can also become targets because they may contain valuable versions of a model’s capabilities.

AI infrastructure additionally creates opportunities for economic denial-of-service attacks. An attacker may deliberately trigger long reasoning operations, recursive agent loops, repeated tool calls, and GPU-memory exhaustion. A relatively inexpensive request can generate substantial costs across inference services, databases, and external APIs.

A recent example comes is the attack on Hugging Face. They reported that an intrusion into production infrastructure was conducted through an autonomous agent framework, which turned out to be OpenAI testing frontier models. Hugging Face’s data pipeline was abused to run code on a processing worker. The agent then escalated privileges, collected more credentials, and moved to other systems, i.e., classic threat actor tradecraft. The incident highlights several risks associated with AI: the pace of frontier model development and an agent's ability to identify and abuse vulnerabilities and build complex attack chains, and the need for adequate guardrails for agentic systems.

Recommendations for Third-Party Risk Management (TPRM) Teams

The uncomfortable reality is that many companies are adopting AI faster than they're building AI threat and risk expertise. As a result, expecting TPRM to independently assess AI security at a deep technical level is often unrealistic. So how do you get your TPRM program up to AI speed?

1. Establish an AI Security-specific Process
Determine how you are going to assess the security of the AI system that can be done in an efficient, repeatable, and comparable process. The challenge is current assurance instruments aren't fit for purpose because they lack scalability, relevance, and reliability. Using these instruments can introduce more delays and friction into the assessment and procurement process because they can't answer the question of how secure the AI system really is with confidence.

2. Assess the complete AI system
Determine how you are going to assess the security of the AI system that can be done in an efficient, repeatable, and comparable process. The challenge is current assurance instruments aren't fit for purpose because they lack scalability, relevance, and reliability. Using these instruments can introduce more delays and friction into the assessment and procurement process because they can't answer the question of how secure the AI system really is with confidence.

3. Require independent assurance
TPRM functions should establish assurance requirements based on the sensitivity of the data, the autonomy of the system, and the potential impact of its actions.

Recommendations for AI System Vendors

These recommendations apply to the broad ecosystem of model developers, model hosts, cloud providers, AI application vendors, agent-platform developers, connector providers, and data suppliers, all of which could be part of an AI system.

1. Understand and document the AI system
Maintain a clear understanding of how the AI system works, including the organizations, technologies, data sources, and services that support it.

2. Ensure transparency and traceability
Be able to identify where important system components, data, and outputs come from, and how they have changed over time.

3. Limit access and authority
Ensure AI systems have only the access and permissions needed to perform their intended functions, with appropriate oversight for significant actions.

4. Protect information used by the AI system
Safeguard the information, knowledge sources, and records that AI systems rely on to make decisions and take actions.

5. Monitor AI system behavior
Maintain visibility into how AI systems operate, what information they use, the actions they take, and any unusual or unexpected behavior.

6. Evaluate the security of the complete system
Assess the security and resilience of the entire AI system, including people, processes, technologies, data sources, and external dependencies.

3. Provide independent, validated assurance
Support prospects and customer trust through independent assessments, certifications, or other forms of validated assurance.  

Conclusion

AI is rapidly becoming one of the largest sources of third-party risk in the enterprise. Unlike traditional software, AI systems rely on interconnected models, agents, external data sources, retrieval systems, cloud infrastructure, and numerous third- and nth-party providers, making accountability and security more difficult to assess. As a result, vendor questionnaires and legacy assessment approaches are often insufficient to evaluate the true risk of an AI-enabled system. TPRM teams that establish AI-specific assessment requirements and demand independent, validated assurance like HITRUST AI Security will be better positioned to identify hidden risks, accelerate procurement decisions, and protect their organizations from emerging threats that can originate anywhere in the AI supply chain.

Vendors standardizing on HITRUST AI Security can gain a competitive advantage that provides transparency to buyers and enables them to remove time and friction in the procurement process. This makes it easier for buyers to perform their evaluation efficiently and effectively to meet the needs at the speed of their organization.

Take the next step

Reach out today to learn how the HITRUST can help your organization evaluate AI security, validate security controls for current and emerging AI threats, and build trust across ecosystems.