12 Days of AI Risk Management: Days 5-8

12-part action plan continues – get in front of AI risk for GenAI, third party risk management, regulatory compliance and more.

By Jacqueline Lebo

SAFE presents a 12-part action plan to get your organization out in front of the #1 cyber risk management challenge of the coming year, AI risk. Take it one “day” at a time, face 2026 with confidence.

Day 5: AI in Third-Party Risk (TPRM): Continuous Scoring, Not Annual Checks

The reality is that a vendor’s risk posture can shift dramatically in a matter of hours.

Day 6: The Model Risk Management (MRM) Crossover

Learn from the financial industry: Ensure that AI in lending, trading, and fraud detection is accurate.

Day 7: The Generative AI Policy Imperative

GenAI risk demands a clear, non-negotiable framework based on the principle of Risk-Based AI Use.

Day 8: AI for Regulatory Compliance Mapping (NIST to EU AI Act)

How to avoid a massive, manual mapping exercise every time a new AI regulation is enacted.

More tips and best practices in Days 1-4 of this series.

Day 5: AI in Third-Party Risk (TPRM): Continuous Scoring, Not Annual Checks

For too long, Third-Party Risk Management (TPRM) has been treated as a compliance exercise: sending annual questionnaires, collecting certifications, and reviewing static reports. This process offers a dangerous illusion of security. The reality is that a vendor’s risk posture can shift dramatically in a matter of hours due to control failures, newly exposed vulnerabilities, or cloud misconfigurations—long before the next annual review.

Strategic Insight: From Attestation to Continuous Behavior Monitoring

The strategic shift required is moving from collecting static attestations to implementing continuous risk scoring based on objective, dynamic data. AI is the only technology capable of facilitating this transition at scale.

The role of AI in TPRM is to analyze and correlate vast streams of data, far beyond what a human analyst can manage:

• Public and Non-Public Data: AI continuously ingests and analyzes external signals, including media reports, breach trends, dark web mentions of the vendor’s domain, and open-source intelligence (OSINT) to look for adversarial activity.
• Technical Posture Signals: AI models analyze security data derived from outside-in monitoring (for external hygiene) and, where applicable, internal control effectiveness signals (for compliance with contract terms). This includes continuous checks for configuration changes, unpatched critical vulnerabilities, and expired certificates.
• Pattern Recognition: Crucially, the AI doesn’t just flag individual events; it identifies anomalies in the vendor’s risk behavior. A combination of a sudden drop in a security score, coupled with an increase in dark web mentions, can flag a systemic risk before a breach announcement.

By quantifying risk based on this dynamic, continuous analysis—a core capability of platforms like SAFE —the organization gains true risk velocity visibility into the supply chain. This proactive approach flags deteriorating security hygiene, allowing the host organization to demand immediate mitigation and renegotiate terms before the vendor’s control failure becomes a systemic risk to its own operations.

The Financial Cost of Static TPRM

The reliance on static questionnaires directly compromises the quantitative risk model. From a FAIR perspective, a static annual check means the Control Strength factor in the equation is only validated once per year. For 364 days, the organization operates with a massive, unquantified uncertainty regarding its Threat Event Frequency (TEF) originating from the third party.

Continuous scoring provides validated, real-time control status, dramatically reducing the uncertainty in the FAIR calculation and providing defensible data for the Board regarding supply chain exposure.

---

Deliverable and Clear Actionable Next Steps

The executive mandate must be clear: risk management cannot be outsourced to a vendor’s self-assessment.

A Mandate for Procurement and Risk Leaders:

“Is your TPRM program monitoring vendor behavior or just collecting attestations?”

Next Steps for Risk Leaders:

1. Demand Continuous Data: Challenge the procurement function to include a contractual requirement for continuous, objective risk data sharing (or monitoring) from all Tier 1 and Tier 2 vendors, moving beyond the simple right-to-audit clause.
2. Integrate Risk Scoring: Retire point-in-time vendor risk scores and integrate a continuous scoring mechanism into procurement decisions, making risk posture a real-time gating factor for contract renewal and access privileges.
3. Define Behavioral Anomalies: Work with the security team to define the top five “behavioral risk anomalies” (e.g., a 20% drop in overall security score, a failure to patch within a defined SLA) that will trigger an automatic, immediate remediation requirement from the vendor.

Day 6: The Model Risk Management (MRM) Crossover

The largest financial institutions have managed Model Risk Management (MRM) for decades and can  ensure that AI and quantitative models used for lending, trading, and fraud detection are stable, unbiased, and accurate. When AI models are used for critical security decisions—like blocking traffic, detecting insider threats, or managing access—the risk is equally acute and demands the same level of discipline.

Security leaders must adopt the rigorous MRM principles of the financial sector. Failing to do so allows security models to degrade silently, turning a sophisticated control into an unreliable liability.

Strategic Insight: Guarding Against Silent Model Degradation

The core MRM challenge in cybersecurity is ensuring the Model Control Effectiveness remains stable over time. This requires mandatory, continuous checks on two non-negotiable elements:

1. Data Drift (The Performance Degradation Risk):
   • What it is: Data drift occurs when the characteristics of the real-world data feeding the model changes significantly from the data the model was originally trained on.
   • The Security Impact: A model trained to recognize “normal” network traffic patterns six months ago will become less accurate today as organizational behavior, cloud migrations, and remote work evolve. This leads to increased false negatives (missed threats) and potential false positives (blocking legitimate users), directly compromising the security posture and operational efficiency.
   • The MRM Mandate: Continuous monitoring is required to detect when the statistical properties of the input data shift beyond acceptable thresholds, triggering an automatic model retraining or validation cycle.
2. Adversarial Robustness (The Intentional Attack Risk):
   • What it is: The model’s vulnerability to intentional, minor modifications in the input data designed to trick the model into misclassifying the input (e.g., adding imperceptible noise to malware to evade detection).
   • The Security Impact: An AI threat detection model must be rigorously tested against adversarial attacks. A lack of adversarial robustness means an attacker can subtly manipulate inputs to bypass controls—a direct failure of the Control Strength in the quantitative risk equation.
   • The MRM Mandate: Models must be regularly subjected to simulated adversarial input tests to ensure they remain robust and resistant to manipulation before deployment and throughout their lifecycle.

By mandating these MRM checks, security leadership ensures that the sophisticated controls they rely on maintain their reliability, moving the security function from a best-effort approach to a defensible, quantified process.

---

Deliverable and Clear Actionable Next Steps

MRM requires the security team to hold the data science teams accountable for model stability, not just initial accuracy.

Four Key Questions to Ask Your Data Science Team about the Security Model’s Performance Stability:

1. “What is the threshold for data drift, and what is the current percentage deviation from the training baseline for our critical security inputs?” (Focuses on input stability)
2. “How often are model predictions formally audited against a ground truth or human review to confirm the Model Control Effectiveness hasn’t degraded?” (Focuses on output accuracy/Model Drift)
3. “What specific adversarial robustness testing techniques (e.g., evasion, data poisoning) are applied to the model before it is pushed to production?” (Focuses on resilience to attack)
4. “What is the formal Model Validation schedule, and who is the non-developer stakeholder responsible for signing off on model health?” (Focuses on governance and accountability)

Next Steps for Risk Leaders:

1. Adopt the Terminology: Begin integrating MRM terms like data drift, model validation, and adversarial robustnessinto procurement contracts and internal governance documents for AI tools.
2. Establish a Validation Pipeline: Mandate that all AI-driven security tools must submit to a formalized, periodic (e.g., quarterly) validation and re-calibration process led by an independent team.
3. Quantify Drift Risk: Work with your CRQ platform (like SAFE) to assign a dollar value to the risk associated with model degradation (i.e., the increase in ALE if a model’s performance drops by 10%).

Day 7: The Generative AI Policy Imperative

The rapid adoption of Generative AI (GenAI) tools presents an immediate, material risk that outpaces existing security policies. The widespread use of these models creates new attack vectors and data handling liabilities. Operationalizing GenAI risk is not a technical project; it is a policy imperative that demands a clear, non-negotiable framework based on the principle of Risk-Based AI Use.

Strategic Insight: A Risk-Based Approach to GenAI

A risk-based approach is essential because not all AI use carries the same financial exposure. The policy should not treat drafting a marketing headline the same as generating code or handling PII.

The core principle is simple: The level of governance and control must be proportionate to the potential for Loss Magnitude (ML).

The policy must address the three most critical and financially damaging risks, using a risk-based lens:

1. Data Leakage (Input Contamination Risk):
   • Risk-Based View: This risk is highly variable. Pasting non-confidential text into a public LLM is low risk. Pasting customer PII or proprietary code is a high-risk activity that directly leads to an uncontrolled export of data.
   • The Consequence: This constitutes an unauthorized data export and a potential breach of contract or regulation (like GDPR or HIPAA), leading directly to high Model Loss Magnitude through fines and legal action. This is the single biggest policy failure point.
2. IP Risk (Copyright and Plagiarism Risk):
   • Risk-Based View: Using GenAI for internal brainstorming is low risk. Using GenAI to generate final source code or external marketing copy introduces a high legal risk of IP infringement, as the output may contain copyrighted material from the training set.
   • The Consequence: The organization loses its legal defensibility over generated assets and inherits a potentially indefinite legal liability, impacting the Legal Cost component of the quantitative risk model.
3. Prompt Injection (Model Manipulation Risk):
   • Risk-Based View: Using a public, unvetted GenAI service to summarize a document is high risk because the model’s security controls are unknown. Using a sandboxed, internally managed model with certified adversarial robustness testing is low risk.
   • The Consequence: A successful prompt injection attack can reduce Model Control Effectiveness and allow for data exposure or system compromise. The use of ungoverned GenAI systems dramatically increases this vulnerability.

By mandating controls that reflect the sensitivity of the data and the purpose of the model, the organization can avoid unnecessary friction for low-risk use cases while locking down the high-risk activities that threaten the balance sheet.

---

Deliverable and Clear Actionable Next Steps

Effective risk management means establishing clear boundaries immediately, treating high-risk activities with the utmost control.

Template for the Top 3 Mandatory Clauses in your new GenAI Usage Policy:

Clause	Name	Mandate (Risk-Based Control)	Rationale (Risk Mitigated)
Clause 1	Sensitive Data Prohibition	Prohibited for ALL external GenAI: Do not input any Confidential, PII, PHI, or CUI into any external, non-sanctioned GenAI tool. Only use approved, internal, or enterprise-licensed models with certified data privacy protections for handling sensitive data.	Mitigates Data Leakage: Immediately blocks the single largest risk source, tying control directly to the sensitivity of the data.
Clause 2	Ownership and Verification	All GenAI-generated content must be treated as unverified draft material. Content used for external communications or production code must undergo mandatory human review and verification for factual accuracy and copyright infringement.	Mitigates IP Risk: Ensures the company retains legal responsibility and control over the final output, treating high-impact content with the necessary legal rigor.
Clause 3	Approved Tools List	Use of all GenAI services must adhere to the Approved Tools List maintained by IT/Security. Use of any other external GenAI service for business purposes is prohibited without documented security and risk approval.	Mitigates Prompt Injection & Shadow AI: Contains the proliferation of ungoverned Shadow AI and allows for continuous vetting of tools based on their risk profile and control stability.

Next Steps for Risk Leaders:

1. Immediate Dissemination: Treat these three clauses as a minimum acceptable standard and distribute them immediately to all employees.
2. Mandate Risk Classification: Work with Data Science and Development teams to classify all existing and proposed AI projects (including Shadow AI) into risk tiers (Low, Medium, High) based on their potential Model Loss Magnitude to determine the necessary governance controls.
3. Technical Enforcement: Begin implementing technical controls to detect and block traffic to the most common external GenAI endpoints to enforce the Approved Tools List until high-risk activities can be adequately monitored.

Day 8: AI for Regulatory Compliance Mapping (NIST to EU AI Act)

The introduction of complex, global AI regulations—such as the EU AI Act, the NIST AI Risk Management Framework (RMF), and pending US legislation—creates a significant compliance burden. Organizations with mature security programs already maintain vast evidence bases for frameworks like ISO 27001 or NIST Cybersecurity Framework (CSF). The critical challenge is avoiding a massive, manual mapping exercise every time a new AI regulation is enacted.

Strategic Insight: Dynamic Mapping for Compliance Velocity

AI is essential for moving compliance from a static, auditor-driven exercise to a dynamic, continuous function. The key insight is using AI/ML to dynamically map existing security control evidence against the requirements of emerging AI-specific frameworks.

Manually mapping controls is slow, error-prone, and provides only a snapshot of compliance status. An AI-driven approach provides compliance velocity by:

1. Semantic Correlation: The AI doesn’t rely on simple keyword matching. It uses natural language processing (NLP) to understand the intent and scope of an existing control (e.g., a control requiring “data lineage documentation” in ISO 27001) and correlate it to a newly mandated requirement (e.g., the EU AI Act’s need for “data and data governance” documentation for high-risk systems).
2. Evidence Utilization: The AI automatically extracts, verifies, and tags existing control evidence (e.g., policy documents, technical configurations, audit logs) against the new requirements, confirming where current operational practices already satisfy the new AI regulatory mandates. For instance, systems like SAFE’s SOC2 Analyzer Agent and Jack Jones’ custom GPT for FAIR-CAM demonstrate how specialized AI tools can quickly analyze hundreds of control documents to find semantic matches and identify gaps, proving the effectiveness of this approach.
3. Real-Time Gap Identification: Most importantly, the AI instantly highlights gaps. For instance, it might confirm control coverage for data security (NIST CSF controls) but flag missing evidence for the EU AI Act’s unique requirements around human oversight or model transparency, directing resources precisely where new controls must be created.

This process drastically reduces the Compliance Cost component of the overall Loss Magnitude (ML) calculation, transforming weeks of manual analysis into an immediate, quantified view of regulatory readiness.

---

Deliverable and Clear Actionable Next Steps

The goal is to provide a transparent, executive-ready view of where the organization is covered and where immediate control investments are required to meet emerging AI regulations.

Visual Map: Overlap and Gaps Between Current Compliance and EU AI Act

Control Domain	Existing Framework (e.g., NIST CSF, ISO 27001)	EU AI Act Requirement	Coverage Status	Risk Action Required
Data Governance	Fully Covered (Data quality, lineage, access control defined).	Chapter 3, Section 3, Article 17:  Data and data governance for high-risk systems.	Overlap	None (Evidence Mapped)
Model Security	Covered (Vulnerability Mgmt, Access Control).	Chapter 3, Section 2, Art. 15: Cyber-security and robustness testing.	Overlap	None (Evidence Mapped)
Transparency	Partial (Standard system documentation).	Chapter 3, Section 2, Art. 13: Technical documentation and record-keeping (including model evaluation reports).	GAP	High: Need formalized policy on XAI results and impact assessments.
Human Oversight	None (No specific human-in-the-loop requirement).	Chapter 3, Section 2, Art. 14: Human oversight requirements.	GAP	Critical: Must develop procedures for continuous human validation and intervention mechanisms.
Auditability	Covered (Logging and monitoring).	Chapter 3, Section 2, Art. 12: Logging of activities (ensuring traceability).	Overlap	None (Evidence Mapped)

Next Steps for Risk Leaders:

1. Prioritize New Frameworks: Mandate the immediate ingestion of key emerging global AI standards (NIST AI RMF, EU AI Act, etc.) into the organization’s existing governance, risk, and compliance (GRC) platform.
2. Pilot AI Mapping: Select a single high-risk AI system and use an AI-driven mapping solution to demonstrate how existing controls map to the new EU AI Act requirements, specifically focusing on the Transparency and Human Oversight gaps.
3. Quantify Gaps: Integrate the identified compliance gaps into your CRQ platform. The lack of Human Oversightor Transparency controls represents a significant increase in Model Loss Magnitude that must be financially quantified and budgeted for remediation.

More tips and best practices for risk-based AI management in Days 1-4 of this series.