Artificial Intelligence has moved from isolated pilot project to a strategic component embedded in financial, commercial, and operational processes. For Internal Audit this creates a concrete challenge: auditing "around" AI is no longer enough. The risk of the algorithm itself must be evaluated. This article proposes a practical framework for identifying, classifying, and auditing AI risks, aligned with current corporate governance requirements.
Why AI requires a different audit approach
A traditional information system is deterministic: given the same input, it always produces the same output, and its logic can be reviewed line by line. A machine learning model does not work that way. Its behavior depends on training data, can evolve over time, and in many cases not even its own development team can fully explain why it produced a particular result.
This shifts the risk away from access controls or data integrity (where Internal Audit already has a consolidated methodology) toward new territory: algorithm opacity, bias in training data, and accountability when an automated decision causes harm.
The shift from ex-post auditing to continuous auditing
The use of AI is also transforming how auditing is done. More and more departments are moving toward a continuous audit model, where control is no longer a point-in-time review but becomes integrated into the organization's own data flows. This requires the audit team not only to evaluate AI as a risk object, but also to understand how to use it as a control tool.
AI risk taxonomy for Internal Audit
Before auditing, classify. Most current reference frameworks group AI risks into three broad categories.
| Category | Typical risks |
|---|---|
| Technical | Training data quality and representativeness, model drift, lack of explainability, AI-specific cybersecurity vulnerabilities |
| Ethical | Algorithmic bias, indirect discrimination, lack of transparency toward the affected user |
| Social / reputational | Loss of trust from customers or employees, regulatory impact, use of generative AI without adequate human oversight |
This taxonomy is the starting point for building the AI audit universe: each organizational use case (a credit scoring model, a conversational assistant, a fraud detection system) must be mapped against these categories to prioritize what to audit first.
AI governance framework: what must exist before auditing
Internal Audit cannot evaluate an AI model in isolation without first assessing the governance framework within which it operates. The minimum elements it should verify are the following.
Inventory and owner for each model
The organization must be able to identify which AI models are in production, what process they support, and who the business owner (not just the technical team) is for their correct functioning.
Pre-deployment approval and validation policy
Before deploying a model with impact on customers, employees, or financial decisions, there should be a committee or validation process that reviews ethical, technical, and compliance risks.
Meaningful human oversight
For the highest-risk use cases, there must be a human control point capable of reviewing, challenging, and if necessary overriding the model's decision. Internal Audit must verify that this oversight is real and not merely formal.
Model traceability and documentation
Training data used, model version, date of last validation, and bias test results must be documented and available for review.
Key control point: if an AI use case has no identified business owner, it most likely also lacks a real risk validation process behind it. This is the first warning sign in any AI audit.
Practical methodology for auditing an AI use case
Once the governance framework has been verified, fieldwork on a specific use case can be structured in four phases.
Phase 1: Understanding the use case and its impact
Identify what decision the model makes or supports, what volume of people or transactions it affects, and what the consequence of an undetected error or bias would be. This impact analysis determines the depth of the audit engagement.
Phase 2: Evaluating input data
Review the provenance, quality, and representativeness of the training data. One of the most frequent causes of algorithmic bias is not in the model itself, but in historical data that already reflected a prior organizational bias.
Phase 3: Testing model behavior
Where possible, request or execute tests that compare model results across different groups or segments to detect unjustified differences in treatment. If the model is not natively explainable, assess what explainability mechanisms have been implemented.
Phase 4: Communicating findings and follow-up
AI findings often require technical language that the Audit Committee does not always master. Translate the finding into business impact and reputational risk, and ensure the action plan includes a model revalidation date, not just a one-off fix.
Common mistakes when auditing AI risks
- Auditing only access and security, not the algorithm. Verifying who can access a system says nothing about whether the model discriminates or has degraded over time.
- Accepting the vendor's explanation without independent verification. When the model belongs to a third party, Internal Audit must require evidence of bias testing and validation, not just a compliance declaration.
- Treating generative AI the same as a traditional predictive model. The risks of a conversational assistant (hallucinations, leakage of confidential information in prompts) differ from those of a scoring model and require specific testing.
- Not reviewing model drift after deployment. A model correctly validated at the outset can degrade over time if its performance is not continuously monitored.
How technology supports continuous AI auditing
Assessing AI risk on a point-in-time basis, once a year, is insufficient when models change versions frequently. An audit management and GRC platform with continuous monitoring capabilities allows the department to:
- Maintain a live inventory of AI models in production, with their owner, risk level, and date of last validation.
- Link each AI use case to its risk assessment and associated action plan, without relying on standalone documents.
- Generate automatic alerts when a model exceeds the revalidation deadline defined by internal policy.
- Centralize evidence of bias testing and explainability so it is ready for any Audit Committee review or regulatory inspection.
AI risk audit checklist
- Is there an up-to-date inventory of all AI models in production?
- Does each use case have an identified business owner, not just a technical lead?
- Have bias tests been performed on training data and on model outputs?
- Is there real, documented human oversight for the highest-risk use cases?
- Is there a periodic revalidation process to detect model drift?
- Does the audit plan include AI use cases within the audit universe, with their own risk assessment?
Conclusion
Auditing AI does not require turning the entire team into data engineers, but it does require extending the traditional control methodology toward new questions: where does the data come from, what human oversight exists, and how can it be demonstrated that the model is still behaving as originally validated. Organizations that incorporate this assessment in a structured way (supported by tools that maintain traceability for each model) will be better prepared for the regulatory scrutiny that is already anticipated in this area.