AI Governance Under the EU AI Act: A Practical Framework

The EU AI Act: A New Era for AI Regulation

The EU Artificial Intelligence Act entered into force in August 2024, making it the world's first comprehensive legal framework specifically governing artificial intelligence systems. While some provisions are already applicable and others phase in over 12 to 36 months, the Act's risk-based approach to AI regulation will shape how organisations develop, deploy, and govern AI for decades to come.

Unlike sector-specific AI rules that existed previously, the EU AI Act applies horizontally across all industries and use cases. It affects not just AI developers but also organisations that deploy AI systems developed by others. For privacy and compliance teams, the Act introduces a new category of obligations that intersects significantly with existing data protection requirements under GDPR.

The Four-Tier Risk Classification System

The EU AI Act's central innovation is its risk-based classification system, which determines the compliance obligations applicable to each AI system. The four tiers — Unacceptable Risk, High Risk, Limited Risk, and Minimal Risk — are not arbitrary categories but reflect the potential for AI systems to cause harm to individuals, society, or fundamental rights.

Unacceptable Risk systems are prohibited outright. These include AI systems that exploit vulnerabilities of specific groups, systems that use subliminal techniques to distort behaviour, real-time remote biometric identification in public spaces for law enforcement (with narrow exceptions), and social scoring systems by public authorities. Any organisation deploying a system that might fall into this category must conduct an immediate assessment.

High-Risk AI: Obligations and Conformity Assessment

High-Risk AI systems — defined by their deployment context in Annex III of the Act — include AI used in critical infrastructure, biometric identification, employment and recruitment, credit scoring, access to essential services (healthcare, education), law enforcement, and administration of justice. These systems can be deployed, but only after a conformity assessment and ongoing compliance obligations are satisfied.

Conformity assessment for most High-Risk systems is self-assessment: the provider conducts the assessment following the requirements of the Act and declares conformity. For certain biometric and law enforcement systems, assessment by an accredited notified body is required. The substance of the assessment covers risk management, data governance, technical documentation, transparency, human oversight, accuracy, robustness, and cybersecurity.

Limited-Risk and Minimal-Risk Obligations

Below the High-Risk tier, most AI systems fall into Limited-Risk or Minimal-Risk. These categories carry lighter obligations, but they are not obligation-free — and the boundary between Limited-Risk and High-Risk is where most classification disputes will arise.

Limited-Risk systems — those that interact with humans, generate or manipulate content, or perform emotion recognition or biometric categorisation — trigger transparency obligations under Article 50. Users must be informed they are interacting with an AI system, deep-fake and AI-generated content must be labelled in a machine-readable way, and providers of general-purpose AI generation systems must implement content watermarking. Minimal-Risk systems, which cover the vast majority of enterprise AI use cases (spam filters, recommendation engines, business analytics), face no specific obligations under the Act — though voluntary codes of conduct are encouraged, and downstream GDPR obligations continue to apply.

General Purpose AI Models and Foundation Model Rules

The Act's provisions on General Purpose AI (GPAI) models — added late in negotiations to respond to the rise of foundation models like GPT-4, Claude, and Gemini — impose obligations directly on model providers rather than only on deployers. Any GPAI model placed on the EU market from August 2025 must meet baseline documentation, copyright compliance, and training-data summary requirements.

GPAI models presenting 'systemic risk' — defined by compute thresholds (10^25 FLOPs currently) or Commission designation — face additional obligations: model evaluations, adversarial testing, incident reporting, and cybersecurity protections. For enterprises deploying rather than building foundation models, the practical takeaway is that vendor selection matters more than ever. A deployer using a GPAI model whose provider has not properly documented training data and evaluation results inherits gaps that will surface during their own conformity assessments.

Building a Risk Management System

Article 9 of the EU AI Act requires providers of High-Risk AI systems to establish, implement, document, and maintain a Risk Management System throughout the AI system's lifecycle. This is not a one-time exercise but a continuous process that includes identification and analysis of known and foreseeable risks, estimation and evaluation of risks that may emerge in intended use and reasonably foreseeable misuse, and adoption of risk management measures.

In practice, an EU AI Act Risk Management System should be integrated with existing enterprise risk management and privacy risk frameworks. Many of the inputs — data quality assessments, impact assessments on individuals, monitoring of model performance — are shared with DPIA processes under GDPR. Organisations with mature privacy programmes have a head start.

Data Governance Under Article 10

Article 10 sets the data governance obligations for training, validation, and testing datasets used in High-Risk AI systems. Datasets must be relevant, representative, free of errors as far as possible, and complete for their intended purpose. Where bias may affect outcomes, the datasets must reflect that risk and mitigation measures must be documented.

Practical compliance requires more than dataset descriptions. Providers must document data collection methods, data preparation processes, formulation of assumptions, availability, quantity and suitability, examination for possible biases, and identification of any gaps or shortcomings and how they were addressed. Where personal data is processed in training, GDPR lawful basis analysis stays in force — the AI Act does not create a new lawful basis. Read the AI and personal data compliance guide for the GDPR side of this specifically.

Technical Documentation and Logging Requirements

High-Risk AI systems must be accompanied by comprehensive technical documentation covering the system's intended purpose, development methodology, performance metrics, training data governance, and testing results. This documentation must be kept up to date and made available to competent authorities on request.

Automatic logging of system operation — capturing events, inputs, and outputs with sufficient detail to enable post-hoc evaluation — is mandatory for High-Risk systems. Log retention periods are prescribed for different system categories. Building documentation habits into the AI development lifecycle from the start is far more effective than retrospective documentation exercises.

Human Oversight: The Core Safeguard

A defining feature of the EU AI Act's approach to High-Risk AI is its insistence on meaningful human oversight. Article 14 requires that High-Risk AI systems be designed and developed so that natural persons can effectively oversee them and, where necessary, override or interrupt their operation. Human oversight measures must be appropriate to the risk and built into the system by design.

Effective human oversight under the Act means more than having a human 'in the loop' nominally. It requires that the overseeing person understands the system's capabilities and limitations, can interpret its outputs critically, and has the authority and means to intervene. Organisations should conduct oversight capability assessments to verify their human reviewers genuinely have the tools, training, and time to exercise meaningful oversight.

Post-Market Monitoring and Serious Incident Reporting

Compliance does not end at deployment. Article 72 requires providers of High-Risk AI systems to establish a post-market monitoring system proportionate to the nature of the AI technologies and risks. The system must actively and systematically collect, document, and analyse relevant data on system performance throughout the lifetime of the system.

Article 73 introduces mandatory incident reporting: providers must report serious incidents to market surveillance authorities within timelines ranging from immediately (for widespread infringement or death) to 15 days for less severe incidents. A serious incident includes any malfunction of an AI system that leads to death or serious harm, a serious and irreversible disruption of critical infrastructure, or a breach of fundamental rights obligations. Building incident detection and reporting into your MLOps pipeline is now a regulatory requirement, not a nice-to-have.

Timelines, Enforcement, and Penalties

The Act phases in over three years. Prohibitions on Unacceptable Risk systems applied from 2 February 2025. GPAI obligations apply from 2 August 2025. Most High-Risk provisions apply from 2 August 2026, with a further extension to 2 August 2027 for AI systems that are safety components of products regulated under existing EU harmonisation law.

Enforcement is a hybrid of national market surveillance authorities and a new European AI Office within the Commission that oversees GPAI. Penalties are tiered and severe: up to €35 million or 7% of global annual turnover (whichever is higher) for violating Unacceptable Risk prohibitions; up to €15 million or 3% for High-Risk violations; up to €7.5 million or 1% for supplying incorrect or misleading information to authorities. These fine ceilings surpass even GDPR's most severe tier, making AI Act non-compliance a board-level financial risk.

Overlap with GDPR: DPIA and FRIA Together

For most High-Risk AI systems processing personal data, two impact assessment obligations run in parallel: GDPR Article 35 requires a Data Protection Impact Assessment (DPIA), and Article 27 of the AI Act requires a Fundamental Rights Impact Assessment (FRIA) for deployers of certain High-Risk systems in the public sector and specified private-sector cases (credit scoring, life and health insurance risk assessment).

The scopes overlap but are not identical. DPIA focuses on risks to data subjects from personal data processing; FRIA focuses on risks to fundamental rights (non-discrimination, human dignity, access to services) from the AI system as a whole. A unified assessment template that satisfies both — one purpose statement, one data flow map, one risk register with rights and privacy risks tagged — is dramatically more efficient than running two parallel exercises. See the AI Privacy Impact Assessment template for a starting point.

Building Your AI Governance Programme

An effective EU AI Act governance programme starts with an AI inventory — a comprehensive list of all AI systems in use across the organisation, including third-party AI embedded in purchased software. Each system should be classified by risk tier, with the classification rationale documented. This inventory forms the foundation for all downstream governance activities.

Governance structures should include an AI risk committee with cross-functional membership (legal, privacy, technology, business), a review process for new AI deployments, ongoing monitoring of existing systems, and an incident response procedure for AI-related harms. Organisations that have already built GDPR compliance governance structures can extend and adapt them for AI Act compliance, leveraging shared data governance infrastructure and privacy expertise.