What is AI governance?

1. What Is AI Governance?

AI stands for artificial intelligence. AI is software that learns from data and then makes choices or predictions. Governance means the rules, roles, and checks an organization uses to control something.

So AI governance is the set of rules, roles, and checks that an organization uses to control how its AI is built and used. The goal is simple: keep AI safe, fair, and trustworthy.

Good governance answers questions like these. Who is responsible if the AI is wrong? How do we test it? Can we explain what it does? Who is watching it over time?

2. Why AI Governance Matters

AI is being adopted very quickly across many industries. That speed creates risk.

AI can make mistakes. It can be biased against certain groups. It can be hard to understand. Without rules, these problems can cause real harm.

Governance helps catch problems early. It builds trust with the public. It also helps organizations follow new laws as those laws arrive.

3. The Main Parts of AI Governance

Most governance plans share the same core ideas. Common parts include:

  • Accountability. Someone is clearly responsible for the AI.
  • Transparency. People can find out how the AI works and when it is used.
  • Fairness. The AI does not unfairly favor or harm certain groups.
  • Safety and security. The AI works as intended and is protected from attacks.
  • Privacy. The AI protects people's personal data.
  • Human oversight. A person can review, pause, or override the AI.
  • Monitoring. The AI is watched over time, because its behavior can change.

4. The Big Frameworks and Rules

Three major efforts shape how the world governs AI today.

NIST AI Risk Management Framework (United States)

NIST is a United States government agency. In January 2023 it released the AI Risk Management Framework, also called the AI RMF. It is voluntary, which means it is guidance and not a law. It groups the work into four jobs: Govern, Map, Measure, and Manage. Many organizations now use it as a starting point for good AI practice.

ISO/IEC 42001 (International)

ISO and IEC are international standards groups. In December 2023 they released ISO/IEC 42001. It is the world's first standard for an AI management system. An AI management system is the set of policies and steps an organization uses to run AI responsibly. A company can be checked by an outside auditor and earn certification to this standard.

The EU AI Act (Europe)

The European Union passed the AI Act, which became law on August 1, 2024. It is the world's first broad law that regulates AI directly. It sorts AI into four risk levels: unacceptable, high, limited, and minimal. Higher-risk AI faces stricter rules. The rules take effect in stages, with major parts arriving in 2026 and later. The law can apply to companies outside Europe if their AI affects people inside the EU.

5. Clinical AI Governance: Governance Meets Health Care

Clinical AI governance is AI governance applied to health care. The word "clinical" means related to the direct care of patients. The same core ideas apply, but the stakes are higher. In health care, an AI mistake can affect a person's health or safety.

Health AI is already common. The FDA is the United States agency that reviews medical products. As of mid-2025, the FDA's public database listed more than 1,250 AI-enabled medical devices cleared for use in the United States, up from about 950 a year earlier. Most of these use prediction models, not chatbots.

Health care raises a few special concerns:

  • Patient safety. A wrong output can lead to a wrong diagnosis or the wrong treatment.
  • Fairness and health equity. A well-known 2019 study in the journal Science found that a widely used health algorithm gave lower risk scores to Black patients who were just as sick as White patients. This reduced their access to extra care. It is a real example of why oversight matters.
  • Privacy. Health data is sensitive and is protected by law.
  • Drift. A model can get worse over time as patients and data change.

Several groups now guide and watch over health AI.

World Health Organization (WHO)

The WHO published Ethics and Governance of Artificial Intelligence for Health in 2021. It lists six principles for health AI: keep people in control of their care; promote safety, well-being, and the public good; make the AI clear and understandable; make people and organizations responsible; ensure fairness and access for everyone; and keep the AI responsive and sustainable. In 2024, the WHO added guidance for large multi-modal models, the kind of AI behind many newer tools.

FDA

The FDA reviews many AI tools that count as medical devices. In January 2025 it released draft guidance on managing AI-enabled devices across their whole life, from design through monitoring after release. This is called a total product lifecycle approach.

CHAI and the Joint Commission

The Coalition for Health AI, or CHAI, is a nonprofit started in 2021 with thousands of member organizations. It writes best practices for responsible health AI, including a written guide and a "model card" that describes a tool's intended use and its risks. In September 2025, CHAI and the Joint Commission, the main accreditor of United States hospitals, released shared guidance to help health systems use AI safely.

Research

A 2026 review in npj Digital Medicine studied 35 health AI frameworks published from 2019 to 2024. It found seven core areas that strong health AI governance should cover, and it built a five-level model to help organizations rate and improve their readiness.

The hardest part is the last step. An AI tool can suggest a note, an order, or a diagnosis. The riskiest moment is when that output is written into the patient's official record and becomes a real action. Many tools focus on building or training the AI. Far fewer focus on this final boundary.

6. Where C_Verified Fits (Open C Health Systems)

Here at Open C Health Systems (OCHS), or "Open_C" for short, we designed one of the most important layers any healthcare organization will need, called C_Verified. C_Verified is a governance and verification layer that sits at the EHR writeback boundary. An EHR is an electronic health record, the digital chart that holds a patient's medical information. The "writeback boundary" is the point where an AI's output would be written into that chart.

C_Verified is built as an independent control point at that boundary. It is designed as a governance layer, not as a tool that diagnoses or treats patients.

OCHS frames the need this way. AI agents are entering clinical work faster than the governance to control them is being built. This includes text, audio, photos, video, and other media. The EHR writeback boundary is where AI output turns into real clinical action. Today there is no standard, independent layer at that point. EHR vendors guard their own systems. AI vendors guard their own products. C_Verified is built to fill that gap as an independent, third-party layer that checks content integrity, enforces governance rules, and keeps a tamper-evident record.

Here is how it works. Every AI writeback attempt passes through one checked path. C_Verified reviews the request against a fixed set of allowed actions. It then makes one of three decisions: allow, hold, or deny. It records that decision in a tamper-evident audit record, which is a log that is hard to change without detection, plus a governance receipt. If the decision is allow, a cleaned version of the output is sent on to the EHR. If the decision is hold or deny, nothing is written to the EHR.

This connects directly to the governance ideas earlier in this report. It supports human oversight, accountability, transparency, and safety, all at the exact moment when AI output could change a patient's record.

References

  1. National Institute of Standards and Technology. AI Risk Management Framework (AI RMF 1.0), NIST AI 100-1. January 2023.
  2. International Organization for Standardization and International Electrotechnical Commission. ISO/IEC 42001:2023, Artificial Intelligence Management System. December 2023.
  3. European Union. Regulation (EU) 2024/1689 (the AI Act). In force August 1, 2024.
  4. World Health Organization. Ethics and Governance of Artificial Intelligence for Health: WHO Guidance. 2021.
  5. World Health Organization. Ethics and Governance of Artificial Intelligence for Health: Guidance on Large Multi-Modal Models. 2024.
  6. U.S. Food and Drug Administration. Artificial Intelligence in Software as a Medical Device, and Draft Guidance, Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations (Docket FDA-2024-D-4488). January 2025.
  7. Bipartisan Policy Center. FDA Oversight: Understanding the Regulation of Health AI Tools. 2026 (citing the FDA public device database).
  8. Coalition for Health AI (CHAI). Responsible AI Guide and Assurance Standards Guide.
  9. The Joint Commission and Coalition for Health AI. Guidance on Responsible Use of AI in Healthcare. September 17, 2025.
  10. npj Digital Medicine. "Advancing healthcare AI governance through a comprehensive maturity model based on systematic review." 2026.
  11. Obermeyer Z, Powers B, Vogeli C, Mullainathan S. "Dissecting racial bias in an algorithm used to manage the health of populations." Science. 2019;366(6464):447-453.