ISO/IEC TR 24028 — AI Trustworthiness (Reliability, Robustness, Transparency & Accountability)

This topic is part of the SG Systems Global regulatory & operations glossary.

Updated November 2025 • ISO/IEC 22989, ISO/IEC 42001, ISO/IEC 23894 • Quality, IT, Data Science, Regulatory, Manufacturing

ISO/IEC TR 24028 is the core technical report that defines what it means for artificial intelligence to be trustworthy. It catalogs the key properties—reliability, robustness, safety, security, privacy, transparency, fairness, and accountability—and explains how they interact across the AI lifecycle. TR 24028 does not certify systems or prescribe one architecture; instead, it gives organizations, auditors, and regulators a shared language to judge whether an AI-enabled process can be trusted. In regulated operations (pharma, medical devices, food, cosmetics, chemicals), that language is what separates “experimental tooling” from AI that is suitable to sit inside a validated QMS, MES, or LIMS.

“Trustworthy AI is not about liking the outcome. It’s about being able to predict how the system behaves, explain why, withstand abuse, and show a paper trail when someone asks who signed off.”

1) Where TR 24028 Lives Across the Lifecycle

TR 24028 is relevant from the first AI idea through to model retirement. In concept and use-case design, it helps decide whether AI is appropriate at all and which trust properties are most critical (e.g., safety vs fairness). In data and model design, it sets expectations for diversity, balance, and robustness to noise or attacks. In training and validation, it drives coverage of normal and abnormal conditions, sub-population performance, and stress scenarios. In deployment and integration, it informs human-in-the-loop design and fallback strategies. In operation and monitoring, it frames how you detect drift, bias, and degradation. And in retirement, it guides safe sunset and evidence retention so that decisions influenced by AI remain traceable years later.

2) Regulatory Anchors & System Controls

While TR 24028 itself is not a certifiable standard, its trustworthiness terminology shows up in law and guidance. The EU AI Act encodes many TR 24028 properties—robustness, cybersecurity, transparency, bias control, human oversight—into mandatory requirements for high-risk AI systems. Regulators in pharma, devices, and food increasingly ask whether AI is reliable, explainable, and subject to human control. Under 21 CFR Part 11 and Annex 11, any AI that touches electronic records must be supported by secure access control, e-signatures with defined meaning, and comprehensive GxP audit trails. TR 24028 is the conceptual bridge between those regulatory expectations and the concrete behavior of your AI-enabled workflows.

3) Trustworthiness Dimensions in TR 24028

TR 24028 organizes AI trustworthiness into a set of recurring themes. Reliability—the AI behaves consistently under expected conditions. Robustness—it gracefully handles noise, variability, and certain malicious inputs without catastrophic failure. Safety—it does not create unacceptable risk of harm to people, product, or environment. Security—it resists tampering, fraud, and unauthorized access. Privacy—it respects data protection principles and minimises exposure of personal or sensitive information. Transparency and explainability—stakeholders can understand what the system does, why, and within what limits. Fairness—it avoids unjust bias and discriminatory impact. Accountability—there are clear lines of responsibility, with evidence of decisions and approvals. Every serious AI deployment in a regulated environment needs a position against each of these dimensions, not just a vague “we tested it” statement.

4) Lifecycle View – Mapping Trust to AI Activities

TR 24028 emphasizes that trustworthiness is not a one-time test; it is a lifecycle property. In requirements and design, you document trust objectives and constraints (e.g., “no single-point AI failure may cause batch release without human sign-off”). In data acquisition and preparation, you ensure datasets meet integrity, diversity, and privacy needs. In model construction, you prefer architectures and feature sets that enable explainability and robustness where required. In verification and validation, you run tailored tests to probe safety, fairness, robustness, and security. In deployment, you embed AI into MES, LIMS, WMS, and ERP in ways that respect existing controls and human roles. In operation, you monitor performance, override patterns, and incidents; and in retirement, you disable or constrain AI safely while preserving logs, training data summaries, and documentation for later review.

5) Trust Criteria & Appetite—Rules, Not Marketing

Declaring AI “trustworthy” means nothing without clear criteria. TR 24028 assumes you will define acceptance thresholds—accuracy ranges, robustness margins, allowable bias deltas, maximum acceptable false-positive/negative rates, privacy guarantees, and explainability requirements—for each use case. Riskier use cases (e.g., AI advising on batch release, exception handling in eBR, or quality overrides) require tighter thresholds, stronger fallbacks, and more conservative human oversight. These trust criteria should be written into QMS documents and risk registers, not improvised by developers under time pressure or treated as soft marketing claims in slide decks.

6) Trust Properties – Controls That Actually Bite

For each trust dimension, TR 24028 expects concrete controls. Reliability is enforced by sound testing, change control, and monitored service-level metrics. Robustness is enforced by stress testing, adversarial probes, input validation, and safe fallback modes when conditions drift outside the trained envelope. Safety is enforced by guards in MES/HMI logic, process constraints, and human sign-off for high-impact actions. Security is enforced by hardened APIs, segregation of duties, and secure configuration of cloud and on-prem components. Privacy is enforced by minimization, anonymization/pseudonymization, and tight access control. Transparency is enforced by model documentation, feature/decision explanations, and user interfaces that show context and confidence. Fairness is enforced by data scrutiny, subgroup testing, and restrictions on use where bias cannot be adequately mitigated. Accountability is enforced by roles, approvals, and audit trails that link specific AI behavior to a responsible owner and version.

7) Typical Trust Failures—And How to Evidence Them

Silent performance drift: model accuracy degrades as processes, materials, or populations change; evidenced by trends in false-positive/negative rates and overrides. Hidden bias: poorer performance on particular groups, sites, or product families; evidenced by subgroup analysis that was never done—or done too late. Security weaknesses: exposed endpoints, weak authentication, or prompt-injection paths; evidenced by penetration tests and incident logs. Lack of transparency: no one can explain what the model is doing or why; evidenced by missing documentation, opaque features, and user confusion. Over-automation: operators treat AI as infallible; evidenced by low override rates even when incidents show the AI was wrong. TR 24028 expects these failure modes to appear explicitly in your trust assessments, together with controls and monitoring, not as surprises in post-incident reports.

8) Disposition of Trust Issues—Risk First, Not Cosmetics

When AI trust issues appear, TR 24028 implies that organizations must address them as they would other critical risks. Technical remediations include retraining on better data, changing architectures, tightening thresholds, hardening input validation, or limiting AI decision scope. Procedural measures include new SOPs, updated training matrices, clarified escalation paths, and revised use-case boundaries. Governance responses include pausing certain AI functions, reclassifying risk levels, or subjecting the system to additional review before resuming use. Cosmetic changes—a new dashboard, a rephrased user message—are not enough; TR 24028 demands that trust issues are treated like any other significant quality or safety concern, with durable fixes and documented outcomes.

9) CAPA, Change Control & AI Lifecycle Stewardship

In regulated environments, meaningful trust gaps belong inside CAPA and Change Control, not only in data-science backlogs. CAPA can target reduced bias, improved robustness, higher transparency, or lower incident rates, with effectiveness checks tied to metrics. Change control should govern new models, significant retraining, architecture changes, and integration modifications that affect how AI influences MES/eBMR, LIMS, or WMS decisions. TR 24028 is essentially telling organizations: treat AI systems as long-lived assets with stewardship responsibilities—not as disposable experiments that vanish after the initial project ends.

10) Prevention by Design—Use-Case, UX & Human Factors

TR 24028 pushes organizations to build trust into AI design, not bolt it on after incidents. At the use-case level, that means refusing AI where data quality, controllability, or oversight cannot meet trust thresholds. At the user-interface level, it means clearly labelling AI output, showing confidence and rationale where appropriate, and making it easy to challenge or override AI recommendations. At the human-factors level, it means ensuring that operators, reviewers, and approvers understand AI capabilities and limitations, know when to distrust it, and are not overloaded with opaque alerts. Trustworthy AI is often less flashy and more conservative than “demo-grade” AI—but it survives audits and real-world stress.

11) Monitoring & Early Warning—Trust via SPC and OOT

Trust is not permanent; it is renewed by monitoring. TR 24028 aligns naturally with SPC control limits and Out-of-Trend (OOT) concepts used in process and quality monitoring. Organizations should trend AI metrics over time: model accuracy, error types, subgroup performance, override rates, latency, incident counts, and security events. OOT detection on these metrics can trigger investigations and re-validation before AI behavior becomes a quality or safety problem. In well-run environments, most AI issues are caught internally via such monitoring, not via customer complaints or inspection findings.

12) Metrics That Demonstrate Trustworthiness Is Under Control

Concrete metrics make TR 24028 tangible. Track how many AI systems are in use, with risk classification and assigned owners. Measure what percentage of them have current trust assessments and monitoring plans. Track AI-linked Deviations/NCR, complaints, and near misses, and how many result in CAPA or change control. Monitor subgroup performance to ensure fairness commitments are met. Track training completion from the Training Matrix for roles using or approving AI outputs. And monitor override patterns: both “zero overrides” and “constant overrides” can indicate trust problems. Taken together, these metrics move “trustworthiness” from aspiration to measurable reality.

13) Validation of AI-Enabled Workflows

For AI embedded in GxP-relevant workflows, TR 24028 trust concepts must show up in validation. Requirements should specify not just functionality but also trust attributes: robustness ranges, transparency expectations, privacy constraints, and acceptable error profiles. CSV and IQ/OQ/PQ test plans should explicitly challenge AI behavior under different conditions (nominal, edge, failure, security). Validation evidence should document which version of the model and data were tested, and how monitoring is configured post-go-live. Retention and archival rules must ensure that AI-related logs, model metadata, and key datasets remain accessible for reconstruction of events years later. If your validation ignores trust dimensions, it is out of step with TR 24028 and emerging expectations.

14) How TR 24028 Fits Operationally Across Systems

Execution (MES, WMS, automation). TR 24028 trust concepts apply to AI in scheduling, routing, anomaly detection, setpoint proposals, and quality gates. Systems should limit AI to advisory roles where appropriate, enforce human review for high-risk decisions, and provide operators with context and explanations. AI changes to batches, labels, or equipment states must pass through the same controls as non-AI logic.

Quality & QRM. Quality teams use TR 24028 as the conceptual basis for AI-related entries in the risk register, Deviations/NCR, and CAPA. It also informs criteria for accepting or rejecting vendor AI capabilities in equipment, lab instruments, or SaaS tools.

Data & IT platforms. Data and IT teams use TR 24028 to structure AI platforms: common pattern for logging, security, model registries, monitoring, and rollback, so that every AI use case inherits baseline trustworthiness controls rather than creating bespoke patterns each time.

Governance & management review. AI steering committees and management review use TR 24028 language to discuss risk, performance, and incidents in a way that makes sense to executive and regulatory audiences. Instead of “our AI works,” they talk in terms of reliability, robustness, transparency, fairness, and accountability, backed by evidence.

15) FAQ

Q1. Is ISO/IEC TR 24028 a certifiable standard?
No. TR 24028 is a technical report, not a certifiable management-system standard. Organizations cannot be “certified to” TR 24028, but they can align their AI governance, risk management, and validation practices with its trustworthiness concepts and use it to support certifications such as ISO/IEC 42001.

Q2. How does TR 24028 relate to ISO/IEC 23894?
TR 24028 defines what trustworthiness in AI looks like—reliability, robustness, safety, fairness, transparency, etc. ISO/IEC 23894 describes how to manage AI risks using a structured risk process. Together, they provide both the properties you want and the process to achieve and maintain them.

Q3. Does TR 24028 cover generative AI and large language models?
Yes in principle. TR 24028 is technology-agnostic; its trustworthiness properties apply to any AI system whose outputs influence decisions or outcomes. Generative AI and large language models raise specific challenges for robustness, safety, transparency, and misuse—but they are still evaluated along the same trust dimensions.

Q4. How does TR 24028 connect to the EU AI Act?
The EU AI Act defines legal obligations and risk categories for AI; TR 24028 provides a structured way to think about whether a system is trustworthy along relevant dimensions. Using TR 24028 concepts helps organizations design and document controls that support AI Act compliance, especially for high-risk systems, but TR 24028 by itself is not a legal compliance framework.

Q5. Who should own TR 24028 implementation in a regulated company?
Ownership should be shared: Quality and Risk/Compliance lead on governance and documentation; IT/Data lead on platform and security controls; Operations own how AI is embedded into MES/WMS/automation; and Regulatory Affairs ensures alignment with external expectations. If “AI trust” sits only with data scientists, it will not survive audits, incidents, or organizational changes.

Related Reading
• AI Governance & Risk:
ISO/IEC 42001 | ISO/IEC 23894 | GxP
• Integrity & Validation:
Data Integrity | ALCOA+ | CSV |VMP | Audit Trail (GxP)
• Execution & Records:
MES | eBR | eMMR | CPV | Deviation/NCR | CAPA