The increasing requirement for trustworthy AI systems across diverse application domains has become a pressing need not least due to AI’s critical role in the ongoing digital transformation addressing urgent socio-economic needs. Despite the numerous recommendations and standards, most AI practitioners and decision-makers, still prioritize system performance as the main metric in their workflows often neglecting to verify and quantify core attributes of trustworthiness including traceability, robustness, security, transparency, and usability. In addition, trustworthiness is not assessed throughout the lifecycle of AI system development so developers often fail to grasp a holistic view across different AI risks. Last, the lack of a unified, multi-disciplinary AI, Data, and Robotics ecosystem for assessing trustworthiness across several critical AI application domains hampers the definition and implementation of a robust AI paradigm shift framework towards increased trustworthiness and accelerated AI adoption. To address this critical unmet needs, FAITH innovation action will develop and validate a human-centric, trustworthiness optimization ecosystem, which enables measuring, optimizing, and counteracting the risks associated with AI adoption and trustworthiness in critical domains, namely robotics, education, media, transport, healthcare, active aging, and industrial processes through seven international Large Scale Pilots. Notably, cross-fertilization actions will create a joint outcome, which will bring together the visions and specificities of all the pilots. To this end, the project will adopt a dynamic risk management approach following EU legislative instruments and ENISA guidelines and deliver tools to be widely used across different countries and settings. At the same time, diverse stakeholders’ communities will be engaged in each pilot delivering seven sector-specific reports on trustworthiness to accelerate AI take-up.

Rational

Artificial intelligence (AI) plays a key role in the ongoing digital transformation and has triggered a global competition for tech leadership. The widespread application across many domains has led to a rising societal awareness around the consequences of misuse and a demand for systems that are ethical and trustworthy. EU parliament stresses that “EU has so far fallen behind with the result that future technological standards risk being developed without sufficient EU contributions, which presents a challenge to political stability and economic competitiveness; concludes that the EU needs to act as a global standard-setter on AI” . Moreover, while many organizations consider AI to be the next frontier in digital transformation, they just cannot shift into the next gear and further their investment in AI-powered processes. AI practitioners, including researchers, developers, and decision makers, have traditionally considered system performance (i.e., accuracy) to be the main metric in their workflows. This metric is far from sufficient to reflect the trustworthiness of AI systems. According to ISO/IEC TS 5723:2022, trustworthiness is the ability to meet stakeholders’ expectations in a verifiable way and its characteristics include: accountability, accuracy, authenticity, availability, controllability, integrity, privacy, resilience, robustness, safety, security, transparency, usability, etc. According to the NIST AI Risk Management Framework (RMF)2 trustworthy AI is: valid and reliable, safe, fair and bias is managed, secure and resilient, accountable and transparent, explainable and interpretable, and privacy-enhanced. Any breach of AI trustworthiness can lead to severe societal consequences given the pervasiveness of these AI systems. By contrast, improving the trustworthiness of AI systems is a shared priority for the private and public sectors, as indicated by prolific recent research and guidelines. , , However, the potential approaches to attain AI trustworthiness remains a challenge. , ,8; w ere many efforts are under development: some notable examples include ongoing standards efforts, such as the “Overview of trustworthiness in artificial intelligence,” published by ISO Technical Committee: ISO/IEC JTC 1/SC 42 on AI. Other works include the ENISA reports on AI cybersecurity , OECD AI Recommendation ; the Responsible AI Certification ; the “Trustworthy AI” white paper by the China Academy for Information and Communication Technology (CAICT) ; the Trustworthy AI process developed by Deloitte13; the Conformity assessment for trustworthy AI (capAI) and the FUTURE-AI: Guiding Principles and Consensus Recommendations for Trustworthy Artificial Intelligence in Medical Imaging . Legal EU instruments (e.g. proposed AI Act, NIS, NIS2, Resilience Act) aim to protect ethical and democratic values of the AI users.

While most active academic research on AI trustworthiness has focused on the algorithmic properties of AI-based models, advancements in algorithmic research alone is insufficient for building trustworthy AI products. From an industrial perspective, the lifecycle of an AI product consists of several stages, including data preparation, algorithmic design, development, and deployment as well as operation, monitoring, and governance. Improving any characteristic of trustworthiness (e.g., robustness) involves efforts at multiple stages of this lifecycle, e.g., data sanitization, robust algorithms, anomaly monitoring, and risk auditing. On the contrary, a breach in any single stage or aspect can undermine the trustworthiness of the entire system. Therefore, AI trustworthiness should be established and assessed systematically throughout the lifecycle of an AI system. In addition, for taking a holistic view of the trustworthiness of AI systems over all stages of their lifecycle, it is important to understand the big picture of different aspects of AI risks related to trustworthiness that are different from traditional ICT systems:

1) the data used for building an AI system may not be appropriate;

2) the intentional or unintentional changes during the training process that may alter the AI system performance;

3) datasets to train AI systems may become detached from original context, or may be outdated;

4) AI system scale and complexity (many systems contain trillions of decision points) housed within more traditional software applications;

5) pre-trained models can increase levels of statistical uncertainty and cause issues with bias management, scientific validity, and reproducibility;

6) higher degree of difficulty in predicting failure modes for emergent properties of large-scale pre-trained models;

7) perceptions about AI system capabilities. In addition to pursuing AI trustworthiness by establishing requirements for each specific aspect, attention should be paid to the combination of and interaction between these aspects, which are important and underexplored topics for trustworthy real-world AI systems. For instance, the need for data privacy might interfere with the desire to explain the system output in detail, and the pursuit of algorithmic fairness may be detrimental to the accuracy and robustness experienced by some groups 17. As a result, trivially combining systems to separately improve each aspect of trustworthiness does not guarantee a more trustworthy and effective end result. Instead, elaborated joint optimization and trade-offs among multiple aspects of trustworthiness are necessary . There are various EU initiatives that provide guidelines and requirements for AI trustworthiness e.g. the reports developed by the High-Level Expert Group on Artificial Intelligence, the proposed European Union Artificial

Intelligence Act (EU AI Act)20; the White House Blueprint for an AI Bill of Rights ; the risk assessment NIST AI RMF . In parallel, the EU Agency for Cybersecurity (ENISA) in a recent report warns that AI may open new avenues in manipulation and attack Figure 1 The FAITH trustworthiness roadmap.
methods, as well as new privacy and data protection challenges; it also presents the AI threat Landscape. A follow up report by ENISA provides detailed analysis of threats targeting machine learning systems and concrete and actionable security controls described in relevant literature and security frameworks and standards. The above fragmented efforts, require a systematic approach to shift the current AI paradigm toward trustworthiness. This requires awareness and cooperation from multi-disciplinary stakeholders who work on different aspects (e.g. technical, societal, legal, business, standards) of trustworthiness and different stages of the system’s lifecycle. This is the central vision of the FAITH proposal. The innovation action aims to provide the practitioners and stakeholders of AI systems not only with a comprehensive analysis of the foundations of AI trustworthiness but also with an operational playbook for how to assess, build trustworthy AI systems and how to continuously measure their trustworthiness.

FAITH adopts a human-centric, trustworthiness assessment framework (FAITH AI_TAF) which enables the testing/measuring/optimization of risks associated with AI trustworthiness in critical domains. FAITH AI_TAF builds upon NIST Artificial Intelligence Risk Management Framework (AI RMF), upon the requirements imposed by the EU legislative instruments, upon ENISA guidelines on how to achieve trustworthiness by design and upon stakeholder’s intelligence and users’ engagement. Seven (7) Large Scale Pilot activities in seven (7) critical and diverse domains (robotics, education, media, transportation, healthcare, active ageing, and industry) will validate the FAITH holistic estimation of trustworthiness of selected sectoral AI systems. To this end, the proposed framework will be validated across two large scale piloting iterations/phases across focusing on assessing: (i) generic threats of trustworthiness, and (ii) domain-specific threats and risks of trustworthiness. In addition, FAITH AI_TAF will be used to identify potential associations (in the context of cross-fertilisation) among the domains towards the development of a domain-independent, human-centric, risk management driven framework for AI trustworthiness evaluation.

To successfully realize the project’s vision FAITH has i) assembled a number of highly innovative organisations representing industry, SMEs, research and academia, ii) proposed FAITH AI_TAF, a robust assessment framework iii) defined seven (7) large scale pilots (LSPs) to validate the FAITH AI_TAF in diverse application domains, ranging from healthcare to education, and from media to critical infrastructure management, and finally iv) has defined a concrete project plan that will allow the consortium to successfully perform the large scale pilots by taking a holistic view of the trustworthiness of AI systems over all stages of their lifecycle.