From Designing For to Designing With

AI governance conversations are frequently framed either as technical compliance exercises or as abstract debates about future existential risk. In both cases, the people most affected by today’s AI systems are the ones least likely to have a seat at the table, especially those with little influence over how systems are designed or deployed.

Taken from the April 2026 Executive Institutes Forum Summary Report, we’re highlighting how participants considered building genuine trust in an era of unpredictable AI capabilities – shifting the paradigm from designing for humans to designing with them. By centering human-centered design, inclusive data stewardship, and institutional accountability, we can ensure that the technologies shaping our daily lives protect and empower the communities they serve, rather than treating them as secondary considerations.  

Who is the “human” in human-centered AI design? To answer this question, participants focused on AI trust through a human-centered design lens, challenging the default assumption of who counts as the “human” in technology design. Participants were asked to consider how do we move from designing for humans to designing with humans, embedding lived experience and participatory design into the technologies that shape daily life? Trust cannot be built only with system operators or enterprise customers; it must also include those whose information, opportunities, or outcomes are shaped by the systems, often without their active participation.

Trust cannot be built only with system operators or enterprise customers; it must also include those whose information, opportunities, or outcomes are shaped by the systems, often without their active participation.

Participants repeatedly returned to the breadth of AI’s impact. Virtually everyone is directly or indirectly affected by AI systems, whether as users, data subjects, employees, or members of communities shaped by AI-enabled decisions. A particularly vivid example involved an online staffing marketplace breach in which highly sensitive applicant materials were exposed, including interview recordings, background-check information, and other personal data. The implications discussed went beyond a standard data leak: participants highlighted the risks created when voice samples, tax information, and identity data are combined into high-fidelity profiles that can later be exploited, risks that were taken on by particularly vulnerable people¾those seeking employment. This case illustrates how trust is not an abstract principle; it is closely tied to data stewardship, security design, and the downstream effects of failure.

Human and Institutional Dimensions of Trust

Companies often invest heavily in training employees to recognize email spoofing and other traditional cybersecurity threats, while providing far less guidance on safe and appropriate AI tool usage. This gap becomes more serious when personal and professional uses of AI systems blur together, such as when employees use the same tools or environments for work research and sensitive personal questions. Many users do not understand how data is stored, separated, or reused across systems, making it difficult for them to make informed decisions about trust. This suggests that AI governance is not only a policy issue at the executive or regulatory level, but also an operational issue involving employee behavior, literacy, and everyday workflow design.

Policy and Governance Dimensions

Participants also explored the difficulty of governing AI because of its uneven and unpredictable capabilities. One concept that resonated was the “jagged frontier” problem: AI can perform at an expert level in some domains while failing in surprisingly basic ways in others. This unpredictability complicates oversight, because the boundaries of competence and failure are not always visible in advance. The issue became increasingly complex when considering whether governance should rely more on proactive testing, stronger certification mechanisms, and incentives for organizations to identify failure modes before public harm occurs. While some companies already have reputational incentives to reduce bias and hallucinations, the conversation suggested that market incentives alone are unlikely to be enough, especially when the effects of failure are distributed across people with little power to demand accountability.

The session closed with a practical challenge to participants: review their own organizations’ AI governance policies and ask two questions. First, do those policies genuinely represent the interests of all the populations affected by the organization’s AI-related activities? Second, if someone is harmed by an AI system connected to the organization, is there a meaningful path for redress, or only informal public complaint? This closing reframed AI governance as an institutional responsibility rather than merely a technical or theoretical issue, pushing participants to evaluate whether their policies are inclusive in design and accountable in practice.

This is the work the AI Institute was created to advance. At the 2026 Marconi Awards Gala & Institute Forums, global leaders from industry, academia, government, and society will convene to translate these complexities into concrete frameworks, partnerships, and standards while recognizing the innovators shaping what responsible AI looks like in practice.