Responsible AI: Trust, Governance and Bias

‍

‍

‍

As organisations become increasingly ambitious with artificial intelligence, the focus is often on what these systems can achieve. But as AI becomes more complex, another question becomes just as important: how do we know we can trust it?

‍

For Sara Gaurav, Responsible AI Engineer at GSK, the answer lies in looking beyond performance alone. From evaluating AI agents to understanding model bias and building effective governance frameworks, responsible AI requires a much deeper understanding of how systems behave, make decisions and affect the people who use them.

‍

‍

‍

Beyond the Hype of AI Agents

‍

One of the biggest trends Sara is currently questioning is the growing enthusiasm around multi agent systems.

‍

With organisations exploring ways to use multiple AI agents to complete increasingly complex tasks, it can be tempting to assume that more agents automatically mean better results.

‍

But Sara argues that this isn't always the case.

‍

"Actually, using multiple agents can reduce performance of certain tasks up to 70%. So I think that we got to start off sometimes with just one agent and then slowly build up."

‍

When agents communicate with one another, information has to pass between different systems. Much like the game of Chinese whispers, the original information can become distorted along the way.

‍

Rather than introducing unnecessary complexity, organisations should start with the simplest system that can achieve the desired outcome, adding more agents only when there is a clear reason to do so.

‍

It's a useful reminder that AI development isn't about building the most complicated system. It's about building the right one.

‍

‍

The Problem with Measuring AI Performance

‍

A major part of Sara's work at GSK involves developing ways to evaluate AI systems, particularly those used in research and healthcare.

‍

One of the challenges is that traditional performance metrics don't always tell the whole story.

‍

A model might achieve high accuracy in one setting, but perform very differently when presented with unfamiliar data or a different set of conditions.

‍

As Sara explains:

‍

"Your model can be very accurate. You can put it under another complete scenario, and suddenly its accuracy drops highly because you haven't made sure that it's robust enough to different conditions."

‍

This is particularly important in clinical applications, where a model's reliability can have significant consequences.

‍

Sara's work has involved developing evaluation metrics that help engineers assess models across different conditions, including fairness metrics that examine whether diagnostic systems perform equally well for different patient groups.

‍

The challenge is not simply to establish whether a model gets the answer right. It's to understand how consistently and reliably it can do so.

‍

‍

Why We Need to Look Inside the Black Box

‍

‍

‍

‍

For more complex AI systems, particularly large language models and agents, evaluating the final output becomes even more difficult. Sara highlights how benchmarks, which are often used to compare AI performance, can give an incomplete picture of what a system is actually doing. A model might produce the correct answer while making mistakes along the way, repeating actions or contradicting itself. This is why her work increasingly focuses on evaluating the entire trajectory of an AI system, rather than just its final response.

‍

This includes examining:

‍

• The tools it uses
• The steps it takes
• Whether its decisions are consistent.

‍

Another concern is how much we really understand about the reasoning behind AI decisions. Research into chain of thought has raised questions about whether the reasoning an AI system produces genuinely reflects how it arrived at an answer, or whether it is sometimes a post rationalisation.

‍

As Sara puts it:

‍

"I think that these systems are more blackbox than we think they are, and that there's much more value in actually going into interpretability research to understand how these systems work better and trust them more."

‍

Understanding what happens inside these systems is becoming increasingly important as organisations give them more responsibility.

‍

‍

Bias Is More Hidden Than We Think

‍

Another important area of Sara's work is understanding how bias can emerge in AI systems, particularly in healthcare. One particularly interesting example comes from diagnostic models. Research has shown that models trained to identify diseases from medical images can sometimes also infer demographic characteristics, such as a patient's age or sex, even when they were never explicitly trained to make those predictions.

‍

This raises important questions about what models are actually learning from the data they are given: If a model can indirectly identify demographic characteristics, those signals could influence its diagnostic decisions and introduce unintended bias.

‍

Sara describes how her team has explored ways of identifying and mitigating these issues, including developing fairness metrics and experimenting with approaches that penalise models for learning certain demographic signals.

‍

But the issue extends beyond healthcare. AI models are trained on data that reflects particular cultures, perspectives and assumptions. As more people rely on these systems to make decisions or learn about the world, there is a risk that existing biases become reinforced. Sara raises a concern that is particularly relevant as AI becomes part of everyday life:

‍

"If people start using these models for all decisions that they make or learning about anything, they're just going to learn these biases directly, and we're going to lose a bit of that individual opinion into things."

‍

The implications go beyond technical fairness. They touch on the diversity of perspectives and opinions that make human decision making valuable.

‍

‍

Governance Needs to Keep Up With Development

‍

For Sara, responsible AI also means thinking carefully about how models are managed throughout their lifecycle. Her introduction to AI came through cybersecurity, where she became particularly interested in dual use models: systems that can create significant positive impact but can also be misused. In some cases, models designed for beneficial scientific applications can be repurposed for harmful outcomes. This makes questions around model access, storage, security and decommissioning particularly important.

‍

However, governance cannot exist purely as a set of documents or frameworks. It needs to translate into the actual development process. Sara's team is exploring ways to automate parts of this process, using AI evaluation systems to monitor changes as engineers develop and update their models. For example, an AI judge can be integrated into the code review process, automatically checking contributions and identifying when a change causes a system to drift away from its intended behaviour.

‍

This creates more opportunities to identify problems early, rather than relying entirely on manual reviews after development.

‍

‍

Keeping Humans in the Loop

‍

Despite the increasing role of automation, Sara doesn't believe human oversight is going away. As AI systems become capable of completing more tasks independently, the challenge will be finding the right balance between automation and human review. AI can help process and evaluate large volumes of information, but human judgement remains important in determining whether those evaluations are meaningful and reliable.

‍

Sara explains that even new evaluation methods need to be compared against human reviews to establish how closely they align. In other words, we shouldn't assume an AI system is a reliable judge simply because it performs well on a benchmark. There still needs to be a way of checking that its assessment reflects the judgement we actually care about.

‍

‍

The Bigger Picture

‍

Sara's perspective highlights an important shift in how organisations should approach AI development. The conversation needs to move beyond what AI can do and towards understanding how well it does it, where it can fail and what happens when things go wrong. Whether it's deciding how many agents a system needs, evaluating model performance or identifying hidden biases, responsible AI is not a separate stage of development... It needs to influence the decisions made throughout the process.

‍

As Sara puts it:

"I'm very much pro experimenting and pushing the limit, but the idea is to always use it with a responsible AI lens, not over rely on it and know when to be critical."

‍

That balance between experimentation and critical thinking will be essential as AI systems become more capable and more deeply embedded in the way we work.

‍

‍

About Passion Labs

‍

Passion Labs is an AI research and development lab building technologies and thought leadership that amplify human potential. Through our In Conversation series, we spotlight diverse voices shaping the future of AI, across industry, ethics and creativity.

‍

‍