Safe and trusted AI: The importance of explainability

By Simon Miles, Head of AI, Aerogility

The importance of a safe and trusted AI approach

An AI system can be viewed as safe if there’s reassurance about its behaviour. It should act in line with our expectations. Likewise, it can be trusted if we have reason to be confident in the decisions it takes.

We gain this confidence when we can rationalise how a decision has been arrived at. As advanced as neural networks are, they require us to trust answers without being able to view their workings. In many instances, that’s a problem because the explanation is a big part of the equation.

Take ‘what-if?’ scenarios. An aviation business might want to determine the effect of taking an aircraft out of service or sending a certain part for repair. A model-based AI approach not only allows us to visualise the future these actions create but also how these predictions have been determined — empowering confidence in the findings.

On the other hand, if an AI doesn’t reveal its workings but simply predicts a given action will harm business performance, no strategic value is provided. Without being able to visualise how the AI arrived at a decision, particularly in a complex business landscape where the past is not always a forecaster of the future, it’s difficult to trust the prediction.

How do we truly achieve safe and trusted AI?

One critical element is explainability. The explainability of AI software is the ability to understand and interpret how it arrived at its outputs — whether actions, recommendations, or categorisations — from the inputs it has received.

While machine learning and neural networks can be hugely powerful tools, they are not always explainable. The way they arrive at an answer cannot be communicated in a way that humans can understand.

For AI to be safe and trusted, we need to comprehend how it has generated its outputs. This is exactly what model-based AI provides.

Combining machine learning and agent-based modelling

The implementation of safe and trusted AI is key for businesses looking to utilise its capabilities. In doing so, model-based AI can be combined with machine learning.

Where there is a significant amount of reliable data and a lack of detailed understanding of a system being modelled, the ability to reveal an answer through machine learning can outweigh explainability concerns.

As such, these AI approaches should be viewed as complementary, with each able to solve different parts of a complex business challenge. Machine learning identifies insights and trends from large datasets. Aerogility, meanwhile, can simulate scenarios in a way that can be visualised and understood by humans and develop actionable plans and responses.

Ultimately, regardless of their use of machine learning, businesses stand to benefit significantly from harnessing their understanding of their operations to realise the transformative power of safe and trusted AI.

Learn more about how Aerogility enables businesses to gain safe and trusted insights.