Home Theme
Principles into Practice 23 23 23 23
Measuring Robustness: how do we decide if an AI system is “suitably” robust?
Filed under:
Reliability

Under the principle of Reliability, the UK AI Ethics Principles say that an AI-enabled system must be sufficiently robust to be able to ‘fulfil its intended design and deployment criteria and perform as expected, within acceptable performance parameters’. For definitions of Reliable, Robust, and Secure, see card: What does Reliability mean in the context of AI development for UK Defence?. 
 
Why might this be challenging?

Just as people might not behave as expected (or hoped) in a deployed situation (See card: Why do good people do bad things? What can we do about it?), an AI-enabled system that has not been developed to be able to cope with the extreme environment of conflict may not perform in the way intended. 
How do you test for an unknown future operating environment? Whilst specifics may be hard to predict, the general uncertainties (known unknowns if you like) are clear. (See card: What does Reliability mean in the context of AI development for UK Defence?) 
To be demonstrate robustness, a system must be able to: 
  • Anticipate edge cases and unforeseen inputs: Developers must account for the fact that AI systems will encounter situations or data outside their intended design parameters. Robustness requires simulating edge cases and stress-testing the system under degraded inputs, environmental noise, or ambiguous conditions. This involves designing scenarios that represent rare but possible events in real-world deployments. 
  • Designed to have adaptive responses: Robust systems should include fallback mechanisms or adaptive features to handle unexpected conditions gracefully. For example, if an AI system in Defence encounters adversarial interference or unexpected inputs, it should have an appropriate response ready, such as an option switch to a predefined safe mode or escalate issues to human operators. 
  • Mitigate adversarial risks: Robustness is closely tied to the ability to withstand adversarial inputs. Developers need to test systems against potential adversarial attacks, such as data manipulation, spoofing, or conflicting signals, ensuring the AI does not behave unpredictably under such circumstances. 
  • Balance flexibility with predictability: While robustness often involves creating systems that adapt to unforeseen inputs, this should not come at the cost of predictability. Developers must carefully design systems to ensure that adaptive responses remain within the bounds of acceptable and intended behaviours. 

Robustness is not a static quality; it must be continuously reassessed as the AI system evolves, learns, or is deployed in new contexts. Regular testing under updated conditions and adversarial scenarios is essential to maintain robustness over time. 

Disclaimer

This tool has been created in collaboration with Dstl as part of an AI Research project. The intent is for this tool to help generate discussion between project teams that are involved in the development of AI tools and techniques within MOD. It is hoped that this will result in an increased awareness of the MOD’s AI ethical principles (as set out in the Ambitious, Safe and Responsible policy paper) and ensure that these are considered and discussed at the earliest stages of a project’s lifecycle and throughout. This tool has not been designed to be used outside of this context. 
The use of this information does not negate the need for an ethical risk assessment, or other processes set out in the Dependable AI JSP 936 part 1, the MODs’ policy on responsible AI use and development. This training tool has been published to encourage more discussion and awareness of AI ethics across MOD science and technology and development teams within academia and industry and demonstrates our commitment to the practical implementation of our AI ethics principles.