This document discusses a tool called COMXAI that uses combinatorial testing methods to help explain the decisions made by artificial intelligence and machine learning models. COMXAI is able to narrow down combinations of input factors that are responsible for certain model outcomes, which can then be presented as understandable rules. Some limitations of this approach are that it may generate very large rulesets with many input combinations, and it is vulnerable to overfitting issues in the underlying models. The authors propose future work to address these limitations, as well as improving the user interface and security of the tool.

1. An Application of
Combinatorial
Methods for
Explainability in
Artificial Intelligence
and Machine Learning
D. RICHARD KUHN
C O M P U T E R S E C U R I T Y D I V I S I O N
I N F O R M A T I O N T E C H N O L O G Y L A B O R A T O R Y
RAGHU N. KACKER
C O M P U T A T I O N A L A N D A P P L I E D M A T H
D I V I S I O N I N F O R M A T I O N T E C H N O L O G Y
L A B O R A T O R Y

2. Tradeoff
between AI
accuracy and
explainability
CONVOLUTIONAL
NEURAL NETS
(CNNS), PROVIDE NO
EXPLANATIONS
UNDERSTANDABLE
METHODS, SUCH AS
RULE-BASED, TEND
TO BE LESS
ACCURATE

3. COMXAI
A TOOL TO EXPLAIN
AI USING FAULT
LOCATION

4. Fault Location
• IF A COMBINATION OF FACTOR
VALUES OCCURS IN A PASSING
TEST, THEN CLEARLY IT DID NOT
TRIGGER THE FAILURE.
• COMBINATIONS THAT OCCUR
ONLY IN FAILING TESTS ARE
THOSE THAT ARE CONSIDERED IN
NARROWING DOWN THE SET OF
SUSPECT COMBINATIONS

5. COMXAI A COMBINATORIAL
TESTING TOOL
Explain why below Traits are REPTILE class

6. Single Value Fail
observations
NO SINGLE TRAIT IS SUFFICIENT
AS ALL ARE MORE THAN 44%
SHARED WITH REPTILES

7. Two Value Fail
observations
NO PAIR IS SUFFICIENT AS ALL
ARE MORE THAN 2% SHARED WITH
REPTILES

8. Three Value Fail observations
3-WAY COMBINATIONS WILL DISTINGUISH THIS ANIMAL FROM OTHER
ANIMAL TYPES IN THIS DATABASE.

9. CONVERT TO RULE

10. Final Output UNDERSTANDABLE RULE

11. Limitations
1. THOUSAND OF
COMBINATIONS
CAN MAKE RULE
SIZE VERY LARGE
2. OVERFITTING, IN
WHICH A LEARNING
MODEL
INCORPORATED
NOISE VARIATIONS
FROM THE
TRAINING DATA
WILL CREATE
PROBLEM FOR THIS
TOOL.

12. Future Work
• DETECT
OVERFITTING
• IMPROVED UI FOR
BETTER
VALIDATION OF AI
MODELS
• SECURITY FROM
MALWARE NEURAL
NET