To test an AI explanation, change the evidence it cites

EDCT-Bench, a preprint posted on 16 September 2026 by Sihao Ding and colleagues, edits visual evidence cited in model explanations and tests the resulting responses. The authors report inconsistencies across the evaluated models.

An answer can remain valid if other evidence supports it, but its explanation must account for the edit. Passing does not prove internal causal faithfulness. The preliminary training analysis does not establish improved held-out faithfulness.

Sources: Ding and colleagues, EDCT-Bench, preprint, 16 September 2026

The June GRACE preprint by Hoang Pham, Dong Le and Anh Tuan Luu examines individual reasoning steps against supplied context. It distinguishes deduction errors from failures to remain grounded in the source. The work reinforces an important evaluation distinction: a correct final answer can coexist with unsupported intermediate reasoning.

These studies ask different questions. GRACE assesses textual reasoning against context; EDCT intervenes on visual evidence cited by a model. Their results should not be combined into a single accuracy number or used as direct evidence about Ackren.

Sources: Pham, Le and Luu, GRACE, preprint, 15 June 2026 · Ding and colleagues, EDCT-Bench, preprint, 16 September 2026

For a developer, the useful question is practical: what observation would reveal that an explanation is wrong? Reading a plausible paragraph is a weak test if the paragraph cannot be challenged. Changing a cited premise, keeping unrelated evidence fixed and checking the resulting answer gives a more specific test.

For Ackren, an analogous experiment would revise a supplied fact or withdraw a rule, then inspect the answer and its dependencies. That is our proposed application of this research, not a result from either paper. An execution log makes such an experiment inspectable, but does not excuse errors in interpretation or in the log itself.

Sources: Ding and colleagues, EDCT-Bench, preprint, 16 September 2026 · Pham, Le and Luu, GRACE, preprint, 15 June 2026 · Recorded prototype example and development checks