A Definition of Good Explanations and the Challenges Explaining LLM Outputs

This arxiv paper proposes a formal definition of what constitutes a good explanation, drawing on counterfactual reasoning while also accounting for the prior beliefs of the person receiving the explanation. The authors apply this framework to AI explainability and argue it illuminates why LLM outputs are particularly resistant to satisfying explanation — a foundational problem for any deployment context where accountability matters. The work is philosophical in orientation but has practical consequences for how we think about transparency requirements.