Content deleted Content added
Added an image |
Filled in 0 bare reference(s) with reFill 2 |
||
Line 3:
{{artificial intelligence}}
'''Explainable AI''' ('''XAI'''), often overlapping with '''interpretable AI''', or '''explainable machine learning''' ('''XML'''), is a field of research within [[artificial intelligence]] (AI) that explores methods that provide humans with the ability of ''intellectual oversight'' over AI algorithms.<ref>{{Cite journal|last=Longo|first=Luca|display-authors=etal|date=2024 |title=Explainable Artificial Intelligence (XAI) 2.0: A manifesto of open challenges and interdisciplinary research directions|url=https://www.sciencedirect.com/science/article/pii/S1566253524000794 |journal=Information Fusion|volume=106|doi=10.1016/j.inffus.2024.102301}}</ref><ref>{{Cite journal |last=Mihály |first=Héder |date=2023 |title=Explainable AI: A Brief History of the Concept |url=https://ercim-news.ercim.eu/images/stories/EN134/EN134-web.pdf |journal=ERCIM News |issue=134 |pages=9–10}}</ref> The main focus is on the reasoning behind the decisions or predictions made by the AI algorithms,<ref>{{Cite journal |last1=Phillips |first1=P. Jonathon |last2=Hahn |first2=Carina A. |last3=Fontana |first3=Peter C. |last4=Yates |first4=Amy N. |last5=Greene |first5=Kristen |last6=Broniatowski |first6=David A. |last7=Przybocki |first7=Mark A. |date=2021-09-29 |title=Four Principles of Explainable Artificial Intelligence |url=https://doi.org/10.6028/NIST.IR.8312 |journal=NIST |doi=10.6028/nist.ir.8312}}</ref> to make them more understandable and transparent.<ref>{{Cite journal|last1=Vilone|first1=Giulia|last2=Longo|first2=Luca|title=Notions of explainability and evaluation approaches for explainable artificial intelligence|url=https://www.sciencedirect.com/science/article/pii/S1566253521001093|journal=Information Fusion|year=2021|volume= December 2021 - Volume 76 |pages=89–106|doi=10.1016/j.inffus.2021.05.009}}</ref> This addresses users' requirement to assess safety and scrutinize the automated decision making in applications.<ref name="auto">{{Cite journal |last1=Confalonieri |first1=Roberto |last2=Coba |first2=Ludovik |last3=Wagner |first3=Benedikt |last4=Besold |first4=Tarek R. |date=January 2021 |title=A historical perspective of explainable Artificial Intelligence |url=https://wires.onlinelibrary.wiley.com/doi/10.1002/widm.1391 |journal=WIREs Data Mining and Knowledge Discovery |language=en |volume=11 |issue=1 |doi=10.1002/widm.1391 |issn=1942-4787}}</ref> XAI counters the "[[black box]]" tendency of machine learning, where even the AI's designers cannot explain why it arrived at a specific decision.<ref>{{Cite journal |last=Castelvecchi |first=Davide |date=2016-10-06 |title=Can we open the black box of AI? |url=http://www.nature.com/articles/538020a |journal=Nature |language=en |volume=538 |issue=7623 |pages=20–23 |doi=10.1038/538020a |pmid=27708329 |bibcode=2016Natur.538...20C |s2cid=4465871 |issn=0028-0836}}</ref><ref name=guardian>{{cite news|last1=Sample|first1=Ian|title=Computer says no: why making AIs fair, accountable and transparent is crucial|url=https://www.theguardian.com/science/2017/nov/05/computer-says-no-why-making-ais-fair-accountable-and-transparent-is-crucial|access-date=30 January 2018|work=The Guardian |date=5 November 2017|language=en}}</ref>
XAI hopes to help users of AI-powered systems perform more effectively by improving their understanding of how those systems reason.<ref>{{Cite journal|last=Alizadeh|first=Fatemeh|date=2021|title=I Don't Know, Is AI Also Used in Airbags?: An Empirical Study of Folk Concepts and People's Expectations of Current and Future Artificial Intelligence|url=https://www.researchgate.net/publication/352638184|journal=Icom|volume=20 |issue=1 |pages=3–17 |doi=10.1515/icom-2021-0009|s2cid=233328352}}</ref> XAI may be an implementation of the social [[right to explanation]].<ref name=":0">{{Cite journal|last1=Edwards|first1=Lilian|last2=Veale|first2=Michael|date=2017|title=Slave to the Algorithm? Why a 'Right to an Explanation' Is Probably Not the Remedy You Are Looking For|journal=Duke Law and Technology Review|volume=16|pages=18|ssrn=2972855}}</ref> Even if there is no such legal right or regulatory requirement, XAI can improve the [[user experience]] of a product or service by helping end users trust that the AI is making good decisions.<ref>{{Cite web |last=Do Couto |first=Mark |date=February 22, 2024 |title=Entering the Age of Explainable AI |url=https://tdwi.org/Articles/2024/02/22/ADV-ALL-Entering-the-Age-of-Explainable-AI.aspx |access-date=2024-09-11 |website=TDWI}}</ref> XAI aims to explain what has been done, what is being done, and what will be done next, and to unveil which information these actions are based on.<ref name=":3">{{Cite journal|last1=Gunning|first1=D.|last2=Stefik|first2=M.|last3=Choi|first3=J.|last4=Miller|first4=T.|last5=Stumpf|first5=S.|last6=Yang|first6=G.-Z.|date=2019-12-18|title=XAI-Explainable artificial intelligence|url=https://openaccess.city.ac.uk/id/eprint/23405/|journal=Science Robotics|language=en|volume=4|issue=37|pages=eaay7120|doi=10.1126/scirobotics.aay7120|pmid=33137719|issn=2470-9476|doi-access=free}}</ref> This makes it possible to confirm existing knowledge, challenge existing knowledge, and generate new assumptions.<ref>{{Cite journal|last1=Rieg|first1=Thilo|last2=Frick|first2=Janek|last3=Baumgartl|first3=Hermann|last4=Buettner|first4=Ricardo|date=2020-12-17|title=Demonstration of the potential of white-box machine learning approaches to gain insights from cardiovascular disease electrocardiograms|journal=PLOS ONE|language=en|volume=15|issue=12|pages=e0243615|doi=10.1371/journal.pone.0243615|issn=1932-6203|pmc=7746264|pmid=33332440|bibcode=2020PLoSO..1543615R|doi-access=free}}</ref>
Line 47:
For images, [[Saliency map|saliency maps]] highlight the parts of an image that most influenced the result.<ref>{{Cite web |last=Sharma |first=Abhishek |date=2018-07-11 |title=What Are Saliency Maps In Deep Learning? |url=https://analyticsindiamag.com/what-are-saliency-maps-in-deep-learning/ |access-date=2024-07-10 |website=Analytics India Magazine |language=en-US}}</ref>
Systems that are expert or knowledge based are software systems that are made my experts. This system consists of a knowledge based encoding for the ___domain knowledge. This system is usually modeled as production rules, and someone uses this knowledge base which the user can question the system for knowledge. In expert systems, the language and explanations are understood with an explanation for the reasoning or a problem solving activity.<ref
However, these techniques are not very suitable for [[Language model|language models]] like [[Generative pre-trained transformer|generative pretrained transformers]]. Since these models generate language, they can provide an explanation, but which may not be reliable. Other techniques include attention analysis (examining how the model focuses on different parts of the input), probing methods (testing what information is captured in the model's representations), causal tracing (tracing the flow of information through the model) and circuit discovery (identifying specific subnetworks responsible for certain behaviors). Explainability research in this area overlaps significantly with interpretability and [[AI alignment|alignment]] research.<ref>{{cite arXiv |last1=Luo |first1=Haoyan |title=From Understanding to Utilization: A Survey on Explainability for Large Language Models |date=2024-02-21 |eprint=2401.12874 |last2=Specia |first2=Lucia|class=cs.CL }}</ref>
| |||