Dr. Siddharth Mehrotra

Postdoc Researcher

Integrity Based Explanations for Fostering Appropriate Trust in AI Agents


Journal article


Siddharth Mehrotra, Carolina Centeio Jorge, Catholijn M. Jonker, Myrthe L. Tielman
ACM Transactions on Interactive Intelligent Systems, Association for Computing Machinery, New York, NY, USA, 2023 Jul


Full-text
Cite

Cite

APA   Click to copy
Mehrotra, S., Jorge, C. C., Jonker, C. M., & Tielman, M. L. (2023). Integrity Based Explanations for Fostering Appropriate Trust in AI Agents. ACM Transactions on Interactive Intelligent Systems. https://doi.org/10.1145/3610578


Chicago/Turabian   Click to copy
Mehrotra, Siddharth, Carolina Centeio Jorge, Catholijn M. Jonker, and Myrthe L. Tielman. “Integrity Based Explanations for Fostering Appropriate Trust in AI Agents.” ACM Transactions on Interactive Intelligent Systems (July 2023).


MLA   Click to copy
Mehrotra, Siddharth, et al. “Integrity Based Explanations for Fostering Appropriate Trust in AI Agents.” ACM Transactions on Interactive Intelligent Systems, Association for Computing Machinery, July 2023, doi:10.1145/3610578.


BibTeX   Click to copy

@article{mehrotra2023a,
  title = {Integrity Based Explanations for Fostering Appropriate Trust in AI Agents},
  year = {2023},
  month = jul,
  address = {New York, NY, USA},
  journal = {ACM Transactions on Interactive Intelligent Systems},
  publisher = {Association for Computing Machinery},
  doi = {10.1145/3610578},
  author = {Mehrotra, Siddharth and Jorge, Carolina Centeio and Jonker, Catholijn M. and Tielman, Myrthe L.},
  month_numeric = {7}
}

Abstract:
Appropriate trust is an important component of the interaction between people and AI systems, in that ‘inappropriate’ trust can cause disuse, misuse or abuse of AI. To foster appropriate trust in AI, we need to understand how AI systems can elicit appropriate levels of trust from their users. Out of the aspects that influence trust, this paper focuses on the effect of showing integrity. In particular, this paper presents a study of how different integrity-based explanations made by an AI agent affect the appropriateness of trust of a human in that agent. To explore this, (1) we provide a formal definition to measure appropriate trust, (2) present a between-subject user study with 160 participants who collaborated with an AI agent in such a task. In the study, the AI agent assisted its human partner in estimating calories on a food plate by expressing its integrity through explanations focusing on either honesty, transparency or fairness. Our results show that (a) an agent who displays its integrity by being explicit about potential biases in data or algorithms achieved appropriate trust more often compared to being honest about capability or transparent about the decision-making process, and (b) subjective trust builds up and recovers better with honesty-like integrity explanations. Our results contribute to the design of agent-based AI systems that guide humans to appropriately trust them, a formal method to measure appropriate trust, and how to support humans in calibrating their trust in AI.

Share



Follow this website


You need to create an Owlstown account to follow this website.


Sign up

Already an Owlstown member?

Log in