AIM seed grant to Naomi and collaborators

Investigating the cognitive underpinnings of social biases in LLMs.

Congratulations to Naomi, who, with Rachel Rudinger and Sarah Wiegreffe, has been awarded an AIM seed grant from the Artificial Intelligence Interdisciplinary Institute at Maryland, to support "Investigating the cognitive underpinnings of social biases in LLMs." Building on work by UMD computer science students Hillary Owusu and Shramay Palta, recently presented at ACL, the project probes biases displayed by large language models, using tools from cognitive psychology, as abstracted below.

• Hillary Owusu and Naomi Feldman / Anchoring Depends on Confidence and Post-Training in Language Models
• Shramay Palta, Peter Rankel, Sarah Wiegreffe and Rachel Rudinger / Arguments that Alter Minds: LLM Rationales Sway Human (and LLM) Notions of Plausibility

Large language models (LLMs) are increasingly used to support decision-making. Unfortunately LLMs, like humans, are not rational decision makers: they exhibit cognitive biases, paying attention to things they should ignore and ignoring things they should pay attention to. Stereotyped reasoning is a special case of irrational decision making, in which LLMs pay attention to social attributes they should ignore, resulting in injustice. In this project we use what we know about human irrationality to study irrationality in LLMs.  We measure LLMs' cognitive biases and social biases and ask what components of the model lead them to show these biases, with the long-term goal of reducing LLMs' irrational behavior.