Elinor Poole-Dayan

I’m currently doing my master’s at the MIT Center for Constructive Communication at the Media Lab supervised by Deb Roy. I did my undergrad at McGill University in Montreal, Canada and did research with Siva Reddy at Mila Quebec.

I’m passionate about equitable, pluralistic, safe AI for the benefit of all. My research interests include evaluation of LLMs for fairness and safety, pluralistic alignment, and developing equitable human-centered AI.

Currently, my master’s thesis focuses on leveraging large language models to understand diverse, evolving perspectives in dynamic deliberation and augmenting Citizens’ Assemblies through identifying bridging statements, fairly representing the collective stance, and amplifying underheard voices.

My background is in mathematics, computer science, and natural language processing (NLP) with a focus on addressing harmful biases in language models. I also have experience in vision-and-language models such as Stable Diffusion. Beyond my academic pursuits, I am enthusiastic about linguistics, playing ultimate frisbee, and nurturing a growing collection of house plants.

Feel free to reach out if you’d like to chat about any of the above!

news

Oct 2024	I will be attending NeurIPS this year! Please reach out if you’d like to meet up :)
Jul 2024	Presented my work at the International Conference for Computational Social Science (IC2S2) in Philadelphia, Pennsylvania.

selected publications

NeurIPS 2024

LLM Targeted Underperformance Disproportionately Impacts Vulnerable Users

Elinor Poole-Dayan, Deb Roy, and Jad Kabbara

2024
EMNLP 2024

On the Relationship between Truth and Political Bias in Language Models

Suyash Fulay, William Brannon, Shrestha Mohanty, Cassandra Overney, Elinor Poole-Dayan, Deb Roy, and Jad Kabbara

In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Nov 2024

Abs

Language model alignment research often attempts to ensure that models are not only helpful and harmless, but also truthful and unbiased. However, optimizing these objectives simultaneously can obscure how improving one aspect might impact the others. In this work, we focus on analyzing the relationship between two concepts essential in both language model alignment and political science: truthfulness and political bias. We train reward models on various popular truthfulness datasets and subsequently evaluate their political bias. Our findings reveal that optimizing reward models for truthfulness on these datasets tends to result in a left-leaning political bias. We also find that existing open-source reward models (i.e., those trained on standard human preference datasets) already show a similar bias and that the bias is larger for larger models. These results raise important questions about the datasets used to represent truthfulness, potential limitations of aligning models to be both truthful and politically unbiased, and what language models capture about the relationship between truth and politics.
ACL 2022

An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models

Nicholas Meade, Elinor Poole-Dayan, and Siva Reddy

In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), May 2022

Abs DOI

Recent work has shown pre-trained language models capture social biases from the large amounts of text they are trained on. This has attracted attention to developing techniques that mitigate such biases. In this work, we perform an empirical survey of five recently proposed bias mitigation techniques: Counterfactual Data Augmentation (CDA), Dropout, Iterative Nullspace Projection, Self-Debias, and SentenceDebias. We quantify the effectiveness of each technique using three intrinsic bias benchmarks while also measuring the impact of these techniques on a model’s language modeling ability, as well as its performance on downstream NLU tasks. We experimentally find that: (1) Self-Debias is the strongest debiasing technique, obtaining improved scores on all bias benchmarks; (2) Current debiasing techniques perform less consistently when mitigating non-gender biases; And (3) improvements on bias benchmarks such as StereoSet and CrowS-Pairs by using debiasing strategies are often accompanied by a decrease in language modeling ability, making it difficult to determine whether the bias mitigation was effective.