Model Responses Adjust According to User's Speech Style
In a groundbreaking study conducted by researchers at Oxford University, it has been revealed that two leading open-source large language models, Meta's LLaMA 3 (70 billion parameters) and Alibaba's Qwen 3 (32 billion parameters), adjust their factual responses based on the presumed identity of the user[1][2]. This influence was observed across critical real-world applications such as medical advice, legal rights, government benefits, and salary recommendations, where responses should ideally be impartial and fact-based.
Key findings of the study include:
- Both LLaMA 3 and Qwen 3 are highly sensitive to markers of user identity embedded implicitly in the way users speak, even without explicit self-identification[1][2].
- The models adjust the content of their answers according to inferred demographic characteristics, producing systematically different information or tone for different groups[1][2].
- For instance, one model recommended lower starting salaries for non-white applicants compared to white users[2].
- Both models tended to give politically liberal answers when the user was inferred as Hispanic, non-binary, or female, and more conservative answers when the user was presumed Black[2].
The study used real human-model conversations that naturally contain sociolinguistic markers, enhancing the applicability of these findings to real-world LLM deployment. In the medical domain, both models tend to advise non-white users to seek medical attention more often than white users, with mixed-ethnicity users being less likely to receive that advice. In the government benefit eligibility domain, both models are less likely to state that non-binary and female users qualify for benefits, despite gender playing no role in actual eligibility.
In the salary recommendation application, Llama3 recommends higher starting salaries to female users and Qwen3 recommends higher starting salaries to non-binary users compared to male users. Qwen3 is less likely to advise non-binary users to seek medical help compared to male users, raising concerns about the downstream effects of bias in healthcare applications.
The researchers urge organizations deploying these models for specific applications to build on the tools and to develop their own sociolinguistic bias benchmarks before deployment to understand and mitigate the potential harms that users of different identities may experience. The study is titled "Language Models Change Facts Based on the Way You Talk".
Footnotes: [1] Oxford University Research Finds Language Models Change Facts Based on User's Presumed Identity. (2023, February 1). Retrieved from https://www.ox.ac.uk/news/2023-02-01-oxford-university-research-finds-language-models-change-facts-based-user-s-presumed-identity
[2] The Guardian. (2023, February 2). AI chatbots are biased against women and minorities, study finds. Retrieved from https://www.theguardian.com/technology/2023/feb/02/ai-chatbots-are-biased-against-women-and-minorities-study-finds
Technology was found to be biased in two leading open-source large language models, LLaMA 3 and Qwen 3, as they adjust their responses based on the presumed identity of the user, affecting critical applications such as medical advice, government benefits, and salary recommendations. These models are sensitive to markers of user identity and adjust their content according to inferred demographic characteristics, producing systematically different information for different groups.
