By: Shivani Choudhary
Pulling the Curtain on AI Safety
Artificial intelligence (AI) jumped to the forefront of the public consciousness in late 2022, with the rising ubiquity of large language models (LLMs) such as ChatGPT. As AI continues to integrate itself into our lives at home, school, and work, concerns about the safety of these models arise. To make the common user more comfortable around AI, lab groups at the University of Wisconsin–Madison delve into the issues of AI safety and explore solutions to these problems.
Sean (Xuefeng) Du, a computer science PhD student whose paper on AI hallucinations was recently accepted to the Conference on Neural Information Processing Systems, explains that AI safety has many dimensions.
“Safety has a really broad scope in the LLM area, including topics such as hallucination, toxicity, and bias detection and mitigation. Researchers are currently exploring different metrics to quantify the safety of LLMs,” Du elaborates.
Professor Shamya Karumbaiah, in the Department of Educational Psychology, works on AI safety in the realms of bias and inequity. In her lab, she explores the limitations of technical notions of bias, like theoretical mismatches, and student privacy tradeoffs. She examines alternative ways to define bias that are reliable in real-world contexts such as education.
“One of the projects we are exploring is how we could define bias as the difference in model performance based on the linguistic variation we see in [student] writing itself, as opposed to the race or gender of the student who wrote it. We are trying to see differences in mono- and multi-lingual writing,” Karumbaiah explains.
Karumbaiah focuses her research on education and the limitations of multilingual large language models (MLLMs) have on students of diverse backgrounds.
“For me, education is fascinating because it’s [part of] a real-world context with its own history of inequities that get brushed under the rug, and [AI] has been amplifying some of the same biases that have been problematic for so long,” Karumbaiah explains.
Another project involves the use of MLLMs in bilingual classrooms. Karumbaiah finds that “despite being multilingual, MLLMs are still limited, especially with translanguaging.”
Translanguaging is when a person switches between two or more languages fluidly to communicate an idea. MLLMs, despite being trained on more than one language, seem to struggle most with assessing student essays written in a mix of languages.
Karumbaiah’s students are working on improving such linguistic and cultural capabilities of MLLMs so it can be used to support monolingual teachers of multilingual students in K-12 classrooms. However, the success of this application is based on the integrity of the model itself.
Du explores these limitations through his work on “hallucinations, toxicity, and bias detection and mitigation.” Hallucinations occur when the model makes up information that is not real, and toxicity happens when the model’s response contains “swearing, biases, or harmful information.”
Du also describes the ways scientists actively look and check for bias in models. Sometimes, they use larger models to verify the output of smaller models.
“One of the projects we are exploring is how we could define bias as the difference in model performance based on the linguistic variation we see in [student] writing itself.”
– Shamya Karumbaiah
“Other papers also use a more statistical push,” Du explains. “They compare a model’s output against a large external dataset that is mostly true and not harmful. If the relation between [the dataset and model output] is tight, the generation is most likely to be clean and not harmful.”
While generative AI is not problem-free, both Karumbaiah and Du agree that it is a positive development and a powerful technology. Du argues that LLMs like ChatGPT help people solve real-world problems, like writing emails and drafting documents. Meanwhile, Karumbaiah thinks of AI as a “tool in the hands of humans,” useful in the context of education, so long as student safety and well-being is prioritized. On the whole, though, both agree that AI can enrich people’s lives.