"Chinese models refuse to answer significantly more of the questions than the American models. (DeepSeek refused 36 percent of the questions, while Baidu's Ernie Bot refused 32 percent; OpenAI's GPT and Meta's Llama had refusal rates lower than 3 percent.) In cases where they didn't outright refuse to answer, the Chinese models also gave shorter answers and more inaccurate information than their American counterparts did."
"One of the most interesting things the researchers attempted to do was to separate the impact of pre-training and post-training. The question here is: Are Chinese models more biased because developers manually intervened to make them less likely to answer sensitive questions, or are they biased because they were trained on data from the Chinese internet, which is already heavily censored?"
"Given that the Chinese internet has already been censored for all these decades, there's a lot of missing data."
Stanford and Princeton researchers conducted a comparative study of Chinese and American large language models by submitting 145 politically sensitive questions to nine models total, repeating the experiment over 100 times. Chinese models demonstrated substantially higher refusal rates, with DeepSeek refusing 36% of questions and Baidu's Ernie Bot refusing 32%, while OpenAI's GPT and Meta's Llama refused fewer than 3%. Beyond outright refusals, Chinese models provided shorter and less accurate responses. Researchers investigated whether bias stemmed from manual post-training interventions by developers or from pre-training on already-censored Chinese internet data. The heavily censored Chinese internet environment has created significant data gaps affecting model training.
#ai-censorship #chinese-language-models #political-content-filtering #ai-bias-and-training-data #internet-censorship
Read at WIRED
Unable to calculate read time
Collection
[
|
...
]