Humans overrely on overconfident language models, across languages

About this paper:

• Neil Rathi, Dan Jurafsky, and Kaitlyn Zhou published this conference paper at the Conference on Language Modeling (COLM 2025) in 2025.
• The study examines how LLMs express certainty and uncertainty in English, French, German, Japanese, and Mandarin, then tests how bilingual users rely on those expressions.
• Its central implication is that aligning confidence language with accuracy is not sufficient for safety: evaluation must also account for how the same expression is pragmatically interpreted in each language.

Introduction

When an LLM begins an answer with “definitely,” “probably,” or “I think,” the wording may appear incidental. In practice, it can influence whether a user accepts the response. The paper calls such phrases epistemic markers: linguistic devices through which a speaker communicates how certain they are about what they say.

Three concepts must first be separated.

1. Accuracy asks whether the model’s answer is correct.
2. Linguistic calibration asks whether the strength of the model’s expressed certainty corresponds to its actual accuracy.
3. Human reliance asks whether a user is willing to accept the answer and act on it.

A generally accurate model can still say “definitely” when it is wrong or hedge when it is right. More importantly, even a technically well-calibrated expression may fail at the interaction level if people do not interpret it as intended. Estimating uncertainty inside a model is only one part of the problem; communicating uncertainty to a human listener is another.

Most previous work has concentrated on English. Yet languages do not use uncertainty expressions in identical ways. English and French speakers commonly use moderate expressions of confidence, while hedges and indirect formulations are more frequent as a conversational baseline in Japanese. Stronger expressions are relatively common in German and Mandarin. A dictionary may align Japanese to omoimasu with English “I think,” but that does not guarantee that listeners treat the two phrases as equally strong warnings of uncertainty.

The paper begins from this mismatch. Its question is not only whether multilingual models are overconfident, but how four stages connect:

language-specific norms → the model’s distribution of confidence expressions → the listener’s interpretation → the risk of relying on a wrong answer

Replacing any one of these stages with an English-language result can distort the estimated risk. The authors therefore combine a large-scale generation analysis with a separate bilingual participant study.

Main Discussion

Dividing epistemic markers into three levels

The authors classify model expressions into three categories:

• Weak certainty: expressions such as “it could be,” “I think,” and “maybe,” which communicate substantial uncertainty
• Moderate certainty: expressions such as “it is likely” or “it is most likely,” which indicate confidence without a categorical commitment
• Strong certainty: expressions such as “definitely,” “I am 100% sure,” and “it must be,” which communicate near-absolute certainty

These categories were not built by mechanically translating an English lexicon. For each language, Prolific participants generated phrases expressing certainty and uncertainty, and the most frequent phrases became few-shot examples. Proficient speakers then rated the perceived certainty of GPT-4o outputs on a seven-point scale. Ratings 1–2 became weak, 3–5 moderate, and 6–7 strong.

This procedure matters because the study is interested in perceived pragmatic force rather than strings alone. Still, Fleiss’ κ among annotators was only 0.45 for English, 0.44 for French, 0.38 for German, 0.37 for Japanese, and 0.26 for Mandarin. Mandarin contained particularly few weak expressions, making the moderate–strong boundary less stable. The numbers themselves show that confidence strength is an interpreted variable rather than a perfectly objective attribute.

Experiment 1: How overconfident are multilingual models?

The first experiment uses multiple-choice questions from all 57 subjects in MMLU. For Japanese, the authors use the machine-translated and manually checked JMMLU subset. The same set of 7,494 questions was translated from English into French, German, and Mandarin with the Google Translate API. The evaluated systems were GPT-4o and the 70B and 8B Instruct versions of Llama 3.1. Each prompt contained ten demonstrations—five using uncertainty expressions and five using certainty expressions in randomized order—and the process was repeated across three prompt sets.

Mean MMLU accuracy was as follows.

Model	English	French	German	Japanese	Mandarin
GPT-4o	80.96	76.32	77.83	77.08	75.75
Llama 3.1 70B	66.69	57.51	34.47	45.90	55.65
Llama 3.1 8B	59.44	37.17	23.05	31.29	42.17

English performance is highest for every model. The language gap is especially large for Llama. This baseline matters when interpreting overconfidence: a higher error rate among strongly worded answers may reflect both confidence miscalibration and a lower underlying ability to solve questions in that language.

The paper defines overconfidence rate as

\[P(\text{incorrect}\mid\text{strong expression}).\]

It is the proportion of strongly expressed answers that are wrong. Averaged across languages, the rate is 15.22% for GPT-4o, 39.15% for Llama 3.1 70B, and 49.04% for Llama 3.1 8B. Even for the strongest model, roughly one in every six or seven strongly worded answers is incorrect. GPT-4o becomes somewhat more accurate when it uses strong rather than weak or moderate markers. For both Llama models, however, accuracy barely differs across marker categories. Their verbal confidence strength provides little useful information about correctness.

Non-English overconfidence rates are consistently higher than the English rate. For GPT-4o, 11.26% of strongly expressed English answers are wrong, compared with roughly 18% in Mandarin. This should not be read as proof that Mandarin expressions cause overconfidence. MMLU accuracy, translation quality, and instruction-following success also differ by language. The safer conclusion is that strong wording is not a dependable safety signal in non-English use.

Figures 1 and 2: Accuracy and expression frequency are different questions

Figure 1 plots MMLU accuracy by weak, moderate, and strong markers for every model-language combination. The important feature is not a single bar but whether accuracy rises with confidence strength inside each panel. It rises to some extent for GPT-4o, while the three bars are broadly similar for Llama 3.1. Selecting occasions when a Llama model says it is certain therefore does not reliably select more accurate answers.

Figure 2 reorganizes the same generations by the frequency of each marker category. In GPT-4o, 96% of English and 91% of French responses contain moderate or strong markers. Weak markers form the largest Japanese category at 59%. Strong markers, by contrast, account for 42% of English, 53% of German, and 80% of Mandarin generations. The models reproduce patterns reported in linguistic research: more hedging in Japanese and stronger formulations in German and Mandarin.

This creates the paper’s central tension. A model can learn language-specific speaking norms without becoming well calibrated. More Japanese hedges do not necessarily track a higher probability of error. Similarly, the conventional frequency of strong wording in German or Mandarin does not turn that wording into evidence of accuracy. Cultural naturalness in expression and truth-sensitive calibration are separate evaluation axes.

Experiment 2: Does the same person rely differently across languages?

The second experiment measures human choice rather than model output. The authors recruited 45 English–French, 45 English–German, and 45 English–Japanese bilingual participants. All self-reported at least C1 proficiency in both languages. Mandarin was excluded because models produced too few weak Mandarin markers to construct a comparable three-category task.

Participants saw difficult geography trivia questions and the beginning of an AI answer. An item might show “What is the capital of Kiribati?” followed by a marker such as “I think it is…”. The participant chose whether to rely on the model or look up the answer independently. Each person completed 30 English and 30 target-language items. Language order and item order were randomized. Within each language, the 30 stimuli comprised five weak, fifteen moderate, five strong, and five plain formulations without an epistemic marker.

The within-person bilingual design is a major strength. Rather than comparing unrelated national samples, it asks how the same bilingual person’s decision changes when the language changes.

Reliance on strong expressions was 66.34% in English, 54.54% in French, 61.90% in German, and 78.91% in Japanese, averaging 65.42%. Figure 3 shows little English–French difference for weak and moderate markers, while German strong markers and plain formulations elicit less reliance than their English counterparts. Japanese differs in the opposite direction: weak, moderate, and strong epistemic markers all elicit more reliance than they do in English.

The Japanese result reverses the intuition that more hedging must make a system safer. If indirect or qualified wording is common in ordinary Japanese interaction, a listener may discount it as politeness or a discourse norm rather than interpret it literally as substantial uncertainty. A model can therefore generate “I think” more often without producing a proportionate increase in user caution. The frequency of a hedge and its perceived warning effect can move in opposite directions.

Combining the two experiments into overreliance risk

The authors combine the generation experiment’s overconfidence rate with the human experiment’s reliance on strong markers to define overreliance risk:

\[R_{m,l}=P(\text{rely}\mid\text{strong},l)\times P(\text{incorrect}\mid\text{strong},m,l),\]

where $m$ denotes a model and $l$ a language. The resulting percentages are:

Model	English	French	German	Japanese
GPT-4o	7.47	8.71	9.55	11.72
Llama 3.1 70B	23.64	23.36	41.10	41.93
Llama 3.1 8B	25.20	34.19	47.53	54.95

The average is close to 10% even for GPT-4o and roughly 40% for Llama 3.1 8B. Japanese has the highest value for all three models. Its higher reliance rate combines with model overconfidence, so the apparent advantage of generating more hedges does not reduce the resulting risk.

This quantity requires careful interpretation. It is an expected-risk proxy created by multiplying probabilities estimated in two different experiments, not the observed rate of real interactions in which someone accepted a wrong answer. The reliance task showed only the beginning of an answer, not its completion or correctness, and the calculation assumes that language-level reliance rates can be combined with model-language overconfidence rates. Thus, 11.72% does not mean that Japanese users accept exactly 11.72% of all GPT-4o answers incorrectly. The metric is more useful for comparison: it reveals an interaction that would be missed by considering either model accuracy or user trust alone.

What the evidence establishes—and what it does not

The study has clear strengths. Native or highly proficient speakers supplied and judged language-specific expressions, the authors connect model generation to human behavior, and the bilingual within-person design gives the language comparison a stronger foundation. Several limitations nevertheless constrain generalization.

• The model and language scope is narrow. The study evaluates GPT-4o and two sizes of Llama 3.1, all released in 2024. The human study covers only English, French, German, and Japanese. The ethics statement explicitly notes that only high-resource languages were examined.
• Translation conditions are not uniform. JMMLU was machine-translated and manually checked, whereas French, German, and Mandarin questions were translated with Google Translate. Language differences may partially reflect translation difficulty or cultural fit.
• The overconfidence measure is entangled with baseline accuracy. $P(\text{incorrect}\mid\text{strong})$ is practically meaningful, but large language-level accuracy gaps mean that calibration and general task ability both contribute to the value. It is not equivalent to a standard calibration error or Brier score.
• The prompts explicitly elicit confidence language. This is not a measurement of how often unprompted models spontaneously use epistemic markers. The authors also note that people may rely heavily on plain responses without any marker.
• Removing prompt failures creates selection risk. Appendix Table 5 reports template-following success of only 29.20% for GPT-4o in English and 1.11% for Llama 3.1 8B in Mandarin. All failures are excluded. Results in low-success conditions therefore characterize a highly selected subset rather than the model’s full output distribution.
• Confidence labels contain measurement error. Annotator agreement is at best moderate, and Llama generations were labeled by language-specific classifiers with a mean held-out accuracy of 78.13%. Classification errors can propagate into estimated marker distributions and overconfidence rates.
• Reliance is only one simplified behavioral decision. Participants see neither a complete answer nor evidence and choose between only “rely” and “look it up.” Domain expertise, time pressure, citations, explanation quality, and consequences would all affect reliance in actual use.

These limitations do not erase the finding; they define its scope. The study does not establish a universal risk percentage for every culture and application. It does show that importing an English relationship between confidence wording and reliance into another language is unsafe.

Conclusion

The most important lesson is that “expressing more uncertainty makes a system safer” is not automatically true. The evaluated models do produce more hedges in Japanese, but Japanese participants treat those expressions as weaker warnings than they treat translated English counterparts. The more naturally a model reproduces a language’s pragmatic conventions, the more that naturalness may sometimes obscure the possibility of error.

Multilingual calibration is therefore not a translation problem. Mapping English “I think” at confidence 0.6 to a phrase in another language is insufficient. At least three layers must be evaluated together:

1. the model’s actual task accuracy in the target language,
2. the confidence expressions it generates at different accuracy levels, and
3. how users of that language interpret those expressions and change their behavior.

The paper’s contribution is to include the third layer. It moves AI safety evaluation beyond output text and toward the effect of that text on human judgment. Whether a model stated uncertainty may matter less than whether the user actually perceived the statement as uncertainty.

In practical systems, natural-language confidence should not bear the full burden of risk communication. Verifiable sources, evidence links, refusal conditions, and language-specific user studies should accompany it. In high-stakes domains such as medicine, law, and education, checking translation quality and target-language accuracy is not enough; designers must also test whether the same interface produces appropriately cautious behavior across languages.

The reported values should be treated as a starting point rather than a finished multilingual safety benchmark. Future work should expand the model and language set, test interaction with complete answers and evidence, and include low-resource languages and more diverse user groups. Even with those qualifications, the paper’s central claim remains persuasive: model confidence is expressed in language, but its risk is determined where language meets human interpretation.