Researchers benchmark gender bias in educational feedback from six large models using 600 essays

How to detect whether LLM formative feedback on student essays exhibits systematic semantic differences due to implicit or explicit gender cues.

• Scenario: The authors state that automated writing feedback is a core direction in learning analytics, and teachers are using LLMs to generate timely, personalized feedback. The authors cite a U.S. survey, stating that a quarter of teachers have used GenAI to evaluate student work, and argue that gender bias may amplify stereotypical inequalities through educational practices; teachers often pass AI feedback to students with little modification.
• Existing gaps: The authors state that there is no mature educational LLM bias benchmark to date. WEAT and its extensions are task-independent and template-based, measuring static representations rather than generative behavior; existing approaches often only add explicit gender to prompts, lacking counterfactuals. The authors state that Borchers and Shou (2025) used sentence embedding cosine similarity for counterfactual evaluation but did not control for output randomness under the same input.
• Hypothesis: The authors define gender bias as systematic preference or prejudice toward one gender. When inputs are identical except for gender, unbiased LLM feedback should not show systematic variance. The authors argue that gender themes in essays, as well as prompt wording, readability, and writing style, may bias outputs toward certain learners.
• Framework proposed in this paper: Use 600 real student essays from AES 2.0, construct counterfactuals via gender word replacement and author background, compute sentence embedding distances for feedback from six LLMs, then perform permutation tests, dimensionality reduction, and qualitative text analysis to audit gender differences in formative feedback.

LLM gender bias in education

• Rowe et al. (2025)'s EuroGEST—across 30 European languages, LLMs associate women with beautiful, empathetic, and tidy, and men with leadership, strength, and professionalism. Choi and Nixon (2025), in collaborative scenarios with identical linguistic content, report that GPT-4o stereotypes men as agentic leaders and women as communal leaders.
• Weissburg et al. (2025)—when educational topics are fixed, LLMs still give different explanations based on demographic attributes such as race, gender, and income. Bai et al. (2024) report that GPT-4 still has implicit gender associations after explicit guardrails. The paper uses these works to support "the latest models still have gender bias," without numerical comparison to this paper.
• Warr et al. (2024), Warr and Oster (2025)—the paper paraphrases their results: GPT-4.0 and others adjust tone and perceived expertise by student group; advantaged learners receive more encouragement and open-ended comments, while marginalized learners receive more directive, compliance-oriented feedback. The authors also cite Cantini et al. (2024, 2025), stating that guardrails are stronger for categories such as sexual orientation, while gender bias remains insufficiently mitigated.

Measurement paradigms and embedding association tests

• Kizilcec and Lee (2022)—three paradigms for educational algorithm bias: similarity, causal, and statistical. The authors argue that causal fairness, operationalized as counterfactual consistency, is more suitable for LLMs where demographic cues can appear both explicitly and implicitly.
• WEAT (Caliskan et al., 2017), SEAT (May et al., 2019), CEAT (Guo and Caliskan, 2021) quantify bias using semantic associations between gender words and stereotyped attributes; StereoSet (Nadeem et al., 2021) evaluates stereotypical tendencies on minimal sentence pairs or completion tasks. The authors state that WEAT and its extensions are task-independent and template-based, measuring static representations rather than generative behavior.
• Blodgett et al. (2020) criticize bias measurement in NLP; Gonen and Goldberg (2019) point out that debiasing can mask rather than eliminate systematic gender bias in word embeddings. After citing these critiques, the authors state that educational evaluation still relies heavily on templates or synthetic probes.

Counterfactual and causal analysis

• Borchers and Shou (2025)—in the same educational setting, use sentence embedding cosine similarity to analyze model behavioral shifts on contextual variables, for benchmarking tutoring adaptability. The authors state that it does not control for output randomness and has not yet been used for gender bias in educational feedback.
• Vig et al. (2020) use causal mediation analysis to investigate gender bias in language models. Verma and Rubin (2018) explain fairness definitions; the causal fairness paradigm adopted in the methods section cites this literature.
• The paper also mentions that linear probing provides only correlational evidence, while counterfactual interventions can observe whether outputs change with gender cues. This is the authors' positioning of the measurement approach, not the cited authors' original words.

Baseline and data

• The experiment has no external bias detector as a control. The robustness baseline is M′: re-prompting the unmodified original essay and using the embedding distance between the two feedbacks to represent randomness and content differences.
• Essays come from AES 2.0, about 24,000 classroom argumentative essays; this study filters out a sample of 600 essays containing gender-related vocabulary. The substitution word list comes from Zhao et al. (2018, 2019), with 192 pairs total.
• The authors state that educational LLM applications lack an established gender bias evaluation baseline. This paper's positioning is: using real student essays rather than template sentences, simultaneously manipulating implicit vocabulary and explicit author background, and using permutation testing to separate output randomness from gender-related shifts.

The authors intervene according to causal fairness: only gender cues are changed, all other inputs remain the same, and then they check whether feedback embeddings systematically deviate. The measurement tool is text-embedding-3-large, mapping each feedback to 3072 dimensions; cosine distance is 1−cos(θ), i.e., 1 minus the cosine of the angle between two vectors, used to represent semantic drift in direction. Euclidean distance represents absolute displacement. Larger distances indicate greater divergence in counterfactual feedback.

Two types of cues and robustness control

• Implicit: Only gender-associated words in the essay are replaced; the prompt does not specify author gender. M vs M-F consists of 300 essays originally containing male-associated words and their female-replaced versions; F vs F-M consists of 300 essays with the reverse replacement. The word list has 192 pairs, mainly titles, pronouns, and occupational words; the authors state that all replacements were manually checked to ensure semantic coherence and grammatical correctness.
• Explicit: Each of the 600 essays is paired with three author backgrounds, with the same feedback request. The male condition includes a boys' school, male name, and male pronouns; the female condition includes a girls' school, female name, and female pronouns; the neutral condition includes a mixed-gender school, neutral name, and they. Table 4 provides the full prompt sentences, not repeated here.
• Robustness group M′: 300 original essays without gender manipulation are submitted again. The authors use it to account for both generation randomness and essay content differences, serving as a control for the main experiment distances.

Models and generation settings

• The six models are GPT-5 mini, GPT-4o mini, DeepSeek-R1, DeepSeek-R1-Qwen, Gemini 2.5 Pro, and Llama-3-8B. The authors state the selection is based on API availability, as well as small distilled models, medium open-source models, and proprietary models in educational settings. Temperature is set to default for all, corresponding to web-style use without tuning.
• Each model generates 300×2×2 implicit, 600×3 explicit, and 300×2 baseline responses. The paper states a total of 3,000 without explaining how this number corresponds to the multipliers. The pipeline is adapted from Borchers and Shou: construct inputs, query models, embed and analyze. Figure 2 shows the prompt template; the full template is not provided in the text.

Permutation tests

• The null hypothesis is that gender cues do not change the feedback distribution; the alternative is that gender cues systematically shift feedback. The observed statistic Tobs is the mean of paired embedding distances.
• The null distribution merges the two groups, randomly shuffles labels, then splits into equal-sized pseudo-groups, repeated B=5000 times. The two-tailed p is the proportion of permutation statistics deviating from the permutation mean at least as much as the observed value; p<0.05 rejects the null. Cohen's d uses 0.2, 0.5, and 0.8 for small, medium, and large effects.
• The authors state that cosine distance is symmetric, so labels cannot simply be swapped between empirical and counterfactual sentences. They also computed Mahalanobis distances for M, F, and N combinations, stating consistency with the current metrics, without providing values.

Qualitative interpretation and text metrics

• t-SNE projects high-dimensional embeddings to two dimensions. The authors interpret clusters separated along gender group boundaries as gender cues shifting the semantic space of feedback, and map vectors with large deviations back to the original text.
• Two researchers independently conduct reflexive thematic analysis. Interpretation categories are linguistic bias (word choice, discourse markers, syntax) and pedagogical bias (autonomy, competence, growth). Quantitative text statistics sample 50 instances from separated clusters.
• Linguistic metrics include academic word list proportion, mean concreteness norms, relative frequencies of first and second person, and sentence type proportions. Pedagogical metrics define a support score as the difference between the relative frequency of autonomy-supportive expressions and controlling expressions divided by total word count. The paper reports frequencies per group, not the score itself.

实验设置

• 被测模型：GPT-5 mini、GPT-4o mini、DeepSeek-R1、DeepSeek-R1-Qwen、Gemini 2.5 Pro、Llama-3-8B。Table 5写作DeepSeek r1、DeepSeek qwen、Llama 3 8B。嵌入模型为text-embedding-3-large，3072维，不是偏见评测对象。
• 数据：AES 2.0约24,000篇议论文中筛出600篇，性别相关词占词数0.84%。隐式为两组各300篇的双向替换；显式为600篇×男/女/中性；基线为300篇再提示。
• 对照：无外部检测器。对照是未操纵作文的M与M′。
• 指标：配对反馈的1-cosine similarity与Euclidean distance；置换检验B=5000的双尾p和Cohen's d。p<0.05拒绝“性别线索不改变反馈”。
• 判定：无LLM裁判。定性由两名研究者独立完成；文本统计来自50个样本。论文未报告编码者间一致性。
• 生成：温度默认。论文未报告top-p、随机种子，也未报告同一提示在主条件下的重复生成次数。

主结果

下表只摘Table 5的三列。隐式M vs M-F的余弦与欧氏p在六种模型上均为<.001。Tobs−Tperm是相对置换均值的超额，不是绝对余弦距离。

Swipe horizontally to read the table

模型	隐式M vs M-F余弦超额	隐式M vs M-F余弦d	显式M vs F欧氏d
GPT-5 mini	0.0094	0.257	0.195
GPT-4o mini	0.0091	0.283	0.580
DeepSeek-R1	0.0084	0.294	0.000
DeepSeek-R1-Qwen	0.0076	0.281	0.000
Gemini 2.5 Pro	0.0107	0.288	0.000
Llama-3-8B	0.0075	0.269	1.116

• 作者称隐式男→女替换在全部模型上应拒绝原假设，效应为小到中等（正文写0.2<d<0.5）；女→男的余弦检验未拒绝原假设。Table 5中F vs F-M余弦p为0.061（Llama-3-8B）至0.325（DeepSeek-R1）。欧氏距离上，作者称Gemini 2.5 Pro与Llama-3-8B为边缘结果；表中二者p为0.046与0.048。作者称Figure 3里M vs M-F的观测值在置换分布右侧，F vs F-M在置换均值左侧。
• 显式条件下，作者称只有GPT-5 mini、GPT-4o mini、Llama-3-8B在含中性组的比较中p<.001；DeepSeek-R1、DeepSeek-R1-Qwen、Gemini 2.5 Pro未拒绝原假设。正文给出的显式效应为0.195、0.58、1.116，对应表中欧氏d（表内GPT-4o mini为0.580）。作者称Figure 4里这三模型的观测值偏右，其余模型贴近置换均值。
• 作者称M–F效应通常最大，但GPT-4o mini的F–N略高于M–F，二者都在中等范围。Table 5余弦d：F–N为0.680，M–F为0.618。

稳健性基线

• 测的是同一批未改写作文的两次反馈（M vs M′，n=300，置换5000次），看随机性或内容差异能否解释主实验偏移。Figure 5给出六模型的1-cosine置换图，图中位置本文不能从转写文本读出。
• Table 6的1-cosine观测值：GPT-5 mini 0.1610、GPT-4o mini 0.1590、DeepSeek-R1 0.1514、DeepSeek-R1-Qwen 0.1512、Gemini 2.5 Pro 0.2050、Llama-3-8B 0.1766。Tobs−Tperm为-0.0002到0.0003。p分别为0.0356、0.0576、0.6788、0.0530、0.0176、0.0112；Cohen's d为-2.2837、-1.8072、0.0048、0.0045、0.0124、0.0163。
• 作者称多数基线无统计差异，仅Gemini 2.5 Pro与Llama-3-8B边缘显著，且Cohen's d<0.02，不改变主结论。该表述与Table 6中GPT-5 mini的p和d，以及GPT-4o mini的d=-1.8072不一致。作者未解释很大的d如何与接近0的超额并存。

嵌入空间可视化

• 隐式t-SNE（Figure 6）：作者称M vs M-F在全部模型上空间分开，trustworthiness 0.968，KL 1.516；F vs F-M高度重叠，KL 0.9012，trustworthiness 0.9916。图注同样写M vs M-F分开、F vs F-M重叠。
• 显式（KL 1.133，trustworthiness 0.9938）：图注称GPT-5 mini、GPT-4o mini、Llama-3-8B各组分开，DeepSeek-R1、DeepSeek-R1-Qwen、Gemini 2.5 Pro大量重叠。正文称GPT-4o mini与Llama-3-8B的簇最分散。

文本与教学偏向

• 作者只用GPT-4o mini和Llama-3-8B作例，样本为分离簇中的50条。女性关联线索下疑问句、感叹句和人称代词更多：GPT-4o mini第一人称F=0.16、M=0.08（每100词）；Llama-3-8B第二人称M=8.54、M-F=15.78（每100词）。男性线索下学术词更多：Llama-3-8B为M=7.95、M-F=7.7（每100词）。
• Figure 7：Llama-3-8B学术词比例M组0.0795、M-F组0.077；具体性均值M-F 2.349、M 2.338。GPT-4o mini学术词比例M 0.054、F 0.051、N 0.0515；F组具体性均值最高，为2.424。图注称M组的agency support指标高于另两组，controlling指标更低，未给出这些指标的柱高数字。
• 作者称教学差异在显式条件下更系统。GPT-4o mini自主支持比：M=28、F=27.1、N=25.5（每100词）。作者称男性线索更多模态动词和间接建议，女性线索更多指令式指导；中性提示在结构上更接近男性组，但自主支持最低。Table 7、Table 8有摘录，此处不复述原句。

其他消融与分析

• Mahalanobis距离用于M、F、N组合，作者称与余弦、欧氏结果一致，论文未给数值。
• 论文未报告温度、词表规模或嵌入模型的消融。
• 负面结果：隐式F vs F-M在余弦距离上六模型均未拒绝原假设；三种模型的显式Cohen's d在Table 5中为0.000。作者称这一男→女与女→男的不对称出乎意料，但在全部模型上一致。

Limitations and future directions noted by the authors

• The analysis mainly quantifies embedding divergence and identifies textual patterns; deeper semantic interpretation and explainable AI are still needed to identify the mechanisms that explain and drive these biases (Discussion, Limitations and future work).
• The extent to which cultural context or situational framing in prompts moderates bias has not been explored; the authors state that existing prompt variant frameworks can be used for future research on these differences (same section).
• The findings point to risks for learners, but connecting these findings to actionable debiasing strategies and teacher training to mitigate AI bias impact still requires future work (same section).

Experimental coverage

• The tested generative models are GPT-5 mini, GPT-4o mini, DeepSeek-R1, DeepSeek-R1-Qwen, Gemini 2.5 Pro, and Llama-3-8B, totaling 6; feedback embeddings use text-embedding-3-large.
• The data are 600 student argumentative essays from AES 2.0 containing gender-related words, with gender words accounting for 0.84%; the implicit word list has 192 pairs, and explicit cues are author names, pronouns, and school gender composition.
• Statistics are paired permutation tests on cosine and Euclidean distances, B=5000, p<0.05; Mahalanobis distances for M, F, and N were also checked. Temperature was set to each model's default.
• Qualitative text statistics sampled 50 instances, with example models being GPT-4o mini and Llama-3-8B; thematic analysis was conducted independently by two researchers.
• The paper does not report the number of repeated generations for the same prompt (except for M′ re-prompting) or inter-coder agreement. The paper discusses WEAT, SEAT, CEAT, and linear probing, stating that the former are task-independent and the latter only provide correlational evidence; the experiments do not report numerical comparisons with these methods.

The authors use sentence embedding distance and permutation testing to provide a computable benchmark for auditing LLM gender bias in educational settings, using formative feedback on student argumentative essays as the case study.

• Problem: When inputs are held constant except for gender, feedback may still vary due to gender-related wording in the essay or the author's background in the prompt. The authors argue that if teachers forward such feedback with little modification, it may affect students' agency, perceived competence, and fairness.
• Method: From AES 2.0, they take 600 essays containing gender-related vocabulary, perform male↔female word substitution (300 each) and male/female/neutral author backgrounds (600×3). Six LLMs generate feedback at default temperature; cosine and Euclidean distances are computed on text-embedding-3-large, with permutation testing at B=5000 against the re-prompted unmodified M′, followed by t-SNE and text metrics to see which linguistic and pedagogical features the differences fall on.
• Implicit results: In Table 5, for M vs M-F, all six models have cosine and Euclidean p<.001, with cosine Cohen's d ranging from 0.257 for GPT-5 mini to 0.294 for DeepSeek-R1. For F vs F-M, cosine p ranges from 0.061 to 0.325, failing to reject the null hypothesis. The authors state that male→female substitution causes a larger semantic shift than female→male.
• Explicit results: Only GPT-5 mini, GPT-4o mini, and Llama-3-8B reject the null hypothesis. Euclidean Cohen's d for M vs F is 0.195, 0.580, and 1.116 (Table 5). DeepSeek-R1, DeepSeek-R1-Qwen, and Gemini 2.5 Pro have explicit d=0.000. The conclusion is that author information can trigger biased feedback on 3/6 models, and gender expression in essays can trigger it on all 6 models.
• Text direction: The authors state that under male cues, feedback is more abstract, explanatory, and more supportive of autonomy; under female cues, it is more emotional, interactive, and also more directive. GPT-4o mini's explicit autonomy support ratio is M=28, F=27.1, N=25.5 per 100 words.
• Authors' conclusion: The authors state that the M′ baseline generally cannot explain the main experimental shift, but in Table 6, GPT-5 mini has p=0.0356 and Cohen's d=-2.2837, which is not fully consistent with the main text's "d<0.02, only two models marginal." The authors argue that if essay feedback is deployed without auditing, learners may internalize biased evaluations; the abstract states that they also propose reporting norms and practical prompt suggestions for counterfactual evaluation, but the discussion text does not list items.