Who Can You Hide Behind? Author-Conditioned Style Transfer and Evasion of a Commercial AI-Text Detector
Who Can You Hide Behind? Author-Conditioned Style Transfer and the Evasion of a Commercial AI-Text Detector
The authors used Gemini to rewrite 991 AI text passages in the styles of 55 writers; the median QuillBot detector score fell from 100% to 0%, and the authors say the results apply only to this detector and rewriting system.
The attacker gave AI-generated answers, along with a writing sample from a named writer, to the Gemini chat interface and asked it to learn that style and rewrite the original text so that the percentage score given by the QuillBot AI detector would drop. The paper does not state what knowledge assumptions the attacker made about the detector's internals; the victim system is the QuillBot AI detector.
- Scores from AI text detectors affect writers, and the authors wanted to know: after AI-generated answers are rewritten according to a named writer's writing sample, how do detection scores change, and do they vary by writer?
- Using the Gemini chat interface, 991 AI-generated answers were each rewritten in the excerpt style of one of 55 writers, then the original and the rewrite were each scored once with the QuillBot AI detector, and paired comparisons, cross-writer comparisons and text reuse analysis were performed.
- After rewriting, the median QuillBot score fell from 100% to 0%, the mean fell from 67.9% to 11.2%, and 66% of rewrites scored 0%; rewrites for 10 writers all scored 0%, rewrites for 6 writers still had mean scores above 50%; the more the samples were reused, the lower the score.
- The authors note: only one detector and one rewriting system were covered, a general-purpose rewriting baseline is missing, semantic preservation was not measured, the rewrites were markedly longer, author and prompt versions were partly confounded, and the results cannot be generalized to all detectors.
- Evaluated models
- QuillBot AI detector
- Benchmarks
- 991 段 AI 生成的日常问题答案(来源数据集未记录)55 位具名作家的已发表作品节选
- Metrics
- QuillBot AI score(百分比)改写后 0% 占比5-gram 复用率长度比
Researchers had Gemini rewrite 991 AI-generated answers in the style of one of 55 named authors, lowering the median score of QuillBot's AI-text detector from 100% to 0%. After rewriting, the mean dropped from 67.9% to 11.2%, and 66% of rewrites scored exactly 0%; among 699 samples with original scores of at least 50%, 71% of rewrites scored below 10%. Effects varied by author used: rewrites corresponding to 10 authors all scored 0, while for 6 authors the mean remained above 50%; the more five-word sequences were reused from the author samples, the lower the score, and rewrites reusing less than 5% still averaged 15.2%. The authors note that the results concern only this one detector and this one rewriting system, that the study lacks a general rewriting baseline, that semantic preservation was not measured, and that the rewritten texts were noticeably longer than the originals.
Paper analysis
What problem does this paper attempt to solve?
The authors examined: after AI text is rewritten in a named writer's style, how QuillBot detection scores change and whether they vary by writer.
Core question: after conditioning the rewriting of AI-generated answers on a named writer's writing samples, how do commercial AI text detector scores change, and does this change vary by writer.
- Problem setting: AI text detector judgments have real consequences for writers, so it is necessary to understand when their scores change and what the changes mean (introduction).
- Gap in prior work: The authors state that prior research has shown paraphrasing can lower detection scores; this paper examines a related question—rewriting conditioned on a named writer's writing samples.
- Core observation: The authors find that QuillBot scores shift downward overall after rewriting, and that rewrite scores differ greatly across writers; however, the authors emphasize that the design cannot attribute the observed changes to writer identity itself.
- Paper output: A paired analysis (scores before and after rewriting), cross-writer comparisons, and text reuse and length analyses; the authors position it as an observational analysis of one detector and one rewriting system.
- Threat model:
- Goal: Lower the percentage score of AI-generated text on the QuillBot AI detector.
- Knowledge and capability: Manual rewriting through the Gemini chat interface, with a prompt containing an excerpt of a named writer's published work; the paper does not state assumptions about knowledge of the detector's internal mechanisms.
- Victim system: QuillBot AI detector; the rewriting system is Gemini (model version not recorded).
What related research exists?
Related work falls into four groups: detection methods and evaluation, rewriting evasion, style and authorship attribution, and prompt-driven evasion.
Detection methods and evaluation
- The survey by Wu et al. (2025) [1] reviews machine-generated text detection approaches such as statistical methods, trained classifiers and watermarking, and their evaluation challenges.
- DetectGPT [2] examines probability curvature, Binoculars [3] compares the scores of two language models; Kirchenbauer et al. [4] embed a detectable watermark at generation time. The paper notes that these methods differ from the commercial detector examined here.
- The M4GT benchmark [5] covers multiple languages, domains and generators; the paper uses it to illustrate the importance of testing beyond a single setting.
Paraphrasing and detection
- Krishna et al. (2023) [6] show that paraphrasing can lower detection rates, and examine retrieval as a defense.
- Sadasivan et al. (2025) [7] study recursive paraphrasing against multiple detector families; Cheng et al. (2025) [8] use detector feedback to guide rewriting.
- Other work tests editing, prompt and paraphrasing attacks, including small adversarial changes and detector ensembles [9–11]; Masrour et al. [12] evaluate the effect of commercial humanizers and paraphrasing tools on detection and on text meaning; in the practical evaluation by Tufts et al. [13], detectors performed unstably on unseen data and adversarial prompts.
Style and authorship attribution
- Doughman et al. [14] show that writing style and text complexity affect machine-text detection; MoSEs [15] proposes a style-aware detection method.
- StyleRemix [16] alters selected style features to obfuscate authorship attribution; Nguyen et al. [17] examine the role of language models in authorship obfuscation, imitation and verification. The paper notes that authorship obfuscation and imitation are related but distinct tasks.
Prompt-driven evasion
- SICO by Lu et al. (2024) [18] optimizes human-written in-context examples to guide the model to generate text that evades detectors.
Positioning of this paper: The authors state that this paper instead examines a manual chat workflow—using excerpts attributed to named writers—and compares score differences across these writers; the authors also note that these differences cannot prove that writer identity itself caused the observed changes. This paper does not compare baseline methods and does not use a public benchmark dataset.
How does the paper address this problem?
Collected 991 pairs of "AI original + writer-style rewrite", scored each once with QuillBot, then performed paired and cross-writer statistical analyses.
Overall approach: construct paired samples—each AI-generated answer is rewritten once by Gemini in the excerpt style of one writer, the original and the rewrite are each scored once with the QuillBot AI detector, and then before/after scores, differences across writers and text reuse are compared (Figure 2).
Data and materials
- Source texts: 991 AI-generated answers to everyday questions (covering wine selection, budgeting, job hunting, cooking, etc.), median length 92 words; the authors state that the source dataset, license and sampling process are not recorded in the available materials.
- Writer set: 55 writers, 5 to 31 text segments each (median 18), covering essayists, journalists, scientists, AI researchers and novelists (Table 2).
Rewriting procedure
- Prompt design: The prompt gives the writer's name, states that the sample is their work, and attaches an excerpt of their published work (median prompt length 436 words), asking the model to learn that style and rewrite the given content; the source text is provided separately.
- Prompt variants: The prompt was adjusted during collection—25 prompts added an instruction to keep the word count close to the original, 23 required avoiding repetitive opening sentence patterns; 18 writers used more than one prompt variant.
- Generation method: Manual rewriting through the Gemini chat interface; the model version, collection date, and whether a new session was used for each item are not recorded.
Scoring and cleaning
- Detection: Each original and rewrite was scored once with the QuillBot AI detector, reporting the percentage returned by the tool directly; the authors caution that this score should not be interpreted as a calibrated probability of AI authorship.
- Cleaning: From 1,099 rows in the collection table, 99 empty rows, 6 rows with missing or non-numeric scores, 2 rows with no author, and 1 anomalous row with a score of 1000% were removed, yielding 991 pairs; one writer spelling error is reported as George Monbiot (Table 4).
Metrics and statistics
- Metrics: before/after scores, share of rewrites scoring 0%, length ratio (rewrite word count/original word count), sample reuse rate (share of 5-grams in the rewrite that appear in the writer's sample).
- Tests: paired changes tested with the Wilcoxon signed-rank test; differences across writers (writers with at least 5 text segments) tested with Kruskal–Wallis; correlation between reuse and score tested with Spearman ρ. The authors state that these tests assess only within-sample associations and do not establish causation.
What experiments did the paper conduct?
On 991 pairs: after rewriting, median score 100%→0%; 10 writers all 0%, 6 still above 50%; the more reuse, the lower the score.
Experimental setup
- System under test: QuillBot AI detector (each text scored once; settings and collection date not recorded).
- Rewriting system: Gemini chat interface (version not recorded), operated manually.
- Data: 991 "original—rewrite" pairs, from the 55-writer condition (5–31 segments each, median 18).
- Metrics: QuillBot percentage score, share scoring 0%, 5-gram reuse rate, length ratio.
- Statistical tests: Wilcoxon signed-rank (paired), Kruskal–Wallis (cross-writer), Spearman ρ (correlation); the paper does not report repeated scoring.
Main results
According to Table 1 (Before/After are means):
Swipe horizontally to read the table
Subset n Before After = 0% < 10% ≥ 50% 全部配对 991 67.9% 11.2% 653 (66%) 708 (71%) 82 (8%) 原文 ≥ 50% 699 95.1% 12.0% 454 (65%) 494 (71%) 67 (10%) - 中位分数从 100% 降到 0%(n = 991;Wilcoxon 符号秩 p < 10⁻¹⁰⁰);Figure 1 显示分数质量从 100% 移到 0%。
- Figure 3 显示 711 对下降、184 对不变、96 对上升;上升的 96 对中有 86 对原文低于 20%。
- 作者指出许多原文改写前分数就低:269 段原文低于 20%,因此单独报告了原文 ≥ 50% 子集。
跨作家差异
- Kruskal–Wallis 检验(55 位作家的改写分数):H = 541,p < 10⁻⁸⁰。
- 10 位作家的改写全部得 0%(包括 Jia Tolentino、George Monbiot、Morgan Housel、Seth Godin、Haruki Murakami);6 位作家的改写均分仍高于 50%:Maria Popova 73.8%、Fareed Zakaria 61.4%、Melanie Mitchell 59.4%、Wendy Smith 56.2%、Rebecca Goldstein 54.9%、Brian Greene 52.3%(Figure 4)。
- 作者称由于作家样本、源文本和提示词变体未被独立控制,不能把这些差异归因于作家身份或风格本身。
样本复用与分数
- 复用率越高分数越低(Spearman ρ = −0.30,p < 10⁻²⁰);作者说明该关联不证明复制导致了低分。
- 据 Table 3:复用率 <5% 的 663 条均分 15.2%(57% 得 0%);5%–20% 的 213 条均分 3.9%(78% 得 0%);>20% of the 115 items averaged 1.5% (96% scored 0%).
- Writer-level reuse correlated with mean score (ρ = −0.39, p = 0.003): Janelle Shane median reuse 42%, Heather Havrilesky 38%, while the 6 writers whose mean score still exceeded 50% had reuse rates of at most 1% (Figure 5).
Length analysis
- Rewrites were about four times as long as the originals (median 92 → 428 words; length-ratio quartiles 2.9, 4.1, 5.8).
- Rewrite length itself was not associated with score (ρ = −0.01, p = 0.75), and the length ratio was weakly associated (ρ = −0.11).
Other ablations and analyses
- The paper does not report other ablation experiments; Table 2 gives the per-writer 0% share, median reuse rate and median length ratio for all 55 writers.
What are some points that can be further explored?
The authors list several design limitations and propose that next steps should add baselines, multiple detectors, semantic-preservation measures and scoring of the writers' own texts.
Limitations noted by the authors and future directions
- The results cover only one detector, one rewriting system, and one manually collected score per text; commercial detectors and Gemini change over time, limiting reproducibility and generalizability (Section 5).
- There is no generic "make this sound human-written" paraphrasing baseline, nor a baseline using anonymous human samples, so the effect of named writers cannot be separated from the effect of rewriting or of providing samples themselves (Section 5).
- Prompts changed during collection, 18 writers had multiple prompt variants, and writer and prompt version are partly confounded (Section 5).
- 0% only means QuillBot did not flag the text; it does not mean the text reads as human-written or preserves the original meaning; rewrites were markedly longer and often restructured into prose, and semantic preservation was not measured (Section 5).
- The 5-gram reuse metric captures only near-verbatim overlap, not approximate paraphrasing (Section 5).
- Future directions: the next study should add a generic paraphrasing baseline, multiple detectors and generators, semantic-preservation measures, and detection scores for each writer's own written texts, to test whether detectors mistake expository style for machine origin (Section 7).
Scope of experimental coverage
- The detector under test is one: the QuillBot AI detector; the rewriting system is one: Gemini (Section 3).
- The sample is 991 pairs of texts, 55 writers, 5–31 segments each (Table 2).
- Each text was scored only once; the paper does not report repeated scoring or scoring consistency (Section 5).
- The source dataset and its license, and the sampling process, are not recorded; the Gemini model version, collection date and detector settings are not recorded (Section 3).
- The paper did not measure semantic preservation and did not set a generic paraphrasing baseline (Section 5).
Summarize the main content of the paper
An observational study: rewriting in a writer's style lowered the median QuillBot score from 100% to 0%, but the authors stress the results cannot be generalized.
An observational study on the robustness of AI text detectors, examining the effect of "rewriting in a named writer's style" on commercial detector scores.
- Problem: Detector scores affect writers; the authors want to know how rewriting conditioned on writer samples changes scores and whether this varies by writer.
- Method: Using the Gemini chat interface, 991 AI-generated everyday answers were each rewritten in the excerpt style of one of 55 writers; the QuillBot AI detector scored the original and the rewrite once each, then paired statistics, cross-writer comparisons and 5-gram reuse analysis were performed.
- Overall effect: Median score fell from 100% to 0%, mean from 67.9% to 11.2% (n = 991, Wilcoxon p < 10⁻¹⁰⁰); 66% of rewrites scored 0%; among the 699 pairs with original ≥ 50%, 71% of rewrites were below 10% (Table 1).
- Differences across writers: Rewrites for 10 writers all scored 0%, while rewrites for 6 writers (e.g., Maria Popova 73.8%) still had mean scores above 50% (Kruskal–Wallis p < 10⁻⁸⁰, Figure 4).
- Reuse and length: The more the samples were reused, the lower the score (ρ = −0.30), but rewrites with reuse rate <5% still had a mean score of only 15.2%; the median rewrite length was about four times the original, and length itself was not associated with score (ρ = −0.01).
- Authors' conclusion: The authors emphasize that the design cannot explain why these patterns occur, and that the results describe only this one detector and rewriting system and should not be generalized to all detectors; the authors suggest that future research add a generic paraphrasing baseline, multiple detectors, semantic-preservation measures, and scoring of the writers' own texts.