Tree-Ising: repairing language models with bit-level Ising edits
Tree-Ising: Repairing Language Models with Bit-Level Ising Edits
The authors propose Tree-Ising, which formulates repairing the reasoning loops of Qwen3.5-0.8B as a row-wise Ising problem solved with GPU annealing; the authors report MATH-500@4 reaches 54.6%, higher than the 51.7% of agent editing.
- The small reasoning model Qwen3.5-0.8B in thinking mode frequently falls into text loops; 93% of MATH-500 responses exhaust the 32k token limit, and accuracy is only 7.9%. Prior work uses an LLM agent to directly rewrite weights for repair, but a single edit costs 158 rounds, 89.7M tokens, and 15.6 hours.
- Tree-Ising formulates the repair as sparse discrete optimization over BF16 weights: only sign flips and powers-of-two scalings that preserve the shared exponent and mantissa are allowed; forward statistics define an objective that weakens the looping direction and protects normal outputs; the problem is exactly decomposed into one QUBO/Ising problem per output row and solved with GPU simulated annealing or simulated bifurcation.
- On Qwen3.5-0.8B, Tree-Ising's MATH-500@4 is 54.6%, higher than agent editing's 51.7%; the authors report leading on all five math benchmarks, and the improvements on MATH-500 and GSM8K remain significant after multiple-comparison correction; 13% fewer weights modified and 2.2× fewer BF16 bit flips. Transferring the same recipe to Qwen3.5-2B, GSM8K greedy accuracy rises from 21.6% to 69.4%.
- The authors note: only a single failure mode (reasoning loops) is covered, with two models from the same model family and mathematical benchmarks; the 0.8B recipe was designed retrospectively after analyzing the agent's public logs, reuses its layer selection, and lacks a strict equal-dose control; the objective is a first-order surrogate of the unedited model's statistics; the quantum experiment has only 8 variables and is not written back into the model, so it does not constitute quantum advantage.
- Evaluated models
- Qwen3.5-0.8BQwen3.5-2B
- Benchmarks
- MATH-500GSM8KAMC23Minerva MathOlympiadBench
- Metrics
- accuracycut at capchanged weightsBF16 bit flipsstate-tree bit flips
Researchers propose the Tree-Ising method, which turns repairing reasoning loops in language models into sparse discrete optimization over BF16 weights; on Qwen3.5-0.8B it raises MATH-500 accuracy from the base model's 7.9% to 54.6%, exceeding the weight repair by the GPT-6 Astra agent in the MIT work (51.7% under the same evaluation). 93% of the base model's answers hit the 32k token limit due to loops. The method uses forward statistics to define an objective, weakens loop directions, strengthens conclusion directions and protects valid outputs, exactly decomposing it into one QUBO (Ising) problem per output row, solved with GPU simulated annealing or simulated bifurcation; edits only perform sign flips and exponent shifts, without changing the shared exponent and mantissa. The recipe requires only 4 forward requests and zero LLM tokens, taking about 31 minutes per run at about $0.90; by comparison, the agent's discovery of the repair took 158 rounds, 89.7M tokens, 50 forwards and $312, and the authors state that their recipe reused the layers and doses selected by the agent.
Why it was selectedThe paper reformulates LLM agent weight repair as row-by-row Ising optimization, using forward statistics to set objectives and GPU annealing to solve, an approach that can be transferred to other model behavior repair tasks.
Paper analysis
What problem does this paper attempt to solve?
How to turn an LLM agent's weight repair of a looping model into a sparse discrete optimization problem solvable in minutes.
Core question: can the defects in the weights that cause reasoning loops be repaired directly using only forward statistics and discrete optimization, without an LLM agent, with smaller and cheaper edits?
- Problem setting: the authors report that Qwen3.5-0.8B in thinking mode frequently falls into short-text loops until the token budget is exhausted; under a 32,768 token limit, 93% of MATH-500 responses are truncated, and accuracy is only 7.9% (Figure 1).
- Existing method and its cost: Takehi et al. [55] have a GPT-6 Astra agent read the weights and, within a budget of 50 forward requests, locate the MLPs of layers 15 and 19 and rewrite 64 columns in each, raising MATH-500 to 48.1%; however, the authors note that a single edit costs 158 rounds, 89.7M tokens, and 15.6 hours (Table 4), and that each step relies on natural-language judgment with no optimality guarantee.
- Core observation: the authors argue that the evidence the agent collects (forward comparisons between looping responses and successful responses) can be formalized as a mathematical objective, and that the edit is a discrete choice over stored bits; lossless BF16 storage is a special case of a state tree, which allows edits to be restricted to bit-level operations that preserve the shared exponent and mantissa.
- Proposed in the paper: Tree-Ising—decomposing the repair into one independent QUBO/Ising problem per output row, solved with GPU simulated annealing or quantum-inspired simulated bifurcation, with an agent-free transfer to Qwen3.5-2B and an 8-variable QAOA quantum hardware proof of concept.
What related research exists?
Related to five lines of work: weight-space programming, model editing, activation steering, repetition suppression, weight-quantization QUBO, and quantum hardware.
Weight-space programming and model editing
- Takehi et al. [55]—a general LLM agent reads and writes network weights, and its agent repaired the reasoning loops of Qwen3.5-0.8B; this paper studies the same repair task but replaces the agent with an optimizer built from the same empirical evidence.
- ROME [41], MEMIT [42], Knowledge Neurons [11], MEND [43], Task Arithmetic [27]—modify factual associations or merge fine-tuning deltas through continuous updates; the paper notes that these methods produce continuous updates, whereas Tree-Ising selects sparse discrete modifications that the storage format can represent exactly, and in Section 4.5 compares against a ROME-style rank-one solution to its own objective.
- Bit-flip attack [50]—flipping a small number of bits can drastically change network behavior; the authors state that this paper uses the same sensitivity to repair rather than to break.
Activation steering and directional intervention
- Steering methods that add directions to the residual stream at inference time [38, 56, 62] and directional ablation that can be folded into weights [5]; the direction in this paper is a comparable contrast direction, but the intervention takes the form of sparse weight edits with a collateral penalty, requiring no change at inference time.
- ThinkEdit [53]—finds a linear direction controlling reasoning length and removes it from the output projections of a few attention heads, changing about 0.2% of parameters; the paper notes that this paper addresses the opposite failure mode (reasoning that never terminates) and edits MLP output projections using discrete moves rather than continuous projections.
Repetition and loops
- Decoding heuristics [22], modified training objectives [58, 59], and analyses of chain of thought and decoding configurations in reasoning models [12, 48]; the paper notes that Tree-Ising changes neither the decoder nor the training process, only editing the weights offline once.
Weight formats and QUBO solving
- Block floating point and microscaling formats [13, 51], lossless BF16 compression [60], state trees [3]; in this paper the state tree defines the set of legal moves for the forward objective.
- AdaRound [46] and Muñiz Subiñas et al. [45]—the latter formulates quantized layer-output reconstruction as an exact QUBO separated by row and solves it with simulated annealing; the paper states that its objective adds a behavioral term along the measured direction and a sparsity cost beyond the reconstruction term.
Quantum hardware: quantum annealing for training binary networks [52], selecting pruning and quantization configurations for CNNs [57], embedding variational quantum circuits as adapters in LLMs [34, 2]; the paper notes that in these works the quantum circuit is a new component of the model architecture, whereas in this paper the model form is unchanged and the quantum device is only responsible for solving which existing BF16 weights should be modified.
Baselines and benchmarks: the comparison baseline is the agent-edited checkpoint of Takehi et al. [55]; the evaluation benchmarks are MATH-500, GSM8K, AMC23, Minerva Math, and OlympiadBench. The authors position this paper as: on the same repair task, replacing the agent with an optimizer yields smaller and cheaper edits.
How does the paper address this problem?
The repair is written as a sparse discrete objective on the BF16 state tree, decomposed exactly into per-row Ising problems, and then solved with GPU annealing or simulated bifurcation.
Overall approach: Tree-Ising mathematizes all three steps of the agent repair pipeline (layer selection, objective definition, weight search), relying only on forward-pass statistics, restricting edits to bit-level moves natively supported by the storage format, and then solving by row decomposition (Figure 2).
Weight representation and legal moves
- Uses the lossless BF16 special case of state trees [3]: every 64 consecutive weights share a maximum exponent E as a shared node, and each weight stores a 4-bit state D = E − e, a sign bit, and a 7-bit mantissa.
- Each weight may only be multiplied by a factor in the set F = {−1, 2, 1/2, 4, −2, −4, −1/2, 1/4} or left unchanged; legal moves must preserve the shared exponent within the group and produce no escape, so only the sign bit and 4-bit state of a single weight change, while the mantissa and shared exponent stay fixed by construction.
- After preselecting one move per candidate weight, the edit becomes a binary decision vector, and the objective becomes a polynomial of degree no more than two, matching the QUBO form.
Layer selection
- The base model generates one response each (8,192 token limit) on 384 practice problems (192 MATH training problems, 192 GSM8K training problems); teacher-forced forward passes are run on looping and correct trajectories, recording the residual stream h(t), MLP output m(t), and output-projection input a(t) at each layer, and Table 2 partitions tokens into the looping set R, protected set P, ending set Z, and continuation set M.
- The direction is defined as uℓ ∝ uℓ,loop − β uℓ,end (β = 0.5), i.e., a combination of the looping direction and the ending direction; the layer score sℓ = uℓ⊤(m̄ℓ,R − m̄ℓ,P) measures how much that layer's MLP writes along u on looping tokens.
- The rule is to take the two highest-scoring layers and exclude the last layer; on Qwen3.5-0.8B this selects layers 15 and 19, matching the agent's choice (Figure 3).
Optimization objective
- The objective is E(Δ) = γ dose(Δ) + coll(Δ) + λ ‖ Δ ‖0: dose is the projection u⊤∆µR of the edit along u on looping tokens, collateral is the output perturbation on protected tokens (i.e., the layer-output reconstruction error commonly used in pruning and quantization), and the last term limits the number of modified weights.
- The target dose is dtarget = −τ u⊤WµR; on Qwen3.5-0.8B it is set to 1.6 times the agent's edit dose (corresponding to τ = 6.7 and 6.1), and new models use τ = 6.5, derived entirely from the base model; the authors note that the comparison with the agent is not strictly equal-dose.
Row-wise Ising decomposition and solving
- Once the statistics are fixed, all three terms sum over output rows, and the problem decomposes exactly into one independent subproblem per row; within each row the top K = 512 columns by score are kept, giving 512 spins per row and 2,048 dense Ising problems across the two layers of 0.8B.
- The default solver is GPU simulated annealing: 16 replicas per row, 300 Metropolis sweeps, with temperature geometrically decreased from 2 to 10⁻⁴ × the row's median |ha|; the alternative is discrete simulated bifurcation (dSB) with 1,200 steps and step size 0.5, with all other components unchanged.
- Calibration: λ is initialized to 2 times the median collateral of a single weight; for each λ, a geometric bisection over γ ∈ [10⁻⁶, 10⁶] finds the smallest γ that reaches the target dose; if over budget, λ is doubled, up to six times.
- The recipe's budget on Qwen3.5-0.8B is 65,536 modified weights per layer; the row problems are standard Ising models that can be handed directly to quantum annealing or QAOA (the 8-variable proof of concept in Section 6).
What experiments did the paper conduct?
Under the same protocol, compared with agent editing on Qwen3.5-0.8B, and transferred to Qwen3.5-2B, plus ablations and an 8-variable quantum hardware validation.
Experimental setup
- Models tested: Qwen3.5-0.8B (editing layers 15 and 19, down_proj, each 1,024 × 3,584) and Qwen3.5-2B (layers 15 and 19, each 2,048 × 6,144; the strengthened variant adds layer 14).
- Baseline: the agent-edited checkpoint of Takehi et al. [55], re-evaluated under the same evaluation framework and the same random seed, with per-question paired tests.
- Data: forward statistics come from 384 practice questions (192 MATH-train + 192 GSM8K-train); ablations use 128 held-out practice questions; the evaluation benchmarks are MATH-500, GSM8K, AMC23, Minerva Math, and OlympiadBench.
- Evaluation protocol: vLLM thinking mode, temperature 0.6, top-p 0.95, top-k 20, presence penalty 1.5, at most 32,768 tokens; after </think>, the last boxed answer is graded with Math-Verify; @k denotes the average accuracy over k samples per question.
- Statistical tests: per-question paired tests, with Bonferroni correction over the agent's 12 comparisons (p < 0.05/12).
- Number of repetitions: the paper reports a single generation seed for each benchmark evaluation and a single run per solver (Section 8).
Main results
According to Table 3 (Qwen3.5-0.8B, accuracy %; the upper block is the numbers originally reported in [55], the lower block is the re-evaluation under the unified framework):
Swipe horizontally to read the table
Model MATH-500@4 GSM8K@1 AMC23@16 Minerva@4 OlympiadBench@1 Base model (reported in [55]) 7.9 13.8 0.5 1.5 1.3 Agent edit (reported in [55]) 48.1 67.1 23.4 25.4 21.8 Agent edit (this paper's framework) 51.7 65.6 22.0 24.2 21.4 Tree-Ising (SA) 54.6 70.5 22.7 26.1 24.3 Tree-Ising (SB) 55.1 70.8 25.0 25.6 24.0 - The authors state that under both solvers Tree-Ising exceeds agent editing on all five benchmarks; the improvements on MATH-500@4 (+2.9 ± 1.0 points, p ≈ 0.003) and GSM8K (155 vs 90 questions, p = 4 × 10⁻⁵) remain significant after Bonferroni correction, while the OlympiadBench improvement (p = 0.045) does not pass correction.
- Truncation rates drop across the board: MATH-500@4 from 31.6% to 23.9%, GSM8K from 8.0% to 3.1%, OlympiadBench from 47.8% to 32.5% (Table 3); the authors believe the accuracy gains mainly come from the model being able to finish within the budget.
- The authors measured 51.7% for agent editing, close to an independent reproduction's 51.0%, and believe the difference from the 48.1% originally reported in [55] comes from its initial evaluation setup; on 400 MATH-500 questions never seen during development, Tree-Ising (SA) scores 53.8% vs agent 50.8% (p ≈ 0.009).
Cost and editing footprint
- Cost (Table 4): the agent discovery process is 158 rounds, 89.7M tokens, 50 forward requests, 15.6 hours, $312; Tree-Ising is 4 forward requests, 21.8 minutes of model GPU time plus 8.8 minutes of annealing, 31 minutes total, about $0.90, zero LLM tokens. The authors explicitly state this compares "the full process of discovering the repair" with "applying a fixed recipe," and the recipe reuses the agent's layer selection and dose.
- Footprint (Table 5): Tree-Ising (SA) modifies 112,630 weights (agent 130,188, 13% fewer), with 278,767 BF16 bit flips (agent 621,388, 2.2× fewer) and 4.0× fewer state-tree bit flips; shared-exponent and mantissa modifications are zero (by construction), whereas agent editing changed 1,261 shared exponents and 129,668 mantissas.
Ablations and continuous-editing comparison
- Ablations (Table 6, 128-question practice screen, single completion, standard error about 4.4 points): removing the finishing direction (β = 0) gives 43.8% accuracy and 53.1% truncation; using only the basic moves f ∈ {−1, 2, 1/2} gives 50.0%; shrinking the budget to 28k per layer gives 49.2%; a dose of 1.2× gives 57.8% (within error of the selected configuration's 57.0% and modifying 31% fewer weights).
- Continuous solution: the closed-form rank-one solution of the same objective (ROME-style) scores 57.8% on the practice screen, comparable to Tree-Ising, but modifies 96% of the weights of the two matrices (7.02M entries, 62×) and 103× the BF16 bit flips; directional ablation at the same dose scores only 38.3%. The authors therefore conclude that the accuracy gains mainly come from the objective itself rather than the discrete solver; the continuous variant was evaluated only on the practice screen.
Transfer and quantum proof of concept
- Qwen3.5-2B transfer (Table 7): layers are self-selected by a forward scan and the 6.5× dose is set by a rule from the base model; GSM8K greedy accuracy rises from 21.6% to 69.4% (1,319 questions), and the truncation rate drops from 77.9% to 26.7%; the new edited solution gets 669 questions wrong and breaks 38 questions the base model originally answered correctly. The strengthened variant (three layers, 9× dose) scores 71.8%, but its advantage over the standard recipe is not significant (p = 0.09).
- Quantum hardware (Table 13): the 8-variable subproblem for one row of layer 15 of 0.8B was run as QAOA on IBM Heron (ibm_fez); at depth 2, 110 of 1,024 samples found the exact optimum (10.74%), versus 4 for the uniform control; the authors explicitly state this does not constitute quantum advantage and that uniform sampling is a weak control.
Other ablations and analyses
- Solver comparison: SB and SA show no statistical difference on any benchmark (MATH-500@4 difference +0.45 ± 0.9 points, p = 0.62), and SB modifies 4% fewer weights (108,327 vs 112,630).
- Layer scan: gradient attribution ranks layers 14, 21, 18, and 16 highest, failing to reproduce the agent's choice (Section 3.2).
- Dose attainment: the actual dose in each layer is 6.2–6.8× its own write, 0.3–2.3% above the target dose (Table 9).
- On the 2B practice screen, accuracy keeps rising with dose (Figure 4 right), so the authors additionally tested a 9× dose.
What are some points that can be further explored?
The authors list limitations including scope, baselines, retrospective design, surrogate objective, and the scale of the quantum experiment, and propose several future directions.
Limitations and future directions noted by the authors
- Narrow scope (Section 8): only one failure mode, the reasoning loop, is addressed, and evaluation is only on two models from the same model family and on math benchmarks; instruction following, general language modeling, and non-math benchmarks were not tested, and side effects other than the collateral items were not measured; evaluating larger models, other model families, and other failure modes is future work.
- Baselines and evaluation (Section 8): the base model's 7.9% MATH-500 score is cited directly from [55] and was not rerun in their own framework; continuous editing was evaluated only on the 128-question practice split; early variants were screened on 100 MATH-500 questions, and the full results include questions seen during development; each benchmark uses a single generation seed; the p-values in the table are nominal, and only the improvements on MATH-500@4 and GSM8K pass Bonferroni correction.
- Retrospective design (Section 8): the recipe was designed after analyzing the agent's public logs; on 0.8B it reuses the agent's layer selection, its dose is 1.6× the agent's, and a strict equal-dose (1.0×) edit was not run; the rule excluding the last layer was set after seeing the 0.8B scan results; the layers selected for both models are full-attention blocks, and the consistency may come from the shared architecture rather than independent validation; evaluating the scan on models without this layout is future work (Section 3.2).
- Surrogate objective and solvers (Section 8): the objective is a first-order surrogate of unedited-model statistics that ignores inter-layer effects, and the dose is only a linear surrogate for behavioral change; candidate columns and move preselection are heuristics; each solver is run only once, timing conditions are not strictly matched, and the authors draw no conclusions about the solvers' relative speed.
- Quantum hardware (Section 8): only 8 variables (8 of the two matrices' 7.3 million weights), a single hardware job, the solution is not written back into the model, and the control is uniform sampling rather than random-angle circuits; the authors state this only demonstrates a correct mapping and does not constitute quantum advantage; scaling to whole-layer repair and measuring downstream accuracy is future work (Section 9).
- Accounting and implementation (Section 8): the state-tree bit count assumes a simple encoder, and direct patching of compressed checkpoints has not been implemented; the time cost is a single run and excludes both parties' method development costs.
Experimental coverage
- The models tested are 2 in total, Qwen3.5-0.8B and Qwen3.5-2B, both from the same hybrid linear/full-attention architecture family (Section 8).
- The evaluation benchmarks are five math benchmarks: MATH-500, GSM8K, AMC23, Minerva Math, and OlympiadBench; for 2B only GSM8K greedy was evaluated (Section 5).
- For 0.8B, layer selection reuses the agent's results and the dose is calibrated with reference to agent editing; the independent pipeline without reference editing was validated only on GSM8K for Qwen3.5-2B (Sections 4–5).
- Statistics use 384 practice questions, and the ablation screen uses 128; each benchmark uses a single generation seed and each solver is run once (Section 8).
- The paper does not report agreement with human grading; answer judging is done entirely by Math-Verify (Section 4.1).
Summarize the main content of the paper
Turning agent-style weight repair into row-wise Ising optimization: smaller, cheaper, and transferable to new models without reference edits.
A model editing methods paper: it rewrites an LLM agent's weight repair for looping models as a sparse discrete optimization problem on a BF16 state tree, Tree-Ising.
- Problem: the authors report that Qwen3.5-0.8B in thinking mode has 93% of MATH-500 responses exhaust 32k tokens due to loops, with accuracy of 7.9%; the GPT-6 Astra agent's repair is effective, but a single edit takes 158 rounds, 89.7M tokens, 15.6 hours, and $312.
- Method: only sign flips and powers-of-two scalings that preserve the shared exponent and mantissa are allowed; forward statistics define an objective that weakens the looping direction, strengthens the ending direction, and protects normal outputs; the problem is exactly decomposed into one QUBO/Ising problem per output row and solved with GPU simulated annealing or simulated bifurcation.
- Main results:
- On Qwen3.5-0.8B, MATH-500@4 reaches 54.6% (SA) and 55.1% (SB), higher than the 51.7% of agent editing; the authors report leading on all five mathematical benchmarks, with the MATH-500 and GSM8K improvements remaining significant after Bonferroni correction (Table 3).
- 112,630 weights modified, 13% fewer than the agent; 2.2 times fewer BF16 bit flips; zero modifications to shared exponents and mantissas (Table 5).
- A single repair takes 4 forward requests, 31 minutes, about $0.90, and zero LLM tokens (Table 4); the authors note this compares a fixed recipe with the agent's full discovery process.
- The same recipe on Qwen3.5-2B, without any reference edits, raises GSM8K greedy accuracy from 21.6% to 69.4% (Table 7).
- 8-variable QAOA on IBM Heron at depth 2 finds the optimum 10.74% of the time; the authors state explicitly that this is only a proof of concept and does not constitute quantum advantage.
- Authors' conclusion: the two solvers perform comparably, indicating that the gains come from the mathematical objective itself; the closed-form continuous solution has comparable accuracy but requires modifying 96% of the weights. The authors believe that the repair task in weight-space programming can be completed by optimization within minutes, and plan to release code.