CoLAT: contrastive adversarial training for robust graph neural networks under label poisoning
Contrastive Adversarial Training for Robust Graph Neural Networks under Label Poisoning
CoLAT uses contrastive disagreement scores to guide label flipping for adversarial training; the authors state that under various label poisonings it outperforms the listed GNN defenses.
Training-time label poisoning: the adversary flips the labels of labeled nodes within budget k to maximize test classification error. The attack surface of Eq. (1) is labels rather than graph structure. The evaluation attacks are LafAK, NTK and multiclass SMulti.
- In semi-supervised GNNs, a small number of tampered training labels spreads through message passing. The authors state that existing defenses mostly target random noise or structural perturbations and struggle to distinguish adversarial poisoning from hard examples.
- CoLAT trains the detector with a margin contrastive loss on edges, selects nodes by neighborhood inconsistency score, then assigns them the label with the largest classification loss, and updates alternately with the classifier.
- Compare NTK and SMulti on GCN. The authors state that CoLAT has higher test accuracy under most budgets; in Table 1, Citeseer's SMulti 5% and 20%, and PubMed's SMulti 10%, are higher with PIGNN. Large-graph attacks are subgraph approximations.
- Appendix F states that fully adaptive label poisoning against this defense was not evaluated, and the authors consider designing such attacks still an open problem. Experiments include GCN, GAT, GraphSAGE and various graphs; Table 2 and Table 3 compare only undefended training.
- Evaluated models
- GCNGATGraphSAGE
- Benchmarks
- Cora MLCiteseerPubMedAmazon PhotoOGBN-ArXivActorCoauthor-CSCoauthor-PhysicsAmazon-ratings
- Metrics
- test accuracyrobust accuracyclean accuracy
The study proposes CoLAT, a contrastive adversarial training framework for robust node classification under label poisoning, which uses a structure-aware detection model to identify high-risk nodes whose labels are inconsistent with structure and guides adversarial label flips during training. The alternating optimization implicitly cleans some of the contaminated labels while performing structure-aware adversarial training; on multiple benchmarks, CoLAT outperforms existing defenses under different poisoning intensities and scales efficiently to large graphs.
Paper analysis
What problem does this paper attempt to solve?
Poisoning a small number of training labels spreads through message passing; CoLAT uses contrastive scores to guide adversarial label flipping.
After a small number of training labels is poisoned, GNN message passing affects a large number of nodes in the graph; the authors state that existing defenses mainly target label noise and struggle to separate adversarial poisoning from hard examples.
- Setting: Semi-supervised node classification has labels only on a subset VL. The authors state that message passing causes a small number of tampered labels to affect most of the graph, thereby amplifying the attack (Lingam et al., 2024).
- Existing gaps: The authors state that defenses against label poisoning are few; apart from regularization methods (Zhang et al., 2020), prior work mostly handles label noise (Qian et al., 2023). Adversarial training previously mainly targeted structural perturbations (Gosch et al., 2023); contrastive learning was mostly used as representation regularization and not to identify adversarial label perturbations.
- Core observation: The authors borrow the homophily prior, i.e., connected nodes tend to be of the same class (McPherson et al., 2001). Nodes whose labels are inconsistent with their local neighborhood are treated as more susceptible to adversarial manipulation.
- Proposed method: They propose CoLAT (Contrastive Label-flipping for Adversarial Training). A contrastive detection model selects highly inconsistent nodes, then assigns them labels that maximize the classification loss, and alternately optimizes them with the classifier to approximate the minimax of label poisoning.
- Threat model:
- Goal: The adversary constructs a perturbed label set Ỹ to maximize test classification error under a budget constraint. Equation (1) uses minimax on the training loss as the direct formalization.
- Capability: Flips labels only on labeled nodes, with the number of flips not exceeding budget k. The paper does not describe this threat as white-box or black-box.
- Victim system: Semi-supervised node classification GNN on an undirected unweighted graph. The evaluated attacks are LafAK and NTK for binary classification, and SMulti for multi-class; the authors state that they use linear or kernel surrogate approximations of the bilevel problem and do not obviously change graph homophily.
- Success criterion: Test accuracy on clean test labels. When training is poisoned it is called robust accuracy; when both training and test are clean it is called clean accuracy.
What related research exists?
The authors state that label poisoning defenses are relatively few, and contrastive learning is mostly used only as representation regularization rather than adversarial point selection.
Adversarial Attacks on Graphs
- Structure attacks: The evasion and poisoning reviewed by the authors are mainly through topology perturbation, including GOttack (Alom et al., 2025), Zang et al. (2020), meta-learning attacks (Zügner and Günnemann, 2019a), Zügner et al. (2018) and Dai et al. (2018). The paper does not criticize these methods one by one.
- Label poisoning: The authors state that such training-time attacks are still relatively understudied. Approaches include gradient-based label propagation attacks (Liu et al., 2022), and approximating GNNs with linearization or neural tangent kernels to make the bilevel optimization solvable (Lingam et al., 2024; Zhang et al., 2020).
Defenses on Graphs
- Mainly structure defenses: The authors list Yuan et al. (2024), Li et al. (2024a), low-rank defense (Entezari et al., 2020), GNNGuard (Zhang and Zitnik, 2020), graph structure learning (Jin et al., 2020) and graph contrastive learning (You et al., 2020), and state that they mainly target structure perturbation.
- Label poisoning and certification: The authors state that work explicitly considering robustness to label poisoning is limited, naming Zhang et al. (2020) and exact certification (Sabanayagam et al., 2025). For adversarial training against label poisoning on standard classification, see Bal et al. (2025), Biggio et al. (2012); the authors state that it remains largely unstudied in the graph setting.
Label Noise
- Difference from poisoning: The authors state that label noise is usually random flips independent of the input; adversarial label poisoning is data-dependent and aims to maximize performance degradation. For a review see Wang et al. (2024).
- Methods under noise: loss regularization (Du et al., 2023; Zhang et al., 2020), graph structure augmentation (Dai et al., 2021; Qian et al., 2023), contrastive learning (Xue et al., 2022; Yuan et al., 2023). The authors state that existing contrastive methods mainly treat contrastive learning as representation regularization, rather than a mechanism for detecting adversarial label sets.
Baselines and Benchmarks
- Baseline methods: undefended clean training; label poisoning defense CP (Zhang et al., 2020); pairwise noise defenses PIGNN (Du et al., 2023), CRGNN (Li et al., 2024b), RTGNN (Qian et al., 2023) included per Lingam et al. (2024). The authors exclude methods that perform data cleaning before training.
- Benchmarks: Cora ML, Citeseer, PubMed, Amazon Photo, OGBN-ArXiv, Actor, and Coauthor-CS, Coauthor-Physics, Amazon-ratings in the appendix. Attacks are LafAK, NTK, SMulti.
The authors state that, to their knowledge, CoLAT is the first method to combine contrastive detection with adversarial training to defend against label poisoning.
How does the paper address this problem?
CoLAT splits the inner maximization into contrastive point selection and loss-maximizing label assignment, then alternately trains the classifier.
CoLAT formulates label poisoning as a bilevel problem between the classifier and the adversary, then approximates the combinatorial search with structure-aware alternating updates.
Problem Setting
Undirected unweighted graph G=(V,A,X), labeled subset VL, clean one-hot labels Y. The GNN obtains representations through L layers of AGGREGATE and COMBINE, then through a linear head and softmax obtains class distribution pθi. Clean training minimizes cross-entropy on VL. The adversary obtains Ỹ within budget k.
The authors write the interaction as a minimax over the training loss:
minθmaxỸ1/(|VL|)∑i∈ VLℓ(pθ i,ỹi)
The constraint is that the number of label flips does not exceed k. The authors state that the inner layer is hard to solve due to the label combination space, so from the bilevel adversarial training perspective it is split into "which points to change" and "what labels to change them to" (Robey et al., 2024; Bal et al., 2025).
Contrastive Detection and Point Selection
- Detection model: gφ maps nodes to ℓ2-normalized embeddings. The main text sets it to the same architecture as classifier fθ with separate parameters, so that selection is not confounded by the classifier's current state; Sec. 5.2 and Appendix D also state that the two need not have the same architecture.
- Contrastive loss: a supervised contrastive loss with margin on edges. The embedding inner product of same-class neighbors is pulled toward 1, and different-class neighbors are pushed beyond margin γ. γ∈(0,1), with searched values 0.3, 0.5, 0.7. The authors state that poisoned labels break this geometric relationship, making the contrastive penalty higher.
- Inconsistency score: a node's score is its average contrastive distance to its neighbors, divided by degree and then min-max normalized over VL to [0,1]; ε>0 is used for stability, and the paper does not give the value of ε.
- Attack set: the k labeled nodes with the highest scores. The k for defense training is searched among 3%, 5%, 10%, which is not the same quantity as the evaluation attack budget. The authors state that directly selecting points by classification loss is unreliable, because high-loss nodes may simply be hard examples.
Worst-case Labels and Alternating Optimization
- Label assignment: for selected nodes, among the one-hot vectors not equal to the current label, choose the label that maximizes the current classification loss. The authors state that this constraint avoids flipping an already flipped label back.
- Outer layer: minimize cross-entropy with stochastic gradient descent on the current adversarial labels. The authors state this is to make the model face worst-case label poisoning (Bal et al., 2025).
- Loop: Algorithm 1 updates the classifier each epoch, then fits the detector with the current labels, computes scores, selects points and assigns worst-case labels. The authors state this decouples the detector parameters from the adversarial labels. Cora ML etc. are trained for 200 epochs, OGBN-ArXiv for 500 epochs. For large graphs, the detector update interval can be set to δ, with experiments taking 1, 5, 10, 20, i.e. the classifier updates δ steps for every 1 detector update.
- Relation to cleaning: The authors state that alternating flips implicitly recover some corrupted labels, but the objective is still adversarial training, not standalone data cleaning.
Success Determination
Test labels remain clean. Robust accuracy is recorded when training labels are poisoned; clean accuracy is recorded when both training and test are clean. Both are tracked on the clean test set during training.
What experiments did the paper conduct?
In Table 1, CoLAT is mostly higher than the baselines, with some SMulti settings on Citeseer and PubMed as exceptions.
Experimental Setup
- Architecture: For the main results, both the classifier and the detector are GCN. GAT and GraphSAGE are also tested; in the Actor ablation, an MLP can be used to score the GCN classifier. Cora ML, Citeseer, PubMed, Actor use 2 layers, hidden dimension 16, dropout 0.5, 200 epochs. OGBN-ArXiv uses 3 layers, hidden dimension 256, 500 epochs. All methods use the same architecture.
- Data: In Table 5, Cora ML has 2810 nodes, train/validation/test 140/140/2530, homophily 0.81; Citeseer has 2110, 120/120/1870, 0.74; PubMed has 19717, 60/60/19597, 0.80; Amazon Photos has 7640, 160/240/7250, 0.80; OGBN-ArXiv has 169343, 90941/29799/48603, 0.63; Actor has 7600, 4028/2432/1140, 0.22. The first three datasets follow the split and evaluation protocol of Lingam et al. (2024), and the largest connected component is taken.
- Attacks and baselines: LafAK, NTK, SMulti, with budgets mostly 5%, 10%, 20%, 30%, and additionally 40%, 50% in the appendix. Baselines are clean training, CP, PIGNN, RTGNN, CRGNN. On OGBN-ArXiv, NTK and SMulti cannot be directly scaled, so the authors split the graph into subgraphs, attack them separately and then merge, calling this an approximation. Amazon Photo's NTK 30% is not reported because the optimization did not converge.
- Protocol: 50 hyperparameter trials per method, 15 for Coauthor; selected and fixed by validation set, CoLAT uses early stopping. Results are the mean of 5 random seeds, with standard deviation in parentheses. The metric is accuracy on clean test labels. The paper did not use LLM review, nor report statistical significance tests.
- CoLAT hyperparameters: flip budget k∈{3%, 5%, 10%}, margin γ∈{0.3, 0.5, 0.7}. The search grid for learning rate and weight decay is in Table 6.
Main Results
Table 1 is the test accuracy of GCN on three citation graphs. The table below lists only the mean for Cora ML; see the original table for standard deviations.
Swipe horizontally to read the table
Training method Clean NTK 30% SMulti 30% Clean 0.8270 0.4626 0.3561 PIGNN 0.8236 0.4581 0.4891 RTGNN 0.6297 0.3817 0.4364 CRGNN 0.8088 0.4636 0.3415 CP 0.8122 0.4657 0.3760 CoLAT 0.8377 0.5342 0.6739 According to Table 1.
- The authors state that CoLAT has higher robust accuracy in almost all settings, and surpasses baselines under stronger poisoning such as 20%, 30%. The same table is not highest everywhere: Citeseer's SMulti 5% is PIGNN 0.7328, CoLAT 0.7095; SMulti 20% is PIGNN 0.6981, CoLAT 0.6200. PubMed's SMulti 10% is PIGNN 0.7983, CoLAT 0.7844.
- In the Clean column of Table 1, CoLAT is 0.8377, 0.7420, 0.8171, and undefended GCN is 0.8270, 0.7410, 0.7940. The authors state that on clean data CoLAT is comparable to vanilla GCN, and higher in the discussed cases. In the OGBN-ArXiv Clean column of Table 2, CoLAT is 0.7055 and undefended is 0.7086.
- The authors state that robustness comes from implicit cleaning and additional adversarial labels. Figure 2: Cora ML, SMulti 10%, of 28 initially poisoned nodes, 10 true labels are recovered, 7 are changed to non-true classes, and 13 clean nodes are flipped; the main text states 36% of poisoned labels are recovered, and test accuracy improves by 12%. Figure 7 corresponds to the main text's statement that for Cora-ML, NTK 10%, even without explicit cleaning, adversarial training still brings a 10% improvement in test accuracy.
Scale, Heterogeneous Graphs and Other Attacks
- OGBN-ArXiv (Table 2, subgraph approximate attack): undefended GCN is 0.4741 at NTK 30% and 0.207 at SMulti 30%; CoLAT is 0.6184 and 0.2485. At NTK 10%, CoLAT is 0.6806 and undefended is 0.6839. The authors state that undefended training degrades faster as poisoning increases. The authors also state that CoLAT computes on edges, with complexity O(E); PIGNN relies on dense n×n reconstruction, and CRGNN computes contrastive loss on all node pairs, which is O(n²).
- Heterogeneous graphs: On Actor in Table 3, at low NTK budgets CoLAT is lower than or close to undefended, e.g. Clean is 0.2777 vs 0.2942; SMulti 10% is 0.2435 vs 0.1997, 30% is 0.1444 vs 0.1079. The authors state the gains are smaller than on homophilous graphs, because the contrastive score treats same-class neighbors as positives, whereas Actor's edges are actor co-occurrence. On Amazon-ratings in Table 14, NTK 50% is CoLAT 0.2903, undefended 0.2806; SMulti 50% is CoLAT 0.050, undefended 0.071.
- Appendix: In Table 7, Amazon Photo, SMulti 30% is CoLAT 0.7935, clean training 0.4771; the authors state improvements over clean training of 22% and 31% at 20% and 30% respectively. In Table 9, Cora-ML, SMulti 50% is CoLAT 0.6114, clean training 0.2764. In Table 10, for LafAK at high budgets CoLAT is mostly the highest in the table, but PubMed, LafAK 5% is PIGNN 0.7443, CoLAT 0.7211. In Table 8, the authors state it outperforms baselines except for one configuration; in the table Coauthor-CS's NTK 10% is PIGNN 0.916, CoLAT 0.8932.
Other Ablations and Analyses
- Architecture (Figure 3, Cora ML): The authors state that behavior on GAT and GraphSAGE is similar to GCN; GraphSAGE exceeds 72% under SMulti at all poisoning rates, with the highest gain reaching 31% at 30%. In Table 11, GAT, LafAK 30% is CoLAT 0.7605, clean training 0.5298.
- logit heuristic (Figure 4): The authors state that for Cora ML, at 30%, this heuristic improves only 0.4% under NTK, and the detection model 13%; under SMulti the heuristic is about 5%, and the detection model at most 31%. See Table 13 for LafAK numbers.
- One-step filtering (Figure 4): The detection model is flipped only once after 100 epochs of training. The authors state that at NTK 20%, adversarial training improves about 13% and filtering about 1%; at SMulti 20%, about 26% and 8%. See Table 12 for LafAK.
- Update interval (Figure 5): δ∈{1,5,10,20}. The authors state that lower frequencies are still higher than clean training and shorten training time; the main text does not give the accuracy for each δ.
- Selecting the most consistent nodes (Table 4): For Cora-ML, SMulti 30%, selecting inconsistent nodes is 0.6739, selecting consistent nodes is 0.3757, clean training is 0.3561.
- Time and memory (Table 16, Coauthor-Physics, A100-SXM4-40GB): CoLAT is 7.67 seconds, 2.77 GB; clean GCN is 4.23 seconds, 2.76 GB; PiGNN memory OOM (69.22 GB).
What are some points that can be further explored?
The authors state that adaptive poisoning against the detector was not evaluated, and regard its construction as an open problem.
Limitations and future directions noted by the authors
- Cannot cover all attacks (Conclusion, Appendix F): The authors state that no defense is universally robust under all attack settings (Papernot et al., 2016), nor can robustness against all possible or adaptive strategies be guaranteed.
- No adaptive attack against this defense was conducted (Appendix F): The authors write that they did not consider fully adaptive label poisoning specifically targeting this defense. Performance may degrade if an attacker targets the detection mechanism or exploits its structural assumptions.
- Construction cost (Appendix F): The authors state that constructing such attacks against adversarial training defenses is computationally expensive and not simple (Mujkanovic et al., 2022). The adversary must simultaneously consider contrastive point selection and changing adversarial training dynamics within a bilevel optimization, which is difficult to scale.
- Open problem (Appendix F): The authors state that designing strong adaptive attacks remains an open problem. The authors also believe that improvements across multiple attack types and budgets indicate that structure-aware adversarial training provides a useful inductive bias.
- Direction (Conclusion): The authors consider incorporating structural consistency into adversarial training to be a promising direction for improving the resilience of graph models to label corruption.
Experimental coverage
- The main classifier is GCN; GAT and GraphSAGE are in Sec. 5.2 and Table 11. Table 15 compares using MLP logits versus GCN logits to select points for the GCN classifier on Actor.
- Datasets include the eight graphs in Table 5 and Amazon-ratings in Table 14. Attacks are LafAK, NTK, and SMulti, with budgets stated up to 5%–30%, and for some data up to 40% and 50%.
- Table 1 reports clean training, CP, PIGNN, RTGNN, CRGNN, and CoLAT. Table 2 and Table 3 compare only CoLAT and clean training.
- Results are the mean of 5 random seeds; Appendix A.3 states that hyperparameter trials were 50 and Coauthor was 15. The metric is accuracy on clean test labels. The paper does not report statistical significance tests.
- Appendix F states that fully adaptive label poisoning against CoLAT was not included. A note to Table 2 states that NTK and SMulti on OGBN-ArXiv are approximations after merging subgraph attacks. Sec. C.1 states that NTK 30% on Amazon Photo is not reported because the attack optimization did not converge.
Summarize the main content of the paper
CoLAT selects points by neighborhood inconsistency score and performs worst-case label flips to train a GNN resistant to label poisoning.
CoLAT is a contrastive–adversarial training method for semi-supervised node classification on graphs that defends against training-label poisoning.
- Problem: A small number of tampered training labels spreads through message passing. The authors believe existing methods mostly handle random noise or structural attacks and struggle to separate adversarial poisoning from hard examples.
- Method: A detector learns embeddings on edges with a margin contrastive loss, selects the top-k nodes by neighborhood inconsistency score, then assigns them one-hot labels that maximize the current classification loss and differ from the current label; the classifier minimizes cross-entropy on these labels. The two alternate per Algorithm 1 to approximate the minimax under budget k.
- Main results: In Table 1, GCN, Cora ML, SMulti 30%, CoLAT's test accuracy is 0.6739 versus 0.3561 for undefended GCN; in the same table, PubMed, SMulti 30% is 0.7114 versus 0.4989. On Citeseer's SMulti 5% and 20%, and PubMed's SMulti 10%, PIGNN is higher than CoLAT.
- Other results: In Table 2, on the subgraph-approximated OGBN-ArXiv, NTK 30%, CoLAT is 0.6184 versus 0.4741 undefended. In Figure 2, on Cora ML, SMulti 10%, it restores 10 of 28 poisoned nodes to their true labels, while also flipping clean nodes. On the heterophilic graph Actor, the improvement under SMulti is smaller than on homophilic graphs (Table 3).
- Authors' conclusion: The authors state that across multiple label poisoning strategies and benchmarks in the literature, CoLAT improves robustness and remains competitive on clean data; at the same time, no defense is universally robust against all attacks, and adaptive attacks against this method remain an open problem.