The limitation is in the input, not the model: state-dependent command safety in natural language interfaces for industrial controllers
The Limit Is in the Input, Not the Model: State-Dependent Command Safety in Natural-Language Interfaces to Industrial Controllers
The authors propose an answerability audit that tests whether language models judging industrial controller commands have the information needed for the decision, and in 15 pairs of tests across three self-built systems found that without device state provided, four language models could not reliably distinguish command pairs requiring opposite answers even with added rules. The audit records model inputs, pairing the same command with two states requiring opposite answers, and counts a pair as correct only if both answers are correct. With state provided, model scores ranged from 0 to 13 correct pairs out of 15, with three models improving; with text alone, one model always allowed and another always refused, their raw accuracies differing by 32 percentage points, but both had a balanced accuracy of 50% and no correct pairs. A rule evaluator using the same rules as the expected answers matched all 15 pairs; the authors state this does not constitute an independent safety test. The audit is intended to complement systems that check actions against safety rules, helping distinguish missing inputs from model errors; small-sample scores cannot establish a model's general reliability.