Toward Safe World Models: Risks, Defenses, and Evaluation

Prediction accuracy alone cannot establish world model decision safety.

• Use scenario: World models predict action-conditioned consequences for planning, learning from imagined experience, simulation, and model-mediated evaluation. The authors argue that errors in observation, state and memory, dynamics, evaluation, execution, or feedback can enter unsafe decisions and persist through updates (abstract, Section 1).
• Gap left by existing work: MBRL, safe learning in robotics, ML lifecycle assurance, and VLA safety surveys are called complementary foundations by the authors. The authors narrow the problem to: how do biases in the learned description of the environment become unsafe decisions, and what evidence supports that link (Section 1).
• Core observation: Bias can arise without an attacker, from incomplete data, partial observability, model bias, or inappropriate objectives; it can also come from deliberate manipulation of data, observations, or other interfaces. The authors argue that both must be traced along the same functional view, but the causal premises must be separated (Section 1, Section 3.5).
• Proposed organization: Six functional stages, plus a literature synthesis of non-adversarial failures, attacks, defenses, evaluation, and deployment conditions; the evidence corpus is 163 deduplicated references (abstract, Section 3.1). Stages represent dependencies, not six mandatory software modules (Section 3.2).
• Threat model: This is not a paper proposing a single attack. Section 3.5 requires attack claims to specify six dimensions.
  • Goal: Integrity violation, targeted unsafe behavior, hidden hazards, plan ranking manipulation, availability degradation, information extraction, or persistent compromise.
  • Knowledge: white-box, gray-box, or black-box, covering the model, planner, defense, deployment version, and environment.
  • Capability and entry points: training samples, observations, memory, parameters, model queries, planning interfaces, action channels, logs, or feedback; budgets include poisoning ratio, perturbation magnitude, query count, physical feasibility, and cross-modal coordination.
  • Victim: learned action-conditioned environment predictors used for learning, planning, simulation, or safety evaluation, and components that construct, consume, execute, and update these predictions (Section 1–2). Static encoders and one-shot predictors that do not enter the temporal decision process are out of primary scope.

The authors organize complementary work according to the introduction and background sections, without experimental comparisons.

World Models for Decision-Making

• Moerland et al. (2023): Organizes model-based reinforcement learning into model learning and model use.
• Hafner et al.'s PlaNet (2019) and Dreamer series (2020–2025): The paper paraphrases that PlaNet uses a compact recurrent state-space model to support planning from pixels; Dreamer and its successors train policies and value functions using large numbers of latent trajectories.
• Schrittwieser et al. (2020)'s MuZero, Hansen et al. (2024)'s TD-MPC2: The paper paraphrases that MuZero supports planning without reconstructing observations; TD-MPC2 uses an implicit world model for multi-task trajectory optimization in continuous control.

Safety Assurance and Lifecycle

• Brunke et al. (2022): Discusses safe learning in robotics, emphasizing constraints, control, and assurance.
• Mohseni et al. (2023): Connects machine learning dependability techniques to engineering safety strategies.
• Ashmore et al. (2021): Organizes assurance according to the machine learning lifecycle. The authors do not write these three as failed methods, only as complementary foundations (Section 1).

Embodied Systems and Software Agents

• Li et al. (2026)'s VLA safety literature: Places embodied threats, defenses, and evaluation in the same discussion.
• Chen et al. (2026): Uses an architecture–lifecycle connection for computer-using agents' perception, decision-making, tool permissions, execution, and maintenance. The authors write that these dependencies help mark where predictions gain action authority; this survey additionally requires action-conditioned dynamics and tracing from prediction to consequences (Section 1).

Adjacent Concept Boundaries

• Simulators, digital twins, policies, value functions, and foundation models: Section 2.5 draws functional boundaries. Learned simulators used to predict decision consequences can fall under the world model definition; world models can also predict only compact latent states. Policies map information states to actions; world models map states and actions to possible consequences. Foundation models are included only when the prediction process meets the above functional criteria.

• Baselines and benchmarks: The paper has no experimental baselines. Section 7 uses SafeBench, Safety-Gymnasium, nuPlan, WorldGym, etc., as narrative anchors. The authors state that these resources differ in agents, harms, denominators, simulators, and oracles, and cannot be compared by metric name alone (Section 7.4, Table 7.1).

• Positioning: The authors' organizational question is more specific: how do biases in the learned description of the environment become unsafe decisions, and what evidence supports this link (Section 1). Section 2.4 states that architectural evolution alone does not establish safety improvements.

The authors organize the literature with a six-stage safety closed loop, rather than proposing a new world model or new attack.

Functional Definitions and Usage Modes

• Under partial observability, the agent acts on history rather than the true state. The world model maps history to an internal state and predicts action-conditioned next latent states, observations, rewards, safety costs, and termination or constraint-related variables (Section 2.1). The predictive distribution can be split into environmental stochasticity and model uncertainty.
• The internal state is not assumed to equal the physical state. The authors distinguish task sufficiency from safety sufficiency: two histories can be equivalent for reward prediction yet differ in variables needed to choose safe actions (Section 2.1).
• Five non-mutually exclusive architecture families are in Table 2.1: latent state space, value-equivalent or task-oriented, observation-generating or interactive, object-centric or spatial, and hybrid world–action or evaluator. Three usage modes are planning with the model, learning from imagination, and model-mediated evaluation (Section 2.2–2.3).

Six Stages and Attribution

• S1 data and observation, S2 state and memory, S3 dynamics and imagination, S4 evaluation and planning, S5 action and control, S6 feedback and updating. The forward path explains harm formation; the feedback path explains correction, persistence, and amplification (Section 3.2).
• Failures are attributed to the stage where the safety-relevant object is first broken; subsequent transmission is not counted again as root cause. When unique determination is impossible, primary stage attribution remains unresolved (Section 4.1).
• Table 3.1 gives protected objects, failure types, and minimum evidence per stage. The authors also define two cross-stage properties: decision-critical fidelity, i.e., whether distinctions and uncertainties that can change plan acceptability, ranking, or safety margins are preserved; and recoverability, i.e., whether one can return from a degraded state to an acceptable operational state in time (Section 3.3).

From Bias to Harm

The authors write the diagnostic chain from risk source to subsequent data changes as

r → Δz_t → Δp_θ → ΔĴ_c → Δrank(α) → Δa_t → H → Δ𝒟_t+1

It marks the first broken object, plan ranking changes, external harm or constraint violation, and feedback changes to subsequent observations, memory, or training data; items need not all be present (Section 3.4).

Decision-critical error is written as

DCE(M, M′) = 1[G(M, z_t, 𝒜_t) ≠ G(M′, z_t, 𝒜_t)]

G returns the safety-relevant relations induced by the model on a given state and candidate plan set. The authors state that this indicator is not a general safety metric and must be reported together with the changed relations, planner, time horizon, and implementation results (Section 3.4).

• Attack claims must also specify lifecycle stage, access surface, capability and budget, persistence, and adaptivity. Testing only fixed perturbations against fixed defenses is coded as a non-adaptive attack (Section 3.5).
• The claim template requires specifying operational assumptions and version, the changed object or decision relation, metrics and test distribution, and the deepest outcome layer. Missing fields are marked incomplete, not upgraded by interpretation (Section 3.6).

Evidence Directness and Outcome Levels

• E1: The evaluated object is the learned environment predictor, its state, imagined rollouts, or its explicit contribution to learning, planning, or monitoring.
• E2: Adjacent observation, prediction, planning, control, or learning interfaces, without directly evaluating the world model dependency in question.
• E3: Behavior or assurance of an integrated agent or controller, without separating the world model dependency. Hardware deployment alone does not change E1 or E2 to E3.
• E4: Terminology, general mechanisms, or secondary synthesis. Directness is not a quality ranking, nor does it equal outcome level. Outcome levels are separately divided into model, planning, behavior, and operational (Section 3.6).
• The coding tuple also includes model family, usage mode, primary stage, risk mechanism, protected object, defense function, and domain (Section 3.7). Defense functions are divided into prevention, detection, mitigation, isolation, recovery, and assurance updates after changes.

How Evaluation Claims Are Judged

• Evaluation is defined as a check of a specified claim, not a model-level safety score. The authors give an example: a pedestrian occupancy predictor calibrated for three seconds on held-out city routes is only a model-layer claim; it neither establishes that the planner maintains clearance nor predicts the collision rate of a deployed vehicle (Section 7.1).
• An oracle can be instrumented ground truth, high-fidelity simulation or digital twin, formal specification, expert annotation, learned classifiers or LLM/VLM reviewers, or log-based estimation with off-policy or causal assumptions. Reports should state the oracle and which aspects of reality it does not represent. The paper does not specify the review model or thresholds used by the authors.
• Section 7.6 writes paired candidate protocols, benchmark cards, conditional stratification, and cross-version re-certification as recommendations, which the authors explicitly state are not standards already validated in the literature.

Experimental Setup

• Study type: Claim-centered literature synthesis, not a frequency-based meta-analysis, and no new model experiments run by the authors (Section 3.1).
• Search: Covered through September 2026 for learned world models, latent dynamics, action-conditioned prediction environments, interactive generative environments, and world-model-supported planning, control, monitoring, or safety evaluation. Queries combined these concepts with safety, robustness, uncertainty, attack, defense, assurance, evaluation, benchmark, etc. The paper does not report hit counts per database or a PRISMA flow.
• Inclusion and exclusion: Included works evaluating action-conditioned predictors or their states, rollouts, or uncertainties; works measuring how these predictions affect planning, learning, execution, monitoring, or updating; adjacent interfaces that can transmit world model risks; and evaluation resources needed to interpret claims. Excluded static perception or one-shot generation without a temporal decision-making role, general performance papers without relevant interfaces, duplicate versions, and reviews without technical objects or evaluable protocols.
• Coding: Each result records the primary stage, risk mechanism, protected object, defense function, outcome level, domain and operational assumptions, E1 to E4, and publication status. The same result is coded consistently across sections (Section 3.1, Section 3.7).
• Corpus: 163 distinct references in this manuscript. Section-level reference counts are not additive. Supplementary language-agent interface searches are coded as E4 by default unless a specific learned world model dependency is measured.
• Metrics: The authors did not compute ASR, nor set review thresholds or repetition counts. Constraints come from claim templates, four outcome levels, and whether decision relationships change.

Main Results

| Stage | Non-adversarial failure | Attack | Defense | | S1 Data and observation | WAKER | BadDreamer | ReSim | | S2 State and memory | PIGDreamer | ARB4WM | RePo | | S3 Dynamics and imagination | Calibrated MBRL | DINO-WM/PPO | MOPO | | S4 Evaluation and planning | CAP | AdvDO | SafeDreamer | | S5 Action and control | Recovery RL | BadWAM | Shielding | | S6 Feedback and updating | Continual updates | TrojanWorld | AdaWM |

Per Table 1.1. Cells are representative works, not performance scores or rankings. The evaluation dimension merges model-layer resources from S1 to S3 into Calibrated MBRL, S4 as SafeDreamer, S5 as SafeBench, S6 as SAFE-SMART / ASIMOV. Table 5.1 marks AdvDO as S2/S4; Table 1.1 places it in the attack dimension S4. DINO-WM/PPO is in S3 in Table 1.1, while Section 5.7 describes it as a fixed world model with the downstream policy absorbing poisoned experience.

• The authors argue that the architecture moving from latent dynamics to physical robots and interactive generative environments does not by itself establish safety improvements (Section 2.4).
• The authors state that pixel or latent variable errors should not be treated as equally important. Attacks or failures should be reported at the deepest causal layer demonstrated; predicting pixel changes alone is not a successful control attack (Section 3.4–3.5).
• The authors state that among the resources examined in Section 7, none simultaneously measures internal state validity, counterfactual ranking, physical harm, time-limited intervention, and repeated release rollback under the same harm definition (Section 7.8).

Evidence Boundaries for Failures and Attacks

• Organization: Section 4 writes non-adversarial failures by the first object broken. The authors state that task reward and visual quality cannot replace constraint satisfaction, simulated cost violations are not observed physical harm, and detecting anomalies is not timely recovery. Expected constraints and per-trajectory rules are not interchangeable.
• Paraphrased separation: The Objective Mismatch Cartpole experiment is paraphrased as control worsening while validation likelihood does not worsen; this manuscript does not give the magnitude (Section 5.4). BadWAM defaults to 17 forward queries per attacked replanning step, increasing action deviation while penalizing predicted future deviation; the authors state that monitoring bypass remains unresolved (Section 5.6).
• Authors' interpretation: Model-layer disruption need not align in magnitude with behavioral disruption. BadDreamer's evidence stops at open-loop waypoints; the DINO-WM/PPO path stops at execution in the original simulator. The authors state that these chains do not jointly show an attack completing the full closed loop, nor do they give physical robot consequences (Section 5.7–5.8).

Defenses, Evaluation Resources, and Deployment

• Defense boundaries: Table 6.1 lists MOPO, SafeDreamer, FOSP, Shielding, predictive safety filters, Safe Deep Policy Adaptation, AdaWM, and world model anomaly detection side by side by stage, protected object, and evidence boundary. The authors state that E1 does not imply formal guarantees, nor does E3 imply weaker outcome evidence.
• What the authors mark as not directly evaluated: Direct world model safety literature on memory poisoning, staleness, and compression loss is scarce; the corresponding memory strategies are engineering requirements synthesized from interfaces and background evidence, not directly evaluated world model memory defenses (Section 6.2.2). Direct E1 evidence for automatic re-certification of continuously updated systems is scarce (Section 6.6.3). Domberg and Schildbach's residual detection is coded as E1 operational-layer detection; the authors state that residuals cannot indicate causes, small residuals do not mean safety is established, and detection is not recovery success (Section 6.6.1).
• Evaluation and deployment: Table 7.1 states the boundaries of each anchor, e.g., Calibrated MBRL's calibration results have no physical claims, and WorldGym's ranking correlation is not closed with deployment. The authors state that SafeBench, Safety-Gymnasium, nuPlan, and HABIT cannot be interchanged by metric name. DayDreamer is coded as E1 behavioral-layer task learning evidence; OccWorld is E1 model-layer occupancy prediction plus planning-layer offline trajectory evaluation, which the authors state does not establish closed-loop driving safety (Section 8.1).

Other Ablations and Analyses

• The paper does not report the authors' own ablations, hyperparameter sweeps, or repeated experiments.
• Tables 4.1, 5.1, 6.1, 7.1 are mechanism and evidence boundary comparisons; the authors state they are not performance rankings or exhaustive catalogs.
• The authors state that in the examined representative set, direct tests of long-lived safety-relevant memory errors reaching operational outcomes, and joint measurement of confidence on dangerous state–action pairs with the risk of the chosen plan, remain scarce; this is not a count of the entire literature (Section 4.8).
• Studies testing only fixed perturbations against fixed defenses are coded as non-adaptive and cannot support robustness claims against attackers who know the defense or can iterate over time (Section 3.5).
• Cognitive-Driven Defense, MirrorShield, LlmFixer, and OpenEvoShield on language interfaces are coded as E4; the authors state that their jailbreak resistance or message-level detection cannot verify visual observations, transferred labels, or physical safety (Section 6.1.1, Section 6.4.1, Section 6.6.2).

Limitations and future directions noted by the authors

• The authors state that world model terminology and architectures remain heterogeneous, direct safety evidence is scarcer than adjacent evidence in reinforcement learning, control, or robotics, and the literature changes faster than any fixed retrieval snapshot (Section 10).
• The authors write that results on a simulation, a specific planner, or a specific update process should not be generalized to unrestricted operational safety (Section 10).
• In Section 3.1 the authors write the main limitations of the corpus as vocabulary drift, uneven publication status, and the scarcity of studies exposing the full chain from prediction to consequences. Section 3.7 writes that the framework cannot infer real safety from model-level metrics alone, cannot dissolve underspecified safety goals, and cannot automatically transfer evidence from one domain to another.
• The authors propose extending decision-critical fidelity from point predictions to multi-step and trajectory level, covering uncertainty accumulation, branching futures, and the effect of model error on planners or controllers, while remaining feasible under deployment latency (Section 9.1).
• The authors propose pairing safety-sufficient states with independent runtime assurance: write down independence assumptions, test failover and degradation information, and measure both harm reduction and usability; closed-loop evaluation should also separate attacker permissions, persistent objects, model changes, plan changes, implementation violations, and behavior after reset or rollback (Sections 9.2–9.3).
• The authors propose comparable evaluation claims, simulation-to-deployment safety transfer, and incorporating continual learning, supply chain, and operational governance into the safety argument, including version approval, rollback authority, incident reporting, and responsibility assignment (Sections 9.4–9.5).

Experimental coverage

• The evidence corpus is 163 deduplicated references as of September 2026; inclusion criteria cover action-conditioned predictors and their states, rollouts, or uncertainty, the impact of these predictions on planning, learning, execution, monitoring, or updating, adjacent interfaces that can transmit risk, and evaluation resources needed to explain claims (Section 3.1).
• Each result is coded by model family and usage, main-loop stage, risk mechanism, protected object, defense function, evaluation level, domain, and E1 to E4 encoding (Section 3.7).
• The main text synthesis covers non-adversarial failures, attacks, defenses, and four evaluation levels from model to runtime on S1 to S6, plus deployment discussions for three categories: manipulation, navigation and driving, and multi-agent and human-interactive (Sections 4–8).
• Attack conditions use white-box, gray-box, and black-box per Section 3.5, and distinguish adaptive versus non-adaptive coding; language agents with retrieval are coded E4 when no learned world model dependence is detected.
• The paper does not report per-library hit counts, the PRISMA screening flow, models run by the authors themselves, number of repetitions, or query budget (Section 3.1). Section 7.8 writes that the examined resources were not jointly measured under the same harm definition for internal state effectiveness, counterfactual ranking, physical harm, time-limited intervention, and repeated release rollback.

World model safety is organized as a property of the prediction-decision closed loop, not as isolated prediction scores.

• Problem: Action-conditioned world models are used for planning, imagined learning, and simulation. The authors argue that between prediction quality and safe behavior lies the decision process; errors accumulate along imagined trajectories, and optimization may favor actions for which the model is overly optimistic.
• Framework: Six functional stages from data and observation to feedback and updating. Five architecture families and three usage modes only describe representation and consumption; safety analysis separately uses stages, protected objects, outcome levels, and E1 to E4. Stages are not six mandatory modules.
• Approach: Search through September 2026, 163 deduplicated references. Each result is coded by entry stage, adversarial or non-adversarial mechanism, defense function, and evidence directness. Claims with missing fields are marked incomplete, not upgraded by interpretation.
• Synthesis findings: Table 1.1 lists only representative works, not comparable safety scores. The authors state that model-layer changes need not align with behavioral disruption. BadWAM is paraphrased as defaulting to 17 forward queries per attacked replanning step, with monitoring bypass still unresolved. Section 7.8 states that the examined resources do not jointly measure internal state, counterfactual ranking, physical harm, time-limited intervention, and repeated release rollback under the same harm definition.
• Authors' conclusion: Defensible claims should link model-related bias to observed decisions or behavior, and specify domain, threat conditions, time horizon, and version. E1 to E4 only describe evidence directness, not safety scores. The authors point future work toward decision-critical fidelity, independent runtime assurance, closed-loop threats, comparable evaluation, and assurance updates after updates.