FAITH: A feasibility-aware safety filtering reinforcement learning framework for high-dimensional systems

Safety and task reward compete when written into the same policy update; without a safe action, the classic hard projection also fails to give the minimal-harm action.

• Deployment scenario: The authors believe reinforcement learning can already support agile behaviors for high-dimensional legged and humanoid robots, but when deployed near people or equipment, every link must remain outside protected zones. Safety for contact-rich, underactuated systems also depends on whether future actions can ultimately avoid violations.
• Two-zone objective: Strong disturbances can make violations unavoidable. The authors want a controller that safely maximizes task reward when safety is feasible, and otherwise only minimizes harm. Current geometric safety (h(s)≤0) is not equivalent to the existence of a policy that can always avoid the unsafe set.
• Writing into the objective: Penalties, primal-dual multipliers, and constraint reduction in primal methods all act on the task policy. The authors believe that too-small weights allow violations, too-large weights are conservative, and primal-dual also swaps in an unstable saddle-point problem; when update directions oppose, improving safety cancels task progress.
• Action filtering gap: Classic minimal-intervention filters require analytic safety functions and dynamics, only minimize instantaneous action deviation, and a fixed nominal policy cannot adapt to the intervention's effect on long-horizon reward. Learned nonlinear constraints make online optimization or sampling over high-dimensional actions difficult; projection is undefined when no feasible action exists.
• Proposed framework: The authors propose model-free FAITH (Feasibility-Aware safety-fIlTered RL for High-dimensional systems). It approximates the optimal state-action safety value, uses a feedforward network to amortize minimal intervention, and lets the task policy optimize the unmodified task reward only on the filtered dynamics; when the acceptance condition has no solution, the same filter turns to actions with lower predicted peak harm.

Feasibility-Aware Safe Reinforcement Learning

• Reachability: Hamilton–Jacobi reachability characterizes which states can avoid future violations (Fisac et al., 2019; Wabersich et al., 2023). Neural solvers approximate its dynamic programming, while reinforcement learning formulations estimate the value from sampled transitions (Bansal and Tomlin, 2021; Feng et al., 2025; Hsu et al., 2021).
• Writing into policy optimization: Subsequent work writes the value into task policy optimization via multipliers, reachability-dependent objectives, or feasibility-guided generation (Yu et al., 2022; Ganai et al., 2023; Zheng et al., 2024), and extends the formulation to stabilize-avoid and unknown feasibility (So and Fan, 2023; So et al., 2026). The paper only describes these works as writing the safety value into policy optimization, without critiquing them one by one.

Safety Filters

• Pre-execution correction: Safety filters correct nominal actions before actuation, thereby separating task control from safety enforcement (Ames et al., 2017; Wabersich et al., 2023).
• Two learning routes: One route learns the filtering function and retains online filtering; the other directly learns corrections, or trains the task policy on filtered dynamics. The paper places Robey et al. (2020), So et al. (2024), Oh et al. (2025, 2026), Thananjeyan et al. (2021), Hsu et al. (2023), and Kim and Kim (2026) in the former route, and Yu et al. (2022), Bejarano et al. (2025), Yang et al. (2025), and Yun et al. (2026) in the latter.

Legged and Humanoid Safety Control

• Barriers and whole-body constraints: Legged safety filtering ranges from reduced-order barrier controllers to full-order methods (Cohen et al., 2024; Nguyen et al., 2025); recent humanoid work handles joint, actuation, and link-level constraints (Yang et al., 2025; Chen et al., 2025, among others).
• Switching and falling: Learned reachability is used for safe-set estimation and recovery policy switching (He et al., 2024; Chen et al., 2026); protective falling policies are used to mitigate robot injury (Meng et al., 2025; Xu et al., 2025).

Baseline Methods and Benchmarks

• Baselines: The double integrator and Safety Gym use RCRL, the authors' feasibility-aware rewrite of CRPO, RC-CRPO, PPO with β[h]+ subtracted from the reward, PPO(β), and the reduced-order control barrier function ROCBF(α). Humanoid experiments use PPO without a safety term (No Filter) and ROCBF(α) with it as the nominal policy.
• Environments: a one-dimensional double integrator, Safety Gym's SafetyCarCircle1-v0, and Walking-Avoid and Push-Avoid for the 29-DoF Unitree G1 in Isaac Lab.

The authors adopt the same two-zone problem, but use a correction network to enforce safety on actions, and the task policy only optimizes return under the filtered dynamics, with no competing safety term in its update. The feedforward filter requires no dynamics model or online solving at deployment; when the learned accepting set is empty, the same loss prioritizes lower predicted harm. The authors also state that this structure prioritizes protective clearance over balance, without requiring dedicated fall rewards, mode switching, or a separate recovery controller.

Overall, FAITH places safety learning in the critic, backup actor, and feedforward filter, while the task policy only maximizes the original task reward on the filtered dynamics. At deployment, only the task policy and filter are computed forward.

Two-Region Problem

• System: discrete-time, unknown deterministic dynamics s_{k+1}=f(s_k,a_k). The policy's task objective is discounted return over an initial distribution, with discount factor γ∈[0,1), and returns are assumed finite.
• Harm and feasible set: signed harm h defines the unsafe set S_u={s:h(s)>0}, with positive values indicating violation severity. The optimal safety value is V_h^⋆(s):=min_πsup_{k≥ 0}h(s_k), and the feasible set is the states where this value does not exceed 0. Because of dynamics and actuation limits, h(s)≤0 does not imply the state is feasible.
• Two-region objective: when feasible, maximize return subject to peak harm not exceeding 0; when infeasible, only minimize peak harm.

Safety Critic and Backup Actor

• Optimal safety critic: state values cannot compare candidate actions under unknown dynamics. Q⋆h(s,a) is the least upper bound on peak harm starting from that state-action, with subsequent actions produced by some policy. Q⋆h(s,a)≤0 indicates the action admits a trajectory that remains safe; taking the minimum over actions recovers the optimal safety value.
• Bellman and approximation: the optimal critic equals the maximum of current harm and the minimum successor safety value over the next state. The backup actor μψ minimizes Q_{h,ϕ}(s,μψ(s)) over replay states, used in place of minimizing over the next action. The critic regresses to y=(1−γ_h)h(s)+γ_h max{h(s), Q_{h,ϕ̄}(s′,μψ(s′))}, with the target network updated by Polyak averaging.
• Not a formal certificate: this objective corresponds to the discounted safety Bellman fixed point; γ_h<1 时可以不同于 Q⋆h。加上近似误差，作者把 Qh,ϕ(s,a)≤0 写成实用的安全估计，而不是形式证书。作者写可以采用集成和目标策略平滑。回放缓冲保留不安全和近边界转移；人形实验还惩罚备份动作偏离已执行动作，以减小外推误差。

前馈安全过滤器

• 摊销投影：接受条件为 Qh,ϕ(s,a)+δ≤0，δ≥0 为裕度。执行动作是 aexec=clipA(anom+Δω(s,anom))，避免在高维动作空间里在线解非线性投影。
• 训练损失：训练分布 ν 由回放状态，以及回放动作与随机名义动作的混合构成。损失为 L_F(ω)=𝔼_ν[|a_exec−a_nom|²+λ[Q_h,ϕ(s,a_exec)+δ]₊]。过滤器初始化接近恒等映射，训练中把 λ 增大到 λmax≫1。
• 两种情形：名义动作已在学习接受集内时，理想过滤器保持它；否则优先满足学习约束，并由偏差项选择改动较小的动作。若所有动作都使 Q+δ>0, the hinge activates over the entire action set, increasing λ prioritizes lower predicted peak harm, and does not switch to a recovery controller.
• Bound given in the paper: when the action set is compact, Q is continuous in actions, and the loss is globally minimized for each input, the difference between this minimizer and the minimum point of Q does not exceed the square of diam(A) divided by λ, at a rate of O(1/λ). For the learned filter, the paper states this relationship holds only up to learning error.

Training Task Policy via Filter

• Filtered dynamics: the task policy samples nominal actions, and the environment executes only the filtered action. Filtered transitions and rewards substitute the executed action into the unknown dynamics and original reward. PPO maximizes discounted return on this filtered trajectory, with filter parameters fixed at each policy update. Safety changes the trajectory experienced by the policy but does not enter the task reward.
• Lemma 1: if the filter is perfect on the feasible set—any nominal action on a feasible state is mapped back to the feasible set, and no modification is made when the original action already makes the successor feasible—and satisfies the conditions of the cited theorem, then the supremum of filtered return equals the supremum of safe policy return. Global optimization contains the policy class with the optimal policy, i.e., the constrained problem recovering the feasible region.
• Not a deployment guarantee: Remark 1 states that exact projection onto Q⋆h≤0 gives a perfect filter; the learned filter is only approximate, so Lemma 1 is not a guarantee for the deployed controller.

Training Loop

1. Online rollouts are used only for PPO updates of the task policy; the critic, backup actor, and filter are updated offline from replay.
2. The backup policy is executed with some probability to avoid out-of-distribution safety values; these trajectories go only to replay, not to PPO. The paper does not give this probability.
3. Deployment requires only forward computation of the task policy and filter. The authors compare in Remark 2: in-objective methods reconcile safety and return with the same policy; FAITH updates safety with the filter loss and return with PPO, and they do not compete in the same task policy objective.

Experimental Setup

• Problems and Environments: Q1 asks how much reward a feasible starting state can retain while satisfying safety; Q2 asks whether harm can be reduced when it is unavoidable; Q3 asks whether it can scale to high-dimensional systems with full-body constraints. The environments are a one-dimensional double integrator, Safety Gym's SafetyCarCircle1-v0, and the Walking-Avoid and Push-Avoid tasks of a 29-DoF Unitree G1 in Isaac Lab.
• Tasks: For DI, the state is (p,v), action a∈[-1,1], h(s)=|p|−1, reward is −min{|p−2|,|p+2|}, initial values p,v∼U[−1,1], so safety and reward conflict. For the humanoid, 31 links are represented by grid-inflated points, and h is the maximum signed penetration of any point into an infinite-height planar protected zone; the simulation observation is analytic 64-beam, 360° planar LiDAR. Walking-Avoid starts stationary outside the protected zone, and the authors state that remaining stationary is feasible; Push-Avoid applies a push at the start of each episode, and the true feasible set is unknown.
• Scale: DI and Safety Gym use 3 seeds per method. DI trains for 2000 iterations, selecting the safest checkpoint among iterations 1800, 1900, and 2000, with ties broken by higher reward; the validation set and evaluation set are independent, and 4096 episodes are evaluated. Safety Gym's checkpoint selection is the same as in Section V-A. The humanoid is reported over 4096 trajectories. The paper does not report the number of humanoid seeds or the number of Safety Gym iterations.
• Baselines: Low-dimensional environments use RCRL, RC-CRPO, PPO(β), and ROCBF(α). The humanoid uses PPO without a safety term (No Filter), and ROCBF(α) that filters planar velocity commands using it as the nominal policy; the body inflation radius is 0.40 m, with α∈{0, 0.1, 1, 5}. The authors state that other baselines were omitted because they performed poorly on the humanoid.
• Metrics: Safety rate is the proportion of safe trajectories, with the unsafe set being h(s)>0. Task reward, the average peak harm over infeasible DI starting states, and peak penetration on the common failure subset of Push-Avoid are also reported. This subset consists of initial states where all evaluated methods violate safety, and the authors do not label it as physically infeasible. A footnote states that safety is defined in terms of personnel protection, and falling itself is not considered unsafe. The paper does not use LLM review.
• Humanoid Reward: Includes velocity tracking, foot slip penalties, and related motion terms, with penalties on contact forces, and head and hand weights greater than the torso. The paper does not provide the individual weights.

Main Results

The main humanoid results are as follows. ± is the episode-level standard deviation; the safety rate is not given a standard deviation. Walking-Avoid is from Table I, and Push-Avoid is from Table II.

Swipe horizontally to read the table

Method	WA Safety %	WA Reward	PA Safety %	PA Reward	Failure Subset Peak Harm
No Filter	80.37	2428±288	21.53	123.2±96.0	0.253±0.149
ROCBF(0)	99.95	2094±342	84.52	115.5±64.3	0.172±0.146
ROCBF(0.1)	99.95	2182±330	83.35	116.3±41.3	0.175±0.140
ROCBF(1)	99.71	2362±307	78.56	117.1±87.0	0.165±0.140
ROCBF(5)	94.80	2373±303	60.11	115.5±96.6	0.166±0.133
FAITH	99.95	2365±345	85.45	80.0±104.9	0.153±0.184

• Walking-Avoid: FAITH's safety rate is the same as the highest value in the table, higher than ROCBF(1) and ROCBF(5). The authors state this is equivalent to retaining 97% of the unfiltered reward; at the same safety rate, its reward is higher than ROCBF(0) and ROCBF(0.1). ROCBF(5)'s reward is higher than FAITH's, with a lower safety rate.
• Push-Avoid: FAITH's safety rate is higher than the other rows in the table, and its reward is lower than each ROCBF and No Filter. The authors state this is because it sacrifices balance to maintain clearance, rather than trading safety for higher reward. The Fig. 6 caption states: when the push is tolerable, the robot stands upright and stays outside the red box; when the push is too strong, it sacrifices balance, falls, and protects the head and hands.
• Common Failure Subset: FAITH's peak harm is lower than the other rows in the table. The authors state that the same policy-filter limits full-body penetration without a fallback controller or explicit mode switching.

Low-Dimensional Environments

• Feasible Starting States: The authors report FAITH achieves 100% safety on DI and the highest reward among all safe methods (Fig. 3 left, error bars are min-max). The text does not give the reward values for each method in that figure. There are 350 infeasible states in the sampled resets.
• Infeasible Starting States: Fig. 3 labels FAITH's average peak harm as 0.1499±9e−6 and PPO(100) as 0.1499±3.4e−5; the authors state both are the lowest. The Fig. 4 caption states that FAITH is close to the analytic feasible set on visited states, feasible starting states achieve the highest reward along the boundary while remaining safe, and infeasible starting states immediately apply maximum braking. The peak harm labels for the other methods in the figure are interleaved with the text and are not transcribed item by item here.
• Authors' Interpretation of Baselines: RCRL has high variance across seeds, which the authors attribute to possibly unstable saddle points; RC-CRPO crosses the boundary, which the authors attribute to possibly competing safety and performance updates; PPO exhibits a safety-performance trade-off; ROCBF ignores acceleration limits, making trajectories suboptimal. All initial states in Safety Gym are feasible; the authors report FAITH achieves 100% safety with high average reward, and all evaluated baselines exhibit violations (Fig. 5). The text does not give the values for each point in Fig. 5.

Design Ablations

• Relative to Analytic Reference: In Table III, Oracle reward is −219.37, safety rate 100.00%, peak harm 0.1499; FAITH is −220.64±0.23, 100.00%, 0.1499±9e−6; PPO+GTCBF trained with ground-truth CBF is −219.40±0.01, 100.00%, 0.1500. The authors state FAITH closely follows the analytic optimum given by these two. PPO+ROCBF(3) has a higher reward of −187.46±0.04, but a safety rate of 0.00 and peak harm of 1.3501±1.7e−4.
• Removing Two Components: After removing training with filtered dynamics, DI reward is −230.45±1.71 and safety rate 97.52±2.74%; Car-Circle reward is 8.96±5.21 and safety rate 98.44±2.07% (Tables III, IV). The authors attribute the reward drop to PPO not adapting to the filter, and the safety drop possibly to the policy outputting more unsafe actions and filling the replay buffer. After removing backup replay, DI safety rate is 83.87±27.92%, and the authors state the safety value may be out of distribution.
• Fixed Filter: Fixing FAITH's filter and then training PPO gives DI reward −221.09±0.22 and safety rate 99.98±0.04%, and Car-Circle reward 18.46±1.39 and safety rate 100.00%. The authors state both environments are close to full FAITH.

Hardware Demonstration

• Pipeline: The authors perform a qualitative demonstration on a 29-DoF Unitree G1, with boxes representing protected zones. After ground-plane removal via SLAM, the 10 Hz LiDAR keeps the nearest point in 64 equally spaced azimuth bins over 360°, matching the simulation observation format, and this observation is held between scans.
• Frequency: FAITH at 50 Hz combines this observation with the current state to generate joint position targets, with 1000 Hz PD tracking. The Fig. 7 caption states that the dashed box marks the parts modeled in simulation, including G1 dynamics and analytically generated beam inputs.
• Phenomena: The text states that under lighter pushes the robot recovers to an upright posture and avoids the boxes (Fig. 1(a)); under stronger pushes it falls onto its legs and torso and leaves the protected zone (Fig. 1(b)), and states this is consistent with the head and hand contact penalties in training. The caption also states that under strong pushes it protects the head and hands. The paper does not report hardware safety rates or harm statistics.

Other Ablations and Analyses

• DI, PPO+ROCBF(0.5): reward −241.39±0.01, safety rate 99.95%, peak harm 0.1506 (Table III).
• DI, numerical search: reward −220.65±0.10, safety rate 100.00%, peak harm 0.1500±1e−4; Car-Circle safety rate 96.88±2.07%, reward 18.84±0.37. The search keeps the nominal action when Q≤−δ, otherwise samples 1025 actions; the authors state both environments are 18 times slower.
• DI, 0.25λmax: safety rate 0.01±0.01%, peak harm 0.3313±0.11, reward −208.23±10.46; 0.5λmax safety rate 99.99±0.01%, 2λmax safety rate 100.00%, reward −221.42±0.44 (Table III).
• Car-Circle, full FAITH: reward 19.20±0.71, safety rate 100.00%; 0.25λmax safety rate 95.31±5.63%, 0.5λmax 99.48±0.90%, 2λmax reward 17.92±0.88, safety rate 100.00% (Table IV).
• Car-Circle, removing backup replay: safety rate 97.66±3.41%, reward 18.17±1.29; the numbers for removing filtered dynamics training are in the previous section (Table IV).
• The authors state λmax needs to be sufficiently large: reducing it to 0.25 times lowers safety and increases peak harm, increasing it to 2 times maintains the safety rate with slightly lower reward, and results at 0.5 times remain close to the full method.

Limitations and future directions noted by the authors

• No formal guarantees: Discounting, finite λ, and learning errors mean the learned controller has no formal safety or optimality guarantees (Limitations).
• Evaluation scope: Evaluation is limited to flat protected zones, without impact severity or human injury (Limitations).
• Hardware statistics: Hardware demonstrations are qualitative, with no safety or injury statistics, and no quantification of LiDAR noise, control latency, or communication latency (Limitations).

The paper's Limitations section does not separately list future experimental plans. The Conclusion restates the method's conclusions without a separate future work section.

Experimental coverage

• Simulation comparisons include a one-dimensional double integrator, SafetyCarCircle1-v0, and Walking-Avoid and Push-Avoid for the 29-DoF Unitree G1 in Isaac Lab (Section V).
• DI and Safety Gym each use 3 seeds; DI trains for 2000 iterations, selects checkpoints at iterations 1800, 1900, and 2000, and evaluates 4096 episodes. Humanoid results report safety rate, return, and peak penetration on the common failure subset across 4096 trajectories. The paper does not report the number of seeds for humanoid experiments or the number of training iterations for Safety Gym.
• Low-dimensional baselines are RCRL, RC-CRPO, PPO(β), and ROCBF(α). Humanoid baselines are No Filter and ROCBF(α) with α of 0, 0.1, 1, 5, and a body inflation radius of 0.40 m; the authors write that other baselines were omitted due to poor performance on the humanoid (Section V-C).
• Ablations are on DI (Table III) and Car-Circle (Table IV), including retraining PPO with a fixed filter, removing filtered dynamics, removing backup replay, numerical search over 1025 actions, and λ at 0.25, 0.5, and 2 times λmax.
• Hardware is a qualitative demonstration on the Unitree G1, with perception at 10 Hz, inference at 50 Hz, and low-level PD at 1000 Hz (Section VI). The paper does not report specific values for γh, δ, and λmax, nor the execution probability of the backup policy.

FAITH is a model-free safe filtering reinforcement learning framework for high-dimensional systems: safety correction and task return are in separate modules.

• Problem: Putting safety and return in the same policy update can cause them to cancel each other. Classical filters also rely on analytic safety functions and dynamics and only minimize instantaneous action deviation; hard projection is undefined when no safe action exists. The authors distinguish between "can still avoid violations forever" and "can only reduce peak harm."
• Approach: A backup actor provides low-harm actions to the safety critic, which approximates the optimal state-action peak harm. A feedforward residual network performs minimal intervention when the learned feasible set is non-empty, and shifts toward lower predicted harm as the penalty coefficient increases when it is empty. The task policy uses PPO and maximizes the unmodified task reward only on the filtered dynamics. With a perfect filter and the cited conditions, Lemma 1 recovers constrained optimality on feasible states; Remark 1 states this is not a guarantee for the learned controller.
• Low-dimensional results: On feasible starting points for the double integrator, FAITH achieves a 100% safety rate, and the authors state it has the highest return among safe methods. On infeasible starting points, the average peak harm is 0.1499±9e−6, tied with PPO(100)'s 0.1499±3.4e−5 as the lowest marked in Fig. 3. All starting points in SafetyCarCircle1-v0 are feasible; FAITH achieves a 100% safety rate, and all evaluated baselines have violations.
• Humanoid results: In Walking-Avoid, FAITH achieves a 99.95% safety rate and a return of 2365±345; the authors state this is 97% of the unfiltered return. In Push-Avoid, the safety rate is 85.45%, the highest in Table II, with a return of 80.0±104.9, lower than No Filter's 123.2±96.0; the peak harm on the common failure subset is 0.153±0.184, the lowest in the table. The authors state the policy sacrifices balance and falls away from the protected zone.
• Authors' conclusion: Safety and return therefore do not compete in the task policy objective; when the learned feasible set is empty, the filter tends toward minimal predicted peak harm. The same controller was qualitatively demonstrated on the Unitree G1. The authors note that discounting, finite λ, and learning errors mean this controller has no formal safety or optimality guarantees.