FAR.AI reproduced RNN planning behavior in Sokoban and open-sourced the agent

FAR.AI reproduced and extended the 2019 work of Guez et al., training a 1.28 million parameter Deep Repeating ConvLSTM (DRC) agent to play Sokoban, and found that giving additional thinking steps during inference improved planning ability and solving efficiency. The study showed that success rate increased when thinking steps were increased to 6, then plateaued or slightly declined afterward; DRC clearly outperformed a non-recurrent ResNet baseline with more than twice its parameter count, and the additional thinking steps solved mostly harder levels with longer optimal solutions. Behavioral analysis found that in levels solved with 6 steps rather than 0 steps, the agent took longer to place the first three boxes and shorter to place the fourth box, indicating it took actions that were better in the long term rather than immediately optimal; when the network was allowed to think for N steps before starting the N-length loop, 82.39% of looping or pacing behaviors disappeared, suggesting these loops were used for planning.