Fangcun AI releases third-party safety evaluation report on DeepSeek V4.1 Flash-1017
Chinese third-party safety evaluation organization Fangcun AI released a safety assessment of DeepSeek V4.1 Flash-1017, which it says is the first report of the organization's long-term third-party safety evaluation and research program, and describes itself as China's first laboratory focused on independent third-party evaluation and technical research on frontier AI safety. Researcher Tomek Korbak reposted the announcement and commented that systematic-card evaluation of open-weight models by third parties is very valuable, and that it would be better if DeepSeek did its homework itself; he endorsed the chain of thought (CoT) monitorability section of the report and its experimental content, considering CoT-Control and No-CoT Reasoning Index to be the two most informative open-source evaluations currently on the market.
It's great that @FangcunLeap is running proper system card evals of open-weight models. (It would be even better if DeepSeek did their homework but oh, well!)
I like the CoT monitorability section and what they ran: I do think CoT-Control and No-CoT Reasoning Index are the two most informative open-source evals on the market right now.
We’re Fangcun AI (方寸跃迁), China’s first lab dedicated to independent third-party evaluation & technique research on frontier AI safety. Today we’re publishing our evaluation of DeepSeek V4.1 Flash-1017, the first report in our long-term third-party safety evaluation and research program.View the quoted post on X