TikTok CEO Shou Zi Chew said AI will bring scientific breakthroughs in physics, biotechnology, chemistry, and materials science within the next two to three years. He also raised the issue of model alignment, warning that users may grant AI agents extensive permissions without understanding the consequences, and noted that improvements in model capabilities are lowering the cost for malicious actors to create safety problems.
ByteDance's Seed team found that the accuracy of the DeepSeek-V4 series in long-context retrieval fluctuates periodically with the position of the target information in the input, and attributed the cause to its chunked KV Cache compression. In a needle-in-a-haystack test with 128K tokens and about 16,000 key-value pairs, the maximum accuracy gap between different positions for DeepSeek-V4-Flash-Base was 40.2 percentage points, and for DeepSeek-V4-Pro-Base it was 34.8 percentage points; the gap narrowed after post-training.
Oct 7Wed
Wednesday · 2 items
TThe Asia Business DailyThe Asia Business Daily · Attention4848
Epoch AI released the InnovationEval benchmark, testing whether frontier AI can independently discover machine learning algorithm innovations that are new to it, benchmarked against the Self-Distillation Policy Optimization (SDPO) proposed in a human paper. The task requires an agent to start from GRPO and develop new post-training improvement methods, improving short-answer and coding task performance on Qwen3-8B, with a budget of 3,000 GPU hours. Results show that GPT-5.6 Sol and Claude Fable 5 both did not come close to SDPO: Sol's method, adjusted for the same wall-clock time, reached only 15% of SDPO's gain, or about 35% under lenient scoring; Fable 5 showed no measurable improvement. Both merely reused known techniques. Epoch AI also estimates that, based on AI chips shipped before 2027, if cheap models are run, the world could support nearly 2 billion concurrent AI agents, while running frontier models would support about 30 million to 170 million; the cost of reaching a fixed AI performance level has fallen about 47% per quarter over the past three years.
