Import AI· Jack Clark·· 15d agoAI score45
Realtime-Bench: Real-Time Task Benchmark Testing, Performance Analysis of LLMs in Low-Latency Tasks
Import AI 479: Spacewar; low-latency tasks versus LLMs; Genie 4
AI digest
The Realtime-Bench benchmark test was jointly constructed by multiple universities and research institutions, aiming to evaluate the performance of modern general AI systems in real-time decision-making tasks. The test results show that OpenAI's GPT-6 Astra achieved the highest pass rate of 36% in the 'wall clock' evaluation setting. This benchmark test helps understand in which tasks AI agents may perform poorly.
Source:Import AI · importai.substack.com