Latency Evaluation
Cross-source consensus on Latency Evaluation from 1 sources and 5 claims.
1 sources · 5 claims
Benefits
Evidence quality
Highlighted claims
- For the OpenAI Realtime API, AsyncIO reduced latency on HotpotQA and TinyAgent with small accuracy drops. — Building Interactive Real-Time Agents with Asynchronous I/O and Speculative Tool Calling
- On TinyAgent with the OpenAI Realtime API, AsyncIO reduced latency from 7.6 seconds to 4.4 seconds while accuracy decreased. — Building Interactive Real-Time Agents with Asynchronous I/O and Speculative Tool Calling
- Across open-source models and tasks, Async-SFT kept accuracy near the synchronous baseline while reducing latency by 1.6 to 2.2 times. — Building Interactive Real-Time Agents with Asynchronous I/O and Speculative Tool Calling
- Latency measurements are partly simulated rather than fully measured in all settings. — Building Interactive Real-Time Agents with Asynchronous I/O and Speculative Tool Calling
- The paper does not provide inferential uncertainty for reported latency and accuracy differences. — Building Interactive Real-Time Agents with Asynchronous I/O and Speculative Tool Calling