Evaluating the next generation of deep research agents requires moving beyond static rubrics and final report scores. Traditional benchmarks often miss the nuance of how an AI arrives at a conclusion, focusing solely on the end product rather than the investigative journey. MiroEval addresses this gap by introducing a framework that audits the research process, verifies factual claims through active reasoning, and adapts to evolving knowledge via a live update pipeline.

The framework tests agents on 100 diverse tasks, including 30 multimodal challenges that require interpreting complex visual attachments like charts or diagrams. By simulating real-world queries rather than synthetic data, MiroEval exposes the hidden friction in current AI workflows. A key insight from the study is that the quality of an agent’s search and reasoning steps (the process) is highly correlated with the reliability of the final output, boasting a strong 0.88 correlation coefficient.

Despite the sophistication of models like Claude or Gemini, multimodal integration remains a major hurdle for deep research. Most systems saw a performance decline of up to 10 points when required to process images alongside text. The MiroThinker-H1 model emerged as the top performer in the study, maintaining the most balanced scores across synthesis and factuality. These findings highlight that while AI agents are becoming more capable at writing, their ability to navigate complex, multi-layered information remains a critical frontier.

Evaluating the next generation of deep research agents requires moving beyond static rubrics and final report scores. Traditional benchmarks often miss the nuance of how an AI arrives at a conclusion, focusing solely on the end product rather than the investigative journey. MiroEval addresses this gap by introducing a framework that audits the research process, verifies factual claims through active reasoning, and adapts to evolving knowledge via a live update pipeline.

The framework tests agents on 100 diverse tasks, including 30 multimodal challenges that require interpreting complex visual attachments like charts or diagrams. By simulating real-world queries rather than synthetic data, MiroEval exposes the hidden friction in current AI workflows. A key insight from the study is that the quality of an agent’s search and reasoning steps (the process) is highly correlated with the reliability of the final output, boasting a strong 0.88 correlation coefficient.

Despite the sophistication of models like Claude or Gemini, multimodal integration remains a major hurdle for deep research. Most systems saw a performance decline of up to 10 points when required to process images alongside text. The MiroThinker-H1 model emerged as the top performer in the study, maintaining the most balanced scores across synthesis and factuality. These findings highlight that while AI agents are becoming more capable at writing, their ability to navigate complex, multi-layered information remains a critical frontier.