This study pre-registered and compared three AI feedback configurations applied to finished economics meta-analyses: a single-pass model, a cross-model adversarial debate, and a cross-agent workshop. Across 55 papers (44 with author rankings), authors most often found the plain single-pass report more useful for improving their work than the more computation-intensive debate configurations. The results held under multiple robustness checks and persisted irrespective of whether author or AI judges performed the rankings. Notably, authors recalled human referee feedback as more useful than the AI reports, while AI judges consistently ranked the human referee last when placed alongside AI-generated reports. An external AI judge could reverse the ordering, underscoring caution against substituting AI judgments for authors. The findings highlight the bounded returns to extra compute in fixed-length feedback and suggest that, at least in this setting, simpler AI feedback can be more valuable to authors than more elaborate multi-agent configurations .
| 配置 | Token 量级 | API等价成本 | 平均排名(越小越好) |
|---|---|---|---|
| 单次通读 | ~27k | $0.20 | 1.59 |
| mad-research | ~238k | $1.87 | 2.25 |
| paper-workshop | ~800k | $7.20 | 2.16 |
题目分析与结构性梳理 以下对题为“Does Multi-Agent Debate Improve AI Feedback on Research Papers?” 的研究论文进行极为细致的解读与批判性分析,覆盖元数据、设计、数据采集、结果解读、图表与表格的内在含义、估值与风险,以及对结论的综合性评估。分析严格按报告的原始结构展开,并在关键论点、数据点、假设与推断处给出逐条溯源标注,便于回溯原文要点。文中每处直接源自原文的结论或推断均给出页码标识(如 ),如引用多页则并列给出,但同一处引用不超过两页。
2.2 单次前瞻(The single pass)
2.3 Cross-model 对抗审阅(mad-research)
2.4 A multi-agent workshop( paper-workshop)
2.5 Deviations from the pre-analysis plan(对照计划的偏差)
4.2 成本与回报(The cost of a report)
4.3 鲁棒性分析(Robustness)
4.4 AI 报告与人类评审(The AI reports and the human referee)
4.5 作者–机器 concordance(Author–machine concordance)
4.6 讨论(Discussion)
4.7 结论性综合(Conclusion)
图表与表格深度解读要点汇总
总体结论(综合应用)
溯源快速索引(按核心结论与可追溯句点给出页码)
最终总结
2607.14713.pdf