Why this benchmark exists.
Quantitative researchers routinely translate ideas from papers into executable strategies. In automated quantitative R&D, coding agents can perform this work at scale, but interactive expert correction cannot scale with them. QuantCodeEval measures whether agents can carry a paper's strategy semantics through a complete implementation under declared data and interface requirements.
The QuantCodeEval paper is the authoritative reference for every claim on this site. Where the wording here differs from the paper, the paper is canonical.
What this benchmark insists on.
Property-based, not metric-based. A backtest can hit the headline Sharpe for the wrong reasons. We check properties of the code and the strategy — temporal alignment, signal formulas, position sizing, weighting — not whether a single end-to-end number matches.
30 leading finance papers across six categories. Momentum, Mean Reversion, Factor Investing, Volatility Management, Event-Driven Trading, and Systematic Allocation. The taxonomy follows practitioner and textbook classifications and was validated by experienced quantitative trading experts. Five papers per category, primarily from top-tier venues such as the Journal of Finance and the Journal of Financial Economics.
A reproducible release with a clear data boundary. Ten tasks include bundled public data; twenty provide one-command fetchers for WRDS-sourced inputs. The release includes task contracts, checkers, traces, canonical result tables, and scripts that regenerate the figures. This site is generated from the same versioned results.
A collaboration with the practitioner community.
QuantCodeEval is built jointly by computer-science researchers and quantitative trading experts. The roles below reproduce the contributor pool described in the paper. Individual identities and institutional affiliations are withheld for double-blind review and will be revealed when the paper exits review.
Selected the 30 benchmark papers (5 per category) from a 60-paper shortlist, validated the six-category taxonomy, and independently annotated paper-specific properties. Each has 10+ years of industry experience.
Reviewed task specifications and golden-reference implementations against the source papers, including the operational choices needed for executable evaluation.
Built the evaluation infrastructure and checker suite. Human annotators also audited checker verdicts on real submissions and reviewed SOTA-5 failures for the root-cause analysis.
Currently under double-blind peer review.
Author identities are withheld. Versioned code, task materials, and result artifacts are linked from the footer; correspondence should go through the review platform.