← All tasks
T13 · Factor Investing

Empirical Asset Pricing via Machine Learning

Shihao Gu et al. (3 authors), 2020 · The Review of Financial Studies

3%
pass@1 across 16 systems
2/16
systems with ≥1 successful attempt
1/5
1/5 attempts passed for the best system
80 attempts · 16 systems × 5
Claude Code + Opus 5
0%
Claude Code + Opus 4.8
0%
Claude Code + Opus 4.7
0%
Codex + GPT-5.6-sol
20%
Codex + GPT-5.5
0%
OpenCode + Kimi-K3
0%
OpenCode + Opus 4.7
20%
OpenCode + GPT-5.5
0%
OpenCode + Sonnet 4.6
0%
OpenCode + Qwen3.7-Max
0%
OpenCode + GLM-5.2
0%
OpenCode + DeepSeek V4-Pro
0%
OpenCode + GLM-5.1
0%
OpenCode + Qwen 3.6 Plus
0%
OpenCode + MiniMax-M3
0%
OpenCode + MiniMax M2.7
0%
all checkers pass Type-A clean, Type-B fail fail checking timeout no result / incomplete
Strategy

What the paper asks for.

A 3-layer feed-forward neural network (NN3, hidden widths 32-16-8) trained on cross-sectionally rank-normalized firm characteristics produces monthly excess-return forecasts whose top-vs-bottom decile equal-weighted long-short portfolio earns a positive risk-adjusted spread.

Type
ml
Implementation
pandas
Rebalance
monthly
Long-short
yes
Declared task setup
Data
One-command fetcher for WRDS-sourced task data
Operational scope
The paper-defined methodological core is retained; the fixed instruction instantiates only the data mappings, interfaces, and evaluation conventions needed for execution.
Six-stage quantitative strategy pipeline
  1. 01 Data loading
  2. 02 Feature engineering
  3. 03 Model fitting
  4. 04 Signal generation
  5. 05 Portfolio construction
  6. 06 Execution
Checker suite

23 properties · 80 attempts.

Each row is one requirement the strategy must satisfy. The bar reports the share of observed checker verdicts that pass; missing checker output is not converted into a property verdict.

TYPE A · DOMAIN-LEVEL (10)
  • A1
    no backfill
    stage · data loading
    76/79
  • A2
    no global normalize
    stage · feature eng
    73/79
  • A3
    no future data access
    stage · feature eng
    77/79
  • A4
    causal consistency
    stage · signal gen
    74/79
  • A5
    no survivorship bias
    stage · portfolio
    79/79
  • A6
    train test temporal order
    stage · model fit
    79/79
  • A7
    data frequency alignment
    stage · data loading
    79/79
  • A8
    transaction timing
    stage · execution
    78/79
  • A9
    signal delay test
    stage · signal gen
    78/79
  • A10
    e2e metric consistency
    75/79
TYPE B · PAPER-SPECIFIC (13)
  • B1
    nn3 architecture 32 16 8
    §§2.2.4 “and 16 neurons, respectively; NN3 has three hidden layers with 32, 16, and 8 neurons, respectively;”
    provenance · paper-explicit
    stage · model fit
    49/79
  • B2
    feature cross sectional rank to neg1 pos1
    §§3.1, footnote 29 “We cross-sectionally rank all stock characteristics period-by-period and map these ranks into the [-1,1] interval”
    provenance · paper-explicit
    stage · feature eng
    51/79
  • B3
    missing chars imputed cross sectional median
    §§3.1, footnote 30 “Another issue is missing characteristics, which we replace with the cross-sectional median at”
    provenance · paper-explicit
    stage · feature eng
    77/79
  • B4
    target variable one month excess return
    §§3.1 “the average number of stocks per month exceeding 6,200. [28] We also obtain the Treasury-bill rate to”
    provenance · paper-explicit
    stage · model fit
    72/79
  • B5
    decile sort n quantiles 10
    §§3.4.2 “one-month-ahead out-of-sample stock return predictions for each method. We then sort stocks into”
    provenance · paper-explicit
    stage · signal gen
    73/79
  • B6
    long top decile short bottom decile
    §§3.4.2 “Finally, we construct a zero-net-investment portfolio that buys the highest expected return stocks”
    provenance · paper-explicit
    stage · signal gen
    73/79
  • B7
    equal weighting within legs
    §§3.4.2 / Table A.9 “to those of Table 7, but the Sharpe ratios are substantially higher. For example, the long-short”
    provenance · paper-explicit
    stage · portfolio
    75/79
  • B8
    monthly rebalance frequency
    §§3.4.2 “of portfolios to directly exploit machine learning forecasts. At the end of each month, we calculate”
    provenance · paper-explicit
    stage · portfolio
    55/79
  • B9
    annual refit with expanding or rolling window
    §§3.1 “machine learning algorithms are computationally intensive, we avoid recursively refitting models each”
    provenance · paper-explicit
    stage · model fit
    74/79
  • B10
    evaluation period 2010 2019
    §Section 3.1 “We divide the 60 years of data into 18 years of training sample (1957-1974), 12 years of validation sample (1975-1986), and the remaining 30 years (1987-2016) for out-of-sample testing.”
    provenance · benchmark-specified
    stage · data loading
    69/79
  • B11
    r2 oos no demeaning zero benchmark
    §§2.7 (Performance Evaluation) “A subtle but important aspect of our _R_ [2] metric is that the denominator is the sum of squared”
    provenance · paper-explicit
    stage · model fit
    79/79
  • B12
    prediction directional sanity
    §Sections 2.2.4, 3.1, and 3.5 “The models produce one-month-ahead excess-return predictions used to sort stocks cross-sectionally.”
    provenance · paper-inferred
    stage · model fit
    74/79
  • B13
    sp sign convention
    §Section 2.2.4 “The last group includes valuation ratios and fundamental signals, such as earnings-to-price (ep), sales-to price (sp).”
    provenance · paper-explicit
    18/79
What this task reveals

The lowest-passing observed requirement is sp sign convention: 18 of 79 observed checker verdicts pass. The full property list above shows whether errors concentrate in domain-level validity or the paper-specific strategy logic.

Open paper → All tasks
← Previous in Factor Investing
T12 · Factor Momentum and the Momentum Factor
Next in Factor Investing →
T14 · Timing the Factor Zoo