Independent research
Twelve books. One benchmark.
Twelve simulated portfolios test whether AI-assisted trading beats simply holding the S&P 500, and which variables matter: model, decision frequency, account size. Four Claude models each manage their own books and place their own trades, fully autonomously, over six months. Every month they report back to the owner on their own record, including the calls they got wrong.
No real money is traded and no orders are placed with any broker. Nothing here is financial advice.
Highest book change is the best single book of the twelve, net of trading costs — currently Sonnet 5, $1,000 daily. The S&P 500 figure is the index over the same period. Every book, ahead and behind, is listed below.
The design
Three variables crossed: which model, how often it decides, and how much it started with. Every book is independent — separate cash, separate positions, separate history — and no model ever sees another’s book, trades, or reviews.
| Model | $1,000 hourly | $1,000 daily | $1,000 10-min | $2,500 hourly | $2,500 daily | $2,500 10-min |
|---|---|---|---|---|---|---|
| Sonnet 5 | — | — | ||||
| Opus 5 | — | — | ||||
| Fable 5 | — | — | — | — | ||
| Haiku 4.5 | — | — | — | — |
Not every model runs every cadence. Fable 5 is daily only: it is the most expensive model here, and halving its rounds kept the experiment inside its budget without losing the comparison it contributes to. Haiku 4.5 runs the ten-minute grid only — it is a separate question about trading speed rather than part of the model comparison, and it is configured differently to run there.
Value over time
Colour identifies the model and its cadence; line style the starting balance — dotted $1,000, solid $2,500. So the two lines sharing a colour are the same model on the same schedule, differing only in the money they began with. The axis is not zero-based, so a $1,000 and a $2,500 book can share it without flattening both.
What the models say about themselves
At the start of each month every model reviews its own record against the benchmark and writes rules for itself. Those rules are then injected into its own future trading prompts, so a review it writes is a review it has to live with.
Fable 5
2026-09-01Both books lost to SPY over the period: s4 ($2,500) returned -3.07% net and s2 ($1,000) returned -1.59% net, versus SPY's -0.46%. That is underperformance of roughly 2.6 and 1.1 percentage points respectively. Five closed trades across both books produced zero winners, which is the single clearest fact in the data.
Rules it now trades by
- — Before entering, write the thesis and the condition that proves it wrong; if you cannot state the invalidation, do not trade.
- — Default to no trade: with 0 winners in 5 closed trades, require a specific, checkable reason why this entry beats holding cash.
- — Exit when the stated thesis breaks, not when the loss becomes uncomfortable — the trigger is the reason failing, not the price alone.
- — Do not add trades to make up prior losses; trade count so far has correlated with worse returns, not better.
Sonnet 5
2026-09-01Both hourly books lost to SPY and lost outright: s1 -1.64% net and s3 -2.20% net versus SPY's -0.46%, while both daily books beat SPY narrowly with s2 +0.38% net and s4 +0.10% net. Trading costs were 3-10x higher on the hourly books ($1.95 and $7.31 vs $0.30-$0.75), directly eating into the only accounts that were losing money anyway.
Rules it now trades by
- — Do not hold a losing position longer than a winning one — if a thesis is invalidated, exit at least as fast as you'd take profit.
- — On hourly cadence, count round-trip costs against the expected edge before entering; skip trades where the edge is thin.
- — Prefer daily-or-slower cadence unless hourly trades show a demonstrated hit rate above 50% over a larger sample.
- — Track hit rate and hold-time asymmetry every review; a repeated losers-held-longer pattern is the single most correctable behavior here.
- — Treat one month of daily-beats-hourly as a hypothesis, not a rule, until it repeats over more decision rounds.
Opus 5
2026-09-01SPY returned -0.46% over the period and beat three of my four books. The two hourly books lost badly on a relative basis: s3 -3.90% net and s1 -2.96% net, i.e. roughly 250-345bp of underperformance. The two daily books were essentially flat and roughly matched the benchmark (s2 +0.01% net, s4 -0.08% net), which in practice means they did the least harm by trading the least.
Rules it now trades by
- — Write the thesis and the invalidation price before entering. No thesis, no trade; thesis broken, exit immediately regardless of P&L.
- — Require expected move of at least 5x round-trip cost to open a position. If the setup only exists because I am looking right now, it is noise.
- — On hourly cadence, doing nothing is the default. Most hours contain no tradeable information; act only when a pre-written condition triggers.
- — Underperforming a falling market means the loss was mine, not the market's. Never use 'conditions were bad' as an explanation.
- — Fewer, larger, independent positions beat many small correlated ones. Do not replicate the same bet across books.
- — Hold winners until the thesis completes; cut losers at the invalidation level. Never widen a stop to avoid taking a loss.
- — One month and eight closed trades prove nothing. Do not adopt or abandon a strategy on this sample; do enforce the process rules above regardless.
Method
- — Every model receives identical market data each round, assembled once and hashed, so “identical inputs” is a recorded fact rather than an intention.
- — Trades decided before the opening bell are parked, not executed, and fill at the official opening price — never at the pre-market quote, which would hand the portfolio the overnight gap for free.
- — Trading costs are charged on every trade and disclosed to the models in advance.
- — Configuration was frozen when the experiment began. Cadence, starting cash, cost parameters and model versions cannot change mid-run.
- — A missed round is never replayed. Achieved decision counts, not intended ones, are the basis of every figure here.