Simulated backtest design: three arms in runs 1 to 5, four in runs 6 and 7, with controlled variables held across the arms of a run. Only the injected context differs between arms. Everything else is identical.
Opus 4.6 was trained through August 2025. Every tick in these backtests runs from September 2025 through March 2026. The model is making trading decisions about market events it was never trained on. PreReason briefings provide the real-time context that bridges this information gap.
Bitcoin price, macro indicators, on-chain metrics, and cross-asset signals. Sourced through PreReason's data pipeline.
Why: Fresh, multi-domain data is the input. Without it, briefings would be stale.
Current BTC spot price. Provided to ALL arms equally. This is the controlled variable.
Why: Ensures every arm sees the same market state. Only the context layer differs.
per-tick loop startsTransforms raw data into structured analysis: regime classification, momentum signals, energy models, breadth indicators. 4 modular briefings per tick.
Why: This is what we're testing. Does structured, pre-analyzed context give the LLM a measurable edge over raw data or web search?
Gets real PreReason briefings + BTC price
Tests the full product
Gets BTC price only. No external context
Baseline: what happens with zero context
Gets web search results + BTC price
Tests: is structured analysis better than raw information?
Gets PreReason briefings from 150-210 days ago
Tests: is it the content or just having context
Opus 4.6 (Anthropic), same prompt structure, temperature 0 for deterministic output. Each arm makes independent decisions. No cross-contamination.
Why: Identical decision-making process ensures the only variable is the injected context.
Each tick simulates one trading day. The model can BUY, SELL, SHORT, COVER, or HOLD. Position sizing is capped at 80% of portfolio per trade and 80% gross exposure. Transaction costs: 0.045% per trade. Short funding: 0.01% per 8-hour period. No slippage model applied (conservative assumption). Portfolio state carries forward between ticks with no resets.
Why: Identical execution rules for all arms. The only variable is the injected context.
loop back to topAll decisions logged with full reasoning text. No manual intervention during any run. All code paths shared between arms. Only the injected context differs.
Why: Full auditability. The experiment is only as credible as its controls.
The Placebo arm receives the identical preamble as Treatment. Same signal framework, same output structure, same analytical vocabulary. In run 7, its briefing data came from 150 to 210 days before each tick.
It receives the same briefing format with old data, to separate the format from the age of the data. Run by run, against the price only arm:
| Run | Placebo | Control | Gap (points) |
|---|---|---|---|
| Run 1 | -5.19% | -8.44% | +3.25 |
| Run 2 | -6.79% | -6.49% | -0.30 |
| Run 3 | -22.11% | -15.69% | -6.42 |
| Run 4 | -16.81% | -14.69% | -2.12 |
| Run 5 | -18.85% | -6.31% | -12.54 |
| Run 6 | -1.55% | -9.51% | +7.96 |
| Run 7 | -7.70% | -8.14% | +0.44 |
Across the seven runs the stale briefings arm ranged from 12.54 points below the price only arm (run 5) to 7.96 above it (run 6).
The Control-WS arm receives live web search results for every tick. These searches surface real-time coverage that Treatment's briefings do not include: FOMC decisions, ETF flow data, Fear & Greed Index readings, exchange-specific news. The web search arm could search the web; the briefings arm read PreReason's briefings and no web results.
In Run 6, Treatment beat Control-WS by +2.24pp. In Run 7, the gap widened to +9.38pp. In these two simulated runs, the arm with narrower but pre-analyzed information ended ahead of the arm with broader but unstructured information.
All arms run through identical trade execution, portfolio tracking, and fee calculation. Transaction costs are 0.045% per trade. Short funding accrues at 0.01% per 8-hour period. Temperature is fixed at 0. A direction-flip cooldown prevents whipsaw within a single tick. The only variable across arms is the context injected before the LLM makes each trading decision.
Each tick decision is executed via Claude Code's Agent tool, which spawns a fresh subprocess with its own context window. The Agent receives only its arm-specific system prompt and a single user message. No conversation history, no shared memory, no cross-arm contamination. Portfolio state is replayed deterministically from completed tick files on disk before each call. Full details in the Research Deep Dive.
For the full methodology rationale, including preamble text and reasoning traces, see the Research Deep Dive.
Research Deep Dive
Research findings and analysis across all backtest runs
Context Engineering for Crypto Agents
How context format drives trading agent performance
Financial Data APIs for AI Agents
API comparison for data sources that power agent decisions