Introduced Control-WS arm. Treatment beats web search by +2.24pp. Proves structured analysis > raw information
Key Statistics
Period: Sep 1, 2025 - Mar 6, 2026
Ticks: 187
Arms: 4
Model: Opus 4.6
BTC Move: -34.3%
Arm Returns
Arm
Return
Treatment (Briefings)
+3.18%
Control (Price Only)
-9.51%
Placebo (Stale Briefings)
-1.55%
Control-WS (Web Search)
+0.94%
Deltas
Treatment vs Control: +12.69pp
Treatment vs Web Search: +2.24pp
Information Value (Treatment minus Placebo): +4.73pp
What Changed
Added 4th arm (Control-WS) that receives web search results instead of PreReason briefings. Tests whether the value is in the analysis, not just having information.
Market Context
Same window as Run 5: Sep 2025 through Mar 6, 2026. BTC dropped -34.3% from $108K to $71K. This run introduced the 4th arm (Control-WS) to test whether the treatment's edge came from having information at all, or specifically from PreReason's structured analysis.
Observations
The headline result is the treatment-vs-control-WS delta: +2.24pp. The Control-WS arm, which received live web search results instead of PreReason briefings, returned +0.94%. This means having raw, real-time information gets you most of the way (control-WS beat price-only control by +10.45pp), but structured analysis adds a further edge on top.
The Control-WS arm's trading pattern was notably different from both treatment and control. It made more trades (higher frequency) and showed a pattern of reacting to headlines: going short after negative news, then reversing quickly. Its +0.94% return was respectable, but the higher trade count meant more fees and more whipsaw losses.
The treatment arm returned +3.18%, lower than Run 5's +8.46% despite the same briefings and window. This variance is within expected bounds for a 187-tick run. The treatment arm missed the Mar 5 bounce that it caught in Run 5, likely due to slightly different prompt sequencing in the 4-arm setup.