From Raw Data to Agent-Ready Context

01 The Pipeline Problem

Building a market view for an AI agent from raw data is a pipeline problem, not a data problem. The data exists. CoinGecko, FRED, Blockchain.com, and a dozen other APIs will give you the numbers. The problem is what happens between "numbers from 5 APIs" and "an agent that can make a decision."

Where the work happens (per decision)

Deterministic work - faster and cheaper on a server

Fetch

3-5 API calls

~800ms

Parse

Normalize schemas

Handle errors

Calculate

Trends, percentiles

Rolling windows

Format

Serialize for LLM

Compress tokens

Reason + Decide

What the LLM is actually good at

An LLM can orchestrate all of these steps through tool calls. The question is whether it should. Computing a 30-day moving average from raw data is deterministic math. An LLM doing that work burns reasoning tokens on its weakest capability instead of its strongest: interpreting ambiguity and making decisions under uncertainty.

Steps 1 through 4 are not trivial. Each API has a different schema. Rate limits vary. Some return nested JSON three levels deep. Some use epoch timestamps, some use ISO strings. Null handling differs. Error responses differ. Every one of these differences is a line of code you write, test, and maintain. And if any API changes its response format or goes down, the entire pipeline breaks.

02 The Operational Cost Nobody Talks About

Token cost gets the attention, but it is the smallest part of the problem. The real cost is operational. Running multiple API connections means managing multiple rate limits, multiple authentication schemes, multiple failure modes.

When CoinGecko's free plan caps you at 100 calls a minute and FRED returns data with a two-day lag, your pipeline needs to handle both gracefully. Every API is a dependency that can break independently.

Then there is the computation. Turning raw numbers into useful context requires trend calculation (7-day, 30-day, 90-day moving averages), percentile ranking against historical distributions, cross-metric correlation detection, and interpretation logic that converts "net liquidity is $5.98T" into "net liquidity is rising 1.2% over 7 days, 62nd percentile, momentum accelerating." This is server-side computation that runs before the LLM ever sees the data.

If you push this work into the LLM prompt instead of doing it server-side, you pay twice. You pay in tokens for the raw data that the LLM has to parse. And you pay in reasoning quality because the LLM is spending capacity on arithmetic and formatting instead of market analysis. An LLM computing a 30-day moving average from a JSON array of 30 price points is wasting its strongest capability (reasoning under uncertainty) on its weakest (deterministic math).

DIY Pipeline

Plus: code maintenance, error handling, rate limit management

Pre-Analyzed API

Plus: zero infrastructure code to maintain

03 What Agents Actually Receive

The difference between raw data and pre-analyzed context is not cosmetic. Pre-analyzed context changes what the LLM spends its reasoning capacity on.

Raw API Response (CoinGecko + FRED)

{
"bitcoin": {
"usd": 84200,
"usd_24h_change": -2.3142,
"usd_24h_vol": 28400000000,
"last_updated_at": 1743552000
}
}
// + FRED WALCL response (different schema)
// + FRED RRPONTSYD (different schema)
// + FRED WTREGEN (different schema)
// + Blockchain.com hash rate
//
// Agent must: parse 5 schemas,
// compute net liquidity formula,
// calculate 7d/30d trends,
// compare to historical ranges,
// THEN start reasoning about markets

~3,000+ tokens across 5 calls. No interpretation.

PreReason Briefing (single call)

get_context btc.quick-check (MCP), as served on October 12, 2026
# Quick Check
*Generated: Oct 12, 2026 08:52 UTC*
...
> **Signal:** BTC down 3.4% 7d, liquidity expanding over 30d: divergence: price falling despite improving liquidity, weak correlation (-0.06)
## Bitcoin Metrics
...
BTC at $83,496.83; 7d -3.4% (falling), 30d +8.2%, 90d +34.2%. Accelerating downward: recent losses exceeding the longer term pace
*Data as of: Oct 12, 2026 08:52 UTC*
## Macro Environment
### Net Liquidity [At-Trend]
Net liquidity at $5.87T: Fed total assets (Wednesday level, 2026-10-07) less the Treasury General Account (week average to 2026-10-07) and overnight reverse repo (daily, 2026-10-09); 7d +1.3% (flat, within ±2%), 30d +1.7%, 90d -1.6%.
*Data as of: Oct 9, 2026 00:00 UTC*
## Cross-Asset Correlations
- **Bitcoin Price vs Net Liquidity:** no meaningful correlation, -0.06 (n=23d)
*Window 30 days; 23 daily returns.*
---
*Sources: Coinbase | PreReason (calculated)*
*btc.quick-check | free | Terms: /api/context/glossary*

A full briefing measured 929 to 4,618 tokens in markdown (September 2026), 1,610 at the median. One call. Decision-ready.

The raw JSON tells the agent that the price is $84,200 and the 24h change is -2.31%. It says nothing about whether that decline is unusual, what the weekly trend looks like, or what the macro backdrop implies. The agent has to figure all of that out, burning reasoning tokens on arithmetic that a server could have done in milliseconds.

The pre-analyzed format delivers the conclusion alongside the evidence. The agent reads "down 2.3% over 24h but up 4.1% over 7d, momentum decelerating, regime: transition leaning risk-on" and goes straight to its own decision. The API comparison covers which providers offer each level of data preparation.

04 Token Math

Input tokens dominate LLM cost. For most financial agent calls, the input-to-output ratio is roughly 100:1. Optimizing the input format is the highest-leverage cost reduction available.

Token Budget Comparison

Estimates based on GPT-4 class tokenizers. Actual counts vary by model and by briefing: measured on 18 live briefings in September 2026, the markdown briefing used 68% to 94% fewer tokens than the raw rows behind it, 84% at the median.

The 94% reduction is not just a cost saving. Those 19,750 saved tokens become room for longer reasoning chains, more conversation history, and more decisions per session before hitting context window limits. For an agent making 50+ decisions per day, that compounds quickly.

05 Why Not Just Use Web Search

LLMs with web search can pull market data on the fly. An agent can search "Bitcoin price today" and get a current number. For a one-off question, this works. For repeated automated decisions at professional scale, it breaks down in ways that are not obvious until you audit the output.

Web search results are not deterministic. The same query returns different snippets depending on ranking changes, featured snippet rotation, and geographic location. An agent that searched for "net liquidity today" yesterday and got a FRED-sourced number might get a blog post estimate tomorrow. There is no schema, no versioning, no guarantee that the source or format will be consistent across runs.

Audit trailing is the deeper problem. In a professional context, every decision needs a traceable data lineage: what data was received, from what source, at what timestamp, and how it was interpreted. A structured API call returns a versioned response with a timestamp and source attribution. A web search returns a snippet that may not exist at that URL by the time the audit happens.

Web Search

Non-deterministic results

No schema or versioning

Source varies per query

No audit trail

Latency varies (1-5s)

LLM parses unstructured HTML

Structured API

Deterministic responses

Versioned schema

Consistent source attribution

Full audit trail per call

Predictable latency (~200ms)

Pre-formatted for LLM consumption

Web search has a role for exploratory questions where the answer space is open. For repeated, auditable financial decisions where the same metrics need to be retrieved the same way every time, a structured API call is the baseline requirement.

06 Practical Setup

MCP (Model Context Protocol) standardizes how agents call data tools. A single config entry gives the agent access to pre-analyzed market data through structured tool calls.

claude_desktop_config.json

{
"mcpServers": {
"prereason": {
"command": "npx",
"args": ["-y", "@prereason/mcp"]
}
}
}

With this configuration, the agent calls get_context with a briefing parameter and receives the full pre-analyzed context in a single response. The agent's system prompt tells it when and how to use the tool. Full setup documentation, including all available tools and response formats, is in the API documentation.

07 Measuring Impact

Pre-analyzed context is only valuable if it measurably improves outcomes. The correct approach is to A/B test the data pipeline itself: run the same agent logic with raw data inputs and with pre-analyzed context, then compare decision quality per trade. Aggregate returns can be skewed by a single lucky position. Per-decision metrics isolate the effect of data format from market randomness.

Controlled experiments across multiple agent configurations compared the two. The evidence page documents the full methodology, including the serial portfolio carryover design.

This applies to any financial AI agent builder, not just PreReason users. Whatever data pipeline you use, measure it. Compare your agent's decisions with and without each context layer to find out which layers actually carry signal. Regime classification is one context layer that has shown measurable impact in controlled settings.

Related Insights