Scalar Markets Data: 3 Feeds for Realistic Agent Backtests

Scalar markets data is the unified real-time and historical feed of range-settled prediction markets, covering orderbook state, trade prints, and settlement outcomes across venues. Ingesting it correctly lets you backtest, run, and scale algos and AI agents against execution-realistic conditions rather than idealized prices. Platforms like Assymetrix aggregate this data across major venues into a single schema, which is the starting point for any serious integration.

TL;DR:

• Capture trade prints, full orderbook depth, lifecycle events, range endpoints, resolution rules, and derived prices; millisecond timestamps preserve event order for deterministic replay.

• Use WebSockets for live top of book and trades, REST for metadata and snapshots, and bulk columnar exports for historical backtests.

• Build canonical IDs from venue, normalized range endpoints, and an internal slug; preserve source provenance and normalize prices to probabilities from zero to one.

• Backtests should model venue specific maker and taker fees by fill, historical depth based fill probability, slippage, latency, and fee schedule versions.

• Keep sizing and risk limits in a deterministic execution layer, cap each position, monitor fills and slippage, and test in paper mode before live deployment.

Assymetrixdata.assymetrix.comBuild More Realistic Market BacktestsAssymetrix brings real-time and historical prediction market data from Polymarket, Kalshi, and Limitless into one API integration.Explore the Data API

Table of Contents

• 1. What data fields you need for scalar markets

• 2. How to access scalar markets data: APIs, streaming, and exports

• 3. Normalizing cross-venue scalar data and canonical IDs

• 4. Orderbook, fee models, and execution realism for agents and algos

• 5. Backtesting and replay: episode formats that stay deterministic

• 6. Engineering practices for AI agents consuming scalar data

• Where belief meets execution in scalar markets

• How the Assymetrix Data API covers these needs

• FAQ

• Sources

1. What data fields you need for scalar markets

A scalar, or range-settled, market resolves to a numeric value inside a continuous range rather than a binary yes or no, so the data you pull has to capture both price and the shape of the distribution traders expect. Reconstructing market state at any point in time requires five field groups.

• Trade prints: timestamp, price, size, side, and wallet ID for every executed fill.

• Orderbook snapshots: bid and ask levels, depth at each level, and visible resting size.

• Lifecycle events: market open, suspension, close, and the settlement window itself.

• Market metadata: numeric endpoints of the range, resolution rules, tick size, and quote currency.

• Derived series: implied probability, midprice, cumulative volume, and liquidity measures computed from the raw feed.

Timestamp precision matters more than it looks. Millisecond or sub-second precision is what lets you order trades against orderbook updates correctly; second-level timestamps collapse fast sequences into ambiguous ordering, which breaks replay determinism later. Get this field list right and the rest of the pipeline, from normalization to backtesting, inherits a solid foundation.

2. How to access scalar markets data: APIs, streaming, and exports

Three transport patterns cover almost every integration need, and most production systems end up using all three at once.

1. REST for metadata and historical snapshots, where request volume is low but payload completeness matters.

2. WebSocket for top-of-book and trade streaming, where latency matters more than payload size.

3. Bulk exports in Parquet or similar columnar formats for large-scale backtests that need months of history in one pull.

Polymarket’s order book endpoint illustrates the REST side well: it returns aggregated bid and ask arrays with configurable depth, typically defaulting to three levels and capping at ten, alongside instrument state and requires bearer-token authentication. Rate limits on these endpoints push most serious integrations toward exponential backoff rather than fixed-interval polling, and a fetch cadence tied to market volatility, more frequent during active hours, sparser overnight, keeps you inside limits without starving your models of fresh state.

A workable integration recipe: subscribe via websocket for top-of-book, pull periodic full snapshots for deeper book reconstruction, and combine trade prints with bulk exports for historical replay. Our own developer guide to real-time and historical access walks through this pattern in more detail.

Pro Tip: Cache orderbook snapshots locally and diff against streamed updates; it catches silent websocket drops before they corrupt your book state.

3. Normalizing cross-venue scalar data and canonical IDs

Polymarket, Kalshi, and Limitless each expose scalar markets with different field names, different settlement conventions, and different ways of describing the same underlying question. Without a canonical schema, joining feeds across venues turns into a matching problem you solve badly, over and over.

• Build canonical IDs from a venue tag, normalized numeric endpoints, and an internal slug, so the same underlying market resolves to one identifier regardless of source.

• Normalize price to implied probability on a 0 to 1 scale, and handle tick-size differences explicitly rather than rounding silently.

• Harmonize metadata fields including settlement window, resolution rules, and question text, since wording differences across venues otherwise look like different markets.

• Detect cross-venue equivalents algorithmically, but retain provenance on every record so you can trace a normalized field back to its original source if something looks wrong.

This is the step most teams underestimate, and it is also the step that makes Smart Money tracking and arbitrage signals possible once it is done right. Our guide to canonical IDs for cross-venue data goes deeper on the matching logic.

4. Orderbook, fee models, and execution realism for agents and algos

A forecast is worthless to an algo if the execution layer cannot translate it into a realistic fill. Modeling that realism requires data most teams don’t collect by default.

• Maker and taker fees differ by venue and sometimes by market, so per-fill fee accounting has to be tracked at the trade level, not assumed as a flat rate.

• Fill probability should be modeled from historical fills against orderbook depth at the time, not assumed as instant and complete.

• Slippage and latency between signal generation and order placement erode edge fast, especially in thin scalar markets.

• Fee schedules change over time, so replay systems need versioned fee models attached to each historical period, not just the current schedule applied retroactively.

Execution realism dominates forecasting gains more often than teams expect. Research on deterministic replay frameworks finds that fee-aware simulation and maker/taker semantics materially affect realized profit and loss and can swamp the improvements from a better forecasting model. A sharper signal with a naive execution assumption often underperforms a mediocre signal with realistic fill modeling. Our orderbook data guide for developers covers depth-aware fill modeling in practical terms.

5. Backtesting and replay: episode formats that stay deterministic

Deterministic replay is what separates a backtest you can trust from one that quietly lies to you. PredictionMarketBench standardizes this with a four-file episode format that is worth adopting directly.

1. metadata.json: market definition, numeric endpoints, resolution rules, and the fee model version in effect.

2. orderbook.parquet: sequenced orderbook updates with UTC timestamps for exact state reconstruction.

3. trades.parquet: every trade print with side, size, price, and sequence number for tie-breaking against orderbook events.

4. settlement.json: the final settlement value and the timestamp the market closed.

Unifying the timeline matters as much as the files themselves: UTC timestamps throughout, sequence numbers to resolve same-millisecond ties, and an explicit fee-model version attached to the episode rather than inferred from the run date. Track profit and loss, drawdown, total fees, slippage, fill ratio, and forecast skill metrics on every replay, since those six numbers tell you whether a strategy’s edge is real or an artifact of unrealistic fills. Our backtesting walkthrough using historical snapshots applies this episode format to a worked example.

6. Engineering practices for AI agents consuming scalar data

The gap between a good forecast and an executed trade is where most agent systems fail, and closing it is an engineering problem, not a modeling one.

• Separate the forecaster from a deterministic execution layer that enforces position sizing, hard caps, and stop-loss rules independently of model confidence.

• Size positions with quarter or fractional Kelly against a hard USD or USDC cap per position, never scaling purely off model-reported confidence.

• Budget for latency between signal and order, cache aggressively on metadata that rarely changes, and respect venue rate limits with backoff rather than retries.

• Keep audit logs of every decision, run monitoring and alerting on fill ratios and slippage, and run a paper mode alongside live trading before promoting any strategy.

Research on belief-to-trade architectures finds that execution layers enforcing deterministic sizing and risk constraints produce better risk-adjusted returns in replay than forecast-only sizing, largely because they prevent overleveraging on model errors. A canary market and a daily loss limit catch most of the failure modes before they become expensive. Teams building this kind of agent infrastructure sometimes bring in outside engineering support; Bitrupt’s AI and data engineering practice builds production LLM and agent pipelines for exactly this kind of work.

Pro Tip: Run new strategies against a canary market with real capital but a tiny cap before scaling position sizes; a model that looks great on 2026 replay data can still behave unpredictably on live orderbook latency.

Where belief meets execution in scalar markets

The hardest problem in this space isn’t forecasting. It’s the belief-to-trade gap: the distance between a model that knows something and a system that acts on it without bleeding the edge away in fees and slippage. Research on autonomous agent wallets finds agents improve price discovery on text-heavy decisions but lose their edge on numerically intensive tasks, a reminder that forecasting strength is domain-specific. Our own feeds span roughly 1.5 terabytes of historical data across the major venues, with Smart Money wallet tracking built on top. The immediate priority for most teams: get canonical IDs and fee versioning right before touching sizing logic.

— Dean

How the Assymetrix Data API covers these needs

We built our Data API at Data around the exact pipeline this guide describes: one integration instead of three separate venue connections, with normalization already handled.

• Unified canonical IDs across venues so cross-venue joins don’t require custom matching logic.

• Real-time streaming and historical bulk access drawn from a large dataset spanning significant rows of trading activity.

• Smart Money wallet tracking and cross-venue arbitrage signals layered on top of the same normalized schema.

Developers can start with our prediction market API documentation to see endpoint structure and authentication before wiring up a first integration.

FAQ

What makes scalar markets different from binary markets?

Scalar markets settle to a numeric value inside a range, while binary markets settle to yes or no. Data for scalar markets needs the numeric endpoints and resolution rules captured explicitly, since the settlement value itself carries information a binary outcome doesn’t.

Which venues does scalar markets data typically cover?

Coverage typically spans Polymarket, Kalshi, and Limitless, each with distinct data characteristics such as orderbook depth conventions and settlement timing. Normalizing across these three is the common starting point for cross-venue analysis.

What’s the minimum data needed for a deterministic backtest?

At minimum you need orderbook snapshots, trade prints, and settlement records tied together with UTC timestamps and sequence numbers, following formats like the one described in PredictionMarketBench. Fee-model versioning should be included so replays stay comparable as venue fee schedules change.

How do I handle rate limits across multiple venue APIs?

Use bearer-token authentication with exponential backoff rather than fixed polling intervals, and adjust fetch cadence to market activity rather than a flat schedule. Bulk exports reduce rate-limit pressure significantly for historical work compared to repeated REST calls.

Does Assymetrix offer a unified API for all three venues?

Yes, our Data API aggregates Polymarket, Kalshi, and Limitless into a single schema with canonical IDs, documented at Assymetrix. Pricing details are available on request through that page.

Sources

• PredictionMarketBench: A SWE-bench-Style Framework for Backtesting Trading Agents on Prediction Markets

• Polymarket API — Get order book

• The Performance and Risks of Autonomous LLM Agents in Financial Prediction Markets — 12 August 2026 Codex Revision