Trader Leaderboard Analysis for Prediction Markets
Trader Leaderboard Analysis for Prediction Markets
Trader Leaderboard Analysis for Prediction Markets
How to read prediction market trader leaderboards: which metrics matter, how to separate skill from luck, and what to check before building on the data.
Trader Leaderboard Analysis for Prediction Markets
A prediction market leaderboard is a ranking, not a verdict. To analyze one well, treat the rank as a starting hypothesis. Rebuild the metrics from raw trades, test them for sample size and payoff structure, and only then decide whether a wallet carries a signal. This guide covers the metrics that matter, the biases that distort rankings, how venues differ, and what a developer should verify before wiring leaderboard data into a pipeline.
Assymetrixdata.assymetrix.comAnalyze Traders Across VenuesUse Assymetrix’s Data API to access unified prediction market data across Polymarket, Kalshi, and Limitless through a single integration.Explore the Data API
TL;DR
Rank alone hides the method. Profit, return, volume and win rate each answer a different question, and a single sort order rewards some behaviors over others.
Win rate without payoff size misleads. Pair it with profit factor, average win and loss, and drawdown.
Skill versus luck is a sample-size and calibration problem. Require many resolved positions, then test forecasts with a proper scoring rule.
Leaderboards differ across venues in identity, fees, settlement and visibility. Compare only metrics you recompute yourself on normalized data.
Before you build, verify the ranking window, the definition of profit, the refresh cadence and whether the history is complete.
Table of Contents
What a prediction market leaderboard actually measures
The metrics worth computing
Skill, luck, exposure and sample size
How leaderboards differ across venues
From leaderboard to signal
What to verify before you build on leaderboard data
Perspective
Build your own ranking with the Data API
Sources
What a prediction market leaderboard actually measures
Most public leaderboards rank accounts by a single number, usually profit over a window (a day, a week, a month, all time). Some let you re-sort by volume or return. That is useful for discovery and weak for analysis, because the number is a summary of a path, and the path is what you need.
Three questions decide how much a ranking can tell you:
What is being counted as profit? Realized profit on settled markets, or realized plus unrealized marks on open positions? The two can diverge sharply in a market that has not resolved.
Who is being ranked? On wallet-based venues the unit is an address, and one person can operate several. On account-based venues the unit is a username or an anonymized handle. Neither is a verified human.
Which window? A 30-day leaderboard rewards whoever was concentrated in the month’s biggest winner. An all-time board rewards survivors and capital size.
If a leaderboard does not state its answers to these, you cannot interpret the rank. Rebuild the figures from trade-level data instead. Our guide to wallet PnL analytics for prediction markets walks through how to reconstruct realized and unrealized profit per wallet from fills and settlements.
The metrics worth computing
A leaderboard row is a handful of columns. A research-grade profile adds the ones the board leaves out. Trading platforms that document their performance reports are a good template, since the same questions apply. TradingView’s strategy report lists net profit and loss, profit factor, Sharpe and Sortino ratios, average profit and loss, return, and maximum drawdown. Most of these map directly onto a prediction market wallet.
Metric | What it answers | Common trap on a leaderboard |
|---|---|---|
Net profit | How much money was made | Scales with capital, so large accounts dominate |
Return on capital | How efficiently capital was used | Tiny accounts post extreme percentages |
Volume | How active the trader is | High volume can mean market making, not edge |
Win rate | How often positions finish positive | Ignores payoff size |
Profit factor | Gross gains divided by gross losses | Unstable on few trades |
Average win / average loss | Payoff shape | Rarely shown on public boards |
Maximum drawdown | Worst peak-to-trough loss | Hidden by a profit-only sort |
Sharpe / Sortino | Return per unit of volatility | Needs a consistent time series of returns |
Why win rate needs a partner
Win rate is the most quoted and most misread column. One trading-metrics reference notes that win rate alone does not tell the whole story because it ignores the size of wins and losses, and suggests reading it alongside profit factor. Prediction markets make this worse, because payoffs are bounded and asymmetric by construction.
Consider two wallets. One buys contracts priced at 92 cents and wins 95% of the time, collecting 8 cents per win and losing 92 cents on each miss. The other buys 12-cent contracts and wins 25% of the time, collecting 88 cents per win. Their win rates are far apart, yet either can be profitable or ruinous depending on calibration. Only the combination of win rate, average win and average loss tells you which.
Drawdown is the column that is missing
A profit-sorted board shows who ended up on top and says nothing about how they got there. A wallet that made its money in one resolved election market and never traded again looks like a star. A wallet with steady small gains and a shallow drawdown looks ordinary and is far more informative. Compute maximum drawdown from the cumulative realized profit series, and compute it on both a calendar basis and a per-market basis.
For a developer-oriented walk through of the reputation fields you can derive, see 5 trader reputation metrics mapped to API fields.
Skill, luck, exposure and sample size
The hard question is whether a ranking reflects skill. A few checks separate the cases cheaply.
1. Count independent bets, not trades
A wallet that placed 400 trades across three correlated markets has three or four independent observations, not 400. Group fills by market and by underlying event. Then count resolved positions. Public boards often show trade counts, which overstate the evidence.
2. Score forecasts, not just outcomes
Profit conflates forecasting ability with execution, sizing and timing. To isolate the forecast, compare the price a wallet paid against the eventual outcome using a proper scoring rule such as the Brier score. A wallet that consistently buys below the final resolution value, across many independent markets, is showing something profit alone cannot. We cover how to do this reproducibly in Brier audits for prediction markets and traders and compare alternatives in 3 proper scoring rules.
3. Look for concentration
Compute the share of total profit that came from the single best market, and from the best five. If one position explains most of the ranking, you are looking at an outcome, not a process. Rank stability is a useful companion: does the wallet stay in the top decile when you drop its best market?
4. Separate market making from directional trading
High volume with a modest margin often points to liquidity provision or spread capture. Some venues also pay rewards to liquidity providers, and aggregate trackers may blend those into a total. A wallet that earns mostly from rewards is not forecasting anything. Split trading profit from incentives before you label a wallet as informed.
5. Check for survivorship
A leaderboard shows the people who are still on it. Wallets that blew up, stopped trading or rotated to a fresh address are gone from the view. If you evaluate a rule such as “follow last month’s top 20,” you must test it on a set of wallets chosen at the start of each period, including those that later disappeared. Otherwise your backtest inherits the bias.
6. Control for exposure
Prediction markets are not independent assets. A wallet long “yes” across a cluster of related political or sports markets was exposed to one underlying factor. Its profit tells you about that factor’s outcome. Estimate how correlated a wallet’s positions are before you compare it with another.
How leaderboards differ across venues
Cross-venue comparison is where most analyses break. Polymarket, Kalshi and Limitless differ in structure, and each difference changes what a leaderboard column means.
Identity. Wallet-based venues expose addresses, which you can follow across markets but which do not map one-to-one to people. Regulated, account-based venues expose less, and what is public depends on the product and the platform’s own settings.
Fees and rewards. Net profit before and after fees are different quantities. Make sure every venue’s figure is on the same basis.
Settlement and resolution. Resolution rules, timing and dispute processes differ, so “settled” does not mean the same thing everywhere. A position that is a loss on one venue may be unresolved on another.
Market definitions. The same real-world event can be listed with slightly different wording, thresholds or expiry. A like-for-like comparison needs a canonical mapping, not a name match.
Window and coverage. A board’s lookback and its tracked population are set by whoever built it.
The practical rule: do not compare published ranks across venues. Compare metrics you compute yourself, from trade-level data, on a common schema. That is the reason we normalize market and trader identifiers across venues, so the same event and the same behavior line up. For the data model, see our guide to cross-venue prediction market data with canonical IDs and the measurement guide for cross-venue price gaps.
From leaderboard to signal
Once a wallet survives the checks above, leaderboard data becomes input to several workflows.
Cohort analysis. Instead of following individuals, group wallets by a measured property (for example, calibration score above a threshold with at least a minimum number of independent bets) and study what the cohort does. Cohorts are more stable than single wallets, and a signal from fifty weakly informed participants is easier to validate than one from a lone outlier. Our write-up on trader cohort analysis across 900 million indexed events shows how.
Smart Money tracking. The goal is to identify wallets whose positions tend to precede price moves or resolutions, then monitor their new activity. The evidence standard is lead-lag: does the price move toward the wallet’s position after it trades, net of the spread you would pay to follow? See tracking Smart Money wallets on Polymarket and how Smart Money moves before event resolutions.
Alerts and copy logic. If you act on a wallet’s new trades, you inherit latency and slippage. Model the delay between their fill and your order. Our guides to Smart Money alerts and building copy trading with canonical data and rules cover the mechanics.
Validation before capital. Backtest any rule on point-in-time data, meaning you only use information that existed when the trade would have been placed. Then replay it. Read backtesting prediction market strategies with the Data API and paper trading with replayed API data before risking money.
What to verify before you build on leaderboard data
A short checklist for developers and quants:
Definition of profit. Realized, unrealized or both? Gross or net of fees and rewards?
Ranking window and refresh cadence. Daily snapshots are fine for discovery. A signal that depends on recent activity needs trade-level data streamed as it happens, not a re-sorted board. Match refresh frequency to the decay of the signal you are testing.
Population. How many wallets are tracked, and how are inactive or tiny accounts handled? A board capped at its top entries cannot support a percentile analysis.
Point-in-time integrity. Can you reconstruct the leaderboard as it stood on a past date? If the source only offers the current state, your backtest uses information from the future.
Identifier stability. If addresses or usernames change, does history follow? Cross-venue work requires stable canonical IDs.
Completeness. Gaps in history, missing resolutions or deleted markets bias every metric downstream. Reconcile totals against raw fills.
Reproducibility. Store the query, the schema version and the snapshot time with every result.
If you prefer to build in Python, the developer guide to Python prediction market data is a practical starting point.
Perspective
Leaderboards are where most people start and where most serious analysis should leave off. The ranking is built by someone else’s definitions, for someone else’s purpose, over a window you did not choose.
Our view is that the useful object is the trade record. With roughly 1.5 terabytes of history spanning nearly a billion rows of trading activity across Polymarket, Kalshi and Limitless, we can rebuild any leaderboard metric from fills and settlements, and add the ones boards omit: drawdown, payoff shape, calibration, concentration.
Venue neutrality matters here. A ranking produced by one exchange reflects that exchange’s rules. A normalized layer lets you ask the same question of every venue and get an answer you can compare. That is what we are designed to provide, and it is why we treat Smart Money scores as hypotheses to test, not as conclusions to follow.
Dean
Build your own ranking with the Data API
If you want to compute these metrics rather than read them, the Assymetrix Data API gives developers, quants and AI agents unified access to cross-venue prediction market data through a single integration, with normalized schemas for markets and wallets. It supports both real-time and historical use, so the same code that scores a wallet today can reproduce its ranking on a past date.
For an overview of the feeds and endpoints, start with the real-time and historical API guide.
FAQ
Which leaderboard metric is the most reliable?
No single metric is. Profit factor, average win and loss, and maximum drawdown together describe payoff and risk far better than win rate or profit alone. Add a calibration score such as Brier when you want to isolate forecasting ability from sizing and execution.
How many resolved positions do I need before trusting a ranking?
There is no universal threshold, and the right number depends on the variance of the payoffs. Count independent resolved bets, not trades, and test whether the result survives removing the best market. If it does not, the sample is too thin to support a conclusion.
Can I compare Polymarket and Kalshi leaderboards directly?
Not by rank. Venues differ in identity, fees, rewards, settlement and market definitions. Recompute the same metrics from trade-level data on a normalized schema, then compare those.
How often should leaderboard data refresh?
Match it to the signal. Discovery and research can use daily snapshots. Strategies that react to new wallet activity need trade-level updates streamed in real time, since a re-sorted board lags the trades that moved it.
Does following a top-ranked wallet work?
It is a hypothesis to test, not a rule. Measure whether prices move toward the wallet’s position after it trades, net of your latency and spread, using point-in-time data. Paper trade the rule before committing capital.
Sources
Trader Leaderboard Analysis for Prediction Markets
A prediction market leaderboard is a ranking, not a verdict. To analyze one well, treat the rank as a starting hypothesis. Rebuild the metrics from raw trades, test them for sample size and payoff structure, and only then decide whether a wallet carries a signal. This guide covers the metrics that matter, the biases that distort rankings, how venues differ, and what a developer should verify before wiring leaderboard data into a pipeline.
Assymetrixdata.assymetrix.comAnalyze Traders Across VenuesUse Assymetrix’s Data API to access unified prediction market data across Polymarket, Kalshi, and Limitless through a single integration.Explore the Data API
TL;DR
Rank alone hides the method. Profit, return, volume and win rate each answer a different question, and a single sort order rewards some behaviors over others.
Win rate without payoff size misleads. Pair it with profit factor, average win and loss, and drawdown.
Skill versus luck is a sample-size and calibration problem. Require many resolved positions, then test forecasts with a proper scoring rule.
Leaderboards differ across venues in identity, fees, settlement and visibility. Compare only metrics you recompute yourself on normalized data.
Before you build, verify the ranking window, the definition of profit, the refresh cadence and whether the history is complete.
Table of Contents
What a prediction market leaderboard actually measures
The metrics worth computing
Skill, luck, exposure and sample size
How leaderboards differ across venues
From leaderboard to signal
What to verify before you build on leaderboard data
Perspective
Build your own ranking with the Data API
Sources
What a prediction market leaderboard actually measures
Most public leaderboards rank accounts by a single number, usually profit over a window (a day, a week, a month, all time). Some let you re-sort by volume or return. That is useful for discovery and weak for analysis, because the number is a summary of a path, and the path is what you need.
Three questions decide how much a ranking can tell you:
What is being counted as profit? Realized profit on settled markets, or realized plus unrealized marks on open positions? The two can diverge sharply in a market that has not resolved.
Who is being ranked? On wallet-based venues the unit is an address, and one person can operate several. On account-based venues the unit is a username or an anonymized handle. Neither is a verified human.
Which window? A 30-day leaderboard rewards whoever was concentrated in the month’s biggest winner. An all-time board rewards survivors and capital size.
If a leaderboard does not state its answers to these, you cannot interpret the rank. Rebuild the figures from trade-level data instead. Our guide to wallet PnL analytics for prediction markets walks through how to reconstruct realized and unrealized profit per wallet from fills and settlements.
The metrics worth computing
A leaderboard row is a handful of columns. A research-grade profile adds the ones the board leaves out. Trading platforms that document their performance reports are a good template, since the same questions apply. TradingView’s strategy report lists net profit and loss, profit factor, Sharpe and Sortino ratios, average profit and loss, return, and maximum drawdown. Most of these map directly onto a prediction market wallet.
Metric | What it answers | Common trap on a leaderboard |
|---|---|---|
Net profit | How much money was made | Scales with capital, so large accounts dominate |
Return on capital | How efficiently capital was used | Tiny accounts post extreme percentages |
Volume | How active the trader is | High volume can mean market making, not edge |
Win rate | How often positions finish positive | Ignores payoff size |
Profit factor | Gross gains divided by gross losses | Unstable on few trades |
Average win / average loss | Payoff shape | Rarely shown on public boards |
Maximum drawdown | Worst peak-to-trough loss | Hidden by a profit-only sort |
Sharpe / Sortino | Return per unit of volatility | Needs a consistent time series of returns |
Why win rate needs a partner
Win rate is the most quoted and most misread column. One trading-metrics reference notes that win rate alone does not tell the whole story because it ignores the size of wins and losses, and suggests reading it alongside profit factor. Prediction markets make this worse, because payoffs are bounded and asymmetric by construction.
Consider two wallets. One buys contracts priced at 92 cents and wins 95% of the time, collecting 8 cents per win and losing 92 cents on each miss. The other buys 12-cent contracts and wins 25% of the time, collecting 88 cents per win. Their win rates are far apart, yet either can be profitable or ruinous depending on calibration. Only the combination of win rate, average win and average loss tells you which.
Drawdown is the column that is missing
A profit-sorted board shows who ended up on top and says nothing about how they got there. A wallet that made its money in one resolved election market and never traded again looks like a star. A wallet with steady small gains and a shallow drawdown looks ordinary and is far more informative. Compute maximum drawdown from the cumulative realized profit series, and compute it on both a calendar basis and a per-market basis.
For a developer-oriented walk through of the reputation fields you can derive, see 5 trader reputation metrics mapped to API fields.
Skill, luck, exposure and sample size
The hard question is whether a ranking reflects skill. A few checks separate the cases cheaply.
1. Count independent bets, not trades
A wallet that placed 400 trades across three correlated markets has three or four independent observations, not 400. Group fills by market and by underlying event. Then count resolved positions. Public boards often show trade counts, which overstate the evidence.
2. Score forecasts, not just outcomes
Profit conflates forecasting ability with execution, sizing and timing. To isolate the forecast, compare the price a wallet paid against the eventual outcome using a proper scoring rule such as the Brier score. A wallet that consistently buys below the final resolution value, across many independent markets, is showing something profit alone cannot. We cover how to do this reproducibly in Brier audits for prediction markets and traders and compare alternatives in 3 proper scoring rules.
3. Look for concentration
Compute the share of total profit that came from the single best market, and from the best five. If one position explains most of the ranking, you are looking at an outcome, not a process. Rank stability is a useful companion: does the wallet stay in the top decile when you drop its best market?
4. Separate market making from directional trading
High volume with a modest margin often points to liquidity provision or spread capture. Some venues also pay rewards to liquidity providers, and aggregate trackers may blend those into a total. A wallet that earns mostly from rewards is not forecasting anything. Split trading profit from incentives before you label a wallet as informed.
5. Check for survivorship
A leaderboard shows the people who are still on it. Wallets that blew up, stopped trading or rotated to a fresh address are gone from the view. If you evaluate a rule such as “follow last month’s top 20,” you must test it on a set of wallets chosen at the start of each period, including those that later disappeared. Otherwise your backtest inherits the bias.
6. Control for exposure
Prediction markets are not independent assets. A wallet long “yes” across a cluster of related political or sports markets was exposed to one underlying factor. Its profit tells you about that factor’s outcome. Estimate how correlated a wallet’s positions are before you compare it with another.
How leaderboards differ across venues
Cross-venue comparison is where most analyses break. Polymarket, Kalshi and Limitless differ in structure, and each difference changes what a leaderboard column means.
Identity. Wallet-based venues expose addresses, which you can follow across markets but which do not map one-to-one to people. Regulated, account-based venues expose less, and what is public depends on the product and the platform’s own settings.
Fees and rewards. Net profit before and after fees are different quantities. Make sure every venue’s figure is on the same basis.
Settlement and resolution. Resolution rules, timing and dispute processes differ, so “settled” does not mean the same thing everywhere. A position that is a loss on one venue may be unresolved on another.
Market definitions. The same real-world event can be listed with slightly different wording, thresholds or expiry. A like-for-like comparison needs a canonical mapping, not a name match.
Window and coverage. A board’s lookback and its tracked population are set by whoever built it.
The practical rule: do not compare published ranks across venues. Compare metrics you compute yourself, from trade-level data, on a common schema. That is the reason we normalize market and trader identifiers across venues, so the same event and the same behavior line up. For the data model, see our guide to cross-venue prediction market data with canonical IDs and the measurement guide for cross-venue price gaps.
From leaderboard to signal
Once a wallet survives the checks above, leaderboard data becomes input to several workflows.
Cohort analysis. Instead of following individuals, group wallets by a measured property (for example, calibration score above a threshold with at least a minimum number of independent bets) and study what the cohort does. Cohorts are more stable than single wallets, and a signal from fifty weakly informed participants is easier to validate than one from a lone outlier. Our write-up on trader cohort analysis across 900 million indexed events shows how.
Smart Money tracking. The goal is to identify wallets whose positions tend to precede price moves or resolutions, then monitor their new activity. The evidence standard is lead-lag: does the price move toward the wallet’s position after it trades, net of the spread you would pay to follow? See tracking Smart Money wallets on Polymarket and how Smart Money moves before event resolutions.
Alerts and copy logic. If you act on a wallet’s new trades, you inherit latency and slippage. Model the delay between their fill and your order. Our guides to Smart Money alerts and building copy trading with canonical data and rules cover the mechanics.
Validation before capital. Backtest any rule on point-in-time data, meaning you only use information that existed when the trade would have been placed. Then replay it. Read backtesting prediction market strategies with the Data API and paper trading with replayed API data before risking money.
What to verify before you build on leaderboard data
A short checklist for developers and quants:
Definition of profit. Realized, unrealized or both? Gross or net of fees and rewards?
Ranking window and refresh cadence. Daily snapshots are fine for discovery. A signal that depends on recent activity needs trade-level data streamed as it happens, not a re-sorted board. Match refresh frequency to the decay of the signal you are testing.
Population. How many wallets are tracked, and how are inactive or tiny accounts handled? A board capped at its top entries cannot support a percentile analysis.
Point-in-time integrity. Can you reconstruct the leaderboard as it stood on a past date? If the source only offers the current state, your backtest uses information from the future.
Identifier stability. If addresses or usernames change, does history follow? Cross-venue work requires stable canonical IDs.
Completeness. Gaps in history, missing resolutions or deleted markets bias every metric downstream. Reconcile totals against raw fills.
Reproducibility. Store the query, the schema version and the snapshot time with every result.
If you prefer to build in Python, the developer guide to Python prediction market data is a practical starting point.
Perspective
Leaderboards are where most people start and where most serious analysis should leave off. The ranking is built by someone else’s definitions, for someone else’s purpose, over a window you did not choose.
Our view is that the useful object is the trade record. With roughly 1.5 terabytes of history spanning nearly a billion rows of trading activity across Polymarket, Kalshi and Limitless, we can rebuild any leaderboard metric from fills and settlements, and add the ones boards omit: drawdown, payoff shape, calibration, concentration.
Venue neutrality matters here. A ranking produced by one exchange reflects that exchange’s rules. A normalized layer lets you ask the same question of every venue and get an answer you can compare. That is what we are designed to provide, and it is why we treat Smart Money scores as hypotheses to test, not as conclusions to follow.
Dean
Build your own ranking with the Data API
If you want to compute these metrics rather than read them, the Assymetrix Data API gives developers, quants and AI agents unified access to cross-venue prediction market data through a single integration, with normalized schemas for markets and wallets. It supports both real-time and historical use, so the same code that scores a wallet today can reproduce its ranking on a past date.
For an overview of the feeds and endpoints, start with the real-time and historical API guide.
FAQ
Which leaderboard metric is the most reliable?
No single metric is. Profit factor, average win and loss, and maximum drawdown together describe payoff and risk far better than win rate or profit alone. Add a calibration score such as Brier when you want to isolate forecasting ability from sizing and execution.
How many resolved positions do I need before trusting a ranking?
There is no universal threshold, and the right number depends on the variance of the payoffs. Count independent resolved bets, not trades, and test whether the result survives removing the best market. If it does not, the sample is too thin to support a conclusion.
Can I compare Polymarket and Kalshi leaderboards directly?
Not by rank. Venues differ in identity, fees, rewards, settlement and market definitions. Recompute the same metrics from trade-level data on a normalized schema, then compare those.
How often should leaderboard data refresh?
Match it to the signal. Discovery and research can use daily snapshots. Strategies that react to new wallet activity need trade-level updates streamed in real time, since a re-sorted board lags the trades that moved it.
Does following a top-ranked wallet work?
It is a hypothesis to test, not a rule. Measure whether prices move toward the wallet’s position after it trades, net of your latency and spread, using point-in-time data. Paper trade the rule before committing capital.
Sources
Other Blog
