Title
Scoring Probability Forecasts by a User's Bets Against a Market Consensus
Abstract
The purpose of our paper is to describe a probability scoring rule that reflects the economic performance of a hypothetical investor who acts upon the probability forecasts emanating from a given model or human expert by trading against a market-clearing consensus of competing models and forecasts. The probability forecasts being compared are aggregated by an equilibrium condition into a market consensus reflecting the wisdom of the crowd. A good forecaster (model or human expert) is measured as one who allows the user to bet profitably against the market consensus. By asking forecasts to beat the market, forecasters are discouraged from herding and motivated to obtain better information than rival forecasters. We illustrate and prove that each trader's personal incentive to hedge or fudge disappears when the number of forecasts in the market is sufficiently large. Our score exhibits the forecaster's ability to assist economically profitable action and reveals how the user's profits depend strongly on the accuracy of the forecasts and the decision rule (boldness) with which they are acted upon.
Year
DOI
Venue
2021
10.1287/deca.2020.0424
DECISION ANALYSIS
Keywords
DocType
Volume
probability forecast, probability of bankruptcy, prediction markets, forecast evaluation, economic scoring rules
Journal
18
Issue
ISSN
Citations 
3
1545-8490
0
PageRank 
References 
Authors
0.34
0
4
Name
Order
Citations
PageRank
David Johnstone100.34
Stewart Jones200.68
Oliver Jones300.34
Steve Tulig400.34