prediction markets

All posts tagged prediction markets

This post is a response to a challenge on Overcoming Bias to spend $10 trillion sensibly.
Here’s my proposed allocation (spending to be spread out over 10-20 years):

  • $5 trillion on drug patent buyouts and prizes for new drugs put in the public domain, with the prizes mostly allocated in proportion to the quality adjusted life years attributable to the drug.
  • $1 trillion on establishing a few dozen separate clusters of seasteads and on facilitating migration of people from poor/oppressive countries by rewarding jurisdictions in proportion to the number of immigrants they accept from poorer / less free regions. (I’m guessing that most of those rewards will go to seasteads, many of which will be created by other people partly in hopes of getting some of these rewards).

    This would also have a side affect of significantly reducing the harm that humans might experience due to global warming or an ice age, since ocean climates have less extreme temperatures, seasteads will probably not depend on rainfall to grow food, and can move somewhat to locations with better temperatures.
  • $1 trillion on improving political systems, mostly through prizes that bear some resemblance to The Mo Ibrahim Prize for Achievement in African Leadership (but not limited to democratically elected leaders and not limited to Africa). If the top 100 or so politicians in about 100 countries are eligible, I could set the average reward at about $100 million per person. Of course, nowhere near all of them will qualify, so a fair amount will be left over for those not yet in office.
  • $0.5 trillion on subsidizing trading on prediction markets that are designed to enable futarchy. This level of subsidy is far enough from anything that has been tried that there’s no way to guess whether this is a wasteful level.
  • $1 trillion existential risks
    Some unknown fraction of this would go to persuading people not to work on AGI without providing arguments that they will produce a safe goal system for any AI they create. Once I’m satisfied that the risks associated with AI are under control, much of the remaining money will go toward establishing societies in the asteroid belt and then outside the solar system.
  • $0.5 trillion on communications / computing hardware for everyone who can’t currently afford that.
  • $1 trillion I’d save for ideas I think of later.

I’m not counting a bunch of other projects that would use up less than $100 billion since they’re small enough to fit in the rounding errors of the ones I’ve counted (the Methuselah Mouse prize, desalinization and other water purification technologies, developing nanotech, preparing for the risks of nanotech, uploading, cryonics, nature preserves, etc).

Book review: Infotopia: How Many Minds Produce Knowledge by Cass R. Sunstein.
There’s a lot of overlap between James Surowiecki’s The Wisdom of Crowds and Infotopia, but Infotopia is a good deal more balanced and careful to avoid exaggeration. This makes Infotopia less exciting but more likely to convince a thoughtful reader. It devotes a good deal of attention to conditions which make groups less wise than individuals as well as conditions where groups outperform the best individuals.
Infotopia is directed at people who know little about this subject. I found hardly any new insights in it, and few ideas that I disagreed with. Some of its comments will seem too obvious to be worth mentioning to anyone who uses the web much. It’s slightly better than Wisdom of Crowds, but if you’ve already read Wisdom of Crowds you’ll get little out of Infotopia.

Predictocracy (part 2)
Book review: Predictocracy: Market Mechanisms for Public and Private Decision Making by Michael Abramowicz (continued from prior post).
I’m puzzled by his claim that it’s easier to determine a good subsidy for a PM that predicts what subsidy we should use for a basic PM than it is to determine the a good subsidy for the basic PM. My intuition tells me that at least until traders become experienced with predicting effects of subsidies, the markets that are farther removed from familiar questions will be less predictable. Even with experience, for many of the book’s PMs it’s hard to see what measurable criteria could tell us whether one subsidy level is better than another. There will be some criteria that indicate severely mistaken subsidy levels (zero trading, or enough trading to produce bubbles). But if we try something more sophisticated, such as measuring how accurately PMs with various subsidy levels predict the results of court cases, I predict that we will find some range of subsidies above which increased subsidy produces tiny increases in correlations between PMs and actual trials. Even if we knew that the increased subsidy was producing a more just result, how would we evaluate the tradeoff between justice and the cost of the subsidy? And how would we tell whether the increased subsidy is producing a more just result, or whether the PMs were predicting the actual court cases more accurately by observing effects of factors irrelevant to justice (e.g. the weather on the day the verdict is decided)?
His proposal for self-resolving prediction markets (i.e. markets that predict markets recursively with no grounding in observed results) is bizarre. His arguments about why some of the obvious problems aren’t serious would be fascinating if they didn’t seem pointless due to his failure to address the probably fatal flaw of susceptibility to manipulation.
His description of why short-term PMs may be more resistant to bubbles than stock markets was discredited just as it was being printed. His example of deluded Green Party voters pushing their candidate’s price too high is a near-perfect match for what happened with Ron Paul contracts on Intrade. What Abramowicz missed is that traders betting against Paul needed to tie up a lot more money than traders betting for Paul. High volume futures markets have sophisticated margin rules which mostly eliminate this problem. I expect that low-volume PMs can do the same, but it isn’t easy and companies such as Intrade have only weak motivation to do this.
He suggests that PMs be used to minimize the harm resulting from legislative budget deadlocks by providing tentative funding to projects that PMs predict will receive funding. But if the existence of funding biases legislatures to continue that funding (which appears to be a strong bias, judging by how rare it is for a legislature to stop funding projects), then this proposal would fund many projects that wouldn’t otherwise be funded.
His proposals to use PMs to respond to disasters such as Katrina are poorly thought out. He claims “not much advanced planning of the particular subjects that the markets should cover would be needed”. This appears to underestimate the difficulty of writing unambiguous claims, the time required for traders to understand them, the risks that the agencies creating the PMs will bias the claim wording to the agencies’ advantage, etc. I’d have a lot more confidence in a few preplanned PM claims such as the expected travel times on key sections of roads used in evacuations.
I expect to have additional comments on Predictocracy later this month; they may be technical enough that I will only post the on the futarchy_discuss mailing list.

Book review: Predictocracy: Market Mechanisms for Public and Private Decision Making by Michael Abramowicz.
This had the potential to be an unusually great book, which makes its shortcomings rather frustrating. It is loaded with good ideas, but it’s often hard to distinguish the good ideas from the bad ideas, and the arguments for the good ideas aren’t as convincing as I hoped.
The book’s first paragraph provides a frustratingly half-right model of why markets produce better predictions than alternative institutions, involving a correlation between confidence (or sincerity) and correctness. If trader confidence was the main mechanism by which markets produce accurate predictions, I’d be pretty reluctant to believe the evidence that Abramowicz presents of their success. Sincerity is hard to measure, so I don’t know what to think of its effects. A layman reading this book would have trouble figuring out that the main force for accurate predictions is that the incentives alter traders’ reasoning so that it becomes more accurate.
The book brings a fresh perspective to an area where there are few enough perspectives that any new perspective is valuable when it’s not clearly wrong. He is occasionally clearer than others. For instance, his figure 4.1 enabled me to compare three scoring rules in a few seconds (I’d previously been unwilling to do the equivalent by reading equations).
He advocates some very fine-grained uses of prediction markets (PMs), which is a sharp contrast to my expectation that they are mainly valuable for important issues. Abramowicz has a very different intuition than I do about how much it costs to run a prediction market for an issue that people normally don’t find interesting. For instance, he wants to partly replace small claims court cases with prediction markets for individual cases. I’m fairly sure that obvious ways to do that would require market subsidies much larger than current court costs. The only way I can imagine PMs becoming an affordable substitute for small claims courts would be if most of the decisions involved were done by software. Even then it’s not obvious why one or more PM per court case would be better than a few more careful evaluations of whether to turn those decisions over to software.
He goes even further when proposing PMs to assess niceness, claiming that “just a few dollars’ worth of subsidy per person” would be adequate to assess peoples’ niceness. Assuming the PM requires human traders, that cost estimate seems several orders of magnitude too low (not to mention the problems with judging such PMs).
His idea of “the market web” seems like a potentially valuable idea for a new way of coordinating diverse decisions.
He convinced me that Predictocracy will solve a larger fraction of democracy’s problems than I initially expected, but I see little reason to believe that it will work as well as Futarchy will. I see important classes of systematic biases (e.g. the desire of politicians and bureaucrats to acquire more power than the rest of us should want) that Futarchy would reduce but which Predictocracy doesn’t appear to alter.
Abramowicz provides reasons to hope that predictions of government decisions 10+ years in the future will help remove partisan components of decisions and quirks of particular decision makers because uncertainty over who will make decisions at that time will cause PMs to average forecasts over several possible decision makers.
He claims evaluations used to judge a PM are likely to be less politicized than evaluations that directly affect policy because the evaluations are made after the PM has determined the policy. Interest groups will sometimes get around this by making credible commitments (at the time PMs are influencing the policy) to influence whoever judges the PM, but the costs of keeping those commitments after the policy has been decided will reduce that influence. I’m not as optimistic about this as Abramowicz is. I expect the effect to be real in some cases, but in many cases the evaluator will effectively be part of the interest group in question.

I just got around to checking out a mailing list devoted to Futarchy. It looks interesting enough that I expect to post a number of messages to it over the next few weeks. But I have some concerns that is focused too much on problems associated with the final stages on the path to a pure Futarchy rather than on what I see as the more valuable goal of implementing an impure system that involves voters relying heavily on market predictions (which I see as a necessary step to take before people will seriously consider pure Futarchy).
I’m in the process of writing comments on the book Predictocracy, probably too many for one post, and I expect I’ll post some of them only on the futarchy_discuss list.

The Politimetrics provides implied probabilities of Clinton or Obama winning in November if they get the nomination, derived from Intrade prices. I’m surprised that it’s been showing recently that the difference in their electabilities has been mostly zero, with occasional indications that Clinton is slightly more electable. Most other sources of information appear to suggest that Obama has more support than Clinton among independents and Republicans.
I just did a little trading to help move the market toward showing Obama as more electable by replacing my small bet against Clinton being nominated with a bet against her becoming president, but the amount I’m willing to trade was small enough that the markets moved in the opposite direction (i.e. showed increased Clinton electability).
What could cause the markets to indicate knowledge that conflicts with what I expect?
It could be that several limitations of Intrade impair market efficiency, such as not making it easy to see what those of us who have noticed the Politimetrics site see, or having margin requirements that are not conducive to exploiting inefficiencies of this nature (even if I were more confident that the market is wrong, the expected return on investment isn’t enough to persuade me to make large trades).
It could be that Obama is sufficiently unusual that there’s more uncertainty in how he will do, so that while the most likely result is that he’d get more votes than Clinton would, there’s a greater chance of a negative surprise with him.
It could be that Clinton is expected to be sufficiently vicious if she’s losing that she would hurt Obama before giving up.
But the history shown on the Politimetrics site has swings that seem unexplained by these guesses.

Politimetrics (associated with the Westminster Business School) has sponsored some additional Intrade contracts which will provide information about the impact of the presidential election on the country if they ever get enough liquidity. So far, there’s been no sign that much liquidity will exist.
One reason I (and presumably other traders) haven’t placed many orders is that the contracts deal with individual candidates. Since the value of the new contracts should fluctuate with the probability of the relevant candidate’s winning, and those fluctuations are currently much larger than any other factor affecting the prices, trading them would require any trader who doesn’t accept the market price to frequently monitor the prices of the underlying contracts. Nobody wants to do that unless the contracts already have significant volume.
Even if they had some liquidity, there’s a good deal of risk that the long-shot bias which appears to be common on Intrade would limit my confidence in the value of the information provided by those prices for all but the two or three candidates who are most likely to win in November (i.e. I’d probably believe what they said about Clinton relative to Obama, but I’d doubt they would be useful for voters in Republican primaries).
When it becomes clear who will win each party’s nomination, these problems will be reduced, and I’ll probably place a moderate number of orders on some of these contracts.
It should be possible to design a better user interface for decision markets of this nature so that users could place orders purely on the probable impact of a candidate’s election. Shock response futures come closer to doing that than contracts of the form “X wins and Y happens”, but can probably only indicate the direction of the impact.
I’ve created web pages at and (which are currently being updated 4 times a day) which show implied prices (i.e. the price of the conditional contract as a percent of the price of the underlying candidate’s contract) that ought to represent what the markets think the probable effects would be if that candidate wins. Ideally traders could place orders expressed in terms of those implied prices, but that’s nontrivial to implement, and unlikely to happen unless someone pays Intrade a fair amount to create.
I’ve commented on Jed Christiansen’s blog about why I doubt the conditional contracts I’m subsidizing have had enough trading yet to produce valuable information. But the trends suggest there will be enough trading within a few weeks.

I have implemented subsidies to encourage trading of some conditional prediction market contracts that may provide useful information about the consequences of the 2008 presidential election, via a simple automated market maker (using an algorithm described near the end of The subsidized market maker ought to provide incentives for traders to devote more thought to these contracts than they would if the liquidity was less predictable.
Intrade has agreed not to charge any trading or expiry fees on these contracts.
Some places to look for extensive description of the motivations behind these subsidies are here and here.

The contracts are:

Please read the detailed specifications at Intrade before trading them, as one-line descriptions are not sufficient for you to fully understand them.
For the first two of those contracts, the market maker will enter bids and asks of 38 contracts, and can lose a maximum of $5187.76 on each contract. For the other four contracts, the market maker will enter bids and asks of 115 contracts, and can lose a maximum of $7906.25 on each contract.
I will maintain a web page here devoted to these contracts.
See also this more eloquent description on Overcoming Bias.

Book review: Information Markets: A New Way of Making Decisions, edited by Robert Hahn and Paul Tetlock
This book contains some good discussions of current issues in the design of prediction markets (aka idea futures).
Since it’s the result of a conference for experts, it is mainly directed toward experts. It shouldn’t be overly hard for laymen to understand, but it probably focuses on issues that are somewhat different from what most laymen would find interesting, so I’d probably recommend reading Surowiecki’s Wisdom of Crowds or some of Robin Hanson’s earlier papers on the subject first.
One surprising result reported here is that the Iowa Electronic Markets show no longshot bias, in contrast to similar markets on Tradesports/Intrade and to widespread types of sports betting. This looks like an important area for research, although that would probably require setting up many variations on those markets (varying things such as the user interface, commissions on trades, limits on how much money can be invested, etc.), which would be expensive and hindered by regulatory uncertainty.
Michael Abramowicz presents an interesting proposal to create incentives to counteract the likely tendency of markets such as prediction markets to discourage people from making public the knowledge that goes into making market prices efficient. I don’t have much of a guess about how well his solution will work. It needs some more thought about how vulnerable it is to manipulation of the intermediate prices used to reward traders who convince others to follow their reasoning (averaging prices over a week or two would be a simple start at deterring manipulation). But I think he understates the importance of the problems he’s trying to solve. He says “while they are endemic to all securities markets, they apparently cause little harm. They are likely to be much more severe, however, in markets with very few active participants.”. I suspect they are significant in most securities markets, and are underestimated because they are very hard to measure. As someone who trades stocks for a living, I’d say that the amount and quality of knowledge that is shared among traders is quite low compared to most professions, although it’s hard to say how much of this is due to desire to keep valuable information secret and how much is due to the difficulty of distinguishing valuable information from misleading information.

This book does an excellent job of reporting important evidence showing that group decisions can be wiser than those of any one individual. He makes some good attempts to describe what conditions cause groups to be wiser than individuals, but when he goes beyond reporting academic research, the quality of the book declines. He exaggerates enough to give critics excuses to reject the valuable parts of the book.
He lists four conditions that he claims determine whether groups are wiser than their individual members. I’m uncertain whether the conditions he lists are sufficient. I would have added something explicit about the need to minimize biases. It’s unclear whether that condition follows from his independence condition, partly because he’s a bit vague about whether he uses independence in the strong sense that statisticians do or whether he’s speaking more colloquially.
Sometimes he ignores those conditions and makes unconvincing blanket statements that larger groups will produce wiser decisions.
He makes exaggerated claims for the idea that crowds are wise due to information possessed by lots of average people rather than the influence of a few wise people. For instance, he disputes a Forsythe et al. paper which argues that a small number of “marginal traders” in a market to predict the 1988 presidential vote were responsible for the price accuracy. Surowiecki’s rejection of this argument depends on a claim that “two investors with the same amount of capital have the same influence on market prices”. But that looks false. For example, if the nonmarginal traders make all their trades on the first day and then blindly hold for a year, and the marginal traders trade with each other over that year in response to new information, prices on most days will be determined by the marginal traders.
It’s not designed to be an investment advice book, but if judged solely as a book on investment, I’d say it ranks in the top ten. It does a very good job of explaining both what’s right and what’s wrong with the random walk theory of the stock market.
He does a good job of ridiculing the “cult of the CEO” whereby most of a company’s value is attributed to its CEO (at least in the U.S.). I was surprised by his report that 95% of investors said they would buy stocks based on their opinion of the CEO. They certainly didn’t get that attitude from successful investors (who seem to do that only in rare cases where they are able to talk at length with the CEO). But his claim that “Corporate profit margins did not increase over the course of the 1990s, even as executive compensation was soaring” looks false, as well as being of questionable relevance to his points about executives being overvalued. And I wish he had also applied his argument to beliefs of the form “if we could just elect a good person to lead the nation”.
Chapter 6 does a good job of combining the best ideas from Wright’s book Nonzero and Fukuyama’s Trust (oddly, he doesn’t cite Trust).
He exaggerates reports that the stock market responded accurately to the Challenger explosion before any public reports indicated the cause. He claims “within a half hour of the shuttle blowing up, the stock market knew what company was responsible.” I don’t know where he gets the “half hour” time period. The paper he cites as the source says the market “pinpointed” Thiokol as the culprit “within an hour”, but it exaggerates a bit. If the percent decline in stock price is the best criterion, then the market provided strong evidence within an hour. If the dollar value of the loss of market capitalization is the best criterion, then the evidence was weak after one hour but strong within four hours.
He also claims “Savvy insiders alone did not cause that first-day drop in Thiokol’s price.”, but shows no sign that he could know whether this is true. He seems to base on the absence of reported selling by executives whom the law requires to report such selling, but he appears to overestimate how reliably that law is obeyed, and to ignore a large number of non-executive insiders (e.g. engineers). He does pass on a nice quote which better illustrates our understanding of these issues: “While markets appear to work in practice, we are not sure how they work in theory.”