This document discusses using prediction markets and continuous lotteries to aggregate predictions from algorithms authored by different people. It outlines several "lottery paradoxes" where a player can earn returns higher than average by buying all options, and resolutions involving how the average is calculated. It proposes using a network of prediction games to continuously refine probability distributions over scenarios as new data arrives. Several categories of potential applications are described, from enhancing market and business intelligence to discovering biases. The document argues that machine learning prediction will increasingly be produced through open networks of algorithmic competition rather than by individual "artisan" data scientists.
Isolation of AMF by wet sieving and decantation method pptx
Lottery paradox csail-dec-2020
1. The Lottery Paradox
A New Use
MIT Computer Science & Artificial Intelligence Lab
Dec 1, 2020
Peter Cotton
Chief Data Scientist
Intech Investments
2. Hello. I work for Intech — a leading equity quant manager
2
3. I am asymptotically the world’s most productive data scientist
3
(Returns measured water height … somewhere … from NOAA)
Creates data stream
… or so I tell my boss
At the conclusion of a “ten minute data science project”, a data stream is predicted
by dozens of competing time series algorithms, written by different authors using
different tools, in different languages, with access to different exogenous data.
4. Outline
4
1. On the lottery paradox:
a. Positive returns
b. Continuous lotteries
c. Indifference to the market distribution
d. Relationship between returns and distance
1. Putting it to work:
a. Real-time distributional prediction
b. Stacking lottery games
c. Implied quantiles and copulas
d. Categories of business applications
1. An existence “proof” for a prediction network (that doesn’t exist)
a. The demise of artisan “data science”
b. Why algorithms will manage the production of prediction
6. Lottery paradox #1
6
Assume 10% rake. Buyer chooses 1 … 10,000. Most enter randomly.
Mary buys every possible ticket once.
16% return !
7. Lottery paradox resolution - simpler example
7
Mary benefits from Alice and Bob tripping on each other’s toes
Only two outcomes
8. Lottery paradox #2
8
Let W denote the average number of people sharing the prize.
Alice is a random ticket buyer.
Alice shares with approximately W other people
9. Lottery paradox #2 - resolution
9
In the case of two tickets (head and tails), Alice shares with W-½ others
(We can count)
10. Lottery paradox #2 - resolution
10
Alice’s average is a population average, not outcome average.
When Alice, Bob and Joe share the prize, it counts three times.
This allows the population average to exceed the average over tickets by almost 1
11. Lottery paradox #2 - better resolution
11
Alice wins with ticket 137 → Mean # of people choosing 137 goes up by almost +1.
(“Approximate Bayes”)
c.f. Mary winning conveys no information at all.
12. Lottery paradox #2 - even better resolution?
12
Consider Mary’s last ticket …. lucky by +1
But all her tickets are the same
13. Lottery paradox #3: Indifference
13
Suppose:
● No rake
● Mary’s investment is small
● Mary optimizes long run wealth
● Mary can see everyone else’s ticket choices
14. Lottery paradox #3: Indifference
14
Suppose:
● No rake
● Mary’s investment is small
● Mary optimizes long run wealth
● Mary can see everyone else’s ticket choices
⇒ Mary still buys one of each ticket
⇒ Mary doesn’t care what anyone else does !
16. Racetrack paradox - resolution #1
16
Maximize
Constraint
First order Lagrange condition:
Thus must not depend on the horse index i
17. Racetrack paradox - elementary resolution
17
Transfer a tiny investment from first horse to the second
Follows that so they must be equal
18. Remark on Entropy and KL-Divergence
18
Entropy … also the term not involving q in Mary’s return
Kullback and Liebler cross-entropy
We can interpret distance of Q from truth P in terms of Mary’s return exploiting it
19. Now for something completely different?
19
Mayor draws from “Normalish” distribution
Participants write a real number down. All those close share the prize.
22. Mary’s Reward for Accuracy - Normalish
22
Market error Mary’s return
10% 20 pts ( 0.2 )
1% 20 bps ( 0.002 )
● Use the fourth root transform to relate exponential
returns to market error … measured as a percentage of
standard deviation
27. Implied Percentiles
27
Every incoming data point implies a new data point …
z = F(x)
where F is the “community” distribution function
Cumulative distribution for NY Electricity Production (Wind) 1 hr ahead
28. Example: Reactions to the presidential debate
Welcome Module 1
28
See https://www.microprediction.com/blog/tears_of_joy_standardizing_streaming_data
29. Stacking Lotteries
29
Market implied percentiles are themselves the subject of lottery games (via
normal quantile function)
Approximately N(0,1)
Algorithms predicting small
deviations from standard normal
30. Combine Percentiles
30
Some seemingly univariate series of games are actually copulas
Pitch and Yaw implied compulas - from MIT SciML helicopula challenge
31. Optics Analogy
31
Keep “lensing” until you get N(0,1)
Composition of monotone functions, each contributed by one or more algorithms
32. Pathways in the Collective Probability Brain
32
Scenarios “thrown” up to top level lottery
U( )
V( )
R( )
W( )
S( )
T( )
Collaboration
Q( )
Competition
Competition
Competition
33. Law of Iterated Expectations
33
Pathways grow and shrink based on the economics
Point estimates are a special case - shift
Exogenous data is a special case - shift arbitrarily (!)
E[Y|X]
E[E[Y|X]|Z]
Y
E[Y|S]
E[E[Y|S]|Z]
E[Y|S,Z,R] = E[E[E[Y|S]|Z]|R]
Scenarios thrown “up” into top level lottery
Management fees charged down from parent to child
36. Use category #1: Auxiliary market predictions
36
Markets predict the mean of a stock well
Everything else (pretty much) is poorly predicted, due to lack
of the discipline imposed by competition.
● Volatilities,
● Correlations
● Bid-offer spreads
● Liquidity
● Trading costs
● Holding periods
● Client flow
● Response to inquiry
● Cover price
37. Use category #2: Prioritizing human work
37
e.g. reference data cleaning
Probability that a record is changed?
Which records will be changed?
38. Use category #3: Enhancing live data feeds
Welcome Module 1
38
Tagging.
Converting sporadic live data to continuous.
Discovering existing relationships
Predicting delayed data and partially filled data
Discovering good embeddings
Finding new exogenous data
Discovering good proxies for truth
39. Use category #4: Live feature discovery
Welcome Module 1
39
Chumming the water
Predicting quantities correlated with the quantity you truly care about
Determining which feature generation algorithms are suited to the task at hand
40. Use category #5: Enhancing business intelligence applications
Welcome Module 1
40
Predicting numbers on dashboards
Highlighting unusual movements
Predicting human reaction to information, or not (false positives)
Enabling humans to track a larger amount of data in real time
41. Use category #6: Fairness and explanation
Welcome Module 1
41
Discovering data that reveals hidden bias
Historical example: proxies for race, redlining
42. Usage category #7: Surrogate models
42
Competing and combining surrogate models for agent based epidemic modeling
https://www.microprediction.org/stream_dashboard.html?stream=pandemic_infected
43. 43
3. An Existence Proof
(for an automated Machine Learning
network replacing artisan data science
in large part)
44. 1 - Motherhood statement
Welcome Module 1
44
Quantitative business optimization will be a survival requirement for companies
(Machine Learning is set to transform all industries)
45. 2 - Slightly more controversial...
Welcome Module 1
45
Quantitative business optimization using ML/AI = frequently repeated prediction
Control theory ~ RL ~ microprediction of value functions
46. 3 - Obvious to MIT folks
Welcome Module 1
46
Strangers can do your ML for you
47. 4 - Orthodox economics (local knowledge)
Welcome Module 1
47
At approximately zero friction, markets >> central planning by humans
48. 5 - The rest is busywork ...
Welcome Module 1
48
Humans will not play a blocking role in the production of prediction
Machine Learning will be orchestrated by hierarchies of real-time generalized contests
50. 50
• Wrote the front end
• Winning crawlers
• Clients in Java, Julia, Rust
• ZK-MUID proofs
• Monotonic NN’s
Thanks to Key Contributors. Join us !
Interested? Join us Friday’s at noon for informal contributor chat
https://www.microprediction.com/contact-us
