Fast algorithm for computing the inverse of the standard normal distribution (CDF) function

1. Fast Inversion of Normal CDF
John D. Cook

2. What we’re computing

3. Basic assumptions
Five or six significant figures is
adequate for many applications
Memory is fast and plentiful
Low-level bit operations are faster than
arithmetic

4. Basic Approach
Tabulate values at sample points and
use low-order approximations to fill in
Take advantage of binary
representation of floating point
numbers to decide where to sample
Change variables to avoid arithmetic

5. Big clever idea
Extract sample points based on the
exponent and mantissa
The extraction is extremely fast
It is biased to sample more frequently
near zero, exactly where cPhi-1 needs
more sampling

6. IEEE Floating Point
Representation (conceptual)
x = +/- 2e m, 1 <= m < 2
Bit 1: sign bit +/-
Bits 2 through 9: exponent e
Bits 10 through 32: mantissa

7. IEEE Floating Point
Representation (details)
Bit 1: 0 for positive, 1 for negative
Bits 2 through 9: exponent of 2 biased
by 127
(values 0 through 255 correspond to
actual exponents -127 through 128)
Bits 10 through 32: mantissa minus 1
(leading bit always 1, so don’t store)

8. Starting Point for Marsaglia’s
algorithm
Represent numbers by
u = 2-k (2-1 + 2-6 j + 2-24 m)
0 <= k < 32, 0 <= j < 32
0 <= m < 224
k = 126 – e, where ‘e’ is the exponent
representation bits
j = first five mantissa representation bits
m = last 18 mantissa representation bits

9. Sample Points
Tabulate cPhi-1 at points corresponding
to m = 0, i.e. at 32 possible values of i
and j, a total of 1024 points.
Use quadratic Taylor approximation
based at these points
A fixed number samples per exponent
samples more finely near zero, just
where cPhi-1 needs more samples

10. Clever indexing
Conceptually, we have a matrix A[i][j] of
tabulated values
This requires two calculations to find indices –
one for i and one for j – and two operations
to lookup values
Combine into a single index
n = 992-32k + j
that can be extracted directly by one bit
manipulation: bits 2 through 14 minus 3040

11. Polynomial evaluation
Taylor approximation:
t = h B(k,j)
x = A(k,j) -0.5 t + 0.125 A(k,j)t2
Rescale B’s by square root of 8:
t = h B’
x = A – c t – A t2 [ c = sqrt(2) ]
Horner’s method:
x = A – t(c – A t)

12. C++ Implementation
double NormalCCDFInverse(double x)
{
float f1 = (float) x;
unsigned int ui;
memcpy(&ui, &f1, 4);
int n = (ui >> 18) - 3008;
ui &= 0xFFFC0000;
float f2;
memcpy(&f2, &ui, 4);
double v = (f1-f2)*B[n];
return A[n] - v*(sqrt2 - A[n]*v);
}

13. Fine Print
This algorithm only valid for p <= 0.5
For p > 0.5, use cPhi-1(p) = -cPhi-1(1-p)
Phi-1(p) = cPhi-1(1-p)
Algorithm not valid for p < 2-33
The maximum error is 0.000004,
which occurs near p = 0.25

14. Implementation
double PrivateNormalCCDFInverse(double x) …
double NormalCDFInverse(double x) {
return (x > 0.5)
? PrivateNormalCCDFInverse(1.0-x)
: PrivateNormalCCDFInverse(x);
}
double NormalCCDFInverse(double x) {
return (x > 0.5)
? -PrivateNormalCCDFInverse(1.0-x)
: PrivateNormalCCDFInverse(x);
}

15. References
Rapid evaluation of the inverse of the
normal distribution function
by Marsaglia, Zaman, and Marsaglia
Statistics and Probability Letters
19 (1994) 259 – 266

16. Notes
This talk presented June 6, 2001
http://www.JohnDCook.com