Fast Inversion of Normal CDF
John D. Cook
What we’re computing
Basic assumptions
  Five or six significant figures is
  adequate for many applications
  Memory is fast and plentiful
  L...
Basic Approach
  Tabulate values at sample points and
  use low-order approximations to fill in
  Take advantage of binary...
Big clever idea
  Extract sample points based on the
  exponent and mantissa
  The extraction is extremely fast
  It is bi...
IEEE Floating Point
Representation (conceptual)

  x = +/- 2e m, 1 <= m < 2
  Bit 1: sign bit +/-
  Bits 2 through 9: expo...
IEEE Floating Point
Representation (details)

  Bit 1: 0 for positive, 1 for negative
  Bits 2 through 9: exponent of 2 bi...
Starting Point for Marsaglia’s
algorithm
  Represent numbers by
  u = 2-k (2-1 + 2-6 j + 2-24 m)
  0 <= k < 32, 0 <= j < 3...
Sample Points
  Tabulate cPhi-1 at points corresponding
  to m = 0, i.e. at 32 possible values of i
  and j, a total of 10...
Clever indexing
  Conceptually, we have a matrix A[i][j] of
  tabulated values
  This requires two calculations to find in...
Polynomial evaluation
  Taylor approximation:
  t = h B(k,j)
  x = A(k,j) -0.5 t + 0.125 A(k,j)t2
  Rescale B’s by square ...
C++ Implementation
double NormalCCDFInverse(double x)
{
      float f1 = (float) x;
      unsigned int ui;
      memcpy(&u...
Fine Print
  This algorithm only valid for p <= 0.5
  For p > 0.5, use cPhi-1(p) = -cPhi-1(1-p)
  Phi-1(p) = cPhi-1(1-p)
 ...
Implementation
double PrivateNormalCCDFInverse(double x) …

double NormalCDFInverse(double x) {
    return (x > 0.5)
     ...
References
  Rapid evaluation of the inverse of the
  normal distribution function
  by Marsaglia, Zaman, and Marsaglia
  ...
Notes
  This talk presented June 6, 2001
  http://www.JohnDCook.com
Upcoming SlideShare
Loading in …5
×

Fast coputation of Phi(x) inverse

1,798 views

Published on

Fast algorithm for computing the inverse of the standard normal distribution (CDF) function

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,798
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Fast coputation of Phi(x) inverse

  1. 1. Fast Inversion of Normal CDF John D. Cook
  2. 2. What we’re computing
  3. 3. Basic assumptions Five or six significant figures is adequate for many applications Memory is fast and plentiful Low-level bit operations are faster than arithmetic
  4. 4. Basic Approach Tabulate values at sample points and use low-order approximations to fill in Take advantage of binary representation of floating point numbers to decide where to sample Change variables to avoid arithmetic
  5. 5. Big clever idea Extract sample points based on the exponent and mantissa The extraction is extremely fast It is biased to sample more frequently near zero, exactly where cPhi-1 needs more sampling
  6. 6. IEEE Floating Point Representation (conceptual) x = +/- 2e m, 1 <= m < 2 Bit 1: sign bit +/- Bits 2 through 9: exponent e Bits 10 through 32: mantissa
  7. 7. IEEE Floating Point Representation (details) Bit 1: 0 for positive, 1 for negative Bits 2 through 9: exponent of 2 biased by 127 (values 0 through 255 correspond to actual exponents -127 through 128) Bits 10 through 32: mantissa minus 1 (leading bit always 1, so don’t store)
  8. 8. Starting Point for Marsaglia’s algorithm Represent numbers by u = 2-k (2-1 + 2-6 j + 2-24 m) 0 <= k < 32, 0 <= j < 32 0 <= m < 224 k = 126 – e, where ‘e’ is the exponent representation bits j = first five mantissa representation bits m = last 18 mantissa representation bits
  9. 9. Sample Points Tabulate cPhi-1 at points corresponding to m = 0, i.e. at 32 possible values of i and j, a total of 1024 points. Use quadratic Taylor approximation based at these points A fixed number samples per exponent samples more finely near zero, just where cPhi-1 needs more samples
  10. 10. Clever indexing Conceptually, we have a matrix A[i][j] of tabulated values This requires two calculations to find indices – one for i and one for j – and two operations to lookup values Combine into a single index n = 992-32k + j that can be extracted directly by one bit manipulation: bits 2 through 14 minus 3040
  11. 11. Polynomial evaluation Taylor approximation: t = h B(k,j) x = A(k,j) -0.5 t + 0.125 A(k,j)t2 Rescale B’s by square root of 8: t = h B’ x = A – c t – A t2 [ c = sqrt(2) ] Horner’s method: x = A – t(c – A t)
  12. 12. C++ Implementation double NormalCCDFInverse(double x) { float f1 = (float) x; unsigned int ui; memcpy(&ui, &f1, 4); int n = (ui >> 18) - 3008; ui &= 0xFFFC0000; float f2; memcpy(&f2, &ui, 4); double v = (f1-f2)*B[n]; return A[n] - v*(sqrt2 - A[n]*v); }
  13. 13. Fine Print This algorithm only valid for p <= 0.5 For p > 0.5, use cPhi-1(p) = -cPhi-1(1-p) Phi-1(p) = cPhi-1(1-p) Algorithm not valid for p < 2-33 The maximum error is 0.000004, which occurs near p = 0.25
  14. 14. Implementation double PrivateNormalCCDFInverse(double x) … double NormalCDFInverse(double x) { return (x > 0.5) ? PrivateNormalCCDFInverse(1.0-x) : PrivateNormalCCDFInverse(x); } double NormalCCDFInverse(double x) { return (x > 0.5) ? -PrivateNormalCCDFInverse(1.0-x) : PrivateNormalCCDFInverse(x); }
  15. 15. References Rapid evaluation of the inverse of the normal distribution function by Marsaglia, Zaman, and Marsaglia Statistics and Probability Letters 19 (1994) 259 – 266
  16. 16. Notes This talk presented June 6, 2001 http://www.JohnDCook.com

×