Stat310               Transformations


                               Hadley Wickham
Wednesday, 10 February 2010
Explorations in
                              Statistics Research

                   http://www.stat.berkeley.edu/~summer/
                   7 day workshop in Boulder, Colorado
                   Travel + room & board covered
                   Large datasets, real research problems,
                   and data visualisation.



Wednesday, 10 February 2010
1. Test info
               2. Normal distribution (theory)
               3. Transformations




Wednesday, 10 February 2010
Test
                   Feb 18. 80 minute in class test. 4 questions.
                   One double sided sheet of notes.
                   Covers everything up to Feb 16: probability
                   and random variables/distributions. See
                   website for exactly what you should know.
                   Approximately half applied (working with real
                   problems) and half theoretical (working with
                   mathematical symbols).


Wednesday, 10 February 2010
Expectations
                   Points will be awarded for fully converting
                   a word problem into a mathematical
                   problem.
                   You should be able to differentiate &
                   integrate polynomials and exponentials
                   and apply the chain rule.
                   I will supply random mathematical facts
                   and tables of probabilities (if needed).


Wednesday, 10 February 2010
Note sheet
                   Much of the usefulness of a note sheet is
                   the process of making it.
                   You want to condense everything we have
                   covered. Pull out ongoing themes. Make
                   tables. Use colour!
                   Not useful: a photocopy of someone else’s
                   notes, a verbatim copy of the textbook


Wednesday, 10 February 2010
The normal
                              distribution


Wednesday, 10 February 2010
0.4                                                                        0.4
         N(-2, 1)                                                                    N(5, 1)
   0.3                                                                        0.3


   0.2                                                                        0.2
f(x)




                                                                           f(x)
   0.1                                                                        0.1


   0.0                                                                        0.0
         −10     −5           0   5       10
                                         0.4                                        −10   −5   0   5   10




                                         0.3                     N(0, 1)

                                         0.2
                                      f(x)




                                         0.1


                                         0.0
   0.4                                            −10   −5   0      5         10
                                                                              0.4

          N(0, 4)                                                                   N(0, 16)
   0.3                                                                        0.3


   0.2                                                                        0.2
f(x)




                                                                           f(x)
   0.1                                                                        0.1


   0.0                                                                        0.0
         −10     −5           0   5          10                                     −10   −5   0   5   10

Wednesday, 10 February 2010
1                               (x−µ)
                                                − 2σ2
                                                      2

        f (x) = √ e
                 2π

                              Is this a valid pdf?
Wednesday, 10 February 2010
Wolfram alpha


                   integrate 1/(sigma sqrt(2 pi)) e ^ (-(x- mu)
                   ^2 / (2(sigma^2))) from -infinity to infinity




Wednesday, 10 February 2010
Not good enough :(
                              Let’s do it by hand...




Wednesday, 10 February 2010
1 2 2
                 M (t) = e                    µt+ 2 σ t



                              A few tricks + lots of algebra
Wednesday, 10 February 2010
Your turn


                                  σ
                   If X ~ Normal(μ,2),use the mgf to
                   confirm that the mean and variance are
                   what you expect.




Wednesday, 10 February 2010
Cheating...

                   d/dt e^(mu*t + 1/2 sigma^2 t^2) at t = 0
                   d^2/dt^2 e^(mu*t + 1/2 sigma^2 t^2) at t
                   =0
                   d^2/dz^2 exp(mu*z + 1/2 sigma^2 z^2) at
                   z=0



Wednesday, 10 February 2010
Transformations
                   If X ~ Normal(μ,   σ2),   and Y = a(X + b)
                   Y ~ Normal(b + μ,    a 2σ2)


                   If a = -μ and b = 1/σ, we often write
                   Z = (X - μ) / σ
                   Z ~ Normal(0, 1) = standard normal



Wednesday, 10 February 2010
Example

                   Let X ~ Normal(5, 10)
                   What is P(3 < X < 8) ?
                   Learn how to answer that question on
                   Thursday.




Wednesday, 10 February 2010
P (Z < z) = Φ(z)
                              Φ(−z) = 1 − Φ(z)

                     P (−1 < Z < 1) = 0.68
                     P (−2 < Z < 2) = 0.95
                     P (−3 < Z < 3) = 0.998
Wednesday, 10 February 2010
Transformations



Wednesday, 10 February 2010
Discrete
                    x         -5       0       5       10    20
                 f(x)         0.2     0.1     0.3      0.1   0.3

                   Let X be a discrete random variable with
                   pmf f as defined above.
                   Write out the pmfs for:
                   A=X+2            B = 3*X   C = X2


Wednesday, 10 February 2010
Continuous
                   Let X ~ Unif(0, 1)
                   What are the distributions
                   of the following variables?
                   A = 10 X
                   B = 5X + 3
                   C=         X2




Wednesday, 10 February 2010
1.0
                                                X ~ Uniform(0, 1)


  0.8




  0.6




  0.4




  0.2




  0.0

                              0.2   0.4   0.6      0.8

Wednesday, 10 February 2010
1.0
                                                X ~ Uniform(0, 1)


  0.8




  0.6




  0.4




  0.2




  0.0

                              0.2   0.4   0.6      0.8

Wednesday, 10 February 2010
0.10
                                              10X


  0.08




  0.06




  0.04




  0.02




  0.00

                              2   4   6   8

Wednesday, 10 February 2010
0.20
                                              5X + 3



  0.15




  0.10




  0.05




  0.00

                              4   5   6   7

Wednesday, 10 February 2010
1.0
                                                X ~ Uniform(0, 1)


  0.8




  0.6




  0.4




  0.2




  0.0

                              0.2   0.4   0.6      0.8

Wednesday, 10 February 2010
X2

  20




  15




  10




    5




    0

                              0.2   0.4   0.6   0.8

Wednesday, 10 February 2010
sqrt(X)



  1.5




  1.0




  0.5




  0.0

                              0.2   0.4   0.6   0.8

Wednesday, 10 February 2010
Next time

                   Computing probabilities
                   Simulation
                   No reading, BUT GOOD OPPORTUNITY
                   TO REVIEW CURRENT MATERIAL




Wednesday, 10 February 2010

09 Normal Trans

  • 1.
    Stat310 Transformations Hadley Wickham Wednesday, 10 February 2010
  • 2.
    Explorations in Statistics Research http://www.stat.berkeley.edu/~summer/ 7 day workshop in Boulder, Colorado Travel + room & board covered Large datasets, real research problems, and data visualisation. Wednesday, 10 February 2010
  • 3.
    1. Test info 2. Normal distribution (theory) 3. Transformations Wednesday, 10 February 2010
  • 4.
    Test Feb 18. 80 minute in class test. 4 questions. One double sided sheet of notes. Covers everything up to Feb 16: probability and random variables/distributions. See website for exactly what you should know. Approximately half applied (working with real problems) and half theoretical (working with mathematical symbols). Wednesday, 10 February 2010
  • 5.
    Expectations Points will be awarded for fully converting a word problem into a mathematical problem. You should be able to differentiate & integrate polynomials and exponentials and apply the chain rule. I will supply random mathematical facts and tables of probabilities (if needed). Wednesday, 10 February 2010
  • 6.
    Note sheet Much of the usefulness of a note sheet is the process of making it. You want to condense everything we have covered. Pull out ongoing themes. Make tables. Use colour! Not useful: a photocopy of someone else’s notes, a verbatim copy of the textbook Wednesday, 10 February 2010
  • 7.
    The normal distribution Wednesday, 10 February 2010
  • 8.
    0.4 0.4 N(-2, 1) N(5, 1) 0.3 0.3 0.2 0.2 f(x) f(x) 0.1 0.1 0.0 0.0 −10 −5 0 5 10 0.4 −10 −5 0 5 10 0.3 N(0, 1) 0.2 f(x) 0.1 0.0 0.4 −10 −5 0 5 10 0.4 N(0, 4) N(0, 16) 0.3 0.3 0.2 0.2 f(x) f(x) 0.1 0.1 0.0 0.0 −10 −5 0 5 10 −10 −5 0 5 10 Wednesday, 10 February 2010
  • 9.
    1 (x−µ) − 2σ2 2 f (x) = √ e 2π Is this a valid pdf? Wednesday, 10 February 2010
  • 10.
    Wolfram alpha integrate 1/(sigma sqrt(2 pi)) e ^ (-(x- mu) ^2 / (2(sigma^2))) from -infinity to infinity Wednesday, 10 February 2010
  • 11.
    Not good enough:( Let’s do it by hand... Wednesday, 10 February 2010
  • 12.
    1 2 2 M (t) = e µt+ 2 σ t A few tricks + lots of algebra Wednesday, 10 February 2010
  • 13.
    Your turn σ If X ~ Normal(μ,2),use the mgf to confirm that the mean and variance are what you expect. Wednesday, 10 February 2010
  • 14.
    Cheating... d/dt e^(mu*t + 1/2 sigma^2 t^2) at t = 0 d^2/dt^2 e^(mu*t + 1/2 sigma^2 t^2) at t =0 d^2/dz^2 exp(mu*z + 1/2 sigma^2 z^2) at z=0 Wednesday, 10 February 2010
  • 15.
    Transformations If X ~ Normal(μ, σ2), and Y = a(X + b) Y ~ Normal(b + μ, a 2σ2) If a = -μ and b = 1/σ, we often write Z = (X - μ) / σ Z ~ Normal(0, 1) = standard normal Wednesday, 10 February 2010
  • 16.
    Example Let X ~ Normal(5, 10) What is P(3 < X < 8) ? Learn how to answer that question on Thursday. Wednesday, 10 February 2010
  • 17.
    P (Z <z) = Φ(z) Φ(−z) = 1 − Φ(z) P (−1 < Z < 1) = 0.68 P (−2 < Z < 2) = 0.95 P (−3 < Z < 3) = 0.998 Wednesday, 10 February 2010
  • 18.
  • 19.
    Discrete x -5 0 5 10 20 f(x) 0.2 0.1 0.3 0.1 0.3 Let X be a discrete random variable with pmf f as defined above. Write out the pmfs for: A=X+2 B = 3*X C = X2 Wednesday, 10 February 2010
  • 20.
    Continuous Let X ~ Unif(0, 1) What are the distributions of the following variables? A = 10 X B = 5X + 3 C= X2 Wednesday, 10 February 2010
  • 21.
    1.0 X ~ Uniform(0, 1) 0.8 0.6 0.4 0.2 0.0 0.2 0.4 0.6 0.8 Wednesday, 10 February 2010
  • 22.
    1.0 X ~ Uniform(0, 1) 0.8 0.6 0.4 0.2 0.0 0.2 0.4 0.6 0.8 Wednesday, 10 February 2010
  • 23.
    0.10 10X 0.08 0.06 0.04 0.02 0.00 2 4 6 8 Wednesday, 10 February 2010
  • 24.
    0.20 5X + 3 0.15 0.10 0.05 0.00 4 5 6 7 Wednesday, 10 February 2010
  • 25.
    1.0 X ~ Uniform(0, 1) 0.8 0.6 0.4 0.2 0.0 0.2 0.4 0.6 0.8 Wednesday, 10 February 2010
  • 26.
    X2 20 15 10 5 0 0.2 0.4 0.6 0.8 Wednesday, 10 February 2010
  • 27.
    sqrt(X) 1.5 1.0 0.5 0.0 0.2 0.4 0.6 0.8 Wednesday, 10 February 2010
  • 28.
    Next time Computing probabilities Simulation No reading, BUT GOOD OPPORTUNITY TO REVIEW CURRENT MATERIAL Wednesday, 10 February 2010