Introduction
        Mathematical Background & Concepts
           Implementation of a MC Simulation
 Post-processing and ...
Introduction
                 Mathematical Background & Concepts
                    Implementation of a MC Simulation
   ...
Introduction
                  Mathematical Background & Concepts
                                                        ...
Introduction
                   Mathematical Background & Concepts
                                                       ...
Introduction
                   Mathematical Background & Concepts
                                                       ...
Introduction
                 Mathematical Background & Concepts
                                                         ...
Introduction
                Mathematical Background & Concepts
                                                          ...
Introduction
                  Mathematical Background & Concepts
                                                        ...
Introduction
                  Mathematical Background & Concepts
                                                        ...
Introduction
                   Mathematical Background & Concepts
                                                       ...
Introduction
                   Mathematical Background & Concepts
                                                       ...
Introduction
                 Mathematical Background & Concepts
                                                         ...
Introduction
                 Mathematical Background & Concepts
                                                         ...
Introduction
                  Mathematical Background & Concepts
                                                        ...
Introduction
                 Mathematical Background & Concepts
                    Implementation of a MC Simulation
   ...
Introduction
                 Mathematical Background & Concepts
                    Implementation of a MC Simulation
   ...
Introduction
                  Mathematical Background & Concepts      Developing a Mathematical Model
                   ...
Introduction
                 Mathematical Background & Concepts      Developing a Mathematical Model
                    ...
Introduction
                 Mathematical Background & Concepts      Developing a Mathematical Model
                    ...
Introduction
                    Mathematical Background & Concepts      Developing a Mathematical Model
                 ...
Introduction
                  Mathematical Background & Concepts      Developing a Mathematical Model
                   ...
Introduction
                  Mathematical Background & Concepts      Developing a Mathematical Model
                   ...
Introduction
                  Mathematical Background & Concepts      Developing a Mathematical Model
                   ...
Introduction
                  Mathematical Background & Concepts      Developing a Mathematical Model
                   ...
Introduction
                  Mathematical Background & Concepts      Developing a Mathematical Model
                   ...
Introduction
                Mathematical Background & Concepts      Developing a Mathematical Model
                   Im...
Introduction
                Mathematical Background & Concepts      Developing a Mathematical Model
                   Im...
Introduction
                  Mathematical Background & Concepts      Developing a Mathematical Model
                   ...
Introduction
                  Mathematical Background & Concepts      Developing a Mathematical Model
                   ...
Introduction
                 Mathematical Background & Concepts      Developing a Mathematical Model
                    ...
Introduction
                  Mathematical Background & Concepts      Developing a Mathematical Model
                   ...
Introduction
                                Mathematical Background & Concepts            Developing a Mathematical Model...
Introduction
                                               Mathematical Background & Concepts                            ...
Introduction
                                               Mathematical Background & Concepts                            ...
Introduction
                                               Mathematical Background & Concepts                            ...
Introduction
                    Mathematical Background & Concepts           Developing a Mathematical Model
            ...
Introduction
                   Mathematical Background & Concepts          Developing a Mathematical Model
              ...
Introduction
                   Mathematical Background & Concepts      Developing a Mathematical Model
                  ...
Introduction
                 Mathematical Background & Concepts         Developing a Mathematical Model
                 ...
Introduction
                  Mathematical Background & Concepts      Developing a Mathematical Model
                   ...
Introduction
                   Mathematical Background & Concepts      Developing a Mathematical Model
                  ...
Introduction
                         Mathematical Background & Concepts             Developing a Mathematical Model
     ...
Introduction
                  Mathematical Background & Concepts      Developing a Mathematical Model
                   ...
Introduction
                  Mathematical Background & Concepts      Developing a Mathematical Model
                   ...
Introduction
                 Mathematical Background & Concepts      Developing a Mathematical Model
                    ...
Introduction
                                     Mathematical Background & Concepts               Developing a Mathematic...
Introduction
                       Mathematical Background & Concepts           Developing a Mathematical Model
         ...
Introduction
                  Mathematical Background & Concepts      Developing a Mathematical Model
                   ...
Introduction
                  Mathematical Background & Concepts          Developing a Mathematical Model
               ...
Introduction
                  Mathematical Background & Concepts      Developing a Mathematical Model
                   ...
Introduction
                   Mathematical Background & Concepts           Developing a Mathematical Model
             ...
Introduction
                  Mathematical Background & Concepts      Developing a Mathematical Model
                   ...
Introduction
                   Mathematical Background & Concepts      Developing a Mathematical Model
                  ...
Review of Methodology and Rationale of Monte Carlo Simulation - Application to Metrology with Open Source Software
Review of Methodology and Rationale of Monte Carlo Simulation - Application to Metrology with Open Source Software
Review of Methodology and Rationale of Monte Carlo Simulation - Application to Metrology with Open Source Software
Review of Methodology and Rationale of Monte Carlo Simulation - Application to Metrology with Open Source Software
Review of Methodology and Rationale of Monte Carlo Simulation - Application to Metrology with Open Source Software
Review of Methodology and Rationale of Monte Carlo Simulation - Application to Metrology with Open Source Software
Review of Methodology and Rationale of Monte Carlo Simulation - Application to Metrology with Open Source Software
Review of Methodology and Rationale of Monte Carlo Simulation - Application to Metrology with Open Source Software
Review of Methodology and Rationale of Monte Carlo Simulation - Application to Metrology with Open Source Software
Upcoming SlideShare
Loading in …5
×

Review of Methodology and Rationale of Monte Carlo Simulation - Application to Metrology with Open Source Software

795 views
732 views

Published on

Review of Methodology and Rationale of Monte Carlo Simulation - Application to Metrology with Open Source Software

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
795
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Review of Methodology and Rationale of Monte Carlo Simulation - Application to Metrology with Open Source Software

  1. 1. Introduction Mathematical Background & Concepts Implementation of a MC Simulation Post-processing and Analysis of a Simulation Discussion Review of Methodology and Rationale of Monte Carlo Simulation Application to Metrology with Open Source Software Vishal Ramnath vramnath@nmisa.org Mechanical Metrology Group National Metrology Institute of South Africa November 6, 2008 Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  2. 2. Introduction Mathematical Background & Concepts Implementation of a MC Simulation Post-processing and Analysis of a Simulation Discussion Overview of Presentation Introduction Review of GUM Methodology Review of Monte Carlo Methodology Mathematical Background & Concepts Implementation of a MC Simulation Developing a Mathematical Model Assigning Uncertainties and PDF’s to the Model Illustrative Mass Unc Example Post-processing and Analysis of a Simulation Analysing and Understanding the Data Reporting Results in GUM Terms Discussion Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  3. 3. Introduction Mathematical Background & Concepts Review of GUM Methodology Implementation of a MC Simulation Review of Monte Carlo Methodology Post-processing and Analysis of a Simulation Discussion Introduction This is an introductory presentation to convey the basic ideas behind the mechanics of the Monte Carlo technique as applied to metrology measurement uncertainty problems. The rationale for the need to understand and implement Monte Carlo (MC) techniques in the context of metrology is that with the advance of science and technology more accurate measurements are for various reasons increasingly necessary in many economies and MC simulations present the most accurate and readily available numerical technology to solve such challenges taking into account certain limitations in existing approaches such as the well known ISO Guide to Uncertainty in Measurement (GUM). Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  4. 4. Introduction Mathematical Background & Concepts Review of GUM Methodology Implementation of a MC Simulation Review of Monte Carlo Methodology Post-processing and Analysis of a Simulation Discussion GUM Review 1 - Essential Information Needed For an input quantity xi in the GUM framework three quantities are needed viz. ◮ the expectation of xi which is just the estimate of this input ◮ the standard deviation of xi which is the standard deviation of this input σ(xi ) ◮ the corresponding degrees of freedom νi associated with xi If there are dependencies with other input quantities xj , j = i then covariances are also required: in the case of two inputs xi and xj the covariance u(xi , xj ) and correlation coefficient r (xi , xj ) are related by (1) u(xi , xj ) = r (xi , xj )u(xi )u(xj ), −1 r (xi , xj ) 1 Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  5. 5. Introduction Mathematical Background & Concepts Review of GUM Methodology Implementation of a MC Simulation Review of Monte Carlo Methodology Post-processing and Analysis of a Simulation Discussion GUM Review 1 cont. - Essential Information Needed q 1 (2) u(xi , xj ) = ¯ ¯ (xi,k − xi )(xj,k − xj ) q(q − 1) k =1 ◮ If r (xi , xj ) = 0 then there is no correlation and if r (xi , xj ) ≈ 1 then there is strong correlation ◮ Most uncertainty calculations assume r (xi , xj ) ≈ 0 for simplicity i.e. no correlation between input quantities but if necessary correlation can be explicitly incorporated into calculations ◮ In the case of correlation between more than two variables e.g. xi , xj , xk with i = j = k then a covariance matrix and not a scalar correlation coefficient is required Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  6. 6. Introduction Mathematical Background & Concepts Review of GUM Methodology Implementation of a MC Simulation Review of Monte Carlo Methodology Post-processing and Analysis of a Simulation Discussion GUM Review 1 cont. - Essential Information Needed The GUM approach is the propagation of uncertainties associated with input quantities in a measurement model to provide estimates of the model output quantity (univariate) or quantities (multivariate) It should be noted that: ◮ Within the framework of the GUM a mathematical model of the measurand is a prerequisite in order to implement an uncertainty calculation Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  7. 7. Introduction Mathematical Background & Concepts Review of GUM Methodology Implementation of a MC Simulation Review of Monte Carlo Methodology Post-processing and Analysis of a Simulation Discussion GUM Review 2 - Standard Calculation Technique (3a) y = f (x1 , . . . , xn ) ⇐ math model ∂f (3b) ci = ⇐ sens coeff ∂xi n 2 ∂f (3c) u 2 (y ) = u 2 (xi ) ⇐ std unc ∂xi i=1 n 4 u (y ) ci4 u 4 (xi ) (3d) = ⇐ calc eff deg freedom νeff νi i=1 νeff +1 − t Γ[ νeff2+1 ] u2 2 k ⇔ √ 1+ du = p −t πνeff Γ[ ν2 ]eff νeff ⇑ (3e) coverage factor Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  8. 8. Introduction Mathematical Background & Concepts Review of GUM Methodology Implementation of a MC Simulation Review of Monte Carlo Methodology Post-processing and Analysis of a Simulation Discussion GUM Review 2 - Brief Comment on Sensitivity coeff’s Various possibilities will arise in practise with real inputs x ∈ Rn : ◮ univariate, explicit, real model or multivariate, explicit, real model ◮ univariate, implicit, real model or multivariate, implicit, real model In the case of an implicit model i.e. where an explicit functional relationship between the input and output(s) is not known then additional matrix algebraic manipulations are necessary and such manipulations require the solution of linear systems of equations In addition as per the above but with complex models i.e. with x ∈ Cn require analogous sensitivity conterparts where now partial derivatives for both the real and imaginary components of an input are necessary Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  9. 9. Introduction Mathematical Background & Concepts Review of GUM Methodology Implementation of a MC Simulation Review of Monte Carlo Methodology Post-processing and Analysis of a Simulation Discussion GUM Review 3 - Assumptions & Limitations of the GUM There are three chief requirements that limit the applicability of the GUM: ◮ the non-linearity for the measurand as modelled by a function f (x ) must be insignificant [GUM Clause 5.1.2] ◮ the Central Limit Theorem must be assumed to apply for the model of the measurand i.e. the PDF for the output must be Gaussian (alternately in terms of a t-distribution) [GUM Clauses G.2.1 and G.6.6] and ◮ the necessary conditions for a Welch-Satterthwaite formula to calculate the effective degrees of freedom must apply [GUM Clause G.4.2] Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  10. 10. Introduction Mathematical Background & Concepts Review of GUM Methodology Implementation of a MC Simulation Review of Monte Carlo Methodology Post-processing and Analysis of a Simulation Discussion GUM Review 4 - When & how will the GUM not work As per the three requirements for the GUM to adequately apply it will then by implication not function adequately when: ◮ non-linearities in the model are significant - when the model can not accurately be represented by a first order Taylor series expansion then the probability distribution of the measurand can similarly not be accurately represented in terms of the convolution integral of the distributions of the input quantities; ◮ the conditions for the validity of the Central Limit Theorem as applicable to the measurement model are not sufficiently strong - theoretically the CLT predicts a Gaussian distribution for the measurand only in the limit as the number of input quantities increases i.e. it is not necessarily a true or accurate representation of the measurand PDF for a small finite number of input parameters Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  11. 11. Introduction Mathematical Background & Concepts Review of GUM Methodology Implementation of a MC Simulation Review of Monte Carlo Methodology Post-processing and Analysis of a Simulation Discussion GUM Review 4 cont. - When & how will the GUM not work As per the three requirements for the GUM to adequately apply it will then by implication not function adequately when: ◮ the conditions for the validity of the Welch-Satterthwaite formula are not present i.e. in the case for a univariate, real output y where the input quantities x are not mutually independent - the GUM does not state how νeff is to be calculated when the input quantities are correlated† i.e. even though correlation coefficients (alternately covariance matrix) may be modelled / calculated from experimental data there is no methodology to estimate νeff and hence a corresponding coverage factor k unless one assumes u(xi , xj ) ≈ 0∀i = j †Correlation coefficients r (xi , xj ) are used for calculating the combined standard uncertainty uc , cf. U = k(p, νeff ) · uc Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  12. 12. Introduction Mathematical Background & Concepts Review of GUM Methodology Implementation of a MC Simulation Review of Monte Carlo Methodology Post-processing and Analysis of a Simulation Discussion Review of GUM Methodology - Background to why MC is being utilized With the three requirements for the GUM to adequately apply and with limitations and lack of applicability that arises when these conditions are not met for many practical measurement models of real measurement systems and standards we then see that: ◮ Due to the sometimes restrictive conditions on the limitations and applicability of the GUM that many NMI’s and possibly even industrial metrology laboratories are starting to investigate and implement Monte Carlo simulations for their own laboratory standards and in inter-comparisons for e.g. CMC justifications Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  13. 13. Introduction Mathematical Background & Concepts Review of GUM Methodology Implementation of a MC Simulation Review of Monte Carlo Methodology Post-processing and Analysis of a Simulation Discussion Review of MC Methodology In a measurement uncertainty analysis one is concerned with propagating uncertainties from inputs to outputs and the GUM propogates uncertainties from a first order approximation from a model of a measurement system with the assumption that the measurand has a Gaussian distribution whilst a MC method directly propogates PDF information without any prior assumptions. A MC method can be accurately described as a statistical sampling technique. Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  14. 14. Introduction Mathematical Background & Concepts Review of GUM Methodology Implementation of a MC Simulation Review of Monte Carlo Methodology Post-processing and Analysis of a Simulation Discussion Outline of MC Process Used for a Univariate Model 1. select M Monte Carlo trials 2. generate M vectors by sampling from the PDF’s for the set of N input quantities 3. for each vector evaluate the model to give the corresponding value of the output quantity 4. calculate the estimate of the output quantity i.e. the measurand and its associated standard uncertainty 5. use the simulation data to build a discrete representation of the distribution function 6. use the distribution function to calculate the coverage interval for the measurand Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  15. 15. Introduction Mathematical Background & Concepts Implementation of a MC Simulation Post-processing and Analysis of a Simulation Discussion Mathematical Background & Concepts For continous random variables recall: f (x ) is a PDF for a random ∞ variable x if (i) f (x ) 0∀x ∈ R, (ii) −∞ f (x )dx = 1, (iii) b P(a < X < b) = a f (x )dx The corresponding cumulative distribution function is x F (x ) = −∞ f (t)dt For MC work we will use the following nomenclature: Let the PDF for input Xi be gi (ξi ), the PDF for the measurand Y be g(η), and η G(η) = −∞ g(z)dz denote the distribution function corresponding to g(η) Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  16. 16. Introduction Mathematical Background & Concepts Implementation of a MC Simulation Post-processing and Analysis of a Simulation Discussion Mathematical Background & Concepts There are additional mathematical definitions and terminology that are necessary to more fully understand how a Monte Carlo simulation works in practice but for our purposes we will not delve too deeply into the finer details in this presentation and rather concentrate on some of the more practical considerations that are needed if one wishes to undertake and implement a MC measurement uncertainty analysis Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  17. 17. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Implementation of a MC Simulation A few preliminary points should be noted: ◮ a good random number generator is essential for reliable work - the MS Excel RNG is not satisfactory and will introduce problems ◮ the software code used should allow definition of a model and the parameters defining the PDF’s for the input quantities ◮ symmetry in the output PDF is not assumed ◮ no derivatives are required ◮ there is an avoidance of the concept of effective degrees of freedom ◮ sensitivity coefficients are not calculated or needed: possible to modify post-processing to calc a sensitivity coeff Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  18. 18. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Developing a Mathematical Model When developing a mathematical model for a MC simulation it should be noted that there is no distinction between Type A and Type B uncertainty contributors and that the measurand is simply defined in terms of a function e.g. (4) y = f (x1 , x2 , . . . , xn ) where the inputs x1 , . . . , xn directly model and describe the influence if an input is changed - it is this variation/change in the input parameters that is propogated through the model and hence influences the output expressible as an uncertainty. Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  19. 19. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Developing a Mathematical Model A MC simulation is therefore different from the GUM in the sense that one has to have a full and complete understanding of the entire measurement system under investigation and one can not simply assign an input uncertainty and a unity sensitivity coefficient without adequate justification Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  20. 20. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Assigning Uncertainties and PDF’s to the Model In a MC simulation the PDF’s of the input quantities g1 (ξ1 ), . . . , gN (ξN ) are required and the following options are possible: ◮ if the input xi is a quantity that has been measured/calibrated then it will have a measurement/calibration certificate that was done with the GUM so the quoted value is the mean µi and its standard uncertainty is obtained by dividing the expanded uncertainty by the applicable coverage factor - this is enough information to infer its PDF since the GUM result is always expressed in terms of a Gaussian PDF which is completely defined in terms of µ and σ ◮ similarly as above for rectangular (particularly if estimated from e.g. literature), triangular, U shaped distributions etc. Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  21. 21. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Assigning Uncertainties and PDF’s to the Model cont. ◮ a statistical analysis based on relevant theory may indicate an inputs PDF e.g. dimensional cosine terms and one then just has to estimate some parameters to fully define the PDF ◮ there may be discrete numerical data for an input parameter which means that input parameter’s PDF can be built up with its frequency data (like a histogram) Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  22. 22. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Mass Meas Unc Example to Contrast the GUM vs. MC Example (Math Model Explanation & Details) Consider a mass calibration to illustrate the principles of a MC simulation and some of the differences with the standard GUM approach with a schematic illustration of the measurement system Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  23. 23. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Mass Meas Unc Example to Contrast the GUM vs. MC Example (Math Model Explanation & Details) Consider a mass calibration to δmR illustrate the principles of a MC simulation and some of the mW mR differences with the standard GUM pivot approach with a schematic illustration of the measurement system Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  24. 24. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Mass Meas Unc Example to Contrast the GUM vs. MC Example (Math Model Explanation & Details) Consider a mass calibration to δmR illustrate the principles of a MC simulation and some of the mW mR differences with the standard GUM pivot approach with a schematic illustration of the measurement system The principle of a mass measurement with an equal arm force balance is of a balance of moments generated which by applying a force balance with Archimedes’ principle for buoyancy effects we then get Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  25. 25. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Mass Meas Unc Example to Contrast the GUM vs. MC Example (Math Model Explanation & Details) Consider a mass calibration to δmR illustrate the principles of a MC simulation and some of the mW mR differences with the standard GUM pivot approach with a schematic illustration of the measurement system The principle of a mass measurement with an equal arm force balance is of a balance of moments generated which by applying a force balance with Archimedes’ principle for buoyancy effects we then get mW mR + δmR (5) mW g − ρair g = (mR + δmR )g − ρair g ρW ρR Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  26. 26. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Developing the Meas Model Example (Mass example cont.) Rearranging Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  27. 27. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Developing the Meas Model Example (Mass example cont.) Rearranging ρair ρair mW 1− = (mR + δmR ) 1 − ρW ρR Symbol Description mW mass of weight piece W mR mass of reference weight piece R δmR small test mass to add onto mR to achieve force balance ρi mass density with i respectively that of the weight W air medium air or that of the reference’s density R Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  28. 28. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Developing the Meas Model cont. Example (Expressing model with lab specific stds) Mass is an invariant quantity as any physical parameter in metrology but the values quoted will differ depending on the lab system in use e.g. mass is“heavier” in air than in water which is why astronauts train under water to simulate weightlessness: Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  29. 29. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Developing the Meas Model cont. Example (Expressing model with lab specific stds) Mass is an invariant quantity as any physical parameter in metrology but the values quoted will differ depending on the lab system in use e.g. mass is“heavier” in air than in water which is why astronauts train under water to simulate weightlessness: In mass metrology laboratories use the concept of “conventional mass” Definition (Conventional Mass) The conventional mass mW ,c of a weight W is the apparent mass of a hypothetical weight of density ρW 0 = 8000 kg.m−3 that balances W in air when the air density is ρair 0 = 1.2 kg.m−3 i.e. mW (1 − ρair 0 /ρW ) = mW ,c (1 − ρair 0 /ρW 0 ) Fact (Usage of Conventional Mass) Conventional mass is simply a measurement tool used to incorporate the invariance of inertial mass i.e. “compare apples with apples” Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  30. 30. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Final Measurement Model We utilize the Open Source computer algebra system Maxima to simplify our calculations due to the substitions that the conventional mass introduces. The reason for why one may prefer to use a CAS is in the cases when fairly complicated expressions and hand calculations carry the risk of lengthy time and error generation. Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  31. 31. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Review of calculas theory - cf. GUM assumptions Recall that from multi-variable calculas that Taylor series expansions for the case of a single variable (6) f ′ (a) f ′′ (a) f (n−1) (a) f (x ) = f (a)+ (x −a)+ (x −a)2 +· · ·+ (x −a)n−1 +Rn (x ) 1! 2! (n − 1)! can be generalized to the case for multiple variables. An example for two variables would be ∂f ∂f f (x , y ) = f (a, b) + (a, b)(x − a) + (a, b)(y − b) ∂x ∂y 1 ∂ 2f ∂2f + (a, b)(x − a)2 + 2 (a, b)(x − a)(y − b) 2! ∂x 2 ∂x ∂y ∂ 2f + (a, b)(y − b)2 + · · · ∂y 2 Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  32. 32. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Approx. of functions - linearized in 1D Let f (x ) = exp(x 2 ) and expand about a = 1 f = exp(x 2 ) 9 8 y-axis 7 2 6 function ex 5 1st e + 2e(x − 1) 4 2nd e + 2e(x − 1) + 3e(x − 1)2 3 2 1 -1.5 -1 -0.5 0 0.5 1 1.5 x-axis Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  33. 33. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Approx. of functions - linearized in 2D Let f (x ) = exp(x 2 + y 2 ) and expand about a = [1, 1]T Linear approx: L(f ) = 2e2 (y − 1) + 2e2 (x − 1) + e2 9 8.5 9 8 8.5 7.5 8 z1 7 7.5 7 6.5 6.5 6 6 5.5 5.5 1.05 1.04 1.03 1.02 0.95 0.96 1.01 1 0.97 0.98 0.99 y1 0.99 1 0.98 1.01 1.02 0.97 x1 1.03 1.04 0.96 0.95 1.05 3d plot 1.05 0 1.04 -0.5 1.03 1.02 -1 1.01 y1 1 -1.5 0.99 -2 0.98 0.97 -2.5 0.96 0.95 -3 0.95 0.96 0.97 0.98 0.99 1 1.01 1.02 1.03 1.04 1.05 x1 % error Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  34. 34. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Approx. of functions - linearized in 2D Let f (x ) = exp(x 2 + y 2 ) and expand about a = [1, 1]T Quadratic approx: Linear approx: L(f ) = 2e2 (y − 1) + 2e2 (x − 1) + e2 f = L(f ) + 1 [6e2 (y − 1)2 + 8e2 (x − 1)(y − 1) + 6e2 (x − 1)2 ] 2 9 8.5 9.5 9 8 9 8.5 9.5 7.5 8.5 8 9 z1 7 8 7.5 8.5 7 6.5 z1 7.5 8 6.5 6 7.5 7 6 5.5 7 6.5 5.5 6.5 6 1.05 6 1.04 1.03 1.05 1.02 1.04 0.95 0.96 1.01 1.03 1 1.02 0.97 0.98 0.99 y1 1.01 0.99 1 0.98 0.95 0.96 1 1.01 1.02 0.97 0.97 0.98 0.99 y1 x1 1.03 1.04 0.96 0.99 1 0.98 0.95 1.05 1.01 1.02 0.97 x1 1.03 1.04 0.96 0.95 1.05 3d plot 3d plot 1.05 0 1.05 0.3 1.04 -0.5 1.04 0.25 1.03 1.03 0.2 1.02 -1 0.15 1.02 1.01 0.1 1.01 y1 1 -1.5 0.05 y1 1 0.99 0 -2 0.99 0.98 -0.05 0.98 0.97 -0.1 -2.5 0.97 -0.15 0.96 0.96 -0.2 0.95 -3 0.95 0.96 0.97 0.98 0.99 1 1.01 1.02 1.03 1.04 1.05 0.95 -0.25 x1 0.95 0.96 0.97 0.98 0.99 1 1.01 1.02 1.03 1.04 1.05 x1 % error % error Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  35. 35. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Approx. of functions - linearized in 2D Let f (x ) = exp(x 2 + y 2 ) and expand about a = [1, 1]T Quadratic approx: Linear approx: L(f ) = 2e2 (y − 1) + 2e2 (x − 1) + e2 f = L(f ) + 1 [6e2 (y − 1)2 + 8e2 (x − 1)(y − 1) + 6e2 (x − 1)2 ] 2 9 8.5 9.5 9 8 9 8.5 9.5 7.5 8.5 8 9 z1 7 8 7.5 8.5 7 6.5 z1 7.5 8 6.5 6 7.5 7 6 5.5 7 6.5 5.5 6.5 6 1.05 6 1.04 1.03 1.05 1.02 1.04 0.95 0.96 1.01 1.03 1 1.02 0.97 0.98 0.99 y1 1.01 0.99 1 0.98 0.95 0.96 1 1.01 1.02 0.97 0.97 0.98 0.99 y1 x1 1.03 1.04 0.96 0.99 1 0.98 0.95 1.05 1.01 1.02 0.97 x1 1.03 1.04 0.96 0.95 1.05 3d plot 3d plot 1.05 0 1.05 0.3 1.04 -0.5 1.04 0.25 1.03 1.03 0.2 1.02 -1 0.15 1.02 1.01 0.1 1.01 y1 1 -1.5 0.05 y1 1 0.99 0 -2 0.99 0.98 -0.05 0.98 0.97 -0.1 -2.5 0.97 -0.15 0.96 0.96 -0.2 0.95 -3 0.95 0.96 0.97 0.98 0.99 1 1.01 1.02 1.03 1.04 1.05 0.95 -0.25 x1 0.95 0.96 0.97 0.98 0.99 1 1.01 1.02 1.03 1.04 1.05 x1 % error % error The error for a linear approximation of the very non-linear model function i.e. using the GUM method is ∼ 3% and for a quadratic approximation ∼ 0.3% Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  36. 36. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Taylor series for f (x1 , . . . , xd ) For n variables we then have the Taylor series T (x1 , . . . , xd ) for f (x1 , . . . , xd ) as ∞ ∞ ∂ n1 ∂ nd f (a1 , . . . , ad ) T (x1 , . . . , xd ) = ··· n1 · · · n n1 =0 nd =0 ∂x1 ∂xd d n1 ! · · · nd ! (7) ×(x1 − a1 )n1 · · · (xd − ad )nd D α f (a) (8) T (x1 , . . . , xd ) = (x − a)α α! i∈N0 Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  37. 37. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Special cases for Taylor series expansions For a model f (x) of the measurement system given with inputs x = [x1 , . . . , xn ]T (x is a column vector with dimensions n × 1) and with nominal value a which is the state that the measurement ystem is in then making use of the general Taylor series expansion for α f (a) multiple variables T (x) = |α| 0 D α! (x − a)α we note that: ◮ First order approximation: ∂f ∂f (9) f (x ) ≈ f (a1 , . . . , an ) + (x1 − a1 ) + · · · + (xn − an ) ∂x1 a ∂xn a Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  38. 38. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Special cases for Taylor series expansions cont. In most cases it is seldom beneficial to construct a multiple variable Taylor series expansion for 3rd or higher order ◮ Second order approximation:   x1 − a1 ∂f ∂f . f (x ) ≈ f (a1 , . . . , an ) + ···  . .  ∂xa a ∂xn a   xn − an  ∂2f ∂2 f ∂2 f  ∂x1 2 ∂x1 ∂x2 · · · ∂x1 ∂xn  ∂2f ∂2 f ∂2 f   1  ∂x ∂x  ∂x2 2 · · · ∂x2 ∂xn  x1 − a1 (10) +  2. 1 ···  . .  2!  . . .. .   . . . .   xn − an ∂2f ∂2 f ∂2 f ∂xn ∂x1 ∂xn ∂x2 ··· ∂x 2 n Comment: the n × n square matrix above is the Hessian matrix for f and all the entries are evaluated at a Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  39. 39. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Final Measurement Model - Calc details Recall that we have a model for the mass to be measured ρair ρair (11) mW 1− = (mR + δmR ) 1 − ρW ρR and we wish to write the model in terms of the ‘conventional’ mass by subsituting the formulae −1 ρair 0 ρair 0 (12a) mW = mW ,c 1 − 1− ρW 0 ρW −1 ρair 0 ρair 0 (12b) mR = mR,c 1 − 1− ρW 0 ρR −1 ρair 0 ρair 0 (12c) δmR = δmR,c 1 − 1− ρW 0 ρR Once the equations are substited we then want to solve for mW ,c which is the mass of the weight that we wish to calibrate Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  40. 40. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Measurement Model - Formulating and solving in Maxima In Maxima we use the following computer code: LHS: mW*(1 - rhoair/rhoW); RHS: (mR + deltamR)*(1 - rhoair/rhoR); LHS1: ev(LHS, mW = mWc*(1 - rhoair0/rhoW0)/(1 - rhoair0/rhoW)); RHS1: ev(RHS, mR = mRc*(1 - rhoair0/rhoW0)/(1 - rhoair0/rhoR), deltamR = deltamRc*(1 - rhoair0/rhoW0)/(1 - rhoair0/rhoR)); soln: solve(LHS1 = RHS1, mWc); mWc: rhs(soln[1]); whence 1 mW ,c = × (ρR − ρair 0 )ρW − ρair ρR + ρair ρair 0 [((mR,c + δmR,c )ρR − ρair mR,c − ρair δmR,c )ρW +(−ρair 0 mR,c − ρair 0 δmR,c )ρR (13) +ρair ρair 0 mR,c + ρair ρair 0 δmR,c ] Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  41. 41. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Measurement Model - GUM Formulation The measurement model consists of five input parameters which we list below: symbol description PDF comments mR,c reference mass Gaussian from meas cert δmR,c balance mass Gaussian from meas cert ρair density of air rectangular estimated from CIPM formula ρW density of weight rectangular estimate that is equally likely ρR density of reference rectangular from literature of physical properties Comment on parameters that are not included: ◮ the density ρW 0 does not explicitly appear in the model equation as it cancels out when the model equation is algebraically solved for mW ,c which would not be obvious in a spreadsheet ◮ the air density ρair 0 is not included in the model as a variable but as a constant since this is known exactly Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  42. 42. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Practical Implementation of MC Model Inputs Definition (Model constants & parameters) In a measurement mathematical model working in SI units one should distinguish between how to incorporate constants and parameters. A parameter is a variable that one is uncertain of and which has a statistical uncertainty (however small) and PDF whilst a constant is exactly known.1 Fact (Theories which use exact constants i.e. zero unc) An example would be the speed of light which was historically measured using various techniques with associated experimental uncertainties and with Einstein’s Special Theory of Relativity fixed and then later defined as def c0 = 299792458 m.s−1 where σ(c0 ) = 0 Fact (Theories which use approx constants i.e. finite unc) An example would be the Avogadro number NA = 6.02214179 × 1023 mol−1 which as a constant of nature is fixed but which is currently experimentally known to an accuracy of σ(NA ) = 0.00000030 × 1023 mol−1 1For details see the CODATA website for physical and chemical reference values at http://physics.nist.gov/cuu/Constants/international.html Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  43. 43. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Math Model - GUM Calcs 1 We will consider both first and second order calc’s using the GUM for comparison with a Monte Carlo simulation. ◮ 1st order: apply the GUM as usual with sums of products of gradients etc. ◮ 2nd order: must build up f with additional terms using Hessian matrix etc. Fact (Practical observation of GUM calc’s) The GUM makes use of the assumption that there is a linearized model of the system to propogate the uncertainties and to be strictly consistent one should apply a linearized model when calculating the standard uncertainty in order not to mix of terms from different assumptions and approximations, however we note that in practice most metrologists would most likely take the original expression to evaluate the model and not its linearization - this is only valid if the model is approximately linear in a neighbourhood of a where a is the state space that the system is in. Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  44. 44. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Math Model - GUM Calcs 2 Since there are five input variables in our mathematical model the distinction and implications of 1st and 2nd order approximations are not immediately obvious to appreciate. Noting the variables mR,c , δmR,c , ρair , ρW , ρR we can reasonably conclude that the two variables that are most likely to be uncertain and vary are ◮ δmR,c because this must be adequately controlled to achieve a balance and equlibrium on the force beam and the equilibrium can be a bit subjective if there isn’t an exact balance and the beam is moving very slowly ◮ ρair the actual air density which will depend and vary with the laboratory’s ambient temperature and pressure Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  45. 45. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Math Model - GUM Calcs 2 cont Setting the nominal conditions for argument as a = (mR,c , δmR,c , ρair , ρW , ρR ) = (0.099 kg, 0.001 kg, 1.17 kg.m−3 , 7800 kg.m−3 , 8000 kg.m−3 ) for mW ,c ≈ 0.100 kg we can then see in a limited sense the implications of the GUM requirement for a linearized model. Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  46. 46. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Mass Unc Math Model - Approx Linear approx of f error [ppm] Quadratic approx of f error [ppm] O(4 × 10−3 ) O(5 × 10−8 ) ρair / kg.m−3 ρair / kg.m−3 1.35 0.004 1.35 5e-008 4e-008 0.003 1.3 1.3 3e-008 0.002 2e-008 0.001 1.25 1.25 1e-008 0 0 1.2 1.2 -1e-008 -0.001 -2e-008 -0.002 1.15 1.15 -3e-008 -0.003 -4e-008 1.1 -0.004 1.1 -5e-008 0 0.0005 0.001 0.0015 0.002 0.0025 0 0.0005 0.001 0.0015 0.002 0.0025 δmR,c / kg δmR,c / kg Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  47. 47. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Uncertainty results using the GUM 1 Assuming for argument that all the inputs for mW ,c = f (mR,c , δmR,c , ρair , ρW , ρR ) have uncertainties of 0.1% and in addition are not correlated we then have that the uncertainty estimate for the mass being weighed mW ,c reported in standard uncertainty is: ◮ 1st order: (14) u(mW ,c ) = 1.0000000740094588 × 10−6 kg ◮ 2nd order: N 2 ∂f u 2 (f ) = u 2 (xi ) ∂xi i=1 N N 2 1 ∂2f ∂f ∂ 3 f + + u 2 (xi )u 2 (xj ) 2 ∂xi ∂xj ∂xi ∂xi ∂xj2 i=1 j=1 = L+H (15) u(f ) = 1.0001135041200297 × 10−6 kg ◮ difference in uncertainty is underestimated by approx. 113 ppm Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  48. 48. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Uncertainty results using the GUM 2 Full uncertainty can be misleading if linearization not accurate if f is very non-linear – in the above it is not too significant since f is not too non-linear It should be noted that H is formed out of double sum over the number of variables N and that such a calculation can only realistically be performed in a computer algebra system due to the excessive number of partial derivatives that must be computed e.g. with N = 5 then 100 partial derivatives must be calculated. The computer code to perform this computation within a CAS e.g. Maxima is relatively straightforward to implement but it should be noted that the full expression can become algebraically large and unwieldy – the non-linear terms correctly evaluate to zero when the model is indeed linear. Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  49. 49. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Including non-linear terms using the GUM Example (A nonlinear functional) 2 2 Work out the uncertainty for f (x1 , x2 ) = exp[x1 + x2 ] assuming u(x1 ) = u(x2 ) = 0.1% at the point a = [x1 = 1, x2 = 1]T and compare the linear and nonlinear answers using the GUM. The linearization of f is 2e2 (x2 − 1) + 2e2 (x1 − 1) + e2 and now the nonlinear term H where u 2 (f ) = L + H is 2 2 2 2 2 2 2 x2 +x1 x 2 +x 2 2 x 2 +x 2 x 2 +x 2 H = u1 u2 8 x1 x2 e + 2 x1 e 2 1 8 x1 x2 e 2 1 + 4 x1 e 2 1 2   2 2 2 2  2 x +x1 + 2 ex2 +x1 4 x1 e 2  2  x 2 +x 2 2 2 3 x +x1 x 2 +x 2   +u1  2 x1 e 2 1 8 x1 e 2 + 12 x1 e 2 1 +     2   2 2 2 2 2 2 2 x2 +x1 x 2 +x 2 2 x 2 +x 2 x 2 +x 2 +u2 u1 8 x1 x2 e + 2 x2 e 2 1 8 x1 x2 e 2 1 + 4 x2 e 2 1 2   2 2 2 2  2 x +x1 + 2 ex2 +x1 4 x2 e 2  2  x 2 +x1 2 2 2 3 x +x1 x 2 +x 2   +u2  2 x2 e 2 8 x2 e 2 + 12 x2 e 2 1 +     2   Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  50. 50. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Non-linear calc - how significant are the GUM approx? 2 2 For f = ex1 +x2 the non-linear contribution H is non-zero as indicated above, and the difference between the linear and non-linear uncertainty estimates is ◮ u(flinear ) = 0.020899406696487 ◮ u(fnon−linear ) = 0.02089964181349 ◮ the difference in uncertainty is therefore underestimated by approx. 11 ppm Fact (GUM linear model underestimates uncertainty) It is seen that a linearized model may underestimate the actual uncertainty by 10 – 100 ppm Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  51. 51. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Comparison of PDF’s for Math Model’s Calc Unc When calculating uncertainties there are three PDF’s that one must consider when interpreting the measurand’s uncertainty: ◮ the actual PDF for the measurand as computed in terms of a convolution integral (Markov formula2 ) ∞ ∞ ∞ (16) g(η) = ··· g(ξ)δ(y − f (ξ))dξN dξN−1 · · · dξ1 −∞ −∞ −∞ ◮ a Gaussian3 like i.e. a t-distribution with νeff degrees-of-freedom via. the Welch-Satterwaithe formula for the calculation of the measurand y ’s PDF as per the GUM approach ◮ a discrete PDF in a Monte Carlo simulation that is built up with sampled data from the input PDF’s g1 (ξ1 ), . . . , gN (ξN ) that will converge to the measurand’s actual PDF (as calculated with a covolution integral) as the number of MC events M → ∞ 2 adequate mathematical statistics working knowledge is necessary to fully understand the conditions/derivation of the Markov formula wrt. GUM 3 A Gaussian PDF i.e. N(µ = 0; σ 2 ) is entirely defined in terms of the variance σ 2 whilst a t-distribution needs ν to define its shape Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  52. 52. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Comment on Application of Markov formula ◮ The GUM is based on the application of the Markov formula to linearized models and all of the results and formulae in the GUM can be derived (with certain assumptions) via. application of the Markov formula ◮ Practical examples: 1. Higher order terms are necessary in the GUM for non-linear models where the GUM will not work yielding incorrect results e.g. Y = X 2 where u(y) = 2x · u(x)∀x if just linear terms of the form u 2 (f ) = N [∂xi f · u(xi )]2 are used i=1 2. The Markov formula will yield the correct result with 4 u(y) = u(x) 4x 2 + 5 u 2 (x) which is true even for x = 0 ◮ In general a direct evaluation is only analytically possible for certain simple cases whilst symbolic evaluation is only feasible with a low order of variables requiring transformations and evaluation/calculation of Jacobians with a numerical approach preferred Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation
  53. 53. Introduction Mathematical Background & Concepts Developing a Mathematical Model Implementation of a MC Simulation Assigning Uncertainties and PDF’s to the Model Post-processing and Analysis of a Simulation Illustrative Mass Unc Example Discussion Comment on Application of Markov formula cont. ◮ The joint PDF gX1 ,X2 ,... (ξ1 , ξ2 , . . .) built up in terms of matrix multiplications requires the use of a Dirac delta function δ as defined in terms of a sum with derivative terms and in addition manipulation of the inputs ξi wrt. the output η ◮ Such calculations in the GUM require the application of further matrix algebra and will not be considered in this presentation ◮ The direct application of the Markov formula is in practice awkward and difficult to implement particularly in the case of non-linear models and the use of a Monte Carlo approach is entirely consistent with the Markov formula and is in fact a more practical calculation method that does not rely on any of the assumptions inherent as in the GUM Vishal Ramnath vramnath@nmisa.org NMISA-08-0121: Review of Methodology & Rationale of MC Simulation

×