Short introduction to the use of non-parametric statistical analysis for quality assessment, general robust analysis of data. Includes a short introduction to the normal distribution and requirements for parametric analysis, review of binomial and poisson distributions, examples of non-parametric data, and example of using Kolomogrov-Smirnov. Final sections include references.

1. Para-normal Statistics:
Analyzing what doesn't add up.
Steven Lembark
Workhorse Computing
lembark@wrkhors.com

2. Normality
We expect data is normal.
It's what we are trained for.
Chi-Squared, F depend on it.
It's the guts of ANOVA.
Theory guarantees it, sort of.

3. What is "normal"?
Normal data is:
Parametric
Real
Symmetric
Unimodal

4. Ab-normal data
Not all data is parametric.

5. Ab-normal data
Not all data is parametric:
"Bold" + "Tide" / 2 ==

6. Ab-normal data
Not all data is parametric: Nominal Data
"Bold" + "Tide" / 2 == ??
"Bald" - "Harry" >= 0 ??

7. Ab-normal data
Not all data is parametric: Ordinal Data
"On a scale of 1 to 5 how would you rate..."
Is the average really 3?
Are differences between ranks unform?

8. Ab-normal data
Not all data is parametric: Ordinal Data
"On a scale of 1 to 5 how would you rate..."
Is the average really 3?
For different people?

9. Ab-normal data
Not all data is unimodal, symmetric.
Bi-modal data has higher sample variance.
Positive data is skewed.

10. Ab-normal data
Counts usually Binomial or Poission.
Binomial: Coin flips.
Poisson: Sample success/failure.

11. Power of Positive Thinking
Binomial: Count of
success from IID
experiments.
Mean = np
Variance = npq

12. Power of Positive Thinking
Poisson: Count of
occurrances in sample
size n.
Mean = np
Variance = np

13. Power of Positive Thinking
Curves all positive.
Right tailed.
Binomial has highest power if sample data is binomial.
Result: Smaller n for given Beta.

14. Kinda normal
Approximations work...

15. Kinda normal
Approximations work some of the time.
Rule: npq > 5 for binomial approximation.
Goal: Keep mean > 3σ so normal is all positive.
Q: How good an approximation?
A: It depends...

16. The middle way
Binomial:
n=20, p=0.5
Normal:
µ 10, σ = 2.23
Decent approximation.

17. Off to one side
Binomial:
n=20, p=0.3
Normal:
µ = 6, σ = 2.0
Drifting negative.

18. Life on the edge
Binomial:
n=20, p=0.1
Normal:
µ = 2, σ = 1.3
Significant negative.

19. Neverneverland
Binomial:
n=20, p=0.0013
Normal:
µ = 0.26
σ = 0.16
Heavily negative.

20. General rule: npq > 5
Small or large p is skewed.
Six-sigma range should be positive.
At that point n > 5 / pq.
For p = 0.0013, n = 3582.
Sample size around 4000?

21. When we assume we make...
Assuming normal data leaves a less robust conclusion.
Stronger, less robust:
Sensitive to individual datasets.
Not reproducable.

22. Non-parametric Statistics
Origins in Psychology, Biology, Marketing.
Analyze counts, ranks.
Tests based on discrete distributions.

23. Common in Quality
Frequency of failures.
QC with No-Go guages.
Variations between batch runs.
Customer feedback.

24. Example: Safety study
Q: Are departments equally "safe"?
Q: Is a new configuration any "safer"?
Compare sample populations.

25. What is "safe"?
Fewer reported injurys?
What is P( injury ) per operation?

26. What is "safe"?
Fewer reported injurys?
What is P( injury ) per operation?
0.5?
0.3?

27. What is "safe"?
Fewer reported injurys?
What is P( injury ) per operation?
0.5?
0.1?
A whole lot less?

28. What is "safe"?
Fewer reported injurys?
What is P( injury ) per operation?
0.5?
0.1?
A whole lot less?
N(0.01, 0.01) is heavily negative.

29. Severe?
Parametric measure of injurys?

30. Severe?
Parametric ranking of injurys?
( Finger + Thumb ) / 2 == ?

31. Severe?
Parametric ranking of injurys?
( Finger + Thumb ) / 2 == ?
( Hand + Eye ) == Arm ?
( Hand + Hand ) == 2 * Hand ?

32. Ordinal Data
Ranked data, not scaled.

33. Ordinal Data
Ranked data, not scaled.
Hangnail < Finger Tip < Finger < Hand < Arm
"Fuzzy Buckets"
Have p( accident ) from history.

34. Kolomogrov-Smirnov
Got tonic?

35. Kolomogrov-Smirnov
Nope, not Vodka.
Like F or ANOVA: Populations are "different".

36. K-S Test
Compare cumulative
data (blue) vs. Expeted
(red).
Measure is largest
difference (arrow).

37. K-S for safety
Rank the injurys on relative scale.
Compare counts by bucket.
Cumulative distribution:
accomodates empty cells.
minor mis-catagorization.

38. A good datum is hard to find,
You always get the other kind.
Apologies to Bessie Smith
Sliding-scale questions:
"How would you rate..."
"How well did..."
"How likely are you to..."

39. A good datum is hard to find,
You always get the other kind.
Apologies to Bessie Smith
Reproducability:
Variable skill.
Variable methods.
Variable data handling.

40. A good datum is hard to find,
You always get the other kind.
Apologies to Bessie Smith
Big Data:
Multiple sources.
Multiple populations.
Multiple data standards.

41. Repeatable Analysis
Variety of NP tests for "messy" data.
Handle protocol, sampling variations.
Robust conclusions with real data.

42. Summary
Non-parametric data: counts, nominal, ordinal data.
Non-parametric analysis avoids NID assumptions.
Robust analysis of real data.
Even the para-normal.

43. Questions?

44. References: N-P
http://www.uta.edu/faculty/sawasthi/Statistics/stnonpar.html
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC153434/
Nice writeups.

45. References: K-S
http://itl.nist.gov/div898/handbook/eda/section3/eda35g.htm
Exploratory data analysis is worth exploring.
https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test
As always... really good writeup of the test definition, math.

46. References: Robust Analysis
https://en.wikipedia.org/wiki/Robust_statistics
https://en.wikipedia.org/wiki/Robust_regression
Decent introductions.
Also look up "robust statistics" at nist.gov or "robust statistical
analysis" at duckduckgo.

47. References: This talk
http://slideshare.net/lembark
Along with everything else I've done...