Presentation and discussion of the results of the GLOSOLAN PT 2019 - Christian Hartmann, IRD France

1. GLOSOLAN
1st global Proficiency Testing
(PT) 2019
Christian Hartmann, Nopmanee Suvannang, Lucrezia Caon
(special thanks to Charles Gowing & Floria Bertsch)
christian.hartmann@ird.fr

2. Question:
What is a proficiency testing (PT)
?

3. proficiency testing:
1. the same soil sample is provided to different labs,

4. Proficiency testing :
1. the same soil sample is provided to different labs,
2. results are statistically analysed to determine
proficiency (i.e. skills, capability, competence, etc).

5. Q: how can you be sure it is the SAME sample?

6. Answer: a careful preparation (time consuming)

7. sample preparation
50 to 100 kg
of homogeneous soil
Splitter or
divider

8. sample preparation
50 to 100 kg
of homogeneous soil
Splitter or
divider

9. sample preparation

10. sample preparation

11. sample preparation

12. sample preparation
50 to 100 kg
of homogeneous soil
30 g/bag

13. The soil samples:
certified materialown samples
4 soil types
SEALNET 2018

14. The soil samples:
certified materialown samples
14 soil bags / lab.
4 soil types
SEALNET 2018

15. The soil samples:
SEALNET 2018 certified materialown samples
14 soil bags / lab.
4 soil types
Sent to 16 labs in Asia

16. The soil samples:
own samples
3 soil bags / lab.
3 soil types
GLOSOLAN 2019

17. The soil samples:
own samples
3 soil bags / lab.
3 soil types
3 replicates / soil
Sub-
samples
GLOSOLAN 2019

18. The soil samples:
GLOSOLAN 2019 own samples
3 soil bags / lab.
3 soil types
3 replicates / soil
Sent to 100 labs in the world
Sample preparation is
a lot of work…

19. We sent the same soil sample to different labs…
GLOSOLAN 2019SEALNET 2018

20. We sent the same soil sample to different labs…
GLOSOLAN 2019SEALNET 2018
Sample posting is also
a lot of work…

21. We sent the same soil sample to different labs…
GLOSOLAN 2019SEALNET 2018

22. We sent the same soil sample to different labs…
GLOSOLAN 2019SEALNET 2018
Results Results

23. We sent the same soil sample to different labs…
GLOSOLAN 2019SEALNET 2018
Results Results
Statistical analysis…

24. We sent the same soil sample to different labs…
GLOSOLAN 2019SEALNET 2018
Results Results
Statistical analysis…
to evaluate performances…

25. 25
Q? How to evaluate performances?

26. 26
Example of the arrows and target:

27. 27
Your objective:
hit the middle
of the target
Example of the arrows and target:

28. 28
LOW performance:
Example of the arrows and target:

29. 29
LOW performance: not in the middle
high dispersion
Example of the arrows and target:

30. 30
Q? How to evaluate performances?

31. 31
Z score
Q? How to evaluate performances?
measure your performances using the

32. 32
Z score
Q? How to evaluate performances?
= how far from the
middle of the target

33. 33
Z score
Z <1
1< Z <2 satisfactory
Q? How to evaluate performances?

34. 34
2 <Z <3 questionable
Z <1
1< Z <2 satisfactory
Q? How to evaluate performances?

35. 35
2 <Z <3
Z >3 UN-satisfactory
questionable
Z <1
1< Z <2 satisfactory
Q? How to evaluate performances?

36. 36
What is good performance?

37. 37
What is good performance?
2 <Z <3

38. 38
What is good performance?
2 <Z <3
high dispersion

39. 39
What is good performance?
LOW performance

40. 40
What is good performance?
LOW performance

41. 41
What is good performance?
Z <1 , limited dispersion
LOW performance

42. 42
What is good performance?
HIGH performance

43. 43
What is good performance?
HIGH performance

44. 44
What is good performance?
It results from daily work
& regular training.
HIGH performance

45. Different ways to present the
analytical results
and performances

46. 1. Quality charts
Different ways to present the
analytical results
and performances

47. 1. Quality charts
pH

48. mean
+/- sd

49. mean
+/- sd
Lab number

50. mean
+/- sd

51. mean
+/- sd
mean + 2sd
mean + 3sd

52. mean
+/- sd
without outliers

53. mean
+/- sd
outliers
OUT OF THE
TARGET

54. Results

55. 3 replicates,
high precision
Results

56. 3 replicates,
high precision
3 replicates,
LOW precision
Results

57. 3 replicates,
high precision
3 replicates,
LOW precision
how is this possible?

58. 3 replicates,
high precision
3 replicates,
LOW precision
how is this possible?

59. 3 replicates,
high precision
3 replicates,
LOW precision
how is this possible?
No quality control….

60. Results = ?

61. mean
+/- sd

62. mean
+/- sd

63. mean
+/- sd
[Dispersion:
1 unit pH

64. mean
+/- sd
Many
outliers

65. mean
+/- sd
Acidic but close to 7
WRONG DATA…
WRONG CONCLUSION,
WRONG RECOMMENDATIONS...

66. mean
+/- sd
Acidic but close to 7
WRONG DATA…
WRONG CONCLUSION,
WRONG RECOMMENDATIONS...

67. mean
+/- sd
Acidic but close to 7
WRONG DATA…
WRONG CONCLUSION,
WRONG RECOMMENDATIONS...

68. mean
+/- sd
Acidic but close to 7
WRONG DATA…
WRONG CONCLUSION,
WRONG RECOMMENDATIONS...

69. mean
+/- sd
Acidic but close to 7
WRONG DATA…
WRONG CONCLUSION,
WRONG RECOMMENDATIONS...

70. mean
+/- sd
Acidic but close to 7
WRONG DATA…
WRONG CONCLUSION,
WRONG RECOMMENDATIONS...
No quality control….

71. mean
+/- sd
Acidic but close to 7
WRONG DATA…
WRONG CONCLUSION,
WRONG RECOMMENDATIONS...
No quality control….

72. 1. Quality control charts
Organic Carbon
Walkley & Black

73. 1. Quality control charts
Organic Carbon
Walkley & Black
need skills in chemistry:

74. Lab number

76. Many ouliers…
Dispersion:

77. 1. Quality control charts
Organic Carbon
COMBUSTION

78. 1. Quality control charts
Organic Carbon
COMBUSTION
No need to have skills in chemistry:
(can technology help us
to get higher performances?)

80. Similar results
Lower ‘sd’

82. Many ouliers…

83. High(expensive) technology,
without Quality Control
is waste of money….

84. Different ways
to present the results
1. Quality charts
2. Z score
(statistical comparison of performances)

85. The same results presented differently:

86. The same results presented differently:

87. Problem of pHmeter calibration ?
acidic acidic neutral

88. The same results presented differently:

89. Different ways
to present the results
1. Quality charts
2. Z score (to compare performances)
3. Individual Z score (each lab performance)

91. as a conclusion….

92. PT must be organised carefully (homogeneity)
as a conclusion….

93. PT must be organised carefully (homogeneity)
Quality Chart & Z scores:
provide complementary information
as a conclusion….

94. PT must be organised carefully (homogeneity)
Quality Chart & Z scores:
provide complementary information
They help to detect mistakes…
as a conclusion….

95. PT must be organised carefully (homogeneity)
Quality Chart & Z scores:
provide complementary information
They help to detect low performances…
Low performances:
happen even with expensive instruments
==> skilled staff is the most valuable
as a conclusion….

96. In case of low performances:
you need to take corrective actions !!!

97. Thanks for your attention