About some issues of refraction in survey engineering.

1. 2018 Trimble Dimensions
November 5–7
The Venetian / Sands Expo Center – Las Vegas

2. How to solve the Refraction
Problem of Long Distance
Measurements ?
Don’t be afraid of the k refraction coefficient!
Thomas Touzé, Nicolas Chevallier & Nicolas Bolzon : HEIG-VD
Matthieu Hansen : SITES

3. 1. Introduction
2. State of the art on refraction modelling
3. Some experiments
4. Generalized model of refraction
5. Conclusion
Table of contents

4. 1. Introduction

5. Issues on the angular effects of refraction
• Well known phenomenon
in vertical in theory
• In practice, the order of
magnitude of k are not
well known
• Even worse for the lateral
effects of refraction
• No existing methodology
in networks adjustment
Objectives:
• Give a clue on orders of
magnitude of k
• Provide a methodology on
the field and in the
adjustment
• Propose a generalized
model
Introduction

6. 2. State of the art on
refraction modelling

7. 𝑛. sin 𝑖 = 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡
Snell’s law
Discrete case
sin 𝑖 𝛿𝑛 + 𝑛 cos 𝑖 𝛿𝑖 = 0
i.-e.
𝛿𝑖 = −
tan 𝑖
𝑛
𝛿𝑛
Continuous case
Optical path
Apparent path

8. 𝛿𝑛 = 𝑔𝑟𝑎𝑑 𝑛 𝛿𝑀 = 𝑔𝑟𝑎𝑑 𝑛 cos 𝑖 𝛿𝑠
Hence
𝛿𝑖 = −
𝑔𝑟𝑎𝑑 𝑛
𝑛
sin 𝑖 𝛿𝑠
−
𝑔𝑟𝑎𝑑 𝑛
𝑛
sin 𝑖 =
1
𝑟
: curvature of the
optical path (r: radius of curvature) [Torge]
Consequence of the (continuous) Snell’s law

9. Let us consider a sight from A to B
∆𝑖 = − න
𝐴
𝐵
𝑔𝑟𝑎𝑑 𝑛
𝑛
sin 𝑖 𝑑𝑠 = 𝜃 𝐴 + 𝜃 𝐵
𝜃 𝐴: Angular deflection at my station in A.
D : Slopped distance
Total angular deflection of the optical path

10. Hypothesis:
• Vertical gradient & sights close to horizontal: sin 𝑖 ≈ ±1
• Consequently, affects only on the zenithal angles ζ: ∆ζ = 𝜃 𝐴
• Symmetrical deflection: 𝜃 𝐴 = 𝜃 𝐵 =
∆𝑖
2
• Optical path = circle arch (radius r): 𝑟 =
𝑅
𝑘
=
𝐷
∆𝑖
Hence, in radian
∆ζ =
∆𝑖
2
=
𝑘𝐷
2𝑅
With k, the refraction coefficient and R the Earth radius.
∆𝑖 = − න
𝐴
𝐵
𝑔𝑟𝑎𝑑 𝑛
𝑛
sin 𝑖 𝑑𝑠 = 𝜃 𝐴 + 𝜃 𝐵
Usual refraction modelling [Torge]

11. • Value deduced from the standard variation of
the index of refraction n with the altitude.
• Good approximation for observations high
enough above the ground (> 40 m) [Torge]
• According to [Hübner] and confirmed by [Hirt
et al.], it can be very different close to the
ground, or at sunrise & sunset.
𝑘 = 0.13… In which conditions ?
Usual value (0.13) of the k refraction coefficient
[Hübner]

12. 3. Some experiments

13. • Same deflection on AB and BA
sights
• 𝛼 ≈
𝑑
𝑅
angle to the Earth center
(d: horizontal distance)
• 𝜋 = 2∆ζ + ζ 𝐴 + ζ 𝐵 − 𝛼
3.1. Reciprocal simultaneous observations
3.1.1. Principle
∆ζ =
𝑘𝐷
2𝑅
=
𝜋
2
+
𝑑
2𝑅
−
ζ 𝐴 + ζ 𝐵
2
• The angular correction can be computed and k can be deduced
• Issue: both optical centers are looking to each other

14. Use of offset prisms, to avoid
reciprocal sights between optical
centers
• Distances AA.1 & BB.1 ≪ AB
• Simultaneous sights AB.1 & BA.1
• Computation of all the height
differences h
3.1. Reciprocal simultaneous observations
3.1.2. Simultaneous and almost reciprocal observations
തℎ 𝐴𝐵 =
ℎ 𝐴𝐵.1−ℎ 𝐵𝐵.1 − ℎ 𝐵𝐴.1−ℎ 𝐴𝐴.1
2
corrected from refraction effects
[Touzé]. Then, the value of k can be deduced.

15. • Offset prisms a few meters from the
total stations
• 600 m (0.4 mi) long base
• 37 m (40 yd) height differences
• Measured 3 times during 3 days by
students
• One session duration: ~ 45 min
• Length of the shortest avoided
leveling path: 1.4 km (0.9 mi)
• Cloudy and rainy weather
3.1. Reciprocal simultaneous observations
3.1.3. Measurements

16. • Excellent ratio precision/duration compare to leveling
• The refraction coefficient k is close to zero, but ≠ 0.13
• Method known but rarely used by surveyors, yet it is accurate
and precise
3.1. Reciprocal simultaneous observations
3.1.4. Results
Session h [m] σh [mm] k [/] σk [/]
1 37.5657 0.6 -0.15 0.02
2 37.5666 0.6 -0.20 0.01
3 37.5675 0.5 -0.06 0.02
Mean Value 37.5667 0.5 (0.02 in)

17. “SITES” is a French
land-surveying society
involved in geomoni-
toring.
3.2. Adjustment of a monitoring network
3.2.1. Network description
In the south of France,
“SITES” is in charge of the
monitoring of 800m of a
highspeed railway line, in
an embankment area.
Data provided by
Matthieu Hansen

18. • Measurements done in august 2017, during two very warm &
sunny days
• Free adjustment (1 fixed point in 2D+1 & 1 fixed azimuth) with
LTOP (2D+1 swiss adjustment software)
• Angular precision: 0.5’’
• Distances precision: 0.6 mm (0.02 in) + 1 ppm
• Centering precision: 0.4 mm (horizontal) and 0.3 mm (vertical)
• Refraction parameters: k = 0.13 with a 0.06 precision
3.2. Adjustment of a monitoring network
3.2.2. Functional & stochastic models

19. • 507 observations, 294 degrees of freedom
• Probability for the 1st kind tests: 99 %
• According to Student’s law, we should have 5 acceptable
outliers between 2.6 and 3.1
3.2. Adjustment of a monitoring network
3.2.3. (Excellent) Horizontal results
• 0 disactivated observations, 10 reweighted observations (2 %)
by a centering precision < 3 mm (0.1 in)
• Final variance factor: 0.69, slightly smaller than its a priori
confidence interval [0.79,1.23] (result 0.8 time more precise
than expected)

20. • 253 observations, 159 degrees of freedom
• Probability for the 1st kind tests: 99 %
• According to Student’s law, we should have 3 acceptable
outliers between 2.6 and 2.9
3.2. Adjustment of a monitoring network
3.2.4. Some trouble in vertical
• First adjustment: 95 outliers (38 %)!
• Huber’s robust adjustment: 69 outliers (27 %)!
69 gross errors? No, we probably have a functional or stochastic
model problem.

21. Close to the ground on warm days:
• Flicker effect due to strong variation
of n in time
• Consequence: noise on zenithal
angles on distant objects
This must be taken into account in the
stochastic model
3.2. Adjustment of a monitoring network
3.2.5. Stochastic model of the zenithal angles

22. With LTOP software, it
can be done.
➢ Zenithal angle
reweighted with a
precision of k at 1.0.
➢ Still 36 outliers (14 %)
3.2. Adjustment of a monitoring network
3.2.5. Stochastic model of the zenithal angles
Now let’s see if the mean value of k (0.13) is the most
appropriate.

23. Adjustment of the
network by changing the
mean value of k
➢ Looking for the value
that minimizes the
number of outliers and
Q = √(variance factor)
3.2. Adjustment of a monitoring network
3.2.6. Looking for optimal mean value of k
Data k std(k)
Min Q -2.114 0.014
Min W -2.165 0.036
Mean -2.121 0.018Refraction coef: k = -2.12 !!!

24. Finally, we get in vertical, by applying k = -2.12
• 0 disactivated observations, 10 reweighted observations (4 %)
by a vertical centering precision < 3 mm (0.1 in)
• Final variance factor: 1.28, inside its a priori confidence interval
[0.74,1.32]
• A posteriori precision of the 90 points of the network:
3.2. Adjustment of a monitoring network
3.2.7. Final results
Precision [mm] Horizontal Vertical
Min 0.5 0.4
Median 2.0 0.8
Max 2.4 1.2

25. Measurement of a triangle with simultaneous
reciprocal observations during a spring sunny
day.
• Base GD parallel with a sunny wall
• What’s happening when base GD is getting
closer to the wall ?
• Tested at 0.05, 0.075, 0.1, 0.2 & 9.5 m to
the wall
Setup description [Chevallier]
3.3. Refraction lateral effects
Experiment proposed in [Wilhelm]

26. • Coefficient k seems
independent with the
distance to the wall
• Effects on the height
differences of ± 10 mm
(0.4 in)
• ෠𝑘 = −2.33 ± 0.92 1𝜎
Results in vertical
3.3. Refraction lateral effects

27. • Measured angles
compared the ones
deduced from the
distances (cos law)
• Effects correlated
with the distance to
the wall
• More important on
the closest station G
Results in Horizontal
3.3. Refraction lateral effects
• NB: at 180 m, 70” = 60 mm (2.4 in)!!

28. • An object, along the sight, disturbs the
temperature gradient
• Angular deflection related to the
distance s to that object
• Let’s apply this asymmetrical model
[Wilhelm] model
3.3. Refraction lateral effects
Dist the
wall [m]
errG/errTot (D – s)/D
0.050 0.86 0.85
0.050 0.82 0.85
0.075 0.90 0.85
0.100 0.86 0.85

29. • No more significant
effects after 200 mm
(8 in) from the wall
• Asymmetrical model
seems efficient
• Refraction reaches -14
at 50 mm from the
wall.
Values of the horizontal refraction coefficient
3.3. Refraction lateral effects
Possible to correct horizontal reciprocal observations !

30. 4. Generalized model of
refraction

31. 𝛿𝑖 = −
𝑔𝑟𝑎𝑑 𝑛
𝑛
sin 𝑖 𝛿𝑠 = −
𝑠𝑖𝑔𝑛 sin 𝑖
𝑛
𝑔𝑟𝑎𝑑 𝑛 × 𝛿𝑀
➢ Rotation of angle 𝛿𝑖 around 𝑔𝑟𝑎𝑑 𝑛 × 𝛿𝑀
➢ Snell’s law: 𝑛 sin 𝑖 = 𝑛 𝐴 sin 𝑖 𝐴
∆𝑖 = −𝑛 𝐴 sin 𝑖 𝐴 න
𝐴
𝐵
𝑔𝑟𝑎𝑑 𝑛
𝑛2
𝑑𝑠 = − sin 𝑖 𝐴
𝑘𝐷
𝑅
➢ k: related to the gradient along my path
➢ sin 𝑖 𝐴: Interaction of my path with the gradient at the station
Back to the standard model of refraction

32. Refraction as a 3D rotation matrix
𝑅 𝐾: 3D rotation matrix:
• Of angle ∆𝑖 1 − κ with κ, the asymmetrical coefficient (default
value = ½, related to the distance to a disturbing object as
Wilhelm model)
• Around the vector ԦΓ × 𝐴′ 𝐵′ where ԦΓ: unit vector of the gradient
direction, 𝐴′ 𝐵′: vector from the station optical center to the
center of the prism
Parameters & constants:
• k: the refraction coefficient
• αΓ, ζΓ: Azimut and zenithal angle of ԦΓ
• κ: Asymmetrical coefficient, assumed as known

33. 𝐴′ 𝐵′ = 𝐴𝐵 + 𝐵𝐵′ − 𝐴𝐴′ with X’: optical center above X point
Hence, where LAF = local astronomical frame, I & S = instrument
and prism heights, 𝑅Ω: orientation of the station and ഥ𝐷, ҧ𝑟, ҧζ: the
true values of the station’s observations
𝐴′ 𝐵′
𝐿𝐴𝐹 𝐴
= 𝑅 𝐸𝐶𝐸𝐹
𝐿𝐴𝐹 𝐴
𝐴𝐵 𝐸𝐶𝐸𝐹
+ 𝑅 𝐿𝐴𝐹 𝐵
𝐸𝐶𝐸𝐹
0
0
𝑆
−
0
0
𝐼
= ഥ𝐷 𝑅 𝐾 𝑅Ω
sin ҧ𝑟 sin ҧζ
cos ҧ𝑟 sin ҧζ
cos ҧζ
From this equation, one can (courageously) compute the
modeling and the derivative functions of the observations
Total station 3D functional model

34. Simultaneous reciprocal
observations:
• Fixed vertical gradient
• No asymmetry
Adjustment of a monitoring
network
• Fixed vertical gradient
• No asymmetry
k = -2.28 ± 0.12 instead of
-2.12
Application on the previous experiments
Param. Initial 3D model
h [m] 37.5667 m ± 0.5 mm 37.5680 m ± 0.5 mm
k1 -0.15 ± 0.02 -0.16 ± 0.02
k2 -0.20 ± 0.01 -0.19 ± 0.02
k3 -0.06 ± 0.02 -0.05 ± 0.02

35. Lateral refraction
• Azimut of the gradient fixed normal to the wall
Application on the previous experiments
Distance to
the wall
k Std(k)
Gradient
zenithal
angle [deg]
Std(zen.
Ang) [deg]
0.200 0.9 0.4 0 x
0.100 5.7 0.5 74.9 4.1
0.075 8.5 0.5 78.0 2.7
0.050 12.5 0.5 95.3 1.8
0.05 14.2 0.5 85.7 1.6

36. 5. Conclusion

37. About my initial objectives
• Refraction coefficient k can be very different from 0.13. Above
the ground on a warm sunny day, -2 seems more realistic
• Reciprocal simultaneous observations can also be used to
estimate the value of k during a network measurement
• Lateral and asymmetrical effects of refraction can be solved by
reciprocal observations, thanks to Wilhelm model
• A 3D generalized model has been successfully implemented
Conclusion

38. 3. [Hirt et al.]: Hirt C., Guillaume S.,
Wisbar A., Bürki B. and Sternberg H.,
Monitoring of the refraction
coefficient in the lower atmosphere
using a controlled setup of
simultaneous reciprocal vertical angle
measurements, Journal of geophysical
research, 2010.
2. [Hübner]: Hübner E., Einfluss des
terrestrischen Refraktion aud den
Laserstrahl in bodennahen Luftschichten,
Vermessungstechnik, 1977.
1. [Torge]: Torge W., Geodesy, third
completely revised and extended
version, De Gruyter, 2001.
5. [Bolzon & Chevallier]: Bolzon N. and
Chevallier N., Quantification de la
refraction latérale, HEIG-VD, Bachelor
semester project, 2017.
4. [Wilhelm]: Wilhelm W., Die
Seitenrefraktion : Ein unbeliebtes
Thema? Oder ein Thema nur für
Insider? Geomatic, 1994.
References
5. [Chevallier]: Chevallier N., Etude en
3D de la refraction sur des visées
tachéométriques, HEIG-VD, Bachelor
thesis, 2017.
5. [Touzé]: Touzé T., Nivellement
trigonométrique, HEIG-VD, Bachelor
course, 2017.

39. Thank you!
Any question?