This document discusses GPS synchronization issues experienced by a customer and solutions implemented to resolve them. It describes the customer configuration, issues with GPS-Out synchronization, limitations of that approach, and improvements from implementing Node Group Synchronization and upgrading software. The upgrades improved the synchronization reference supervision and robustness, reducing outage times on both the Distributed Unit W and S sides.

1. GPS-OUT LIMITATIONS AND NGS
IMPROVEMENTS
Understanding the basics of synchronization
– Narendra Kumar Sahu, Tier-2

2. Agenda
• Case study of CSR 3658463
• Customer configuration
• Issue with GPS-Out
• Solution for customer issue.
• Limitations with GPS-Out Solution
• New improvements in SYNC area

3. Customer Configuration:
RNC
(NTP
Source)
DUW DUS
RRU-1
RRU-2
RRU-3
NTP Sync
GPS-Out
GPS-Out solution is not applicable for BB nodes.
GOG GOS

4. With the fluctuation in transmission DUS (L21)
sites goes down. DUW (U21) sites remain un-
impacted.
Customer set timeAndPhaseSynchCritical to false
on L21 sites. However, it doesn’t make any
difference.
For DUW (U21) sync source is NTP which have
precision in milliseconds.
DUW (U21) sites does sync adjustments to
compensate frequency drifting from external
reference source and sync remained in Locked
mode on DUW (U21). This is probably due to
freqDeviationThreshold is set to 5000 (max).
DUS (L21) node receiving GPS signal over the
GPS-Out. DUS cannot follow rate of the
frequency drifting from DUW node which is
providing GPS signal with precision of 1 PPS.
So DUS node enters FREE RUNNING MODE.
Problem Description & Observation

5. LOCKED
MODE
HOLDOVER
MODE
FREE
RUNNING
MODE
External reference lost Holdover timer expired
External reference regained
Radio Clock State Machine
FREQUENCY_LOCKED_MODE
TIME_LOCKED_MODE
TIME_OFFSET_LOCKED_MODE
FREQ_HOLDOVER_MODE
TIME_HOLDOVER_MODE
FREERUNNING_MODE

6. Investigation at DUW Side
• Below events seen in the DUW node.
<<nssinfo all>>
[2020-02-03 07:21:37] NORMAL -> MONITORING due to REFERENCE_UNSTABLE
[2020-02-03 07:26:42] MONITORING -> ADJUSTING due to SLOPE_OK event
[2020-02-03 07:45:40] ADJUSTING -> NORMAL due to FREQ_LOCKING_EVENT
[2020-02-03 08:39:09] NORMAL -> MONITORING due to REFERENCE_UNSTABLE
[2020-02-03 08:44:14] MONITORING -> ADJUSTING due to SLOPE_OK event
[2020-02-03 08:44:20] ADJUSTING -> NORMAL due to FREQ_LOCKING_EVENT
• Fluctuation in transport lead to deviation in NTP sync reference at DUW which further impact to DUS.
• DUS is not able to follow the frequency drifts due to GPS out solution.
• Whereas, when there is no deviation in NTP sync reference then DUS is not impacted.
• We can notice unstable reference events on the IpSyncRef at the observed time period on the DUW node.
• Due to instability system perform ADJUSTING frequency to compensate frequency drifting from external
reference source.

7. DUW (U21)
DUS (L21)

8. Proposal to mitigate the outages
• Node Group Synchronization (NGS) was recommended
instead of GPS-Out for improved flexibility and
stability.
RNC
(NTP
Source)
DUW (SP) DUS (SR)
RRU-1
RRU-2
RRU-3
NTP Sync
NGS

9. Issue seen after NGS implementation
• After implementing NGS there was no
issue see on the DUS (L21) nodes. As
it can keep the system clock in
HOLDOVER MODE. Even during the fiber
cut or fluctuations the L21 cells
remain up. Although there is cell
outage for few seconds due to BFN
JUMP. However, they are very short
as compared to previous outages.
• However, now the issue shifted to
DUW (U21) side. As during the fiber
cut the U21 cells were going down.
This was due to DUW entering FREE
RUNNING MODE.
Now the customer wants to know how to mitigate this problem completely, so
that there is no outage on the DUW side and avoid BFN jump on the DUS side.

10. Further analysis
• Collected traces for 24 hours to capture the scenario when the DUW
transitions from FREQUENCY LOCKED to FREE RUNNING MODE.
• From graph of “Time residual variance”
extracted from traces collected on SP
DUW node it can be observed that Sync
reference on SP node is unstable.
• Time residual variance represents the
ratio between measurements predicted
by algorithm and the real measurements
obtained from external Sync references.
• The curve on that graph should be really
close to horizontal line in case of stable
external Sync reference, but as it could
be seen that is not the case in graph
shown below.

11. • Another graph extracted from the traces showing the deviation in ppb.
• Deviation has been seen in
the transport network around
17ppb which is higher than
the limit, is the main root
cause.
• The difference between the
frequency of the output of
the OCXO and the incoming
reference is continuously
monitored by SW.
• Every 2.1s the output is
adjusted, to match the
reference.
• At 2020-04-01, around 2020-
04-01 18:15 the incoming
frequency of the NTP had a
sudden disturbance.
• There is a limit for how fast
the SW can regulate the
output. In this case the
limit was exceeded.

12. Suggestion: upgrade the nodes to SW version C19.Q2 EP12 or C20.Q1 on which
MR5172 (SP8) is delivered.

13. Improvements introduced in MR5172
MR5172-SP1:
• Feature should supervise frequency deviation defects
(frequency jumps, drifts, frequency inaccuracy) on
Baseband nodes for active reference.
• Synchronization reference supervision will declare
defect and deselect active reference when calculated
frequency deviation is above freqDeviationThreshold MO
attribute value. Deselected reference will be
supervised for recovery (if defect is ceased, reference
will be considered for re-activation by sync SW).
• SoCPRI algorithm on Baseband will be more robust to
frequency change of SP.
• User has ability to decide threshold for frequency
defects on active external reference.
• Introduced in 18.Q2.

14. MR5172…
MR5172-SP2:
• Supervision of active references for frequency
jump and fast frequency drift artifacts for G1 and
new alarm introduction for G1 and G2.
• The purpose of this feature is to improve
supervision of active external synchronization
reference in DU Radio Node. Supervision should
detect defects with frequency jumps or drifts.
• New alarm "Sync Reference Deviation" is introduced
for these kinds of defects for both DU Radio Node
and Baseband Radio Node.
• Introduced in 18.Q4.

15. MR5172…
MR5172-SP8:
• Improved Robustness of the Supervision of active
frequency references (the SP2 feature) for DUW/DUS. SP8
can be considered as add-on to SP2 functionality.
• This system improvement makes synchronization reference
supervision more robust when working with lower quality
synchronization references and more tolerant to
unstable sources.
• Instead of immediate traffic degradation,
synchronization holdover capability is retained for a
longer time, increasing the chance that the node re-
aligns with the sync source. It is expected this
results in less impact on network traffic ability.
• DUW: RBS W19.Q2.9 (C19.Q2- EP12-1) and C19.Q2 EP13.

16. Traces used in this case:
SP (DUW):
========
te default *
te e trace5 trace6 trace7 trace9 NSS_CBM_TUM2_TUREG
te save *
mon mp
SR (DUS):
========
te default *
te e trace1 NSS_CBM_TUM2_PLL
te save *
mon mp

17. Counters
PM Counter Name Attribute Value
pmfreeRunningSec FREE_RUNNING
FAILED
pmfreqHoldOverSec FREQUENCY_HOLDOVER
pmtimeFreeRunningSec FREQUENCY_LOCKED
FREQUENCY_HOLDOVER
pmtimeHoldoverSec RNT_TIME_HOLDOVER
TIME_OFFSET_HOLDOVER
pmtimeAssistedHoldoverSec RNT_TIME_LOCKED
TIME_OFFSET_LOCKED
when RadioEquipmentClock.timeSyncAssistanceSt
ate is ASSISTANCE_ACTIVE