This was presented in ISO IEC/JTC 1/SC 24/WG 9 working session on July 20, 2021.

1. Benchmarking of indoor localization
and tracking systems (LTSs)
Takeshi Kurata, Ryosuke Ichikari
Deputy Director
Human Augmentation Research Center
AIST, Japan
ISO IEC/JTC 1/SC 24/WG 9 (2021/7/20)

2. Shopping behavior analysis with indoor LTSs
To compare the shopping behavior in detail, we made heat-map visualization of the stay
time for each 50 cm grid in the real and virtual store. The read area indicates subjects spent
longer time than other area. Because position data of the real store situation is recorded by
hand, we only have the discrete position and timestamp data. Therefore, we could not
compare both of them strictly, but we found out we could get the similar results.
Comparison of heat-map visualization of stay time
Okuma Takashi, Ichikari Ryosuke, Tagai Keiko, Shimakura Hitomi, Isobe Hiroko, Kurata Takeshi, Customer Behavior Analysis Using Service
Field Simulator, Journal of Serviceology, Vol.3, No.2, pp.18-24, 2019.
Real store Virtual store in SFS (Service Field Simulator)
2

3. Work analysis in building maintenance service
with indoor LTSs
• Floor-level detection with 133 man-day
data
• Correspondence rate b/w a BLE-based
method and 10-axis PDR: 96.5%
Bldg. A
Bldg. B Bldg. C
Ryosuke Ichikari, Haruka Nishida, Ching-Tzun Chang, Takashi Okuma, Takeshi Kurata, Katsuko Nakahira, Muneo Kitajima,
Akio Hakurai, and Takuya Misugi: A case study of building maintenance service based on stakeholders’ perspectives in the
service triangle, Proc Joint Conf. of ICSSI2018 & ICServ2018 (2018)
3

4. Target stakeholders of LTS benchmarking
activities
• A benchmarking service provider, a benchmark provider or a
benchmarking competition organizer who wishes to align their
benchmarking activities including self-benchmarking and
open/closed competitions to be consistent with some standards;
• A technology developer/supplier who wishes to estimate and
evaluate the performance of indoor LTSs appropriately with a
benchmarking service provider, a benchmark provider or a
benchmarking competition organizer who aligns their
benchmarking activities to be consistent with some standards; or
• A technology user who wishes to obtain benchmarking results
based on a benchmarking activity, which is consistent with some
standards, or to compare the existing indoor LTS methods in
terms of their performance.
4

5. National Institute of Advanced Industrial Science and Technology
PDR Benchmark Standardization Committee
Support Organizations (45, as of July, 2021):
• Asahi Kasei Corporation, Asia Air Survey Co., Ltd. (Y. Minami), INTEC Inc.,
NEC Networks & System Integration Corporation, MTI Ltd., KDDI R&D
Laboratories, Inc., KOKUSAI KOGYO CO., LTD., SHIBUYA KOGYO CO.,
LTD., Koozyt, Inc., GOV Co., Ltd., SITESENSING, inc., Sharp Corporation,
Sugihara Software and Electron Industry Co., Ltd. (SSEI), ZENRIN
DataCom CO., LTD., Information Services International-Dentsu, Ltd.
(ISID), TOYO KANETSU SOLUTIONS K.K., IBM Japan, Ltd., Hitachi, Ltd.,
Frameworx, Inc. (S. Watanabe), MULTISOUP CO.,LTD., Milldea, LLC,
Murata Manufacturing Co., Ltd., MegaChips Corporation, Recruit Lifestyle
Co., Ltd. (K. Ushida), RICOH COMPANY, LTD., Rei-Frontier Inc.,
• Aichi Institute of Technology (K. Kaji), NARA Institute of Science and
Technology (NAIST) (I. Arai), Kanagawa Institute of Technology (H.
Tanaka), Keio University (S. Haruyama, N. Kohtake, M. Nakajima), Kyushu
University (A. Shimada, H. Uchiyama), University of Tsukuba (T. Kurata),
Japan Advanced Institute of Science and Technology (JAIST) (S. Okada),
Nagoya University (N. Kawaguchi), Niigata University (H. Makino),
Ritsumeikan University (N. Nishio), National Institute of Advanced
Industrial Science and Technology (AIST) (T. Kurata, M. Kourogi), Human
Activity Sensing Consortium (HASC), Location Information Service
Research Agency (LISRA)
5

6. Relevance of LTSs to SC 24
• MAR (e.g. Vision-based spatial registration and
tracking (vSRT) methods for MAR)
– ISO/IEC 18520: Benchmarking of vSRT methods for
MAR
– Also can be used for benchmarking of VPS (Visual
Positioning System)
• Smart Cities (WG 10)
• We focus primarily on indoor LTSs.
6

7. y
Indoor
Locali
zation
Comp
etition
s
ISO/IEC 18520:2019
Information technology — Computer graphics, image processing and
environmental data representation — Benchmarking of vision-based
spatial registration and tracking methods for mixed and augmented
reality (MAR)
• Terms and Definitions
• Benchmarking processes
• Benchmark indicators
• Trial set for benchmarking
• Conformance
Ryosuke Ichikari, Takeshi Kurata, Koji Makita, Takafumi Taketomi, Hideaki Uchiyama, Tomotsugu Kondo, Shohei Mori and Fumihisa Shibata:
Reference Framework on vSRT-method Benchmarking for MAR, Proc. of International Conference on Artificial Reality and Telexistence,
Eurographics Symposium on Virtual Environments (ICAT-EGVE) (2017.11)
7

8. Smart City: Kashiwa-no-ha
(Japanese-oak-leaf)
Consortium members
Living Lab Data Platform
Energy Mobility Wellness Public Space
LTS: Key technology for smart-cities
https://www.kashiwanoha-smartcity.com/en/
8

9. Indoor
LTS
Map
Takeshi
Kurata,
et
al.,,
IoH
Technologies
into
Indoor
Manufacturing
Sites.
IFIP
International
Conference
on
Advances
in
Production
Management
Systems
(APMS),
pp.372-380
(2019)
9

10. Indoor
LTS
Map
ISO/IEC 18520
New work item?!
10

11. xDR (PDR/VDR) Challenge Series
• Offsite indoor localization competitions mainly in
industrial scenarios
• Hosted by PDR benchmark standardizing committee
• Providing dataset measured in actual industrial fields
• Target fields: Warehouse (2017, 2018), Restaurant
(2019) and Manufacturing (2019, 2020)
• Conducted as IPIN (Int. Conf. on Indoor Positioning
and Indoor Navigation) official competition track
(except 2018)
• Evaluating practical performance in industrial scenarios
by multi-faceted evaluation metrics
https://unit.aist.go.jp/harc/xDR-Challenge-2020/
Ryosuke Ichikari, Katsuhiko Kaji, Ryo Shimomura, Masakatsu Kourogi, Takashi Okuma, Takeshi Kurata: Off-Site Indoor
Localization Competitions Based on Measured Data in a Warehouse, Sensors, vol. 19, issue 4, article 763, 2019.
11

12. PerfLoc
https://perfloc.nist.gov/perfloc.php
Winners of the PerfLoc
Prize Competition
(Texas A&M University
Corpus Christi, the US
and Wuhan University,
China)
Organizer:
Dr. Nader Moayeri
12

13. y
Indoor
Locali
zation
Comp
etition
s
ISO/IEC 18305: 2016
Information technology -- Real time locating systems –
Test and evaluation of localization and tracking systems
ISO/IEC 18305 is an international standard for testing Localization and Tracking
Systems (LTSs). NIST initiated the development of this standard in October 2012
and led the development process through the completion of the project in November
2016 with the publication of the standard. Besides the members of the subcommittee
ISO/IEC JTC 1/SC 31, Automatic identification and data capture techniques, which
were directly responsible for the development of ISO/IEC 18305, many individuals
from industry, various user communities, standard developing organizations,
academia, and US federal government reviewed various drafts of the standard and
made invaluable comments/contributions.
NIST activities in LTS testing are based on ISO/IEC 18305. The testing activities are
use cases for ISO/IEC 18305 and a means of validating the standard.
13

14. Contents of ISO/IEC 18305: 2016
3. Terms and definitions, 4. Abbreviated terms
5. LTS taxonomy
1 Types of location sensors, 1.1 Unimodal systems, 1.2 Multimodal systems
2 Reliance on pre-existing networking / localization infrastructure, 2.1 LTSs requiring
infrastructure, 2.2 LTSs capable of infrastructure-less operation, 2.3 Real-time deployment
of nodes facilitating localization, 2.4 Opportunistic use of infrastructure/environment
3 Off-line, building-specific training, 3.1 LTSs requiring off-line training, 3.2 LTSs not
requiring off-line training
4 Ultimate consumer(s) of location information, 4.1 Introduction, 4.2 The ELT, 4.3 The
tracking authority, 4.4 Both the ELT and the tracking authority
6. LTS privacy and security considerations
1 Privacy, 2 Security
7. T&E methodology taxonomy
1 System vs. component testing, 1.1 System testing, 1.2 Component testing
2 Knowledge about LTS inner-workings, 2.1 T&E designed with full knowledge of LTS
inner-workings, 2.2 Black-box testing
3 Repeatability, 3.1 Repeatable testing, 3.2 Non-repeatable testing
4 Test site, 4.1 Building-wide testing, 4.2 Laboratory testing
5 Ground truth, 1 Off-line surveyed test points, 2 Reference LTS
xDR testing not included
14

15. Contents of ISO/IEC 18305: 2016
8. LTS performance metrics
2 Floor detection probability, 3 Zone detection probability
4 Means of various errors, 5 Covariance matrix of the error vector
6 Variances of magnitudes of various errors, 7 RMS values of various errors
8 Absolute mean of the error vector
9 Circular Error 95% (CE95) and Circular Error Probable (CEP=CE50) [Horizontal error]
10 Vertical Error 95% (VE95) and Vertical Error Probable (VEP=VE50)
11 Spherical Error 95% (SE95) and Spherical Error Probable (SEP=SE50) [Error in 3D]
12 Coverage
13 Relative accuracy [distance between two ELTs (Entity to be Localized/Tracked)]
14 Latency
15 Set-up time
16 Optional performance metrics, 16.1 Location-specific accuracy, 16.2 Availability
9. Optional performance metrics for LTS use in mission critical
applications
2 Susceptibility, 3 Resilience
ISO 5725:
Accuracy (Trueness and Precision)
Used in xDR Challenge
Good for flow-line/work analysis w/ heatmap and zone stay-time/transition
15

16. Contents of ISO/IEC 18305: 2016
10. T&E considerations and scenarios
1 Building types, 1.2 Wooden structure single-family house, 1.3 Medium-size brick &
concrete office building, 1.4 Warehouse/factory, 1.5 High-rise steel structure, 1.6
Subterranean structure
2 Effects of mobility, 2.2 Stationary object/person, 2.3 Walking, 2.4 Running, 2.5 Backward
walking, 2.6 Sidestepping, 2.7 Crawling
3 Failure modes and vulnerabilities of location sensors
4 T&E scenarios
11. T&E reporting requirements
2 Test place and date, 3 Environmental conditions, 4 LTS product tested, 5 Equipment
used by the LTS, 6 ELTD features, 7 Location data format, 8 Location update rate and
system capacity, 9 RF emission and interference issues, 10 Set-up procedure, 11 Building
information needed by the LTS
12 LTS GUIs, 12.1 ELTD GUI, 12.2 Tracking authority GUI
13 Maintenance, 14 Floor plans of test buildings, 15 Characterization of T&E scenarios
involving entities in motion, 16 Presentation of numerical T&E results, 17 Visualization of
T&E results
16

17. Contents of ISO/IEC 18305: 2016
Annex A (normative) Conversions between local Cartesian and WGS
84 coordinates
2 Establishing a local 3D Cartesian coordinate system and other preliminaries
3 Conversion from WGS 84 coordinates to local 3D Cartesian coordinates
4 Conversion from the local 3D Cartesian coordinates to WGS 84 coordinates
5 Verification and validation of MATLAB coordinate conversion codes
Annex B (informative) Location sensors and their failure modes
2 RF-based location sensors, 2.1 General, 2.2 RSS, 2.3 Proximity, 2.4 TOA, 2.5 TDOA,
2.6 AOA, 2.7 Signals of opportunity
3 Range/pseudo-range finder, 4 GPS/GNSS, 5 Differential GNSS, 6 Accelerometer, 7
Gyroscope, 8 Magnetometer, 9 IMU, 10 Pedometer, 11 Inclinometer, 12 Altimeter, 13
Acoustic sensor
14 Imager 14.1 General 14.2 Optical 14.3 Infrared 14.4 Lidar
17

18. IPIN-ISC
ISO/IEC JTC 1/SC 31/WG 4
(ISO/IEC 18305: 2016)
ISO/TC 211
(Geographic
information/Geomatics)
ISO/IEC JTC 1/SC 24/WG 9
(ISO/IEC 18520: 2019)
NIST
AIST
What’s the next action?
Submitting the NWIP form ISO/IEC JTC 1/SC 24/WG 9 (but
after preparing the first draft)
• As an extention of ISO/IEC 18305 and 18520
• The title might be like: Benchmarching of xDR-based indoor LTSs
IPIN-ISC: International Standards Committee in International Conference on Indoor Positioning and Indoor Navigation
PDR benchmark
standardization committee
NWIP?!
18

19. Contents of NWI (tentative)
• Terms and Definitions
• Benchmarking Process
– Providing guidelines by applying ISO/IEC 18305 and 18520
• Benchmark Metrics：
– Indicators for xDR
• EAG (Error Accumulation Gradient)
• ALIP (Absolute Localization Inapplicable Period) based indicators
– Requirements (Negative Checks)
• Moving velocity
• Obstacle avoidance
– Providing guidelines by applying ISO/IEC 18305 and 18520
• CE
• Indicators considering flow-line/work analysis w/ heatmap and zone stay time
• Other reliability/temporality/variety indicators
• Representation and Visualization of Benchmarking Results
– Providing guidelines by applying ISO/IEC 18305, 18520, and other
standards
• Conformance
Cf. G. M. Mendoza-Silva et al., "Beyond Euclidean Distance for Error
Measurement in Pedestrian Indoor Location," in IEEE Transactions
on Instrumentation and Measurement, vol. 70, pp. 1-11, 2021.
19

20. Another topic: Four‐year project accepted by NEDO
• Multisensory XR‐AI technology platform development for tele‐
rehabilitation and mutual healthcare coordination with health
guidance
• XR‐AI: XR powered by AI (éksrèɪ)
• MR3: MultiModal Mixed Reality for Remote Rehab
• NEDO: New Energy and Industrial Technology Development
Organization in Japan
20
Remote XR
Service Encounter
User
Therapist
MR3 Wear
MR3
mannequin

21. Annex
21

22. IPIN 2020 Competition Track5
“xDR Challenge in Manufacturing 2020”
award ceremony
Ryosuke Ichikari1, Ryo Shimomura1,2
1:AIST, Japan 2:Univ. of Tsukuba, Japan
Final event and prize awarding was held on December 14, 2020.
http://evaal.aaloa.org/images/2020/videos/04-Track5+winner.mp4

23. Benchmarking for Indoor Localization
• Fair evaluation and comparison between indoor localization
methods is difficult because their performance depends on the
technology and situation where they used
• Required to standardize evaluation method
• We established PDR benchmark standardization committee.
• Indoor localization competitions：
Organizer prepare shared testing environment for comparing
competitors' localization methods with evaluation method
Benchmark
Indicators ＋
Benchmarking
Process
Trial Set
(Dataset)
＋
23

24. Characteristics of PDR/xDR Challenges
• Other Indoor localization competitions: Focusing of accurate evaluation of
accuracy of the positioning methods
• PDR/xDR Challenges: Evaluating practical performance in industrial scenarios
• Main characteristics: data measurement in actual industry and the integrated evaluation
24

25. xDR Challenge in Manufacturing 2020 (Off‐site)
• Target industry: Manufacturing
• Two sub‐tracks
• PDR sub‐tracks for tracking operators
• VDR sub‐tracks for tracking forklifts
• Dataset: Sensor data measured by Android devices
MAP, Reference pos. data, BLE beacon
pos. data (for localization with xDR and BLE)
• Devices:
• BLE beacon: PulsarGum (FUJITSU)
Battery Free, Interval of signal emission: longer than 1.26sec.
• Sensor measurement: BL‐02 (BIGLOBE)
• Organizers：
Ryosuke Ichikari, Ryo Shimomura, Satsuki Nagae,
Nozomu Ohta, Takeshi Kurata (AIST, JP), Antonio Ramon
Jimenez Ruiz (CSIC‐UPM, ES)，Soyeon Lee(ETRI,KR）
• Sponsors:
Floor plan and pos. of BLE beacon
(Size: 140m×80m）
PulsarGum BL‐02
https://unit.aist.go.jp/harc/xDR-Challenge-2020/
25

26. Evaluation of error accumulation caused by xDR in ALIP
(Absolute Localization Inapplicable Period)
• We would like to evaluate the error accumulation unique to xDR
• Dataset for our competition includes data for positional correction (BLE signals)
⇒ BLE signal is partially intentionally deleted for pure evaluation of xDR
• ALIP (Absolute localization Inapplicable period）
⇔ :ALAP（AL Applicable period）
example of AL: BLE beacon
• GT is provided at the borders of the ALIP and ALAP for correction.
ALIP
ALIP ALIP
RSSI
of BLE tag.
Correct position is hidden = Evaluation points
GT is provided for correction
t
Evaluating error accumulation in ALIP
ALAP ALAP ALAP ALAP
26

27. Evaluation indicators in xDR Challenge
• Three evaluation indicators about error ：
• Absolute error：CE（Circular Error), unit：m
• Error distribution bias：Circular Accuracy (CA), unit：ｍ
• Error accumulation：EAG (Error Accumulation Gradient) in ALIPs,
unit：m/s
• Three negative checks
• Requirements for Moving Velocity: checking if moving velocity is
within the decent range
• Requirements for Validity of the Trajectory: checking if points
consist of trajectory are in valid area
• Coverage Ratio: checking if the positional estimation is submitted
whole data
27

28. Evaluation of errors（using evaluation indicators about error）
• I_ce: evaluation for absolute error with CE50
• I_ca : evaluation for error distribution bias with CA
• I_eag : evaluation for error accumulation with EAG50
Evaluation with Negative checks
• I_velocity: speed evaluation with 1.5m requirement of moving
velocity
• I_obstacle: evaluation for obstacle collision with requirement of
validity of trajectory.
• I_coverage: evaluation for the coverage of result submission with
coverage ratio.
Calculation of the final score in xDR Challenge
Final Score＝ I_coverage ( 0.25 I_ce＋0.2 I_ca＋ 0.25 I_eag＋
0.15 I_velocity＋ 0.15 I_obstacle）
28

29. Indicator 1：Circular Error (CE)
• Explanation：Absolute 2D positional error compared to
Ground Truth
• Definition：2D Euclid distances between evaluation points
(Ground truth) and corresponding estimated positions at
the closest time
• Unit：meter
• Adopted indicator：
CE50（median of CEs）
29

30. “Accuracy” and “Precision” in localization
• Evaluation of error bias of localization method is demanded for staying area
analysis in the industrial scenario.
• Error of localization can be divided into elements of “Accuracy” and “Precision”
• Accuracy: degree of trueness or closeness to the correct position
• Precision: degree of variability or repeatability of the estimation
• Common absolute positional error (Circular Error) includes elements both of
Precision and Accuracy.
• We call 2D absolute positional error as CE according to the terminology in ISO 18305
• We evaluate the element of Accuracy by a dedicated indicator (Circular Accuracy)
cf. Indicator related to accuracy in ISO18305：The mean of the error vector
cf. Indicator related to precision in ISO18305：Variances of magnitudes of various errors
30

31. Indicator2：Circular Accuracy (CA)
• Explanation：Degree of bias of error distribution in 2D
error space
• Definition：Distance between peak of the probability
distribution of 2D error and origin of the error space
• Unit：meter
• Adopted indicator：
As is or Area‐Weighted CA
Error X
Error Y
31

32. Indicator 3: Error accumulation gradient (EAG)
• Explanation：Speed of error
accumulation from the
correction points
• Definition：ratio. of error and
elapsed time from the nearest
correction points which are
border of ALIP and ALAP
• Unit：ｍ/s
• Adopted indicators：
EAG50（median of EAGs）
• Our original indicator not
introduced in ISO18305
32
Time elapsed from CP
Close to
Ref. Points
Circular (2D) Error
Far from
Ref. Points
32

33. Negative Check1：Requirement of moving velocity
• Explanation：Requirement checking if local moving
speed of the trajectory is in the decent range
• Definition：Checking the local moving speed (delta
movement /delta time) is less than the defined valu
• Adopted requirement：1.5m/s requirement of moving
velocity
33

34. Negative check 2：Requirement for Validity of Trajectory
• Explanation：Checking the degree of incursion the
trajectory of submitted result into un‐walkable area
• Definition：Calculating ratio the incursion of trajectory into
un‐walkable area in the whole trajectory.
34

35. Negative check3：Coverage ratio
• We stop using frequency evaluation because competitors
have enough interest for submitting result as frequent as
possible.
• Time of the checking points are hidden
• No indictor to deduct points for the uncompleted submission
• Adopting metric calculating ratio of the coverage of
the submitted result to corresponding check points
and multiply the ratio by the total score
• I_coverage：Checking if each checking points have corresponding
submitted results and calculate ratio in the whole trajectory.
• Threshold：+/‐ 1 sec from the checking points
Submitted Results
Checking points Time
× I_coverage：80%
(Example )
35

36. # of application, competitors
• Pre‐admission(Required for providing sample dataset)：9
(Ireland，Japan3，China3 ，Portugal，US)
• Admission(Required for providing test dataset)：4
(Japan2，China１， Portugal)
• Result submission：2
(Japan2)
• Kawaguchi Lab Team (Nagoya University，Japan)
• Yonayona Team（Keio University，Japan）
36

37. Example of Submitted Trajectories
Trajectory of operator（PDR_No.5)
Trajectory of forklift（VDR_No.2)
AIST(as reference)
AIST(as reference)
Yonayona
Kawaguchi Lab.
Yonayona
Kawaguchi Lab.
37

38. eCDF
Operator（PDR）
38

39. Forklift（VDR）
eCDF
39

40. Scores and indicators (Operator (PDR)，average)
Team
I_ce
(CE50)
[CE75]
I_ca
(CA)
I_eag
(EAG50)
I_
velocity
I_
obstacle
I_
coverage
Final
Kawaguchi
Lab.
87.00
(4.77m)
[6.83m]
60.89
(3.91m)
99.84
(0.026m/s)
99.52 99.80 100 88.79
(Winner)
Yonayona
68.18
(10.23m)
[12.36m]
21.90
(8.99m)
99.89
(0.033m/s)
94.51 93.45 100 74.59
AIST
(Ref.)
90.61
(3.72m)
[7.17m]
65.80
(3.42m)
100
(0.018m/s)
98.12 99.21 99.94 90.36
40

41. Scores and indicators (Forklift(VDR)，average)
Team
I_ce
(CE50)
[CE75]
I_ca
(CA)
I_eag
(EAG50)
I_
velocity
I_
obstacle
I_
coverage
Final
Kawaguchi
Lab.
34.24
(20.07m)
[27.05m]
0
(18.58m)
92.01
(0.206m/s)
89.11 100 100 59.93
(Winner)
Yonayona
0
(34.63m)
[60.23m]
0
(26.83m)
70.55
(0.624m/s)
79.55 73.47 100 40.59
AIST
(Ref)
39.02
(18.69m)
[34.20m]
40.71
(5.93m)
86.86
(0.306m/s)
72.92 95.73 100 64.91
41

42. Findings from the results
• Clarified the evaluation indicators which we would like to
promote (CE,CA, EAG etc.) and used in the competition.
• EAG didn’t work well for evaluating the difference between the
competitors
• The length of ALIP (about 30min.) might be too long
• The results of trajectory of forklift were worse than we expected
• Difficulty of the VDR and low‐awareness of VDR
• # of BLE beacons for forklifts area is small.
• Parameters for calculating final score should be re‐adjusted.
• Visibility and repeatability of the evaluation method are
improved.
• We evaluated the evaluation scripts by actually using for evaluation and
sharing with competitors (Although minor changes exist during
competitions)
42

43. Evaluation scripts and documents
• Evaluation scripts are available on github
• For standardizing the evaluation method and getting feedback.
• https://github.com/PDR‐benchmark‐standardization‐committee/
• Open‐Access journal papers available
1. MDPI’s Sensors (Journal)
• Off‐Site Indoor Localization Competitions Based on Measured Data in a Warehouse
• Previous xDR Challenge (2017, 2018) @ warehouse
• Includes a survey of existing indoor localization competition
• https://www.mdpi.com/1424‐8220/19/4/763
2. IEEE Sensors Journal
• Off‐line Evaluation of Indoor Positioning Systems in Different Scenarios: The
Experiences from IPIN 2020 Competition
• xDR Challenge 2020 @ manufacturing site
• https://ieeexplore.ieee.org/document/9439493
43