The document discusses testing methodologies for both narrowband and broadband eco-systems in mission-critical communications, emphasizing the need for conformance, interoperability, and performance testing. It highlights the challenges posed by the fragmented broadband environment compared to narrowband systems, including testing for multi-vendor and multi-operator scenarios. Current testing initiatives and future phases, including ETSI plugtests and certification efforts, are also outlined to address these complexities.

1. The Importance of Testing
Difference between Narrowband and Broadband Eco-Systems
Harald Ludwig, Chairman TCCA Technical Forum
EENA Conference 2021, 7 Oct 2021, Riga

2. 2
Presentation on behalf of TCCA www.tcca.info @TCCAcritcomms
Types of Testing
• Conformance Testing
• to ensure the Implementation under Test conforms to the 3GPP Standard
• Interoperability Testing
• to ensure that two or more Implementations under Test interoperate (work together)
• Performance Testing
• to ensure that Implementation(s) under Test meet Key Performance Indicators (KPI)
• e.g. Call Setup Delay, Voice Latency, Voice Quality, Battery Life, etc.

3. 3
Presentation on behalf of TCCA www.tcca.info @TCCAcritcomms
How we test TETRA?
• Hardware and Software tied together (Network / Terminal)
• Testing & Certification
• Manual testing of 1000+ test cases
• Witnessed by an independent third party (ISCOM)
• No TETRA Conformance Test Set available
• Interoperability Testing + Checking of Conformance together
• Interoperability: Testing of combinations of Implementations (SwMI + Terminal)
• Conformance: Log of Air Interface Messages is compared to TIPs / Standard

4. 4
Presentation on behalf of TCCA www.tcca.info @TCCAcritcomms
What is Different with Broadband?
• Larger, more fragmented Eco-System
• Devices, Operating System, Mission Critical Applications
• Mission Critical Application Servers
• IP Multimedia Subsystem (IMS)
• Multicast/Broadcast Systems (eMBMS)
• LTE Core Network, eNBs, PCRF
• Control Rooms
• Multi-Vendor Environment + Multi-Operator Environment
• Lab Testing, Network Simulators, Live Network Testing

5. 5
Presentation on behalf of TCCA www.tcca.info @TCCAcritcomms
Separation of Testing
• Conformance Testing
• TCCA and GCF work together in the Mission Critical Services Work Stream
• Mission Critical Conformance Testers under development
• GCF Certification for Mission Critical Devices
• Control Rooms (?)
• Interoperability Testing
• ETSI Mission Critical Plugtests
• Next Phases
• Server-Server Testing, Interworking with Narrowband Systems

6. 6
Presentation on behalf of TCCA www.tcca.info @TCCAcritcomms
What do we have today?
3GPP RAN5 Test Cases (3GPP TS 36.579-x)
for MCPTT, MCVideo, MCData
GCF Work Item for MCPTT
ETSI MCX Plugtests
(no certification)
ETSI Plugtests
Test Cases &
Observer Scenarios
(ETSI 103 564)
Test Equipment
(LTE Simulator)
Test Equipment
(IP Simulator)
IUT:
1) Device + Client
2) Client only
TTCN3 Code, developed by 3GPP TF160
Certified Product

7. 7
Presentation on behalf of TCCA www.tcca.info @TCCAcritcomms
Overview MCX Plugtests
Nov 2021, Malaga, Spain + remote
June 2021, remote only
September 2020, remote only
September 2019 in Kuopio, Finland
Dec 2018 – Jan 2019, remote only
June 2018 in College Station, Texas
June 2017 in Sophia Antipolis, France MCPTT#1 Plugtests
MCPTT#2 Plugtests
MCX#3 Plugtests
MCX#4 Plugtests
MCX#5 Plugtests
FRMCS Plugtests
MCX#6 Plugtests