Deep_Learning_Empowered_Semi-Blind_Joint_Detection_in_Cooperative_NOMA.pdf

1. Received March 27, 2021, accepted April 14, 2021, date of publication April 20, 2021, date of current version April 29, 2021. Digital Object Identifier 10.1109/ACCESS.2021.3074350 Deep Learning Empowered Semi-Blind Joint Detection in Cooperative NOMA AHMET EMIR 1, FERDI KARA 1,2, (Member, IEEE), HAKAN KAYA 1, AND HALIM YANIKOMEROGLU 2, (Fellow, IEEE) 1Wireless Communication Technologies Laboratory (WCTLab), Department of Electrical and Electronics Engineering, Zonguldak Bülent Ecevit University, Zonguldak 67100, Turkey 2Department of Systems and Computer Engineering, Carleton University, Ottawa, ON K1S 5B6, Canada Corresponding author: Ferdi Kara (f.kara@beun.edu.tr) ABSTRACT In this paper, we propose a multi-user symbol detection in cooperative-non-orthogonal multiple access (C-NOMA) schemes via deep learning (DL). We use a DL-based detection (DLDet) in both users instead of conventional detectors. Therefore, an iterative detector (i.e., successive interference canceler (SIC)) at the near user (UE1) and a combining plus optimum detector (i.e., maximum ratio combining (MRC) and maximum-likelihood (ML) detector) at the far user (UE2) are not required anymore. The proposed DLDet can detect symbols at the both users simultaneously. Besides, the DLDet does not require an additional channel estimation algorithm to acquire the channel state information (CSI) at the receivers. The multi-user symbol detection is performed based on the received pilot responses simultaneously, thus calling semi-blind. We train the proposed DLDet offline over Rayleigh fading channel and then, we use the offline-trained DLDet as an online detection algorithm. We compare the error performance of the DLDet with two benchmarks: conventional C-NOMA and threshold-based selective C-NOMA (TBS-C-NOMA). With the extensive simulations over Rayleigh fading channels, we reveal that the DLDet outperforms both C-NOMA and TBS-C-NOMA and achieves the full diversity order (i.e., 2). Besides, this performance gain is up to ∼ 10 dB and ∼ 3 − 7 dB in C-NOMA and TBS-C-NOMA, respectively, which is very promising for the energy-limited networks. Moreover, in order to improve the error performance of the C-NOMA, the TBS-C-NOMA introduces two disadvantages, which are a signaling overhead to obtain the optimum threshold value and a capacity performance decay due the silence of the relay when the threshold is not satisfied. To this end, without using a threshold, outperforming TBS-C-NOMA is essential to revoke these disadvantages. The DLDet accomplish this; hence, the power of the DLDet is unveiled in coping with the error propagation. Then, to reveal the robustness of the DLDet against different fading conditions, we use the offline-trained DLDet as an online detector over different fading channels (i.e., Nakagami-m and Rician). We show that the DLDet outperforms conventional detectors over those fading channels and again outperforms the conventional C-NOMA and TBS-C-NOMA performances, although it is trained over Rayleigh fading channel. Indeed, the DLDet has better performance than the C-NOMA and TBS-C-NOMA even though they are assumed to have perfect CSI whereas the DLDet uses only a single pilot signal. The DLDet again provides the full diversity order (i.e., 2m for Nakagami-m and 2 for Rician). This proves the robustness of the DLDet and shows that the DLDet performs well regardless of the fading conditions once it is trained for any fading channel. INDEX TERMS Cooperative NOMA, deep learning, error performance, joint symbol detection, semi-blind detection. I. INTRODUCTION 5G and beyond networks are keen to divided into three main aspects/vertical sectors in terms of quality of The associate editor coordinating the review of this manuscript and approving it for publication was Cunhua Pan . service criteria. These pioneer areas are grouped as: enhanced mobile broadband communication (eMBC), ultra reliable low latency communication (uRLLC) and massive machine-type communication (mMTC) [1], [2]. All these aspects can not be achieved by a single physical layer technique; hence, the multiple access techniques will also diverse between 61832 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ VOLUME 9, 2021

2. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA these vertical sectors contrary to previous wireless genera- tions where one ubiquitous technique covers all clients [3]. Non-orthogonal multiple access (NOMA)1 is one of the key technologies for mMTC due to its high spectral efficiency [3], [4]. Hence, NOMA has attracted tremendous attention from the academia [5] and industry [6]. NOMA is based on transmitting all users’ symbols on the same resource block (time/frequency/code) with different power allocation coefficients. Thus, all resource blocks are shared by users and overall capacity is improved. The detection at the users is succeeded by implementing iterative successive interference cancelers (SIC) to eliminate intentionally created inter-user- interference. In addition, the interplay between NOMA and other physical layer techniques (e.g., cooperative communication, cognitive radio, MIMO, etc.) has been investigated since NOMA has a flexibility to be implemented in other techniques [7]. In cooperative-NOMA (C-NOMA), the users with the better channel conditions act as relays for the users with poor channel conditions and forward their symbols obtained during SIC processes. The C-NOMA concept was firstly proposed for short range communication in [8] and the authors have proved the superiority of C-NOMA over conventional NOMA and orthogonal multiple access (OMA) networks in terms of achievable rate and outage probability. Then in [9], the authors have analyzed the bit error probability of C-NOMA and have showed that the error propagation from near user to far user dominates the error performance of network and the diversity can not be achieved, although a cooperative communication is implemented. Then, to minimize the effect of error propagation in C-NOMA networks, the authors in [10] have proposed threshold-based selective cooperative-NOMA (TBS-C-NOMA) where the near user forwards the far user’s symbols only if the received signal-to-interference plus noise ratio (SINR) is above pre-determined threshold value otherwise it remains silent in the second phase of communication. Although the proposed TBS-C-NOMA decreases the effect of error propagation on C-NOMA, the system model is depended on the pre-determined threshold value. The authors have also analyzed the optimum threshold value to minimize the error performance of TBS-C-NOMA and derived an analytical expression for it. This optimum threshold value is to be used at the near user (relay) and to determine the optimum threshold value, the near user requires extra information such as channel state information (CSI) of far user and transmit power. However, these cost a signaling overload. Besides, there is a trade-off in TBS-C-NOMA that is: the capacity and outage probability performances get worse in TBS-C-NOMA whereas the error performance is increased [11]. Hence, increasing the error performance of C-NOMA rather than 1Although, NOMA has divided into three main concept as: power domain NOMA (PD-NOMA), code domain NOMA (CD-NOMA) and pattern division multiple access (PDMA), we focus on PD-NOMA concept and use NOMA abbreviation for PD-NOMA. implementing TBS-C-NOMA seems more beneficial in terms of all performance aspects. On the other hand, machine learning techniques have been investigated for decades for prediction, recognition and classification applications. There have also been some studies which investigate machine learning in physical layer communications [12]. Nevertheless, the implementation complexity/cost made them impractical for physical communications although it has taken an important role on the higher-level algorithms. However, with the development in the digital signal processors and the improvement of computational capacity, machine learning techniques gain an important role for physical layer communications. Deep learning (DL) is seen as the milestone of machine learning techniques and has great recent attention in communication applications. The authors in [13] have built a DL network to estimate channel and to detect orthogonal frequency division mul- tiplexing (OFDM) symbols jointly. It is shown that using DL networks for estimation and detection has performed better than well-known estimation (least square -LS, minimum mean-square error -MMSE)/detection (maximum likelihood-ML) algorithms. This study has proven the power of DL networks for physical layer communications and has led the researchers to investigate DL networks in physical communications [14]–[16]. State-of-art for DL implementation in physical layer communication is drawn for both data-driven and model-driven networks in [17]–[20] and the potential of DL has been shown. Then, this high potential of DL networks has attracted great attention by the community [21], [22] and the DL networks applications have been implemented in more timely topics such as index modulations [23] and mmWave communications [24]. Therefore, the DL-based algorithms have also been implemented in NOMA schemes where grant-free and multi-user detection are investigated in CD-NOMA [25]–[27] and along with PD-NOMA schemes for classification/ optimization [28]–[30], resource allocation [31], [32], modulation design [33], [34], signal detection [35]–[40]. However, all aforementioned studies assume basic downlink and/or uplink NOMA schemes and to the best of the authors’ knowledge, there is no study to design DL-aided joint signal detection to improve the error performance of C-NOMA, yet. Based on above discussions, in this paper, we propose a data-driven DL network to detect users’ symbols jointly in C-NOMA. The main contributions of this paper are sum- marized as follow. • We propose a DL-based detection (DLDet) in C-NOMA where the proposed DLDet is able to detect both users’ symbols jointly at the near user without an iterative (successive) detection. Hence, we eliminate the usage of SIC process at the near user and this could provide less latency. Besides, the DLDet performs a cooperative detection at the far user without requiring a combining technique. • The DLDet performs the symbol detection based on the received pilot responses; hence no additional channel VOLUME 9, 2021 61833

3. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA estimation algorithm (e.g., LS, MMSE) is required at both users, such that calling a semi-blind joint detection. • Based on the extensive simulations over Rayleigh fading channels, the DLDet provides a better error performance than conventional detectors in C-NOMA. The effect of the error propagation is eliminated completely. The DLDet provides more than ∼ 10 dB gain over C-NOMA which is superb for the energy-limited networks. Indeed, the DLDet also outperforms the TBS-C-NOMA even with optimum threshold usage up to ∼ 2 − 6 dB. Moreover, these performance gains are obtained when very low pilot sizes are used in DLDet whereas the perfect CSI is assumed in C-NOMA and TBS-C-NOMA. Therefore, not only the error performance of the C-NOMA is improved but also the capacity degradation has not been introduced due to the pilot size. In addition, an additional signaling overhead is not required to obtain the optimum threshold value to increase the performance as being in TBS-C-NOMA. • Furthermore, the DLDet outperforms conventional detectors over different fading channels (i.e., Nakagami-m and Rician), although it is trained offline over Rayleigh fading channels. Besides, the DLDet has better performance than conventional C-NOMA and TBS-C-NOMA with the perfect CSI even if only one pilot signal is used in the DLDet. The DLDet achieves the full diversity order (i.e., 2 for Rayleigh and Rician and 2m for Nakagami-m) regardless of the fading conditions. This unveils the robustness of the DLDet against fading conditions. The remainder of this work is as follows. In Section II, we introduce signal and channel models for the C-NOMA. We have also presented benchmark detection for C-NOMA and TBS-C-NOMA in this section. Then, in Section III, we introduce the proposed DLDet: DL-based detection. The model parameters and training details for the DLDet are presented. In Section IV, we provide numerical results and comparisons with the well-known detectors (SIC and ML) are provided. Finally, Section V discusses the results and the paper is concluded. Notation: The normal font and lowercase letters (i.e., y) represent complex scalar values. The bold uppercase (i.e., Y) and lowercase (i.e., y) letters denote the matrices and the vectors, respectively. Re{} and Im{} are the operations for in-phase and quadrature components of a scalar/vector/matrix; hence, we use (.)I and (.)Q for in-phase and quadrature components of a scalar/vector/matrix. ˆ denotes the estimated symbol/vector/matrix. (.)∗ is used for conjugate of a complex scalar/vector. ||.|| is the Euclidean norm of a matrix/vector. We use |.| for the absolute value of a scalar/vector. In the representations, (x/x/X)λ,ψ , ψ = p, d represents signals/responses in the λ link for ψ where p refers to the pilot symbols/responses and d refers to the data symbols/responses. CN(µ, σ2) is a complex Gaussian distribution which has independent real and imaginary random variables with the µ mean and the σ2 2 variance. Lastly, P (A,k) i (e) denotes the error probability of the ith symbols at the user k by implementing A detector. II. SYSTEM AND CHANNEL MODELS In C-NOMA, a downlink NOMA network is considered where a base station (BS) and two mobile users are located (i.e., UE1 and UE2). The users are denoted as the near (intra-cell) user and far (cell-edge) user according to their large-scale fading coefficients (e.g., distances to the BS). As shown in Fig. 1, the intra-cell user also acts as a half-duplex relay for cell-edge user and a cooperative phase is implemented. The total communication covers two time slots and in the first time slot, BS implements a conventional NOMA by superimposing signals of the users. Thus, the received signal by users in the first time slot are given as yλ = p Pshλ( √ a1x1 + √ a2x2) + nλ, λ = s1, s2, (1) FIGURE 1. The illustration of the considered system model. where hλ and nλ denote the complex channel fading coefficient and the additive white Gaussian noise (AWGN), respectively. nλ is distributed as CN(0, N0). The channel coefficient between each node is assumed to follow2 CN(0, σ2 λ ), λ = s1, s2, r where s1, s2 and r denote the links between BS-UE1, BS-UE2 and UE1-UE2, respectively, and σ2 λ is related to the distance between nodes. a1 and a2 are power allocation coefficients for the users’ baseband symbols x1 and x2, respectively. Without loss of generality, UE1 and UE2 are assumed to be intra-cell user and cell-edge user according to distance to the BS i.e., σ2 s1 ≥ σ2 s2; thus, a1 < a2 is determined. 2E[|hλ|2]] = σ2 λ = µd−τ λ is defined where µ, dλ, and τ are the propagation constant, the Euclidean distance, and the path-loss exponent, respectively. Nevertheless, we use σ2 λ for notation simplicity. 61834 VOLUME 9, 2021

4. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA A. C-NOMA After detection of symbols at UE1, UE1 forwards the symbols of UE2 in the second phase of communication (see Fig. 1.a). Hence, the received signal by UE2 in the second phase of C-NOMA is given as yr = p Pr hr x̂ (1) 2 + nr , (2) where Pr is the power of the relay-UE1- and hr is the fading coefficient between users. x̂ (1) 2 denotes the detected symbols of the UE2 at UE1. B. TBS-C-NOMA In TBS-C-NOMA, to minimize the effect of error propagation from UE1 to UE2, the second phase of communication is succeeded only if the SINR at the UE1 is above the pre-determined threshold value, otherwise UE1 remains silent in the second phase of communication (see Fig. 1.b). Hence, the received signal by UE2 in the second phase of TBS-C-NOMA is given as yr = ( 0, γ < γth, √ Pr hr x̂ (1) 2 + nr , γ ≥ γth, (3) where, γth is the threshold value. γ denotes the SINR at UE1 in the first phase for the symbols of UE2 and it is defined as γ = ρsa2 |hs1|2 ρsa1 |hs1|2 + 1 , (4) where ρs = Ps/N0 is defined. C. BENCHMARK DETECTORS IN C-NOMA AND TBS-C-NOMA In the first phase, since the both users’ symbols are conveyed simultaneously, an interference occurs. Therefore, an interference mitigation technique is required. To this end, the UE1 implements an SIC to detect its own symbols. The UE1 firstly detects x2 symbols by pretending own symbols (i.e., x1) as noise since x2 symbols have higher power allocation (i.e., a2 > a1). The maximum-likelihood (ML) detection at the UE1 for x2 symbols is given as x̂ (1) 2 = argmin j

7. ys1 − p Psa2ĥ1x2,j

10. 2 , j = 1, 2, . . . , M2, (5) where x2,j is the jth constellation point in M2-ary modulation order of the UE2. ĥλ, λ = s1, s2, r is the estimated channel coefficient acquired by a channel estimation algorithm (e.g., LS, MMSE). Then, the detected x̂ (1) 2 symbols are subtracted from the received signal and an ML detector is performed for x1 symbols. Thus, the detection structure of x1 symbols is given by x̂1 = argmin j

13. y (SIC) s1 − p Psa1ĥ1x1,j

16. 2 , j = 1, 2, . . . , M1, (6) where y (SIC) s1 = ys1 − p Psa2ĥ1x̂ (1) 2 . (7) On the other hand, a cooperative communication is considered for the UE2 and the UE2 receives two copies within two time phases. Thus, as firstly, these copies are combined by using maximum ratio combining (MRC). The total received symbol at the UE2 after the MRC is given by y2 = ys2ĥ∗ s2 + yr ĥ∗ r , (8) where ()∗ is the conjugate operation. Lastly, to detect x̂2 symbols at the UE1, an ML detector (as being in (5)) is implemented based on the obtained y2 signal. III. THE PROPOSED DLDet In the proposed model, we use a four-layered DL network based on Long Short Term Memory (LSTM). The LSTM networks perform well in predicting single or time-series data and resolve the learning deficiency of long-term correlation in conventional recurrent neural networks (RNNs). As it will be discussed in detail in the next subsection, in this paper, we perform a frame-based detection and within the frame, the data is correlated due to the channel fading. Besides, the pre-trained LSTM networks perform well in different fading/channel environments as long as one of the inputs includes an informative data belonging to that fading/channel condition (e.g., pilot responses). Thanks to our frame design and representative inputs (will be explained in the next subsection), the proposed DLDet could perform well in not only the training channel conditions but also various channel/fading conditions which have not been used in the training. This advantage of the LSTM provides a robustness design against fading environments and once the training is completed, it can be used on different fading/channel environments. It is given by the LSTM inventors as: ‘‘Unlike previous approaches, ours quickly learns to distinguish between two or more widely separated occurrences of a particular element in an input sequence, without depending on appropriate short time lag training exemplars.’’ [41]. Therefore, we choose the LSTM-based DL network thanks to its aforementioned powers.3 Each layer of the proposed DLDet is explained as follows. • The first layer is the input layer where the inputs of the network are transferred to the further layer with weight coefficients likewise in simple artificial neural networks (ANNs). • In the next two layers, we implement two LSTM layers which have 30 and 10 LSTM cells, respectively. 3Nevertheless, we would like to note that other DL approaches (e.g., deep neural networks (DNN), convolution neural networks (CNN), generative adversarial networks (GAN), etc.) have also been used for symbol detection of communications systems in the literature. The performances of the chosen DL approaches are heavily based on dataset generation and the training. Thus, no one can guarantee that one of the DL approaches outperforms others unless the specific applications such reinforcement learning. Other DL approaches could have also been tried for detection of C-NOMA, but it is beyond the scope of this paper. VOLUME 9, 2021 61835

17. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA FIGURE 2. The proposed DLDet networks. As being in all previous data-driven communication applications [25]–[40], the number of LSTM layers and the number of LSTM cells in each layer are empirically determined, such that increasing the sizes do not provide a noteworthy gain in learning performance and the network performance converges. In an LSTM cell, the output is produced by not only the current inputs but also the previous cell status. To memorize the previous status and decide whether the previous statuses are used or not, the LSTM cell includes <gate units>. These are the input gate, the candidate gate, the forget gate and the output gate as seen in Fig. 2 and are explained briefly as follow. – The input gate collects the data from the previous layer and previous cells within the LSTM layer. – The candidate gate computes the new coefficient based on the activation function (i.e., sigmoid function) and the forget gate status. – The forget gate has knowledge how much the previous cell status will be used (e.g., 0 for forget, 1 for full usage and any value between 0 − 1 for the usage to the some extent). – Finally, at the output gate, the cell output is com- puted according to cell input, candidate gate value and activation function (for detailed analysis for an LSTM cell, please refer to [41]). • The last layer in the DLDet is a fully-connected layer (i.e., ANN) and it computes the outputs of the network with trained-weighs and biases. On the other hand, we use a feedback regression layer in the training to optimize the network weights, biases and cell status. A. FRAME DESIGN AND REPRESENTATION In wireless communications, the fading channel has a domi- nant effect on the performance of the systems. Thus, the CSI at the receiver is required to implement signal detection in all wireless systems. Likewise, the signal detection in NOMA needs the CSI at the receiver as explained in the Section II. C and a channel estimation algorithm (e.g., LS, MMSE) should be implemented at the users to acquire the CSI. On the other hand, the DL networks have potential to detect symbols without an additional channel estimation algorithm to acquire the CSI. The signal detection can be performed simultaneously based on the pilot responses [13]. Therefore, in order to perform a semi-blind multi-user signal detection, we insert known-pilot symbols within the data frames. Based on these pilot responses, the proposed DLDet learns the correlation within the data and predicts the transmitted data symbols. 61836 VOLUME 9, 2021

18. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA To this end, the total pilot plus data frame of users at the BS are given as x1 =   xp, xp, . . . , xp | {z } Lp , x1(Lp + 1), . . . , x1(L) | {z } Ld    | {z } L , x2 =   xp, xp, . . . , xp | {z } Lp , x2(Lp + 1), . . . , x2(L) | {z } Ld    | {z } L , (9) where xp is the known-pilot symbol and xi(t), i = 1, 2, t = Lp + 1, Lp + 2, . . . , L is the base-band symbol of the ith user in the tth order within the frame. Lp, Ld and L denote the number of pilot symbols, number of data symbols and total frame size, respectively. The Lp pilot symbols could be also interleaved within the frame. However, this does not change the performance, thus giving in the beginning for the representation simplicity. We assume that all channels undergo a block-fading.4 The channel coefficient within a frame is constant and change from frame to frame. According to (1) and (9), the received frames in the first phase are given as yλ =   yλ,p(1), yλ,p(2), . . . , yλ,p(Lp) | {z } Lp , yλ(Lp + 1), . . . , yλ(L) | {z } Ld    | {z } L = p Pshλ √ a1 xp, xp, . . . , xp, x1(Lp + 1), . . . , x1(L) + √ a2 xp, xp, . . . , xp, x2(Lp + 1), . . . , x2(L) + nλ, (10) where yλ,p(t), t = 1, 2, . . . , Lp and yλ(t), t = Lp + 1, Lp + 2, . . . , L denote the received pilot responses and data symbols in the tth order within frame at the users. nλ is a random process where each sample has an AWGN. Based on the received ys1 frame, the proposed DLDet at the UE1 detects x̂1 and x̂ (1) 2 symbols simultaneously. By the same frame generation in the first phase, the detected x̂ (1) 2 symbols are formed with the known-pilot symbols and the frame is given as x̂ (1) 2 =   xp, xp, . . . , xp | {z } Lp x̂ (1) 2 (Lp + 1), . . . , x̂ (1) 2 (L) | {z } Ld    | {z } L , (11) 4The C-NOMA scheme is assumed in short-range communication [8]. Hence, this assumption is quite reasonable in a short-range communication. where x̂ (1) 2 (t), t = Lp +1, Lp +2, . . . , L denotes the detected x2 symbol in the tth order within the frame at the UE1 by the implemented DLDet. Likewise the first phase, xp is the known-pilot symbol. Then, the detected x̂ (1) 2 symbols and pilot symbols are for- warded to the UE2 in the second phase. Therefore, according to (2) and (11), the received frame at the UE2 in the second phase is given as yr =   yr,p(1), yr,p(2), . . . , yr,p(Lp) | {z } Lp yr (Lp + 1), . . . , yr (L) | {z } Ld    | {z } L = p Pr hr h xp, xp, . . . , xp, x̂ (1) 2 (Lp + 1), . . . , x̂ (1) 2 (L) i + nr , (12) where nr is a random process where each sample has an AWGN. Finally, the proposed DLDet at UE2 detects x̂2 symbols based on the received frames in two phases given by (10) and (12), respectively. B. DATASET GENERATION AND TRAINING DLDet FOR UE1 At the UE1, the detection should be implemented in the first phase. Thus, the detection is performed based on the received frame ys1 in the first phase in (10) so that ys1 is the input of the DLDet. Nevertheless, within the received L-length ys1 frame, the data symbols cover only Ld length. The remaining Lp-length pilot symbols are inserted to acquire channel responses and these are non-informative symbols. We should detect the informative x̂1 and x̂ (1) 2 symbols which are both Ld -length. Thus, the outputs of the DLDet are x̂1 and x̂ (1) 2 symbols with Ld -length. In the LSTM networks, the frame sizes of the inputs and the outputs should be the same. To this end, we should reform the received frame ys1 to have Ld length. Hence, we split the received frame into two groups as pilot responses, Lp-length ys1,p, and received data symbols, Ld -length ys1. Then, to have the same size, we extend the pilot responses by adding copies at the end of the frame until to have Ld -length. These reformed pilot responses and received data frames are the inputs of the DLDet. On the other hand, the DL networks can not perform with complex numbers. Thus, we take in-phase (i.e., ()I ) and quadrature (i.e., ()Q) components of each variables as different inputs and outputs. The illustration for the system model of the DLDet at the UE1 is given in Fig. 2.a. Based on above reforming steps, the inputs of the DLDet at the UE1 are given in (13), as shown at the bottom of the next page. The DLDet is expected to detect data frames. The outputs of proposed DLDet are the in-phase and quadrature components of the detected frames; therefore, the detected VOLUME 9, 2021 61837

19. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA frames are obtained by x̂1,d = x̂I 1,d + 1j ∗ x̂ Q 1,d = x̂1(Lp + 1), x̂1(Lp + 2), . . . , x̂1(L) | {z } Ld , x̂ (1) 2,d = x̂ (1),I 2,d + 1j ∗ x̂ (1),Q 2,d = h x̂ (1) 2 (Lp + 1), x̂ (1) 2 (Lp + 2), . . . , x̂ (1) 2 (L) i | {z } Ld , (14) where x̂I 1,d , x̂ Q 1,d , x̂ (1),I 2,d , and x̂ (1),Q 2,d are the outputs of the DLDet at the UE1 as given in Fig. 2.a. In order to make the DLDet learn better and cope with various signal-to-noise ratio (SNR) values, we consider different SNR values where we define an SNR = [0 : 5 : 30] vector and for each SNR values in this vector, the dataset is re-obtained to train the network. In the dataset generation, we assume all the average channel qualities are the same (i.e., E[|hs1|2] = E[|hs2|2] = E[|hr |2] = 0 dB). The dataset generation for the DLDet at the UE1 is given in Algorithm 1 where S is the number of samples for each scenario. The outputs of Algorithm 1 are the users’ symbols, the pilot responses and the received signals in the first phase. They are given as matrices form and each rows of these matrices are the Ld -length in-phase and quadrature components of the frames obtained in Steps 5 and 8 of Algorithm 1. Hereby, it is worthy noting that the transmitted xi,d , i = 1, 2 frames are included in the dataset since they will be used as the target (desired) outputs at the regression layer during the training process. After obtaining the dataset, the DLDet is trained to detect UE1 and UE2 symbols jointly. In the training process, it is aimed to minimize the number of the erroneous-detected symbols. Thus, the optimization problems for the DLDet at Algorithm 1: Dataset Generation for DLDet at UE1 Data: S, L, Lp, Ld , SNR 1 for each SNR value in SNR do 2 for s = 1 : S do 3 IGenerate Lp-length xp known-pilot symbols. 4 IGenerate random Ld log2 Mi, i = 1, 2 bits for UE1 and UE2 and obtain the Ld -length x1,d and x2,d by mapping random bits to xi by Mi-ary modulation. 5 IObtain the L-length xi, i = 1, 2 frames by using (9). 6 IGenerate random Rayleigh channel coefficients (hs1) and generate random AWGN process (ns1). 7 IAccording to (10), calculate the ys1 with L length and then based on this ys1, reform the Ld -length yI s1,p, y Q s1,p,yI s1,d , y Q s1,d by using (13). 8 end 9 end Result: XI 1,d ,X Q 1,d ,XI 2,d ,X Q 2,d ,YI s1,p,Y Q s1,p,YI s1,d ,Y Q s1,d UE1 are given as {P1} = min ( 1 2Ld Ld X i=1

21. |x1,d − x̂1,d |

23. 2 ) , (15) and {P2} = min ( 1 2Ld Ld X i=1

29. x2,d − x̂ (1) 2,d

35. 2 ) . (16) On the other hand, if the proposed DLDet can not reach the performance of the conventional C-NOMA and/or TBS-C-NOMA, it makes no sense. To this end, we also define yI s1,p = Re                    ys1,p(1), .., ys1,p(Lp) | {z } Lp , . . . , ys1,p(1), . . . , ys1,p(Lp) | {z } Lp    | {z } Ld                  , yI s1,d = Re      ys1,d (Lp + 1), ys1,d (Lp + 2), . . . , ys1,d (L) | {z } Ld      , y Q s1,p = Im                    ys1,p(1), . . . , ys1,p(Lp) | {z } Lp , . . . , ys1,p(1), . . . , ys1,p(Lp) | {z } Lp    | {z } Ld                  , y Q s1,d = Im      ys1,d (Lp + 1), ys1,d (Lp + 2), . . . , ys1,d (L) | {z } Ld      . (13) 61838 VOLUME 9, 2021

36. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA two optimization problems as {P3} = min n P (DLDet) 1 (e) − P (SIC) 1 (e) o , (17) and {P4} = min n P (DLDet,1) 2 (e) − P (ML,1) 2 (e) o , (18) where P (DLDet) 1 (e) and P (DLDet,1) 2 (e) denote the bit error rate (BER) performances of the DLDet in detecting UE1 and UE2 symbols at the UE1 when it is implemented as a detector in C-NOMA. On the other hand, P (SIC) 1 (e) and P (ML,1) 2 (e) denote the BER performances of conventional SIC and ML detectors for UE1 and UE2 symbols at the UE1, respectively, and it can be found in [9].5 According to above optimization problems, we train the DLDet and optimize DL parameters. The training algorithm is given in Algorithm 2. The training settings in Algorithm 2 and the optimized parameters are given in Table 1. Algorithm 2: Training and Parameter Optimization of DLDet at UE1 Data: XI 1,d ,X Q 1,d ,XI 2,d ,X Q 2,d ,YI s1,p,Y Q s1,p,YI s1,d ,Y Q s1,d 1 Initialize DL Parameters (mini batch size, learning rate, maximum epoch) 2 ITrain the network according to P1 and P2 in (15) and (16). 3 if The P3 in (17) P4 in (18) do not converge then 4 IUpdate DL parameters and repeat Step 2 5 end Result: The DLDet at UE1, optimized DL parameters and X̂I 1,d ,X̂ Q 1,d ,X̂ (1),I 2,d ,X̂ (1),Q 2,d C. DATASET GENERATION AND TRAINING DLDet FOR UE2 Since a cooperative communication is included, the detection at the UE2 is performed in the second phase based on the received L-length frames (i.e., ys2 and yr ) within two phases given in (10) and (12), respectively. On the other hand, the UE2 detects only own frames (i.e., x̂2); hence, the output has two frames with Ld -length: in-phase and quadrature components of the detected x̂2. As explained in detail at the training of the UE1, the inputs of the DLDet should be reformed to have non-complex frames and the same size with the output frames. To this end, the illustration for the system model of the DLDet at the UE2 is as given in Fig. 2.b. The input frames are given in (19), as shown at the bottom of the next page, and the detected frame is obtained by the outputs of the DLDet as x̂2,d = x̂I 2,d + 1j ∗ x̂ Q 2,d 5In TBS-C-NOMA, the BER performances at UE1 are exactly the same with C-NOMA; thus, we define optimization problems only for C-NOMA. TABLE 1. Training settings and optimized parameters for DLDet at UE1 and UE2. = x̂2(Lp + 1), x̂2(Lp + 2), . . . , x̂2(L) | {z } Ld , (20) where x̂I 2,d and x̂ Q 2,d are the output frames of the DLDet at the UE2. As discussed in the dataset generation of UE1, we consider different SNR values to make the network better in learning. Nevertheless, we should note that the received frame in the second phase in (12) (i.e., yr ) is based on the detected x (1) 2 frame at the UE1 given in (14). Therefore, to generate a dataset for the DLDet at UE2, the proposed DLDet at UE1 should be implemented and the detected frames should be obtained. Considering this, the dataset generation for training of the DLDet at the UE2 is given in Algorithm 3. The outputs of Algorithm 3 are the UE2’s symbols, the pilot responses, and the received signal at the UE2 in both phases. They are given in the matrix form and each rows of these matrices are the Ld -length in-phase and quadrature components of the frames obtained in Steps 5, 8, and 11 of Algorithm 3. It is again worthy noting that the transmitted x2,d frames are included in the dataset since they will be used as the target (desired) outputs at the regression layer during the training process. As being at UE1, the DLDet at UE2 is expected to detect x2 symbols with minimum error. Thus, the optimization problem for the network is given as {P5} = min ( 1 2Ld Ld X i=1

38. |x2,d − x̂2,d |

40. 2 ) . (21) Again, we define the optimization problem where DLDet should outperform the conventional detectors as {P6} = min n P (DLDet) 2 (e) − P (TBS−C−NOMA) 2 (e) o , s.t, P (DLDet) 2 (e) P (C−NOMA) 2 (e), (22) VOLUME 9, 2021 61839

41. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA where P (DLDet) 2 (e), P (C−NOMA) 2 (e) and P (TBS−C−NOMA) 2 (e) denote the BER performances of the DLDet at UE2, conventional detectors in C-NOMA and TBS-C-NOMA, respectively. As explained in the previous section, the MRC + ML is used to detect x2 symbols in C-NOMA and TBS-C-NOMA. As seen in (22), we seek a DLDet which outperforms the C-NOMA and is competitive with the TBS-C-NOMA in terms of error performance. Based on these optimization problems, the training procedure of the DLD for the UE2 is given in Algorithm 4. The training settings in Algorithm 4 and the optimized parameters are given in Table 1. D. PARAMETER SETTING AND HYPER-TUNING In the DL networks, the network design (e.g., number of hidden layers and cells in each layer) and the training parameters (e.g., learning rate, mini-batch size and maximum epoch) are empirically determined. There is no known rule which gives the optimum network setup and/or training parameters. Therefore, a hyper-parameter tuning is required for all DL-aided designs. To this end, in this paper, we have also applied a hyper-tuning for both network design and training parameters. According to the initial trials, we firstly determined the DLDet network setup as a four-layered LSTM network with two hidden LSTM layers which have 30 and 10 cells, respectively. Then, according to Algorithm 1 and Algorithm 3, we have obtained the dataset for both users.6 After obtaining the dataset, the training procedures are performed by Algorithm 2 and Algorithm 4 for UE1 and UE2, respectively. To hyper-tune the DL parameters, in Step 3 in both algorithms, the networks are re-trained unless the performance converges. As given in the Table 1, we train the both networks by using an ADAM optimizer according to their objective functions (i.e., P1 and P2 in Algorithm 2, and P5 in Algorithm 4) in the Step 2 of both algorithms. In this step, we applied a stopping criterion which ceases training when the validation accuracy does not improve in consecutive epochs. Then, we have updated DL parameters and then re-trained the networks according to these updated DL parameters (see Step 4 in both algorithms) unless the objective functions (i.e., P3 and P4 in Algorithm 2 and P6 in Algorithm 4) converge. In order to reveal the effects of the DL parameters, for some of the training DL parameters, we present the training 6In the dataset generation, any frame size and pilot-to-frame ratio ( φ = (Lp/L)) can be used. These parameters do not affect the training performance much since our frame design and input/output representations remove the effects of them and make the DLDet a flexible detection algorithm which performs regardless of frame size and/or pilot-to-frame ratio. We have selected L = 64 and φ = (1/L) (i.e., the worst case). Hereby, it is worthy noting that for a good training, the amount of the data is important; hence, the sample size S should not be too low. yI s2,p = Re                    ys2,p(1), .., ys2,p(Lp) | {z } Lp , . . . , ys2,p(1), . . . , ys2,p(Lp) | {z } Lp    | {z } Ld                  , yI s2,d = Re      ys2(Lp + 1), ys2(Lp + 2), . . . , ys2(L) | {z } Ld      , y Q s2,p = Im                    ys2,p(1), . . . , ys2,p(Lp) | {z } Lp , . . . , ys2,p(1), . . . , ys2,p(Lp) | {z } Lp    | {z } Ld                  , y Q s2,d = Im      ys2(Lp + 1), ys2(Lp + 2), . . . , ys2(L) | {z } Ld      , yI r,p = Re                    yr,p(1), .., yr,p(Lp) | {z } Lp , . . . , yr,p(1), . . . , yr,p(Lp) | {z } Lp    | {z } Ld                  , yI r,d = Re      yr (Lp + 1), yr (Lp + 2), . . . , yr (L) | {z } Ld      , yQ r,p = Im                    yr,p(1), . . . , yr,p(Lp) | {z } Lp , . . . , yr,p(1), . . . , yr,p(Lp) | {z } Lp    | {z } Ld                  , y Q r,d = Im      yr (Lp + 1), yr (Lp + 2), . . . , yr (L) | {z } Ld      . (19) 61840 VOLUME 9, 2021

42. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA TABLE 2. Training results of the DLDet for different DL parameters in 5 dB SNR. results7 of the Step 2 in Algorithm 2 and Algorithm 4 in Table 2 and in Table 3 for the 5 dB and 20 dB SNR values, respectively. We provide performances for two different SNR 7During optimizing the DL parameters, we require the BER performances of the conventional detectors as shown in (17), (18) and (22). Although the BER performances of NOMA are provided with imperfect CSI in [42], [43], to the best of the authors’ knowledge, there is no study for the BER performances of C-NOMA with the imperfect CSI. Therefore, we optimize the DL parameters for the perfect CSI case where the BER performances of the conventional detectors are given in [10]. To this end, for fairness, the proposed DLDet for both users in Step 3 of Algorithm 2 and Algorithm 4 are also implemented with the perfect CSI where yI s1,p = hI s1 and y Q s1,p = h Q s1 in Algorithm 2 and yI s2,p = hI s2, y Q s2,p = h Q s2, yI r,p = hI r and y Q r,p = h Q r in Algorithm 4 are used. values to show the effect of the SNR since, especially in the low SNR region, the training parameters may not show the best performance in DL. However, in relatively higher SNR region, the performance converges and the DLDet outperforms the conventional detectors. In Table 2 and the Table 3, the best performances (minimum P3, P4 and P6) are shown with blue cells. In addition, if we look both tables in detail, we see that in the training, the constraint in P6 is satisfied for almost all training parameters and the DLDet outperforms the conventional C-NOMA. Nevertheless, to select the same parameters for both networks, we consider the overall performance improving for all SNR region. Besides, in this paper, we mainly aim to maximize the performance gain VOLUME 9, 2021 61841

43. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA TABLE 3. Training results of the DLDet for different DL parameters in 20 dB SNR. for cooperative user and to eliminate the error propagation. Therefore, we select the DL parameters of having the best performance for UE2 which are shown as grey rows in tables (i.e., settings number 21) and these are the DL parameters are given in Table 1. We aware that the selected DL parameters may not give the best P3 performance for UE1, but it gives the best P6 for UE2. In addition, considering this trade-off, one can easily see that the gain in P6 is much more than the loss in P3 since the DLDet provides very similar performance for P3 at UE1 regardless of the DL parameters. The effects of the DL parameters are explained as follow. As seen in Tables 2 and 3, with lower learning rate, the DLDet performs better since it computes the backward error propagation with a lower coefficient. However, this causes a very slow learning process and it converges after some point and not provides further improving. Therefore, we optimize the learning rate as 0.001 which provides a good performance with a relatively faster learning procedure. The mini batch size defines how many samples will be taken from the dataset in every training epoch. Therefore, there is a reverse relationship between mini batch size and the maximum epoch. Indeed, if the maximum epoch is not chosen largely enough for a mini batch size, the learning will not converge. On the other hand, if a too large maximum epoch is chosen, the DLDet may start to memorize 61842 VOLUME 9, 2021

44. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA Algorithm 3: Dataset Generation for DLDet at UE2 Data: S, L, Lp, Ld , SNR 1 for each SNR value in SNR do 2 for s = 1 : S do 3 IGenerate Lp-length xp known-pilot symbols at the BS. 4 IGenerate random Ld log2 Mi, i = 1, 2 bits for UE1 and UE2 and obtain the Ld -length x1,d and x2,d by mapping random bits to xi by Mi-ary modulation. 5 IObtain the L-length xi, i = 1, 2 frames by using (9). 6 IGenerate random Rayleigh channel coefficients (hλ, λ = s1, s2) and generate random AWGN process (nλ, λ = s1, s2). 7 IAccording to (10), calculate the yλ, λ = s1, s2 with L length and then based on the yλ, reform the Ld -length yI λ,p, y Q λ,p,yI λ,d , y Q λ,d by using (13) and (19). 8 I Implement the DLDet at the UE1 and detect O x (1) 2,d as given in (14). Generate Lp-length xp known-pilot symbols at the UE1. Then, obtain the transmitted frame in the second phase given in (11). 9 IGenerate random Rayleigh channel coefficient hr and the random AWGN process (nr ). 10 IAccording to (12), calculate the yr . Then, with respect to yr , obtain yI r,p, y Q r,p,yI r,d and y Q r,d by using (19) 11 end 12 end Result: XI 2,X Q 2 ,YI s2,p,Y Q s2,p,YI s2,d ,Y Q s2,d , YI r,p,Y Q r,p,YI r,d ,Y Q r,d the dataset rather than learning. Likewise, if the mini batch size is chosen too low, the learning may not be performed. In other words, unless too small or too large parameters are chosen, the training procedure will eventually converge to almost the same performance regardless of these parameters. This determines mostly the training time. However, if too small or too large parameters are chosen, the training may not be completed successfully. Similar explanations are also given in [41] as: ‘‘LSTM works well over a broad range of parameters such as learning rate, input gate bias, and output gate bias. However, a large learning rate pushes the output gates towards zero, thus automatically countermand- ing its own negative effects.’’ Based on these discussions and the results given in Table 2 and Table 3 (obtained by Algorithm 2 and Algorithm 4), we optimize the parameters as given in Table 1. Algorithm 4: Training and Parameter Optimization of DLDet for UE2 Data: XI 2,X Q 2 ,YI s2,p,Y Q s2,p,YI s2,d ,Y Q s2,d ,YI r,p, Y Q r,p,YI r,d ,Y Q r,d 1 Initialize DL Parameters (mini batch size, learning rate, maximum epoch) 2 ITrain the network according to P5 in (21). 3 if The P6 in (22) does not converge then 4 IUpdate DL parameters and repeat Step 2 5 end Result: The DLDet at UE2, optimized DL parameters and X̂I 2,d ,X̂ Q 2,d To further present the effects of the DL parameters on the training, we present BER performances of the DLDet at UE1 and UE2 in Fig. 3 and Fig. 4, respectively. In Fig. 3, we present BER performances for detecting UE1’s and UE2’s symbols with respect to transmit SNR for five different training DL parameter sets (i.e., Settings numbers 9, 18, 21, 25 and 26) from Tables 2 and 3. These five DL parameter sets are selected in a way that they have the closest performances to the selected setting. Likewise, in Fig. 4, we present BER performance at the UE2 for the same DL parameter sets. In both figures, we can easily see that the selected DL parameters (i.e., Setting number 21) provide the best BER performances (training performance) in overall, although some of the other DL parameters could give better performance for only specific cases (e.g., SNR or only one signal detection). E. COMPLEXITY The proposed networks are trained offline and then are implemented as online detection algorithms. The online implementation complexity is more meaningful. An LSTM cell is local in space and time; its computational complexity per time step and weight is O(1) [41]. Thus, the feed-forward calculation (implementation) complexity is obtained by the number of weights (i.e., W) in the hidden layer. Thus, the end- to-end implementation complexity of the DLDet is given by O(W1W2) = O(30 × 10). Then, for comparisons, we calculate the computational complexity for conventional estimation/detection algorithms. On the one hand, as given in Section II.C, the UE1 should implement a channel estimation algorithm to acquire CSI, an ML detector to detect UE2’s symbols and an SIC to detect own symbols. According to [44], the computational complexity of an MMSE channel estimation algorithm can be found as O(3Lp + 4). The computational complexities of an ML detection and SIC are given by O(4M2) and O(4M1 +2), respectively, where Mi, i = 1, 2 is the modulation order for the users’ symbols. Thus, the total complexity for UE1 is obtained as O(3Lp +4(M1 +M2)+6). It is noteworthy that this total complexity is given for conventional C-NOMA, if the TBS-C-NOMA is implemented to increase the error performance, additional complexity will VOLUME 9, 2021 61843

45. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA FIGURE 3. BER comparisons at the UE2 for different DL parameter settings (given setting numbers in Tables 2 and 3) in the training. be introduced to obtain the optimal threshold value along with latency to acquire CSI knowledge of all links. On the other hand, the UE2 should implement two times channel estimation algorithms to acquire CSI (i.e., one for between BS-UE1 and one for between UE1-UE2), an MRC technique and an ML detector. The computational complexity of the MRC could be simplified as O(3). Therefore, the total computational complexity for UE2 in conventional schemes turns out to be O(6Lp + 4M2 + 11). Based on the above comparisons, the computational complexity of the DLDet might be a bit higher than the conventional detectors, especially when the modulation orders and pilot size are relatively low. However, one can easily see that the complexity of DLDet is fixed, whereas the complexity of the conventional detectors is increased by the length of pilot size and modulation orders. Besides, as it will be discussed in detail in the next section, the BER performance of the conventional detectors is severe when low pilot sizes are used. Thus, the pilot size should be increased so that similar computational complexities are FIGURE 4. BER comparisons at the UE1 for different DL parameter settings (given setting numbers in Tables 2 and 3) in the training. required for both proposed and conventional methods. More- over, the proposed DLDet outperforms conventional detection/estimation algorithms significantly even if very high pilot sizes are used for conventional detectors. Therefore, we can afford this slight increase in complexity in return for considerable performance gain (i.e., up to ∼ 10 dB in some cases). Lastly, with the last technological developments nowadays, even mobile devices have very high computational capacity; hence, the proposed DLDet can be easily implemented in end-devices. IV. SIMULATION RESULTS In this section, we implement the proposed DLDet for both users and perform experiments on synthetic datasets. In the simulations, we use a PC with an Intel Core i7-9750H CPU and an Nvidia Quadro P6000 GPU for acceleration on training and it shows that the DLDet can be trained and implemented on an ordinary laptop PC. In the simulations, we use an offline training and an online implementation. Once the training is completed, the proposed DLDet network models are used as online detectors for the users instead of the conventional detectors.8 The link-level Monte-Carlo simulations are conducted on MATLAB with 106 iterations for each scenario to validate the performance comparisons with the existing C-NOMA schemes in terms of BER. In all figures, we assume that the power of the relay (near user) is equal to half of the power of BS (i.e., Pr = (Ps/2)). We present the simulations in two parts according to the fading conditions, where in the first section we evaluate the performances over 8In the practical scenarios, the mobile users are moving so that the near user (UE1) and far user (UE2) may change from time to time. Since both users should implement different detector schemes (e.g., SIC in UE1 whereas MRC+ML in UE2), this information should be conveyed to the users whether they are the near or far users. This information is transferred to the users with Forward Control Channel (FCC). Likewise the conventional C-NOMA, with the proposed DLDet scheme, according to FCC signal, the users choose one of the offline-trained DLDets in the online implementation. Since we use exactly the same frame and input representations, both users could use the correct DLDet scheme without any additional process. 61844 VOLUME 9, 2021

46. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA TABLE 4. Channel Conditions in the Simulations. Rayleigh fading channels by using the same and different channel conditions as those of the training. Nevertheless, it is important to note that the channel fading coefficients in the simulations are newly randomly generated and not the same in the training dataset. Then, in the second subsection, we use the proposed DLDet over different fading channels (i.e., Nakagami-m and Rician). In all simulations, in case the perfect CSI is not available for the conventional detector, we implement the minimum mean square error (MMSE) channel estimation [44] which has the best channel estimation performance within the conventional channel estimation algorithms. Nevertheless, it is worthy noting that the MMSE is implemented based on that the second-order statistics of the channels are known, whereas the proposed DLDet does not require this and provides a semi-blind symbol detection simultaneously. Besides, in all simulations, to reveal the effect of pilot size on the performances of both the proposed DLDet and conventional detectors, we present results for various pilot-to-frame size ratios, where φ = (Lp/L) is defined. Unless otherwise is stated, we use L = 64 in all simulations. A. RAYLEIGH FADING CHANNELS In this subsection, we implement the proposed DLDet for C-NOMA over Rayleigh fading channels as being in the offline training procedure. We present simulation results for various channel scenarios where one of them (i.e., Scenario I) has the same second-order statistics (i.e., E[|hλ|2], λ = s1, s2, r) with the training and two of them have different (i.e., Scenario II and Scenario III). The channel conditions in each scenario are given in Table 4. In the given scenarios, channel qualities are chosen equal (i.e., Scenario I and Scenario III) and different (i.e., Scenario II) to represent the practical scenarios where the second-order statistics change mainly based on the distance between nodes (see Footnote 2). 1) THE SAME SECOND-ORDER STATISTICS WITH THE TRAINING Firstly, we present BER comparisons of both symbols at the UE1 for Scenario I in Fig. 5. One can easily see that the proposed DLDet at UE1 performs well in detecting both symbols jointly. Especially, with the low pilot-to-frame ratio (φ), the proposed DLDet outperforms conventional estimation plus detection algorithms substantially. This is quite promising since a pilot-assisted (semi-blind) joint symbol detection can be performed simultaneously with a very low pilot signal size; therefore, the achievable capacity is not reduced due to pilot signals. Besides, an iterative detection (i.e., SIC) is not required anymore since the DLDet performs a joint symbol detection. In both figures, we also provide the BER results for the perfect CSI case where it is seen that the performances of both methods converge. This is compatible with the results in Tables 2 and 3 and proves that the proposed Algorithm 2 not only optimizes the results in training but also in online implementation. Furthermore, the DLDet outperforms the conventional detector in detecting UE2 symbols at the UE1 regardless of the pilot-to-frame ratio or perfect CSI case. This improvement in the BER performance of UE2 symbols is quite important to reduce the error propagation in C-NOMA [9]. In this paper, we aim to improve the BER performance of C-NOMA. Therefore, after this point, we will not provide the BER results at the UE1 since Fig. 5 already proves that the DLDet at UE1 provides remarkable BER performances for both users’ symbols. Then in Fig. 6, we provide BER comparisons for the UE2 in Scenario I. In the comparisons, we present BER performances of conventional detectors for both C-NOMA [9] and TBS-C-NOMA [10]. One can easily see that the DLDet outperforms C-NOMA regardless of the CSI case or φ. This performance improvement is more than 10 dB. In other words, the DLDet achieves the same BER performance of C-NOMA with 10 dB less power consumption, which is a superb gain by considering the energy-limited networks or green communication concept in the future wireless networks. This proves the power of the DLDet in detecting cooperative signals even though an error propagation exists in the cooperative phase. Besides, this detecting is performed based on the pilot signals simultaneously. This shows that the proposed DLDet is capa- ble of a semi-blind detection of cooperative signals without any additional channel estimation and/or combining techniques. On the other hand, the DLDet also outperforms the VOLUME 9, 2021 61845

47. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA FIGURE 5. BER comparisons between conventional detectors and proposed DLDet at UE1 for Scenario I. conventional detector in TBS-C-NOMA. The performance improvement in this case changes between ∼ 2−6 dB according to φ. This is lower compared to the gain over conventional C-NOMA. However, we should note that the TBS-C-NOMA performances are obtained by using the optimum threshold values (in terms of error performance) in [10] and it is clear that this performance improvement will increase when the threshold values are optimized by considering all performance metrics [11]. Besides, it is utmost to note that there is a trade-off in TBS-C-NOMA where the BER performance is increased with the penalty of capacity and outage degradation compared to the C-NOMA [11]. Furthermore, to determine the threshold at the UE1, a signaling overhead is introduced where the channel conditions of UE2 is required. Therefore, outperforming the TBS-C-NOMA makes more impact since the DLDet implements a conventional C-NOMA without introducing a threshold at the UE1. Hence, the BER performance improvement is guaranteed without paying any cost in terms of capacity/outage performance or signaling overhead. FIGURE 6. BER comparisons between conventional detectors and proposed DLDet at UE2 for Scenario I. To investigate the effect of the frame size (i.e., L), in Fig. 7 and Fig. 8, we present the same performances evaluations of Fig. 5 and Fig. 6 for three frame sizes (i.e., L = 32, 64, 128) and various pilot-to-frame ratios (i.e., φ). It is noteworthy that the DLDet for both users is trained offline with L = 64 and φ = (1/L). Nevertheless, it can be used for any frame size and/or pilot-to-frame ratio. This is achieved thanks to the frame design and input representation (given in Section III.A) since with the reforming steps in the input design, we guarantee all inputs to have the same length with the outputs (data symbol length (Ld )) and this provides a flexible online implementation. In the light of these explanations, in Fig. 7 and Fig. 8, one can easily see that the frame size does not change the performance much for both users. Nevertheless, as expected, with the increase of φ, the performance is improved. This is explained as follows. By increasing φ, the pilot response inputs (i.e., yI λ,p, y Q λ,p) have more informative knowledge on the exposed AWGN so that the higher correlation is achieved and the performance is increased. It is also a known phonema for the conventional estimation/detection algorithms. As seen in Fig. 7 and Fig. 8, the frame size does not affect the performance much. Thus, for the illustration simplicity, after this point, we provide simulation results for only L = 64. Besides, as seen in Fig. 6, since the performance gaps between the proposed DLDet and the conventional detectors in C-NOMA are very high, again for better illustration, we present comparisons for only TBS-C-NOMA in the fol- lowing figures. Nevertheless, for a reference point, we still give the results of C-NOMA for only perfect CSI. 2) DIFFERENT SECOND-ORDER STATISTICS THAN THE TRAINING Then, to reveal the robustness of the DLDet to the various channel conditions, we present BER comparisons for the UE2 in Scenarios II and III (which were not included in training) in Fig. 9 and Fig. 10, respectively. As seen in both 61846 VOLUME 9, 2021

48. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA FIGURE 7. BER comparisons of the DLDet at UE1 for different frame size (L) and pilot-to-frame ratio (φ) in Scenario I. FIGURE 8. BER comparisons of the DLDet at UE2 for different frame size (L) and pilot-to-frame ratio (φ) in Scenario I. figures, the DLDet still outperforms conventional C-NOMA remarkably even though the results of conventional C-NOMA are given in perfect CSI. The DLDet is also superior to the TBS-C-NOMA in all pilot-to-frame size ratio (φ) scenarios. FIGURE 9. BER comparisons between conventional detectors and proposed DLDet at UE2 for Scenario II. FIGURE 10. BER comparisons between conventional detectors and proposed DLDet at UE2 for Scenario III. The performance gain is relatively more with the low φ. This again shows that the DLDet performs well in semi-blind joint channel estimation and symbol detection with a very low pilot usage which ensures the capacity improvement. Indeed, the DLDet can outperform the TBS-C-NOMA even with the perfect CSI, although a semi-blind detection is performed in DLDet with very low φ (for only φ ≥ (1/16)). In both figures, it is clearly seen that the DLDet outperforms the TBS-C- NOMA; hence, the above discussions in Scenario I and the provided advantages by the proposed DLDet are still valid when the channel conditions change. In addition, as seen in both figures, the full diversity order (i.e., 2) is achieved by the DLDet, although it can not be observed in conventional C-NOMA due to the error propagation. The performance improvements in Fig. 9 and Fig. 10 show that the proposed DLDet can be used as an online detection algorithm although the channel conditions change once it is trained offline. This robustness against the channel conditions is ensured thanks to VOLUME 9, 2021 61847

49. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA FIGURE 11. BER comparisons between conventional detectors and proposed DLDet at UE2 for Scenario IV. the power of LSTM networks and our frame design and representation. As it is discussed in detail in Section III, we have such a frame design and input/output representation in the training that the LSTM-based DLDet can learn dependencies between inputs and within frame. The DLDet can adapt any channel condition even though it is not trained on that channel condition. Thus, the DLDet could be preferred as an online detection/estimation algorithm as long as the same frame structure and input representations with those of training are used in the online implementation. B. IMPLEMENTATION OVER DIFFERENT FADING CHANNELS In this subsection, we implement the DLDet for C-NOMA over different fading channels although it is trained with Rayleigh fading channel in the offline training. We present the result for Nakagami-m and Rician fading channels. The channel conditions for those fading channels are given in Table 4. In Table 4, λ and mλ denote the spread and shape parameters for Nakagami-m fading channels where λ = E[|hλ|2] and mλ = 2 λ Var[|hλ|2] are defined. For Rician fading channel, Kλ denotes the Rician factor (i.e., line-of-sight-to-multi-path power ratio) and λ = E[|hλ|2] is the scaling factor. In the given scenarios, as being in Rayleigh fading environments, we cover both equal (i.e., Scenarios IV, VI, VII, and IX) and different (i.e., Scenarios V and VII) channel qualities to represent the practical distance-dependent (see Footnote 2) scenarios. In the comparisons, we present the performances of TBS-C-NOMA with optimum threshold values of Rayleigh fading channels given in [10] since there is no work which provides the optimum values over Nakagami-m and Rician fading channels. Nevertheless, we have empirically tried other threshold values and observed that these values are most close to the optimum for also Nakagami-m and Rician fading channels. FIGURE 12. BER comparisons between conventional detectors and proposed DLDet at UE2 for Scenario V. FIGURE 13. BER comparisons between conventional detectors and proposed DLDet at UE2 for Scenario VI. In Figs. 11-13, we present comparisons over Nakagami-m fading channels in Scenarios IV, V, and IV, respectively. In all figures, one can easily see that the DLDet still outperforms conventional detection algorithms although it is not trained/optimized for Nakagami-m fading channels. For all scenarios, the performance gain of the DLDet over conventional C-NOMA is impressive even though the perfect CSI is assumed in conventional C-NOMA. The DLDet can beat the conventional C-NOMA for any φ value. This means that the DLDet is better than the conventional C-NOMA even with only one pilot signal. In Fig. 11 and Fig. 12, m = 0.5 represents the worse channel conditions than Rayleigh fading whereas in Fig. 13, m = 2 represents better channel condition. One can easily see that the DLDet is also better than TBS-C-NOMA. The performance gain over TBS-C-NOMA is between ∼ 3−5 dB, ∼ 1−3 dB and ∼ 5−8 dB in Fig. 11, Fig. 12 and Fig. 13, respectively. This reveals the power of the DLDet and shows that we can achieve better BER 61848 VOLUME 9, 2021

50. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA FIGURE 14. BER comparisons between conventional detectors and proposed DLDet at UE2 for Scenario VII. FIGURE 15. BER comparisons between conventional detectors and proposed DLDet at UE2 for Scenario VIII. performance than TBS-C-NOMA without paying any cost (i.e., capacity/outage performance decay and/or signaling overhead) introduced by the TBS-C-NOMA. Especially with the low pilot-to-frame ratio (i.e., φ) usage, the performance gain of DLDet becomes greater. The only case that the TBS-C-NOMA has similar performance to the DLDet is the Scenario V with a very high pilot-to-frame ratio (φ = (1/2)). However, this use-case is not effective due to the high pilot- to-frame ratio (since it decreases informative (data) spectral efficiency) and the DLDet has a superb gain in any other other pilot-to-frame ratio usage. Indeed, with the increase of m (see Fig. 13) parameter, the DLDet outperforms even the perfect CSI case in TBS-C-NOMA notably with all pilot-to-frame size ratio (except φ = (1/64)). When we compare the results within Fig. 11-Fig. 13, the DLDet achieves the full diversity order (i.e., 2m); thus, the Scenario VI in Fig. 13 has the best performance. On the other hand, as expected, when the channel qualities are improved, the Scenario V in Fig. 12 has FIGURE 16. BER comparisons between conventional detectors and proposed DLDet at UE2 for Scenario IX. better performance than the Scenario IV in Fig. 11. Never- theless, this improvement is a horizontal gain and provides no additional diversity since the diversity order is related to m parameter. Lastly, we present BER comparisons over also Rician fading channels for Scenarios VII, VIII, and IX in Fig. 14, Fig. 15 and Fig. 16, respectively. As being in all above comparisons, the DLDet performs better than conventional C-NOMA for all φ. In this regard, we can easily say that the DLDet is superior to the conventional C-NOMA regardless of the channel conditions, although the perfect CSI is assumed in conventional C-NOMA. Again, the DLDet can also outperform TBS-C-NOMA over Rician fading channels. Since the Rician fading channel represents better channel conditions than Rayleigh fading channel (i.e., K = 0 → Rayleigh), the performance gain of DLDet gets better over Rician fading channels as being in Nakagami-m simulations (for m = 2 in Fig. 13). In addition, likewise in Nakagami-m simulations in Fig. 11- Fig. 13, with the increase of K (see Fig. 16), the DLDet can achieve better performance than the perfect case of the TBS-C-NOMA for all pilot-to- frame size ratio (φ). This is quite promising. Based on the evaluations in Fig. 11-Fig. 16, one can easily see that the DLDet, as an online detection algorithm, is completely robust to different fading conditions, although it is only trained for Rayleigh fading channel conditions in the offline training stage. This, as repeated above, is achieved by our frame representations and the power of the LSTM networks in learning dependencies/correlation within inputs/data-frame. V. CONCLUSION AND FUTURE WORKS In this paper, we propose a data-driven DL network to detect users’ symbols jointly in C-NOMA. The proposed DL-based detection (DLDet) is trained to be able to detect both users’ symbols jointly at the near user (UE1) without an SIC process. Besides, the same network is also optimized to detect VOLUME 9, 2021 61849

51. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA cooperative signals at the far user (UE2) instead of using and MRC plus ML. Furthermore, the proposed DLDet is able to detect symbols jointly without acquiring an additional channel estimation algorithm since it can perform a simulta- neous detection based on the pilot responses, thus providing a semi-blind joint detection. Besides, this is achieved with a very low pilot-to-frame size ratio. In both users, the DLDet outperforms the conventional detectors. Indeed, the performance increase for the UE2 is superb over C-NOMA (i.e., ∼ 10 dB) and TBS-C-NOMA (i.e., ∼ 3−7 dB) over Rayleigh fading channels. This less power consumption is very crucial for the green communication concept in 5G and beyond networks. Moreover, outperforming the TBS-C-NOMA without introducing a threshold is quite important since the TBS-C-NOMA introduces a signaling overhead and capacity performance decay. These problems have been resolved by the DLDet. Moreover, we implement the offline-trained DLDet over different fading channels (i.e., Nakagami-m and Rician). With extensive simulations, we reveal that the DLDet still outperforms conventional detectors over different fading channels and provides the full diversity order (i.e., 2 for Rayleigh and Rician, and 2m for Nakagami-m ) although the offline trained is completed with only limited Rayleigh fading conditions. This reveals the robustness of the DLDet to different fading conditions. In other words, the offline- trained DLDet can be used as an online detection algorithm regardless of the fading conditions once it is trained offline for any fading channel. Based on the specific results of this paper, we believe that the proposed DLDet could be further improved to cover other aspects of the communications systems. To this end, we summarize the future works as follows. Practical Implementation: As discussed in this paper, the pre-trained DLDet performs well under various channel conditions although it is only trained for Rayleigh channel conditions. Therefore, the proposed DLDet can be used an online detection algorithm in any practical cooperative-NOMA use-case as long as the same structure (e.g., frame design and input/output representations) is used for the transmission. To this end, an online testbed (e.g., software defined radio (SDR)) implementation [45] of the pre-trained DLDet is considered a future work. Coded Scenarios: In this paper, we consider an uncoded C-NOMA scheme and prove that the DLDet outperforms the conventional estimation/detection algorithms. On the other hand, the power of the DL networks has been also demonstrated in coded scenarios [46]. However, the existing coded works consider only a point-to-point communication. Thus, the extension for much complex networks such as C-NOMA (an interference due to the NOMA and two phases due to cooperative communication are included) requires more effort. The coded scenario was beyond the scope of this paper, but, it is one of the main directions of the future works. With a true dataset and training, we believe that the DLDet could be extended for all coded cooperative and NOMA schemes such as LPDC, Turbo and Polar coding. Fully-Blind Detection: In this paper, to improve the performance of the DLDet, we perform a semi-blind multi- user detection by using pilot responses such that giving a channel-informative input to the network. However, the pilot signals cause an inefficient resource usage due to non-informative data transmission. Although the proposed DLDet outperforms the existing algorithms with a very low pilot-to-frame ratio (i.e., non-informative-to-informative data ratio), the question may raise that could we perform detection without any pilot signal? For now, we would answer that question for the DLDet as: No. Nevertheless, the recent initial results for fully-blind detection are promising. In [47], the authors demonstrated that a DL-aided fully-blind detection is possible for a point-to-point communication. How- ever, as discussed in the coded scenarios, the extensions for such complex networks (e.g., C-NOMA) are still to be explored. Besides, for a fully-blind detection, an autoencoder and autodecoder are required rather than only a detection solution. Thus, the transmitter should be also re-designed for a fully-blind detection. This is the another aspect of the future works. DL-aided Detection for Other Physical Layer Techniques: This paper reveals the power of the DLDet for signal detection in C-NOMA where an interference due to the NOMA and an error propagation from near user to far user exist. To this end, the DLDet can be used in other NOMA-involved systems such as NOMA-based mobile edge computing and reconfigurable intelligent surfaces (RIS)/backscatter communication (BCOM) assisted-NOMA schemes [48], [49]. Besides, the proposed DLDet can also be applied for other interference-introduced schemes. Hence, a lightweight DL-aided detection for faster than Nyquist signaling (FTNS) [50] is to be next step of this paper. REFERENCES [1] A. Osseiran, F. Boccardi, V. Braun, K. Kusume, P. Marsch, M. Maternia, O. Queseth, M. Schellmann, H. Schotten, H. Taoka, H. Tullberg, M. A. Uusitalo, B. Timus, and M. Fallgren, ‘‘Scenarios for 5G mobile and wireless communications: The vision of the METIS project,’’ IEEE Commun. Mag., vol. 52, no. 5, pp. 26–35, May 2014. [2] M. Agiwal, A. Roy, and N. Saxena, ‘‘Next generation 5G wireless networks: A comprehensive survey,’’ IEEE Commun. Surveys Tuts., vol. 18, no. 3, pp. 1617–1655, 3rd Quart., 2016. [3] F. Boccardi, R. W. Heath, Jr., A. Lozano, T. L. Marzetta, and P. Popovski, ‘‘Five disruptive technology directions for 5G,’’ IEEE Commun. Mag., vol. 52, no. 2, pp. 74–80, Feb. 2014. [4] L. Dai, B. Wang, Y. Yuan, S. Han, I. Chih-Lin, and Z. Wang, ‘‘Non- orthogonal multiple access for 5G: Solutions, challenges, opportunities, and future research trends,’’ IEEE Commun. Mag., vol. 53, no. 9, pp. 74–81, Sep. 2015. [5] Y. Liu, Z. Qin, M. Elkashlan, Z. Ding, A. Nallanathan, and L. Hanzo, ‘‘Nonorthogonal multiple access for 5G and beyond,’’ Proc. IEEE, vol. 105, no. 12, pp. 2347–2381, Dec. 2017. [6] A. Benjebbour, A. Li, Y. Kishiyama, H. Jiang, and T. Nakamura, ‘‘System- level performance of downlink NOMA combined with SU-MIMO for future LTE enhancements,’’ in Proc. IEEE Globecom Workshops (GC Wkshps), no. 1, Dec. 2014, pp. 706–710. [7] M. Vaezi, G. A. A. Baduge, Y. Liu, A. Arafa, F. Fang, and Z. Ding, ‘‘Interplay between NOMA and other emerging technologies: A survey,’’ IEEE Trans. Cognit. Commun. Netw., vol. 5, no. 4, pp. 900–919, Dec. 2019. 61850 VOLUME 9, 2021

52. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA [8] Z. Ding, M. Peng, and H. V. Poor, ‘‘Cooperative non-orthogonal multiple access in 5G systems,’’ IEEE Commun. Lett., vol. 19, no. 8, pp. 1462–1465, Aug. 2015. [9] F. Kara and H. Kaya, ‘‘On the error performance of cooperative- NOMA with statistical CSIT,’’ IEEE Commun. Lett., vol. 23, no. 1, pp. 128–131, Jan. 2019. [Online]. Available: https://ieeexplore. ieee.org/document/8515248/ [10] F. Kara and H. Kaya, ‘‘Threshold-based selective cooperative-NOMA,’’ IEEE Commun. Lett., vol. 23, no. 7, pp. 1263–1266, Jul. 2019. [Online]. Available: https://ieeexplore.ieee.org/document/8707000/ [11] F. Kara and H. Kaya, ‘‘Threshold-based selective cooperative NOMA: Capacity/outage analysis and a joint power allocation- threshold selection optimization,’’ IEEE Commun. Lett., vol. 24, no. 9, pp. 1929–1933, Sep. 2020. [Online]. Available: https://ieeexplore. ieee.org/document/9097857/ [12] F. Kara, H. Kaya, O. Erkaymaz, and F. Ozturk, ‘‘Prediction of the optimal threshold value in DF relay selection schemes based on artificial neural networks,’’ in Proc. Int. Symp. Innov. Intell. Syst. Appl. (INISTA), Aug. 2016, pp. 1–6. [Online]. Available: http://ieeexplore.ieee. org/document/7571823/ [13] H. Ye, G. Y. Li, and B.-H. Juang, ‘‘Power of deep learning for channel estimation and signal detection in OFDM systems,’’ IEEE Wireless Commun. Lett., vol. 7, no. 1, pp. 114–117, Feb. 2018. [14] H. He, C.-K. Wen, S. Jin, and G. Y. Li, ‘‘Deep learning-based channel estimation for beamspace mmWave massive MIMO systems,’’ IEEE Wireless Commun. Lett., vol. 7, no. 5, pp. 852–855, Oct. 2018. [15] S. Dorner, S. Cammerer, J. Hoydis, and S. T. Brink, ‘‘Deep learning based communication over the air,’’ IEEE J. Sel. Topics Signal Process., vol. 12, no. 1, pp. 132–143, Feb. 2018. [16] H. Kim, S. Oh, and P. Viswanath, ‘‘Physical layer communication via deep learning,’’ IEEE J. Sel. Areas Inf. Theory, vol. 1, no. 1, pp. 5–18, May 2020. [17] T. Wang, C.-K. Wen, H. Wang, F. Gao, T. Jiang, and S. Jin, ‘‘Deep learning for wireless physical layer: Opportunities and challenges,’’ China Commun., vol. 14, no. 11, pp. 92–111, Nov. 2017. [18] T. O’Shea and J. Hoydis, ‘‘An introduction to deep learning for the physical layer,’’ IEEE Trans. Cognit. Commun. Netw., vol. 3, no. 4, pp. 563–575, Dec. 2017. [19] Z. Qin, H. Ye, G. Y. Li, and B.-H.-F. Juang, ‘‘Deep learning in physical layer communications,’’ IEEE Wireless Commun., vol. 26, no. 2, pp. 93–99, Apr. 2019. [20] H. He, S. Jin, C.-K. Wen, F. Gao, G. Y. Li, and Z. Xu, ‘‘Model-driven deep learning for physical layer communications,’’ IEEE Wireless Commun., vol. 26, no. 5, pp. 77–83, Oct. 2019. [21] J. Ren and Z. Wang, ‘‘A novel deep learning method for application identi- fication in wireless network,’’ China Commun., vol. 15, no. 10, pp. 73–83, Oct. 2018. [22] A. Zappone, M. Di Renzo, and M. Debbah, ‘‘Wireless networks design in the era of deep learning: Model-based, AI-based, or both?’’ IEEE Trans. Commun., vol. 67, no. 10, pp. 7331–7376, Oct. 2019. [23] T. V. Luong, Y. Ko, N. A. Vien, D. H. N. Nguyen, and M. Matthaiou, ‘‘Deep learning-based detector for OFDM-IM,’’ IEEE Wireless Commun. Lett., vol. 8, no. 4, pp. 1159–1162, Aug. 2019. [24] H. Huang, Y. Song, J. Yang, G. Gui, and F. Adachi, ‘‘Deep-learning-based millimeter-wave massive MIMO for hybrid precoding,’’ IEEE Trans. Veh. Technol., vol. 68, no. 3, pp. 3027–3032, Mar. 2019. [25] M. Kim, N.-I. Kim, W. Lee, and D.-H. Cho, ‘‘Deep learning-aided SCMA,’’ IEEE Commun. Lett., vol. 22, no. 4, pp. 720–723, Apr. 2018. [26] N. Ye, X. Li, H. Yu, A. Wang, W. Liu, and X. Hou, ‘‘Deep learning aided grant-free NOMA toward reliable low-latency access in tactile Internet of Things,’’ IEEE Trans. Ind. Informat., vol. 15, no. 5, pp. 2995–3005, May 2019. [27] N. Ye, X. Li, H. Yu, L. Zhao, W. Liu, and X. Hou, ‘‘Deep- NOMA: A unified framework for NOMA using deep multi-task learning,’’ IEEE Trans. Wireless Commun., vol. 19, no. 4, pp. 2208–2225, Apr. 2020. [28] M. Choi, D. Yoon, and J. Kim, ‘‘Blind signal classification for non- orthogonal multiple access in vehicular networks,’’ IEEE Trans. Veh. Tech- nol., vol. 68, no. 10, pp. 9722–9734, Oct. 2019. [29] N. Zhang, K. Cheng, and G. Kang, ‘‘A machine-learning-based blind detection on interference modulation order in NOMA systems,’’ IEEE Commun. Lett., vol. 22, no. 12, pp. 2463–2466, Dec. 2018. [30] J. Zhang, X. Tao, H. Wu, N. Zhang, and X. Zhang, ‘‘Deep reinforcement learning for throughput improvement of the uplink grant-free NOMA system,’’ IEEE Internet Things J., vol. 7, no. 7, pp. 6369–6379, Jul. 2020. [31] N. Yang, H. Zhang, K. Long, H.-Y. Hsieh, and J. Liu, ‘‘Deep neural network for resource management in NOMA networks,’’ IEEE Trans. Veh. Technol., vol. 69, no. 1, pp. 876–886, Jan. 2020. [32] M. Liu, T. Song, and G. Gui, ‘‘Deep cognitive perspective: Resource allocation for NOMA-based heterogeneous IoT with imperfect SIC,’’ IEEE Internet Things J., vol. 6, no. 2, pp. 2885–2894, Apr. 2019. [33] G. Gui, H. Huang, Y. Song, and H. Sari, ‘‘Deep learning for an effective nonorthogonal multiple access scheme,’’ IEEE Trans. Veh. Technol., vol. 67, no. 9, pp. 8440–8450, Sep. 2018. [34] M. AbdelMoniem, S. M. Gasser, M. S. El-Mahallawy, M. W. Fakhr, and A. Soliman, ‘‘Enhanced NOMA system using adaptive coding and modulation based on LSTM neural network channel estimation,’’ Appl. Sci., vol. 9, no. 15, pp. 1–19, Jul. 2019. [35] Narengerile and J. Thompson, ‘‘Deep learning for signal detection in non- orthogonal multiple access wireless systems,’’ in Proc. UK/China Emerg. Technol. (UCET), Aug. 2019, pp. 1–4. [36] A. Emir, F. Kara, and H. Kaya, ‘‘Deep learning detectors with pilot interval reduction in uplink non orthogonal multiple access,’’ in Proc. 28th Signal Process. Commun. Appl. Conf. (SIU), Gaziantep, Turkey, Oct. 2020, pp. 1–4. [Online]. Available: https://ieeexplore.ieee. org/abstract/document/9302252 [37] A. Emir, F. Kara, and H. Kaya, ‘‘Deep learning-based joint symbol detection for NOMA,’’ in Proc. 27th Signal Process. Commun. Appl. Conf. (SIU), Sivas, Turkey, Apr. 2019, pp. 1–4. [Online]. Available: https: //ieeexplore.ieee.org/document/8806600/ [38] J.-M. Kang, I.-M. Kim, and C.-J. Chun, ‘‘Deep learning-based MIMO- NOMA with imperfect SIC decoding,’’ IEEE Syst. J., vol. 14, no. 3, pp. 3414–3417, Sep. 2020. [39] C. Lin, Q. Chang, and X. Li, ‘‘A deep learning approach for MIMO- NOMA downlink signal detection,’’ Sensors, vol. 19, no. 11, p. 2526, Jun. 2019. [40] A. Emir, F. Kara, H. Kaya, and H. Yanikomeroglu, ‘‘DeepMuD: Multi- user detection for uplink grant-free NOMA IoT networks via deep learning,’’ IEEE Wireless Commun. Lett., early access, Feb. 19, 2021. [Online]. Available: https://ieeexplore.ieee.org/document/9359664, doi: 10.1109/LWC.2021.3060772. [41] S. Hochreiter and J. Schmidhuber, ‘‘Long short-term memory,’’ Neural Comput., vol. 9, no. 8, pp. 1735–1780, Nov. 1997. [42] F. Kara and H. Kaya, ‘‘Error probability analysis of non-orthogonal multiple access with channel estimation errors,’’ in Proc. IEEE Int. Black Sea Conf. Commun. Netw. (BlackSeaCom), Odyssey, Ukraine: IEEE, May 2020, pp. 1–5. [Online]. Available: https://ieeexplore. ieee.org/document/9234956 [43] F. Kara and H. Kaya, ‘‘Improved user fairness in decode-forward relay- ing non-orthogonal multiple access schemes with imperfect SIC and CSI,’’ IEEE Access, vol. 8, pp. 97540–97556, 2020. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/9099559 [44] Y. Li, ‘‘Pilot-symbol-aided channel estimation for OFDM in wireless systems,’’ IEEE Trans. Veh. Technol., vol. 49, no. 4, pp. 1207–1215, Jul. 2000. [45] S. Gecgel, C. Goztepe, and G. K. Kurt, ‘‘Transmit antenna selection for large-scale MIMO GSM with machine learning,’’ IEEE Wireless Commun. Lett., vol. 9, no. 1, pp. 113–116, Jan. 2020. [46] Y. He, J. Zhang, S. Jin, C.-K. Wen, and G. Y. Li, ‘‘Model-driven DNN decoder for turbo codes: Design, simulation, and experimen- tal results,’’ IEEE Trans. Commun., vol. 68, no. 10, pp. 6127–6140, Oct. 2020. [47] H. Ye, G. Y. Li, and B.-H.-F. Juang, ‘‘Deep learning based End- to-End wireless communication systems without pilots,’’ IEEE Trans. Cognit. Commun. Netw., early access, Feb. 23, 2021, doi: 10.1109/ TCCN.2021.3061464. [48] T. Hou, Y. Liu, Z. Song, X. Sun, Y. Chen, and L. Hanzo, ‘‘Reconfigurable intelligent surface aided NOMA networks,’’ IEEE J. Sel. Areas Commun., vol. 38, no. 11, pp. 2575–2588, Nov. 2020. [49] J. Guo, X. Zhou, S. Durrani, and H. Yanikomeroglu, ‘‘Design of non-orthogonal multiple access enhanced backscatter communication,’’ IEEE Trans. Wireless Commun., vol. 17, no. 10, pp. 6837–6852, Oct. 2018. [50] E. Bedeer, M. H. Ahmed, and H. Yanikomeroglu, ‘‘A very low complexity successive symbol-by-symbol sequence estimator for faster-than-Nyquist signaling,’’ IEEE Access, vol. 5, pp. 7414–7422, 2017. VOLUME 9, 2021 61851

53. A. Emir et al.: DL Empowered Semi-Blind Joint Detection in C-NOMA AHMET EMIR received the B.Sc. degree in electronics and communications engineering from Kocaeli University, Turkey, in 2007, and the M.Sc. degree in electrical and electronics engineering from Zonguldak Bülent Ecevit University, Turkey, in 2014, where he is currently pursuing the Ph.D. degree in electrical and electronics engineering. Since 2012, he has been working with the Dis- tance Education Research and Application Center (DERAC), Zonguldak Bülent Ecevit University, as a Lecturer. His research interests include NOMA and machine learning in physical layer. FERDI KARA (Member, IEEE) received the B.Sc. degree (Hons.) in electronics and communication engineering from Suleyman Demirel Univer- sity, Turkey, in 2011, and the M.Sc. and Ph.D. degrees in electrical and electronics engineering from Zonguldak Bulent Ecevit University, Turkey, in 2015 and 2019, respectively. Since 2011, he has been working with the Wireless Communication Technologies Research Labora- tory (WCTLab), where he is currently a Senior Researcher. He is also a Postdoctoral Fellow with the Department of Systems and Computer Engineering, Carleton University, Ottawa, ON, Canada. His research interests include wireless communications specified with NOMA, MIMO/RIS/LIS systems, cooperative communication, index modulations, energy harvesting, faster than Nyquist signaling, aerial networks, and machine learning algorithms in communication. He was awarded the 2020 Premium Award for Best Paper in IET Communications, and the Exemplary Reviewer Certificate from the IEEE Communications Letters, in 2019 and 2020. He serves as an Editor for Physical Communication (Elsevier). HAKAN KAYA received the B.Sc., M.Sc., and Ph.D. degrees in electrical and electronics engineering from Zonguldak Bülent Ecevit University, Turkey, in 2007, 2010, and 2015, respectively. Since 2015, he has been working with Zonguldak Bülent Ecevit University as an Assistant Profes- sor, from 2015 to 2020, where he is currently an Associate Professor. He is also the Head of the Wireless Communication Technologies Research Laboratory (WCTLab). His research interests include cooperative communication, NOMA, turbo coding, and machine learning in physical layer communication. He was awarded the 2020 Pre- mium Award for Best Paper in IET Communications. HALIM YANIKOMEROGLU (Fellow, IEEE) is currently a Professor with the Department of Sys- tems and Computer Engineering, Carleton Uni- versity, Ottawa, ON, Canada. His current research interest includes many aspects of 5G/6G wireless networks. His collaborative research with industry has resulted in 37 granted patents. He is a Fel- low of the Engineering Institute of Canada (EIC) and the Canadian Academy of Engineering (CAE). He received several awards for his research, teach- ing, and service, including the IEEE Communications Society Wireless Communications Technical Committee Recognition Award, in 2018, and the IEEE Vehicular Technology Society Stuart Meyer Memorial Award, in 2020. He was the Technical Program Chair/Co-Chair of WCNC 2004, Atlanta, WCNC 2008, Las Vegas, and WCNC 2014, Istanbul, and the General Chair of IEEE VTC 2010-Fall, Ottawa, and VTC 2017-Fall, Toronto. He has also served as the Chair for the IEEE’s Technical Committee on Personal Communications. He is serving as the Chair for the IEEE Wireless Commu- nications and Networking Conference (WCNC) Steering Committee. He is also a Distinguished Speaker of IEEE Communications Society and IEEE Vehicular Technology Society. 61852 VOLUME 9, 2021

Deep_Learning_Empowered_Semi-Blind_Joint_Detection_in_Cooperative_NOMA.pdf

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Deep_Learning_Empowered_Semi-Blind_Joint_Detection_in_Cooperative_NOMA.pdf

Similar to Deep_Learning_Empowered_Semi-Blind_Joint_Detection_in_Cooperative_NOMA.pdf (20)

Recently uploaded

Recently uploaded (20)

Deep_Learning_Empowered_Semi-Blind_Joint_Detection_in_Cooperative_NOMA.pdf