sensors Article Machine Learning Techniques Applied to Multiband Spectrum Sensing in Cognitive Radios Yanqueleth Molina-Tenorio 1 , Alfonso Prieto-Guerrero 2, * , Rafael Aguilar-Gonzalez 2 and Silvia Ruiz-Boqué 3 1 Master of Sciences and Information Technologies, Metropolitan Autonomous University Iztapalapa, Mexico City 09360, Mexico; yanqueleth@xanum.uam.mx 2 Electrical Engineering Department, Metropolitan Autonomous University Iztapalapa, Mexico City 09360, Mexico; r.aguilar@xanum.uam.mx 3 Department of Signal and Theory Communications, Universitat Politècnica de Catalunya, 08860 Barcelona, Spain; silvia.ruiz@upc.edu * Correspondence: apg@xanum.uam.mx  Received: 6 September 2019; Accepted: 28 October 2019; Published: 30 October 2019  Abstract: In this work, three specific machine learning techniques (neural networks, expectation maximization and k-means) are applied to a multiband spectrum sensing technique for cognitive radios. All of them have been used as a classifier using the approximation coefficients from a Multiresolution Analysis in order to detect presence of one or multiple primary users in a wideband spectrum. Methods were tested on simulated and real signals showing a good performance. The results presented of these three methods are effective options for detecting primary user transmission on the multiband spectrum. These methodologies work for 99% of cases under simulated signals of SNR higher than 0 dB and are feasible in the case of real signals. Keywords: cognitive radios; multiband spectrum sensing; machine learning; neural networks 1. Introduction Spectrum scarcity has been an issue in past years, due to the increasing demand for wireless communications services. Several instances have demonstrated that a solution is highly necessary in order to provide spectrum spaces for future technologies. In this regard, among engineering and regulatory proposed solutions, cognitive radio (CR) has been an outstanding option. CR is based on software-defined radio (SDR) and dynamic spectrum access (DSA) technologies, both allowing to improve drastically the spectrum occupancy [1,2]. Users that consider the utilization of CR are known as secondary users (SU). They do not have assigned a spectrum space; however they have the capacity to use available spectrum spaces adapting to the environment, which provides an affordable communication service. This process should be done without provoking an interference to the licensed users, called also primary users (PU) [3]. In order to accomplish its promises, CR has to perform mainly four tasks such as spectrum sensing, spectrum sharing, spectrum decision, and spectrum mobility [4]. The primary task is spectrum sensing (SS), this activity is fundamental to detect PU presence and to determine if a spectrum space is occupied or empty. SS has been a widely studied topic, present in the literature with many different techniques, being most of them based on single band SS [5]. However, future communications services require high data rates, which can be reached only by considering non-contiguous spectrum bands. The last means that single band SS algorithms should be improved to include multiple bands, this paradigm is called multiband spectrum sensing (MSS). Sensors 2019, 19, 4715; doi:10.3390/s19214715 www.mdpi.com/journal/sensors Sensors 2019, 19, 4715 2 of 22 According to the literature, a branch of MSS techniques corresponds to wide-band spectrum sensing (WBSS) [6]. At the same time, WBSS is divided into sub-categories, for example: wavelet-based spectrum sensing, compressive sensing, and angle-based spectrum sensing among others. Several important contributions related to MSS are mentioned in [7–9]. All these methods and techniques are considered in order to improve the performance of SS and CR. Authors of this paper presented a previous work describing how to detect efficiently PU presence by considering new alternative methods [10], based on hybrid linear and non-linear techniques. In this contribution, a couple of MSS algorithms based on multiresolution analysis (MRA) and the Higuchi fractal dimension (HFD) were proposed and discussed. However, there are several basic issues which still represent a challenge, for example, the threshold setting for distinguishing PU presence or absence [11] and the new paradigm to convert CR in an really smart entity [12]. In [10], the detection threshold of PUs was manually placed, derived from the results obtained with simulated signals. Nevertheless, considering the high variability of the spectrum occupancy, this option was not optimal. Therefore, in this paper, a new proposal is analyzed, reconsidering the way in which the CR should act to automatically place the optimal threshold while dynamically adapting it to follow this spectrum occupancy. To do so three different machine learning (ML) algorithms are proposed, implemented and compared, showing that they are useful and efficient techniques to detect occupied or empty bandwidth spaces in a multiband spectrum sensing scenario. Previous works have included ML techniques in their CR proposals. In [13], it is mentioned an overview of the ML techniques that have been used for different CR tasks. SS and MAC protocols have used reinforcement learning (RL) because is the optimal solution for Markov decision process and game theory-based learning. For signal classification and feature detection non-parametric learning, Dirichlet Process mixture model has been used, given that does not require prior knowledge about the number of mixture components. Artificial neural networks (NN) do not require prior knowledge of the distribution of the observed process. For the case of support vector machines (SVM) algorithm, which has better performance for small training examples compared to artificial NN, it requires prior knowledge of the distribution of the observed process and requires data labeling. For power allocation and rate adaptation, it is considered the theory-based learning strategy. Finally, for the reconfiguration of system parameters, the threshold learning is used, which is suitable for controlling specific parameters under uncertainty conditions with the restriction of training data. Interesting works for CR and ML techniques are available in the literature. For example, in [14] a cooperative spectrum sensing (CSS) scheme based on ML techniques applied to an energy vector is proposed, where each component is an energy level estimated at each CR device. Results show how CSS techniques, considering ML, are capable of implicitly learning from the surrounding environment. In [15], an ML-based multiband spectrum sensing policy has been proposed using the greedy method to track the occupancy statistics of PUs and to estimate the detection performance of the SUs. The policy considering the greedy method can select those sub-bands which provide spectrum opportunities with high throughput for the secondary network. In [16], a technique for sensing the primary radio signal in a cognitive environment using a learning algorithm based on artificial NN is analyzed. In [17], a channel state predictor is implemented for multi-SU in a CR using NN. The Dirichlet process has been used as a framework for non-parametric Bayesian learning in CRs in [18]. A contribution related to SS and ML appears in [19], where a scheme based on the algorithm K-nearest neighbors is proposed. The method includes the training and classification phase, and each user takes a decision that is processed in a fusion center. The results show advantages in the detection of PU compared to traditional methods. In [20] the spectrum occupancy is analyzed by several supervised and unsupervised ML techniques, concluding that SVM is the best classification technique. Also, the SVM technique has shown good results in the allocation of resources such as power and channel, in cognitive radio networks [21]. This work is organized as follows: in Section 2, the theoretical bases of the implemented ML techniques are briefly presented. After, in Section 3, a summary of the methodology of our previous Sensors 2019, 19, 4715 3 of 22 3 of 22 work is described. Besides, in this same section, the new methodology considering the ML techniques isisexplained. explained. Section Section 44 shows shows the the simulation simulation environment environment and and results. results. Finally, Finally,Section Section55mentions mentionsthe the conclusion and a brief discussion. conclusion and a brief discussion. 2. Theoretical 2. Theoretical Bases Bases of of the the Implemented Implemented ML ML Techniques Techniques Machine learning Machine learning provides provides computers computers withwith the the ability ability to to learn learn without without being being explicitly explicitly programmed. ML methods are very effective when programmed. when the the data data set set is is large, large, diverse diverse and and fast-changing. fast-changing. These algorithms These algorithms give give aa deep deep and and predictive predictive analysis analysis of of data data [22] [22] and and can can be beclassified classifiedinto intotwo twobig big groups: supervised (clustering techniques) and unsupervised learning (classification groups: supervised (clustering techniques) and unsupervised learning (classification and regression and regression techniques). This techniques). This section section describes describes briefly briefly the the background backgroundof of the the techniques techniquesassociated associatedwith withmachine machine learningthat learning thatwill willbe beimplemented implementedin inthe theanalysis. analysis. 2.1. 2.1. Neural Neural Networks Networks Artificial ArtificialNNNNarearecomputational computationalmodels modelsinspired inspiredby bythe thecentral centralnervous nervoussystem, system,specifically specificallythe the brain. NN have the capacity to perform pattern recognition. NN usually are presented brain. NN have the capacity to perform pattern recognition. NN usually are presented as a system of as a system of interconnected interconnected “neurons” “neurons” that that cancan compute compute values values from from inputs inputs by by feeding feeding information information through through the the network [23]. The NN are supervised algorithms that require training with network [23]. The NN are supervised algorithms that require training with labeled data. Theselabeled data. These networks networksarearebased basedononempirical empiricalrisk riskminimization minimizationandandthey theyrequire requireprior priorknowledge knowledgeof ofthe theobserved observed process processdistribution. distribution.Figure Figure1 1shows shows a NN a NNconsisting consistingof an input of an layer, input one one layer, or more hidden or more layerslayers hidden and an output layer. A NN is used for both classification and prediction considering and an output layer. A NN is used for both classification and prediction considering a back- a back-propagation algorithm propagationby algorithm weight adjustment by weight ofadjustment each edge ofoflayers. each edge of layers. X1 Y1 W1 j O1 X2 W2 j Yj Wj k Ok Wn j Xn Ym Figure 1. Neural network concept. Figure 1. Neural network concept. 2.2. K-Means 2.2. K-means The K-means algorithm allows the classification of a data set. This algorithm requires prior The K-means knowledge algorithm of the number K ofallows groups the classification to be of a data classified. K-means set. This defines algorithm K number requires of centroids, prior finds a knowledge of the number K of groups to be classified. K-means defines K number of partition that minimizes the squared error between the empirical mean of the centroid of a cluster andcentroids, finds a partition the points inthat the minimizes the squared group. Iteratively error between recalculates the empirical the centroid meanuntil of each cluster of the centroid finding of a cluster a partition that reaches the convergence [24]. K-means usually converges to a local minimum, but in [25], it finding and the points in the group. Iteratively recalculates the centroid of each cluster until has beena partition shown that that whenreaches the convergence the clusters [24]. K-means are well separated, usually it could skip converges to a localconverging the local minimum minimum, to butthe in [25], it has global one. been shown that when the clusters are well separated, it could skip the local minimum converging to the global one. 2.3. Expectation-Maximization 2.3. Expectation-Maximization The Expectation-Maximization (EM) algorithm consists of two major steps: an expectation step, The by followed Expectation-Maximization a maximization step. The(EM) algorithm expectation consists is with of to respect twothemajor steps:underlying unknown an expectation step, variables, followed using by a maximization the current estimate of the step. The expectation parameters is with and conditioned upon respect to the unknown the observations. underlying The maximization variables, using the current estimate of the parameters and conditioned upon the observations. The maximization step provides a new estimation of the parameters. These two steps are iterated until 4 of 22 convergence. Sensors 2019, 19, The 4715 EM algorithm was discovered and employed independently by several different 4 of 22 researchers. However, the authors of [26] brought their ideas together, proved convergence, and coined the term “EM algorithm”. In this work, EM is used to delimit 2 events and can be described step as provides a new estimation of the parameters. These two steps are iterated until convergence. follows. The Given EM algorithm x1 ,..., xn wasnumberdiscovered and employed of observations belonging independently to K typesby of several groups.different researchers. K Gaussians with However, the authors of [26] brought their ideas together, proved convergence, and coined the term  2  are created randomly. In this work only two events ( K  2 ) are considered, resulting in “EM algorithm”. In this work, EM is used to delimit 2 events and can be described as follows. G1(1Given  12 ) and x1 , . .G xn2number . ,2( 22 ) . For xi , P(b  xito each point belonging of observations ) Kistypescalculated. Where of groups. b   A, Bwith K Gaussians  are(µ,the σ2 ) randomly. In this work only two events (K = 2) are considered, resulting in G1(µ1 , σ1 ) and are createdevents: considered 2 G2(µ2 , σ22 ). For each point xi , P(b|xi ) is calculated. Where b = {A, B} are the considered events: P( xi  ) P(B) P(B  xi )  (1) P( xi  ) PP(B) P (xi |B )P( x(B i )  ) P(A) P(B|xi ) = (1) P(xi |B)P(B) + P(xi |A)P(A) 1 ( x   )2 P( xi  )  1 exp(  (xi − 2µB ))2 (2) P(xi |B) = √2 B exp(− 2 B2 2 i B ) (2) 2πσB 2 2σB Recalculating 1,2 1,2 2  for A and B : Recalculating (µ1,2 , σ21,2 ) for A and B: b x  ...  b x 1  b11x11 + . . . +n bnn xn (3) µ1 = b1  ...  bn (3) b1 + . . . + bn b ( x   )2  ...  b ( x   )2 12 b1 (1x1 1− µbb)2 + . . . +n bnn(xn −n µn )2 (4) σ21 = b1  ...  bn (4) b1 + . . . + bn with (µ2 ,σ22).. 2 Analogously Analogouslywith 2 2 3.3.ML-based ML-BasedMethodology Methodology First, First,ititisispresented presentedaashort shortsummary summaryofofthe thetechnique techniquedeveloped developedby bythe theauthors authorsinin[10]. [10].Basically, Basically, this thisoriginal originalmethodology methodologyconsiders considersthethemultiresolution multiresolutionanalysis analysis(a(awavelet-based wavelet-baseddyadic dyadicfilter filterbank) bank) and andthe theHiguchi Higuchifractalfractaldimension dimensionto todetect detecttransmissions transmissionsof ofPUs. PUs.The Theflowchart flowchartofofthe theimplemented implemented methodology methodologyof ofthis thisprevious previouswork workisisdescribed describedininFigure Figure2 2where whereXX( (f f) )represents representsthe thepower powerspectral spectral densityin density inaawide widefrequency frequencyrange rangeor orthethereceived receivedmultiband multibandspectrum spectrumby byaa SU. SU. Multi resolution Reconstructed signal from approximated coefficients analysis Frequency Frequency Approximation edge locator edge coefficients Normalized Approximation Interpolation Dynamic coefficients windows NAC Higuchi fractal yes no Higuchi fractal NAC > 0.7 dimension dimension HFD no Primary HFD > 1.85 user yes no primary user Figure 2. The second methodology described by a flow diagram, developed in [10]. Figure 2. The second methodology described by a flow diagram, developed in [10]. Sensors 2019, 19, 4715 5 of 22 This original methodology is described by the following steps: Step 1. The received multiband spectrum by a SU, X ( f ) , is decomposed via the MRA at defined level 3 with a Haar wavelet, giving the respective approximation and details coefficients. Step 2. From the obtained approximation coefficients at level 4, the spectrum is reconstructed eliminating in this way the broadband noise and only keeping the tendency (or smooth shape X app ( f ) ) of the multiband spectrum X ( f ) . Step 3. The frequency edges locator is constructed by detecting the values changes of the same approximation coefficients passing through the threshold of 0.7, i.e., the approximation coefficients going from down to up of the 0.7 or vice versa. These changes will be the frequency edges necessary to construct the dynamic windows for further analysis. Step 4. The same approximation coefficients obtained in Step 1 are normalized e interpolated (to have the same samples that the dynamic windows). With these normalized and interpolated approximation coefficients (NAC), each conformed window is then processed to detect noise or a possible PU transmission. If the NAC are, on average, under a defined threshold of 0.7 then the values of the analyzed windows is probably a PU transmission. If NAC is greater than 0.7 is practically sure that the transmission corresponds to noise. Step 5. If NAC is greater than 0.7 then the Higuchi fractal dimension is applied directly on the analyzed section of the multiband spectrum X ( f ) . In the other case, the HFD is applied to the reconstructed signal from Step 2 (i.e., X app ( f ) ). Step 6. For each window, if the calculated HFD is lower than 1.85 (decision threshold) a PU transmission is detected. In another case, no PU is detected (only noise). The inclusion of ML techniques in the original methodology described above has the main goal to improve the detection process of PUs. Basically, the two first steps and the last one of the previous work remain the same, however, in this new approach the NAC are processed by ML techniques improving the detection process before applying the HFD, modifying the original methodology in Steps 3 and 4 (marked by a dashed red line). The threshold of 0.7 is modified with these ML techniques and instead of being fixed to a constant value, is adapted dynamically enhancing the frequency edges. Figure 3 shows the introduction of this new block, and the red dashed line points out the place for ML techniques. All techniques, described in Section 2, are implemented in this block. As a result, the frequency edges are improved and a Test window (TW) is used to determine the presence or absence of PUs instead of NAC directly (Step 5 also marked by a dashed red line). Next, the implementation of each ML techniques is described starting with the training process for detection of PUs, and describing later how each technique is introduced in the original methodology. 1 Step 5. If NAC is greater than 0.7 then the Higuchi fractal dimension is applied directly on the analyzed section of the multiband spectrum X  f  . In the other case, the HFD is applied to the reconstructed signal from Step 2 (i.e. Xapp  f  ). Step Sensors 6.19, 4715 2019, Foreach window, if the calculated HFD is lower than 1.85 (decision threshold) a6 PU of 22 transmission is detected. In another case, no PU is detected (only noise). Multi resolution Reconstructed signal from approximated coefficients analysis Frequency edge Machine learning Test Frequency Normalized techniques edge locator approximation coefficients Test window Dynamic 6 of 22 windows Steps 3 and 4 (marked by a dashed red line). The threshold of 0.7 is modified with these ML Higuchi fractal yes no Higuchi fractal techniques and instead of being fixed toTWa=constant 1 value, is adapted dynamically enhancing the dimension dimension frequency edges. Figure 3 shows the introduction of this new block, and the red dashed line points out the place HFD for ML techniques. All techniques, described in Section 2, are implemented in this block. As a result, the frequency edges are improved and a Test window no (TW) is used to determine Primary the presence or HFD > 1.85 user absence of PUs instead of NAC directly (Step 5 also marked by a dashed red line). Next, the implementation of each ML yes techniques is described starting with theno training primary process for detection of PUs, and describing later how each technique is introduced inuser the original methodology. Figure 3. Improving of the original methodology in [10] using ML techniques. Figure 3. Improving of the original methodology in [10] using ML techniques. 3.1. Neural Network with Manual Threshold 3.1. Threshold Setting Setting The inclusion of ML techniques in the original methodology described above has the main goal In this In to improve thisthe work, work, an NN an NN detection is used is processused as ofas aaclassifier. PUs. classifier. This Basically,This classifier classifier the two usesand uses first steps thethe the normalized normalized last one of approximation approximation the previous coefficients coefficients work to remaintothe determine determine the presence the presence same, however, of PU. in this of new PU. To Toapproach make make a better a better detection, the NAC detection, the the approximation are processed by ML approximation coefficients techniques coefficients for the for signals the with signals PU with transmissionsPU transmissions are rescaled are in therescaled interval in of the [0, improving the detection process before applying the HFD, modifying the original methodology interval 1], where of0 [0, 1], where represents in0 the represents with coefficient the coefficient more power withand more power andthe 1 represents 1 represents coefficientthe coefficient with with less less power. power. Figure Figure 4 shows the4 shows the implemented implemented scheme for scheme this MLfor this ML technique. technique. Artificial Neural Network Training stage Training Normalized normalized Manual Test rescaled Training rescaled threshold coefficients of coefficients signal to evaluate Classification stage Test window Classification (in function of of coefficients dynamic windows) Figure 4. The general scheme for the NN-based Figure 4. NN-based technique technique with with aa manual manual threshold threshold setting. setting. First, the NN is trained based on a single frame from the whole wideband spectrum, i.e. X  f  First, the NN is trained based on a single frame from the whole wideband spectrum, i.e., X( f ) and its MRA. The considered frame for training is chosen randomly but must contain one or more PU and its MRA. The transmission. considered Figure 5a shows frame for training an example is chosen of the randomly signal used butNN for the must containwhile training, one orFigure more PU5b transmission. Figure 5a shows an example of the signal used for the NN training, while represents the normalized and rescaled approximation coefficients, obtained from the MRA that will Figure 5b represents be the normalized used to train and the NN. This rescaled step approximation provides coefficients,about to the NN, information obtained from the frequency bandMRA that will appearance be used to train the NN. This step provides to the NN, information about frequency band with and without PUs activities. To separate both classes (PUs or SUs), it is necessary to set an appearance with and initial withoutbefore threshold PUs activities. To separate NN training. A manualboth classeswas threshold (PUsinitially or SUs),proposed, it is necessary to set but this an initial means that threshold before NN training. A manual threshold was initially proposed, but this means the threshold setting will depend on the criteria of an external user, making the process extremely that the threshold setting will depend on the criteria of an external user, making the process extremely unpredictable. However, for a signal with a large difference among coefficients (low noise), the threshold setting is not a big deal. Sensors 2019, 19, 4715 7 of 22 7 ofthe unpredictable. However, for a signal with a large difference among coefficients (low noise), 22 threshold setting is not a big deal. 7 of 22 a) a) b) b) Figure 5. (a) Random signal used for the NN training, (b) Normalized and rescaled approximation coefficients Figure 5.5. (a) Figure which (a) are used Random Random toused signal signal trainfor used thethe for NN. the NN NNtraining, training, (b) (b)Normalized Normalized and andrescaled rescaledapproximation approximation coefficients coefficientswhich whichare areused usedto totrain trainthe theNN. NN. In Figure 6, the user set the threshold at 0.6, and the NN was trained. In this figure, according to this manual In threshold, In Figure Figure 6, 6, the the approximation the user user set set the the threshold threshold coefficients at at0.6, andwere 0.6,and the classified theNN NN was and represented wastrained. trained. In In this this figure,with figure, 1 for noise according according to to and this 0 for manual PUs, identifying threshold, the correctly approximation the PUs presence. coefficients Thus, were the trend classified of and coefficients represented this manual threshold, the approximation coefficients were classified and represented with 1 for noise is clear with and 1 for helps, noise besides and and 00 for toPUs, for separate PUs, both classes, identifying identifying correctly correctlyto determine the the PUs the frequency PUs presence. presence. Thus, edges Thus, the the trend trendof of a PU of transmission. coefficients coefficients is is clear clearThis andoccurs and helps, helps, when besides a change to separate is present both from classes, toa coefficient determine associated the frequency to noise edges and of a PU besides to separate both classes, to determine the frequency edges of a PU transmission. This occurs the immediately transmission. next This is a occurs coefficient when a associated change is to present a PU from (or a vice versa), coefficient as can associated be to shown noise by and the green immediately when a change is present from a coefficient associated to noise and the immediately next is a circles in next Figure is a 6. These coefficient values associated (green coefficient to a circles) PU (or vice associated will permit to a versa), PU tocan (or as vice construct be shown versa), asthe dynamic by can theshown be greenwindows circles by thein (marked Figure green 6.asThese circles W1,…,W5 values6.in in Figure this (green These case) circles)that willwill be permit further to processed construct the individually dynamic windows to determine (marked asif the W1, . values (green circles) will permit to construct the dynamic windows (marked as W1,…,W5 in this . . signal ,W5 segment in this corresponds case) that will or be not to further a PU. processed individually to determine if the signal segment corresponds case) that will be further processed individually to determine if the signal segment corresponds or or not to a PU. not to a PU. Tendency of NN Manual threshold 0.6 Tendency ofofNN + Coefficients noise Manual threshold 0.6 o Coefficients of PU + Coefficients of noise o Coefficients of PU W1 W2 W3 W4 W5 W1 W2 W3 W4 W5 Figure 6. The result of training the NN with a manual threshold set at 0.6 over an entire frame. frame. The normalized and andrescaled rescaledapproximation coefficients approximation havehave coefficients been been classified. Also, the classified. borders Also, the of dynamic borders of Figure 6. The result of training the NN with a manual threshold set at 0.6 over an entire frame. The windows dynamic are determined. windows are determined. normalized and rescaled approximation coefficients have been classified. Also, the borders of dynamic windows are determined. With the NN trained, now it is possible to apply this NN to other frames and obtain the frequency With edges the NN from the normalized trained, and rescaled now it is possible approximation to apply this NN to coefficients. other framesThese frequency and obtain the edges permit to build the dynamic windows. According to Figure 3, prior to decide if applying frequency edges from the normalized and rescaled approximation coefficients. These frequency the edges permit to build the dynamic windows. According to Figure 3, prior to decide if applying the Sensors 2019, 19, 4715 8 of 22 8 of 22 With the NN trained, now it is possible to apply this NN to other frames and obtain the frequency edges HFD to from X(the normalized f ) or Xapp ( f ) , itand rescaled approximation is necessary coefficients. to do the classification of These frequency establishing the coefficients, edges permitthe to build the dynamic windows. According to Figure 3, prior to decide if applying the HFD to X( f ) or “test signal” as the complete output resulting by the trained NN and the “test window” as the specific Xapp ( f ), it is necessary to do the classification of the coefficients, establishing the “test signal” as the segments of the test signal associated to dynamic-sized windows. Steps 3 to 5 of the original complete output resulting by the trained NN and the “test window” as the specific segments of the methodology have been modified as: test signal associated to dynamic-sized windows. Steps 3 to 5 of the original methodology have been modified as: Step 3. A training frame is selected: the approximation coefficients are normalized and rescaled. These Step 3.coefficients A training areframe used is selected: to train the approximation coefficients NN with an initial are normalized threshold and rescaled. set by the user. After NN These coefficients are used to train the NN with an initial threshold training, the normalized and rescaled approximation coefficients from another set by the user. Afterrandom NN training,frames analysis the normalized are obtained andand rescaled approximation evaluated by the trained coefficients from another NN, resulting random in the test which analysis frames are obtained and evaluated by the trained NN, resulting in contains the classification of coefficients (0 for a possible PU transmission and 1 for noise).the test which contains the classification of coefficients (0 for a possible PU transmission and 1 for noise). Step 4. The frequency edges and dynamic windows are determined from the output of the NN. The Step 4. The frequency edges and dynamic windows are determined from the output of the NN. The edges are the result of the change from one state to another (from one to zero and vice versa). edges are the result of the change from one state to another (from one to zero and vice versa). Step 5. If the test in each dynamic window (spectrum signal segment) corresponds to 1, it is practically Step 5. If the test in each dynamic window (spectrum signal segment) corresponds to 1, it is practically X f .  sure surethat thatthe thetransmission transmission corresponds tonoise, noise,that thatmeans meansthe theHFD HFD willbebe applied to to X f corresponds to will applied ( ) . On Onthe the contrary contrary case, the analyzed case, the analyzedwindows windowsare arehighly highlyprobably probably a PU a PU transmission transmission andand thethe HFD will be applied to Xapp ( f) . HFD will be applied to X f app . The result The result of of this this whole whole process process isis shown shown in in Figure Figure7.7. First, First, in Figure Figure 7a a real random random multiband multiband signal is signal is presented, presented, this this signal is just an example to demonstrate demonstrate the the functionality functionality of of the the methodology. methodology. Figure 7b Figure 7b shows shows the the classification classification of of the the approximation approximation coefficients coefficients using using the the NN. NN. Here, Here, itit is is possible possible to appreciate to appreciatethethedifference differencebetween betweencoefficients coefficients associated associated with with PUPUandand noise. noise. Finally, Finally, in Figure in Figure 7c 7c the the result result of determination of the the determination of the of the availability availability of aofbandwidth a bandwidth space space is indicated. is indicated. In this In this figure, figure, it isit is clear clear howhowthethe trained trained NN NN classifies classifies thethe spectrum spectrum signal signal is occupied is occupied or empty. or empty. a) b) c) Figure 7. (a) Figure 7. (a) Real Real random random multiband multiband signal. signal. (b) (b) Classification Classification of of coefficients, coefficients, the the result result of of using using the the trained NN mentioned in Figure 6. (c) Result of applying the methodology using NN as a trained NN mentioned in Figure 6. (c) Result of applying the methodology using NN as a classifier. classifier. 3.2. Neural Network with an EM Threshold Setting 3.2. Neural Network with an EM Threshold Setting The NN training threshold that was used in Section 3.1 depends on the criteria of the user. To The NN training threshold that was used in subsection 3.1 depends on the criteria of the user. improve it, an Expectation Maximization (EM) algorithm is included to select automatically and To improve it, an Expectation Maximization (EM) algorithm is included to select automatically and optimally the threshold. Figure 8 shows the new block diagram including EM. optimally the threshold. Figure 8 shows the new block diagram including EM. 1 Sensors 2019, 19, 4715 9 of 22 99 of of 22 22 Artificial Artificial Neural Neural Network Network Training Training stage stage Expectation Expectation Training maximization maximization Training Training Training Normalized Normalized normalized Test normalized Test rescaled rescaled rescaled Decision rescaled Decision coefficients coefficients of of coefficients threshold coefficients threshold signal signal to to evaluate evaluate Classification Classification stage stage Test Test window window Classification Classification (in (in function function of of of of coefficients coefficients dynamic dynamic windows) windows) Figure Figure 8. 8. The The general general scheme scheme for for the the NN-based NN-based technique technique with with aa threshold threshold setting threshold setting using setting using EM. using EM. EM. Figure Figure999shows Figure showsthe shows the normalized the normalized and normalized and rescaled and rescaled approximation rescaled approximation coefficients approximation coefficients of coefficients of aa generated generated random random multiband multiband spectrum used to train the NN. When applying EM to these coefficients, two specific limits spectrum used to train the NN. When applying EM to these these coefficients, coefficients, two specific limits are are obtained. obtained. The The threshold threshold isis then then selected selected randomly randomly between between these these limits limits (a (a detailed detailed explanation explanation of of this this decision decision is is explained explained in subsection in Section 4.1).4.1). subsection 4.1). a) a) b) b) Figure Figure 9. 9. (a) (a) Multiband Multiband spectrum spectrum generated generated randomly. randomly. (b) (b) Approximated, Approximated, normalized normalized and and rescaled rescaled approximation approximation coefficients coefficients which are which are used areused to usedto train totrain the trainthe NN. theNN. NN. InFigure In In Figure10 Figure 1010the theapplication the applicationof application ofofthe thetheEM EMEMalgorithm algorithm algorithm is is is exemplified. exemplified. exemplified. Here,Here, Here, thethe the firstfirst first twotwo two classesclasses classes are are are randomly randomly chosen chosen withwith Gaussian Gaussian distribution distribution (marked (marked inin green), green), randomly chosen with Gaussian distribution (marked in green), the normalized and rescaled the the normalized normalized and and rescaled approximation approximation coefficients approximation coefficients are are plotted plotted on on the the XXX axis axis (marked axis (marked with (marked with yellow with yellow bullets). yellow bullets). The bullets). The result result of of applying EM to these coefficients are the two Gaussians in blue, where applying EM to these coefficients are the two Gaussians in blue, where it is clearly seen that they it is clearly seen that they correspond correspond to to two two groups groups that that can can be separated. be separated. separated. The Thelimits The limits L limits L1 and L2 L11 and LL22are areset are setatat set 1 µ atL1LL1= 111+ 33 3σ111 and and L2 = LL22  µ + 3σ when L1 < L2 since the probability of occurrence is 0.001 for each class and where 2  33 0.001 2 2when L1 L2 since the probability of occurrence is for each class and where 22 when L1  L2 since the probability of occurrence is 0.001 for each class and where (0.211, 20.1143) and (0.872, 0.0446) are the mean and standard deviation (STD) of Gaussians 1 and 2, (0.211, (0.211, 0.1143) and and (0.872, 0.1143)Finally, (0.872, 0.0446) 0.0446) are are the the mean mean and and standard standard deviation deviation (STD) (STD) of Gaussians 11 and and 2, respectively. the threshold is chosen randomly between L1 and L2. Theof Gaussiansbetween uncertainty 2, respectively. Finally, the threshold is chosen randomly between L 1 and L 2 . The uncertainty between respectively. what the optimalFinally, the value threshold would be is is chosenin clarified randomly When L1L1>and Section 4.between L2 . The L2 means uncertainty that the SNR isbetween close to what the optimal value would be is clarified in Section 4. When LL11  LL22 means that the SNR is close 0 dB. In this case, L1 = µ1 + nσ1 and L2 = µ2 + nσ2 are adjusted with n = 2, 1, 0 untilthe what the optimal value would be is clarified in Section 4. When means that L1SNR< L2iswhere close to 0 dB. In to 0probability this dB. In this of case, case, L 1    n L1  11 for and n1each L2    2  22 be andn Lwould n  are  n0.022, adjusted 22 are0.158, adjusted with n  2,1,0 with n  2,1,0 until L1  L2 where until L 1  L 2 where the occurrence 1 0.499. the the probability of occurrence for each n would be 0.022,0.158,0.499 .. probability of occurrence for each n would be 0.022,0.158,0.499 Limit L1 Limit L2 Sensors 2019, 19, 4715 10 of 22 10 of 22 Figure 10. The power of the approximation coefficients in axis X and the result to classify them in two classes. EM threshold Limit L1 In the case of theLimit analyzed L2 example (Figure 9), the threshold is set to 0.654 (Figure 10). Applying this threshold to a training signal allows the classification of coefficients and the determination of frequency edges that facilitate the building of the respective dynamic windows as it is shown in Figure 11. This improvement only modifies Step 3: Step 3. A training frame is selected: the approximation coefficients are normalized and rescaled. These coefficients are used to train the NN where coefficients will be classified and two limits ( L1 and L2 ) estimated by the EM algorithm. Then, a random value between L1 and L2 is selected as threshold. 10. Thepower Figure 10. Figure powerofofthe the approximation coefficients in axis X and the the result to classify them in After The NN training, approximation the normalizedcoefficients in axis and rescaled X and result approximation to classify coefficientsthem in two from another two classes. classes. random analysis frames are obtained and evaluated by the trained NN, resulting in the test which contains the classification of coefficients (0 for a PUistransmission and 1 for10).noise). InInthe the caseof case ofthe theanalyzed analyzed example(Figure example (Figure 9),the 9), thethreshold threshold set to is set to 0.654 0.654 (Figure (Figure 10). Applying Applying this threshold to a training signal allows the classification of coefficients and the this threshold to a training signal allows the classification of coefficients and the determination ofdetermination of The result frequency edges of applying edges that this that facilitate technique facilitate the is the building shown building ofof the in Figure the respective 12. Here, respective dynamic a multiband dynamic windows windows as random as itit is signal is shown shown in is in frequency used to Figure 11. determine 11. This its states This improvement of improvement only occupancy only modifies based modifies Step on Step 3: 3: the mentioned trained NN clearly detecting the Figure presence of PUs. Step 3. A training frame is selected: the approximation coefficients are normalized and rescaled. These coefficients are used to train the NN where coefficients will be classified and two limits ( L1 and L2 ) estimated by the EM algorithm. Then, a random value between L1 and L2 is selected as threshold. After NN training, the normalized and rescaled approximation coefficients from another random analysis frames are obtained and evaluated by the trained NN, resulting in the test EM threshold Limit L1 which contains the Limit L2 classification of coefficients (0 for a PU transmission and 1 for noise). Tendency of NN + Coefficients of noise The result of applying this o Coefficients of technique PU is shown in Figure 12. Here, a multiband random signal is used to determine its states of occupancy based on the mentioned trained NN clearly detecting the presence of PUs. W1 W2 W3 W4 W5 11. The result of training Figure 11. training the NN NN with with the the EM-based EM-based threshold threshold set set to to 0.654. 0.654. Determination Determination of frequency edges and and construction construction of of the the dynamic dynamic windows. windows. Step 3. A training frame is selected: the approximation coefficients are normalized and rescaled. EM threshold These coefficients are used to train the NN where coefficients will be classified and two limits Limit L1 ( L1 and Limit L2 ) estimated L2 by the EM algorithm. Then, a random value between L1 and L2 is selected Tendency of NN as threshold. + Coefficients of noise After NNo Coefficients training, ofthe PU normalized and rescaled approximation coefficients from another random analysis frames are obtained and evaluated by the trained NN, resulting in the test which contains the classification of coefficients (0 for a PU transmission and 1 for noise). W1 W2 W3 W4 W5 The result of applying this technique is shown in Figure 12. Here, a multiband random signal is usedFigure to determine its states 11. The result of occupancy of training based the NN with theon the mentioned EM-based thresholdtrained NN Determination set to 0.654. clearly detecting of the presence of PUs. frequency edges and construction of the dynamic windows. Sensors 2019, 19, 4715 11 of 22 11 of 11 of 22 22 a) a) b) b) c) c) Figure 12. Figure Figure 12. (a) 12. (a) Multiband (a) Multiband spectrum Multiband spectrum generated spectrum generated randomly. generated randomly. (b) randomly. (b) Classification (b) Classification of Classification of the of the approximation the approximation approximation coefficients, the coefficients, the result result of of using using the the trained trained NN trained NN mentioned NN mentioned in mentioned in Figure in Figure11. Figure 11. (c) 11. (c) Result Result of of applying applying the the complete methodology using a NN as a classifier. classifier. complete methodology using a NN as a classifier. 3.3. Methodology Using 3.3. Methodology 3.3. Using aaa K-means K-Means K-means Classifier Classifier Anotherunsupervised Another Another unsupervisedmethod unsupervised method used used inin this this work work as as aa classifier of the classifier of the approximation approximation coefficients coefficients coefficients from a random from aa random from multiband random multiband spectrum multiband spectrum spectrum isis the is the K-means the K-means technique. K-means technique. technique. InIn order In order to order to determine to determine the the multiband determine the multiband multiband availability of availability of availability the of the spectrum the spectrum signal, spectrum signal, the signal, the approximation the approximation coefficients, coefficients, normalized approximation coefficients, and normalized and normalized rescaled, rescaled, are and rescaled, are first are first first classified classified with with K-means. K-means. The The flowchart flowchart of of this this process process is isshown shown in in classified with K-means. The flowchart of this process is shown in Figure 13. Figure Figure 13. 13. K-means K-means Reconstructed Reconstructed signal from from Clusters selection Clusters selection stage stage signal approximation approximation Power > -90 dBm Power > -90 dBm no no coefficients coefficients yes yes Decision of Decision of Test Test Bandwidth number of number of Bandwidth analysis clusters clusters analysis Classification stage Classification stage Normalized Normalized Test window Test window rescaled rescaled Classification Classification (in function (in function of of coefficients of coefficients of of coefficients of coefficients dynamic dynamic signal to signal to windows) windows) evaluate evaluate Figure 13. Figure 13. The The general The scheme general general for the scheme scheme for the approximation forapproximation coefficients the approximation classification coefficients coefficients based on classification classification based on the K-means based the K-means on the technique. K-means technique.technique. In In this In this technique, this technique, the technique, the objective the objective is objective is is to to classify to classify the classify the approximation the approximation coefficients approximation coefficients into coefficients into 222 groups, into groups, one groups, one one inferring inferring the the PU PU transmission transmission and and one one corresponding corresponding to to the the noise, noise, inferring the PU transmission and one corresponding to the noise, when the multiband spectrum when when the the multiband multiband spectrum spectrum contains contains PUs contains PUs PUs transmissions. transmissions. On transmissions. On the the other the other hand, other hand, they hand, they must they must must bebe classified be classified classified inin in aaa single single single groupgroup group when when the when the the same radio-electric same radio-electric same radio-electric spacespace contains space contains only contains only noise. only noise. However, noise. However, one However, one of one of the of the disadvantages the disadvantages disadvantages foundfound found in in K-means in K-means K-means is that is it it is that that is necessary it is is necessary necessary totoindicate to indicate indicate thethenumber the numberofof number ofclusters clustersthat clusters thatare that areexpected, are expected,which expected, whichintroduces which introducesaaa difficulty introduces difficulty difficulty since the since the since frames the frames that frames that are that are evaluated are evaluated evaluated by by the by the algorithm the algorithm are algorithm are random. are random. random. Then, Then, to Then, to indicate to indicate indicate howhow how many many clusters many clusters clusters the the K-means K-means algorithm algorithm needs, needs, it it is isnecessary necessary aa prior prioranalysis analysis the K-means algorithm needs, it is necessary a prior analysis of the behavior of Xapp of of the the behavior behavior ofofX Xapp (( ff)). . As As it it is is shown shown app( f ) . As it is shown in Figure 13, the possible presence of a PU in the signal Xapp ( f ) is first evaluated setting a simple in Figure in Figure 13, 13, the the possible possible presence presence of of aa PU PU inin the the signal signal X Xapp ( f ) is first evaluated setting a simple app( f ) is first evaluated setting a simple 12 of 22 Sensors energy 2019, 19, 4715at detector −90 dBm. If there is no detection, the number of clusters is set to 1. On the12other of 22 hand, if some transmission exceeds the threshold, we have three possible cases which are solved with a smalldetector energy analysis at of −90 bandwidth: dBm. If there is no detection, the number of clusters is set to 1. On the other hand, if some transmission exceeds the threshold, we have three possible cases which are solved with Case a small 1: impulsive analysis noise in the signal. Owing to the small bandwidth of this anomaly, i.e. less than of bandwidth: 7 samples * 0.1 [MHz], the number of clusters chosen is still 1. Case 1: impulsive noise in the signal. Owing to the small bandwidth of this anomaly, i.e., less than 7 Casesamples 2: possible * 0.1PU transmissions [MHz], the numberin the signal. chosen of clusters With the is understanding still 1. that a transmission has at least 7 samples * 0.1 [MHz] then the number of clusters will be 2. Case 2: possible PU transmissions in the signal. With the understanding that a transmission has at Caseleast 7 samplesnoise 3: impulsive * 0.1 and [MHz] then the possible PU number of clusters transmissions will bein2.the frame. In this case, when coexisting Case 3: impulsive locatingnoise that and therepossible PU transmissions is a possible transmissioncoexisting with at leastin the frame. In* this 7 samples case, when 0.1 [MHz], that locating that there is a possible transmission means the chosen number of clusters will be 2. with at least 7 samples * 0.1 [MHz], that means the chosen number of clusters will be 2. The methodology is modified in Steps 3 and 4, regarding the proposed steps in subsection 3.1 as The methodology is modified in Steps 3 and 4, regarding the proposed steps in Section 3.1 follows: as follows: Step 3. 3.AApriori Step priorianalysis analysisdirectly directly on on the the approximation coefficientstotoselect approximation coefficients select the the number number of of clusters clusters is is done: done: If the reconstructed signal ( X app( f ) ) does not exceed −90 dBm, the cluster number willwill If the reconstructed signal ( X ( f ) ) does not exceed -90 dBm, the cluster number app bebe1 1(only (onlynoise). noise).Else, Else,the the bandwidth bandwidth of of the thesignals signalsthat thatexceeded exceeded this this threshold threshold is studied is studied (3 (3 cases casesmentioned mentionedbefore). before). To To confirm that ititisisnot confirm that notimpulsive impulsivenoise noise(2(2 clusters). clusters). Then, Then,the thenormalized normalizedand andrescaled approximationcoefficients rescaled approximation coefficientsfrom from another another random random analysis analysis frames framesare areobtained obtained and and evaluated evaluated byby the the K-means K-meansalgorithm, algorithm,resulting resulting in in thethe testtest which which contains containsthe theclassification classification of of coefficients (0 for coefficients (0 foraaPU PUtransmission transmissionand and 1 for 1 for noise). noise). Step Step 4. 4.The Thefrequency frequency edges edges and anddynamic dynamicwindows windowsareare determined determinedfromfrom the output of the of the output classifier. The the classifier. edges The are the edges are result of theof the result change from one the change stateone from to another state to(from one to another zero one (from and vice versa). to zero and vice versa). The result of applying K-means to coefficients is shown in Figure 14. In Figure 14a we can observe The resultwindows how dynamic of applying andK-means to coefficients the frequency edges areis shown chosen.inInFigure Figure14.14a, In Figure 14a we can the clustering observe is given in howgroups two dynamic windows where and theof the presence frequency edges PUs exists. are chosen. In Figure 14b, itInisFigure shown14a,thethe clustering power is givenin of coefficients in two groups where the presence of PUs exists. In Figure 14b, it is shown the power of coefficients Y-axis, where 1 corresponds to low power (bullets in red color) and 0 to a higher power (bullets in color in Y-axis,K-means blue). where 1classifies corresponds to low correctly thepower given (bullets in red color) the approximation and 0 to anormalized coefficients, higher power and(bullets in rescaled, color of blue). multiband a random K-means classifies spectrum. correctly the given the approximation coefficients, normalized and rescaled, of a random multiband spectrum. a) b) W1 W2 W3 W4 W5 Figure14. Figure 14.(a) (a)Classification Classificationof of coefficients coefficients withwith K-means K-means and respective and their their respective frequency frequency edges. edges. (b) (b) Power Power of coefficients in Y-axis the result of use K-means as a classifier. of coefficients in Y-axis the result of use K-means as a classifier. The result of applying K-means to the multiband spectrum monitoring technique using the technique as a classifier is shown in Figure 15, where Figure 15a corresponds to a randomly generated 13 of 13 of 22 22 Sensors 2019, 19, 4715 13 of 22 The result The result ofof applying applying K-means K-means toto the the multiband multiband spectrum spectrum monitoring monitoring technique technique using using the the technique as technique as aa classifier classifier is is shown shown in in Figure Figure 15, 15, where where Figure Figure 15a 15a corresponds corresponds toto aa randomly randomly generated generated multiband spectrum, while Figure 15b shows the occupation multiband spectrum, while Figure 15b shows the occupation along occupation along the along the frequency, the frequency, result of applying frequency, result of applying the complete the complete methodology. methodology. methodology. a) a) b) b) Figure 15. Figure 15. (a) (a) The The random randommultiband random multibandspectrum multiband spectrumwhich spectrum whichcorresponds which corresponds corresponds to toto coefficients coefficients coefficients plotted plotted plotted in Figure in Figure in Figure 14. 14. 14. (b) (b) (b) The TheThe result result result of applying of applying of applying the the the complete complete complete methodology. methodology. methodology. Results 4. Results 4. In this section, In this this section, the section, thesimulation the simulationenvironment simulation environmenttoto environment toanalyze analyzeand analyze andcompare and compare compare the the pros pros the and and pros cons cons and in in in cons the thethe use useuse of of the the three three ML ML techniques techniques is is completely completely described. described. Performance Performance is also is also checked checked using of the three ML techniques is completely described. Performance is also checked using signals using signals signals captured captured from from a real captured aa real real environment. environment. from environment. 4.1. Simulated Signals 4.1. Simulated Simulated Signals 4.1. Signals The The methodology methodologydescribed described in Section 3 with in Section Section the ML techniques, was applied was to a globaltosimulation The methodology described in 33 with with the the ML techniques, ML techniques, was applied applied to aa global global of 500,000 simulation frames of for 500,000 SNR values frames for in SNR the interval values in [−10, the 20] dB, interval as it [−10, is shown 20] dB, in as Figure it is 16. shown In in each frame, Figure 16. simulation of 500,000 frames for SNR values in the interval [−10, 20] dB, as it is shown in Figure 16. the In number, each frame,typethe(OFDM number,or NRZ type associated (OFDM orwith NRZ CDMA) and position associated with (frequency CDMA) and edges) of(frequency position simulated In each frame, the number, type (OFDM or NRZ associated with CDMA) and position (frequency symbols edges) ofofwere generated simulated randomly. symbols Each frame were generated generated consists Each randomly. of 1024 samples frame spaced consists by 0.1 of 1024 1024 MHz (i.e., samples an spaced edges) simulated symbols were randomly. Each frame consists of samples spaced entire by 0.1 band 0.1 MHz consist MHz (i.e.(i.e. an of 102.4 an entire MHz). entire band band consistSimulation consist of parameters of 102.4 102.4 MHz). are MHz). Simulationsummarized Simulation parameters in Table parameters are 1. The are summarizedanalyzed summarized in in by blocks Table correspond 1. The analyzedto studies blocks ofcorrespond noise, symbols, to and frequency studies of noise, edges. These symbols, and studies frequency serve to analyze edges. These Table 1. The analyzed blocks correspond to studies of noise, symbols, and frequency edges. These the behavior studies serveof to symbols, analyze the noise the and the behavior ofaccuracy symbols,of detected noise and thefrequency the accuracyedges. of detected detected frequency frequency edges. edges. studies serve to analyze behavior of symbols, noise and accuracy of Methodology Methodology Multiband Multiband spectrum spectrum Generator Generator sensing sensing SNR SNR of 500,000 of 500,000 Percentage Percentage technique technique [-10, 20] [-10, 20] random random of success of success Classification Classification frames frames based on based on ML ML techniques techniques Occupation of Occupation of signal signal Study of Study of noise noise and symbols and symbols Study of Study of edges edges Figure 16. Figure Scheme of 16. Scheme of the the global global simulation. simulation. Sensors 2019, 19, 4715 14 of 22 14 of 22 Table 1. Simulation parameters. Table 1. Simulation parameters. ML Technique NN-Manual Threshold ML Technique NN-EM Threshold K-Means Software NN – Manual threshold MATLAB NN 2014b– EM threshold K-means Software Value for SNR MATLAB 2014b −6 a 20 dB spaced by 2 dB Value for per Number of frames SNR each SNR value −6 a 20500,000 dB spaced by 2 dB Number of frames per each SNR value 500,000 Number of symbols per each frame Randomly between [0, 2] Number of symbols per each frame Randomly between [0, 2] Bandwidth 102.4 MHz Bandwidth 102.4 MHz Samples per frame 1024 Samples per frame 1024 From Section 3.2, From 3.2, aarandom randomthreshold thresholdisischosen chosenbetween between the values the values L1 L1 of of and andL2,L2, after applying after applyingthe EM EM the algorithm. The chosen algorithm. The chosenthreshold is the result threshold is the of the study result of theshown studyinshown Figure in17.Figure Here, a17. simulation Here, a of 10,000 randomly simulation of 10,000 generated randomly frames was performed generated frames was for each SNR value. performed for In thisSNR each simulation, value. the two In this levels L1 andthe simulation, L2two were chosen levels L1 asandtheL2maximum were chosen or theas minimum the maximum threshold or thevalue, respectively, minimum thresholdnecessary value, to the NN training. respectively, In Figure necessary to the 17, NNthe averageInand training. STD17, Figure of the average frequency edges and STDdetection for each edges of the frequency of the limits of this detection threshold, for each of the is shown. limits Here, of this choosing threshold, any of these is shown. Here,limits doesany choosing notof represent a bigdoes these limits change not in the estimation represent of frequency a big change edges, which in the estimation means that of frequency if any edges, random which meansvalue is ifchosen that between any random one value of chosen is these two limits,one between it will of not these represent a big two limits, it difference. In practice, will not represent it is a big extremelyIncomplicated difference. practice, it to is determine complicated extremely the optimumto value for thisthe determine threshold, optimum because value the for premise of multiband this threshold, because spectrum sensing the premise of technique isspectrum multiband based onsensing the ignorance techniqueof the characteristics is based of the received on the ignorance signal (frequency of the characteristics edges, of the SNR, received presence signal of UP, bandwidth, (frequency edges, SNR, etc.)presence For SNRof < −2 UP,dB the proposed bandwidth, etc.)methodologies For SNR < −2dodB notthe have a good proposed performance since methodologies the have do not signals are embedded a good performance in noise. since the signals are embedded in noise. Figure Figure 17. The mean and and STD STDofofthe thefrequency frequencyedges edges location location estimation estimation using using thethe threshold threshold asand as L1 L1 L2 considered and as the L2 considered asmaximum and and the maximum minimum values, minimum respectively, values, for the respectively, forNN the training. NN training. In Figure In Figure 18a, 18a, it it is is highlighted highlighted aa random random multiband multiband spectrum spectrum that that represents represents aa challenging challenging case case because of the contained power noise. In Figure 18b, we can observe that the three methodologies because of the contained power noise. In Figure 18b, we can observe that the three methodologies have an have an excellent excellent performance performance in in aa quite quite noisy noisy environment. environment. The frequency edges for the original methodology and the different proposals for the example of Figure 18, are shown in Table 2. The edges where the transmissions of the PUs were placed in the simulation have been also included. It is clearly observed that the proposals presented in this paper prove to be more accurate than the original methodology. Sensors 2019, 19, 4715 15 of 22 15 of 22 a) 15 of 22 a) b) b) (a) Multiband Figure 18. (a) Multiband spectrum generated randomly. (b) (b) Original methodology methodology and proposed methodologies applied methodologies appliedtotodetermine determinethe theoccupation occupationbyby PUs. PUs. Where Where allthem all of of them detect detect correctly correctly UPs UPs transmissions. transmissions. Table 2. Frequency edges detection with the different techniques of the example shown in Figure 18. The Figure frequency 18. (a)edges for the Multiband original spectrum methodology generated randomly. and the different (b) Original proposals methodology for the example and proposed of Figuremethodologies 18, are shown in Table applied 2. The the to determine edges where the Frequency occupation by transmissions PUs.Edges Detected Where of[MHz] all of them the PUs detect were UPs correctly placed in the simulation have Simulation been also included.0.1It is 14.2 transmissions. clearly19.4 observed 40.7 that the 60.7 proposals 73.7 presented 93.7 in this paper 102.4 prove to be more accurate than the original methodology. Original The methodology frequency edges for the 0.1 14.2 original 18.4 40.5 methodology 60.2 and the 73.4proposals different 93.2 for the 102.4 example of Figure 18, are shown NN-Manual in Table 2.0.1 threshold The edges 14.4 where 19.2 the transmissions 40.8 60.8 of the PUs 73.6 were placed 93.6 102.4 in the Table 2. Frequency edges detection with the different techniques of the example shown in Figure 18. simulation have been also included. It is clearly observed that the proposals presented in this paper NN-EM threshold 0.1 14.4 19.2 40.8 60.8 73.6 93.6 102.4 Frequency Edges Detected [MHz] prove to be more accurate than the original methodology. K-meansSimulation 0.1 14.4 19.2 19.4 0.1 14.2 40.840.7 60.8 60.7 73.7 73.693.7 93.6 102.4 102.4 Table 2. Frequency Original methodology 0.1 14.2 18.4 40.5 60.2 73.4 93.2 102.4in Figure 18. edges detection with the different techniques of the example shown Figure 19 shows NNthe - Manual meanthreshold and STD 0.1 14.4 of the 19.2 Edges frequency Frequency 40.8edges 60.8 detection Detected73.6 [MHz]93.6 as102.4 a function of the SNR NN - for each proposed ML technique EM threshold Simulation 0.1 14.4 described0.1in 14.2 19.2 40.8 19.4 3.40.7 Section 60.8 73.6 60.7 73.7 the Apparently, 93.793.6 102.4 mean 102.4 of each of them tends K-means Original methodology 0.1 0.1 14.4 19.2 14.2 18.4 40.8 40.5 60.8 60.2 73.6 73.4 93.293.6 102.4 to have the same behavior; however, the variance for each SNR differs in some of the cases. Table 3 102.4 NN - Manual threshold 0.1 14.4 19.2 40.8 60.8 73.6 93.6 102.4 shows the values of mean and STD, where0.1 NN - EM threshold it is14.4 possible to appreciate how for higher SNR values all 19.2 40.8 60.8 73.6 93.6 102.4 ML techniques tend to similar values, K-means being K-means the 0.1 14.4 19.2 40.8 most 60.8stable 73.6 method. 93.6 102.4 Figure 19. Mean and STD for each method. It is noteworthy that K-means shows a constant behavior, Figure Figure 19. Mean Mean 19. with compared and the and STDSTD others. forforeach eachmethod. method. It It isisnoteworthy noteworthythat K-means that shows K-means a constant shows behavior, a constant behavior, compared with the others. compared with the others. Figure 19 shows the mean and STD of the frequency edges detection as a function of the SNR for each proposed ML technique described in Section 3. Apparently, the mean of each of them tends to have the same behavior; however, the variance for each SNR differs in some of the cases. Table 3 shows the values of mean and STD, where it is possible to appreciate how for higher SNR values all Sensors 2019, 19, 4715 16 of 22 ML techniques tend to similar values, being K-means the most stable method. Table 3. Mean and STD of frequency edges detection per SNR. Table 3. SNR SNR Technique Technique −2 −20 02 42 46 68 810 10 12 12 14 1416 1618 1820 20 Mean Mean 1.49 1.49 2.00 2.00 1.07 1.07 1.99 1.99 1.10 1.10 1.18 1.18 2.99 2.99 1.16 1.16 1.18 1.18 1.16 1.16 1.66 1.66 1.33 1.33 NN-Manual threshold NN-Manual threshold STDSTD 3.03 10.86 10.86 1.47 1.68 1.53 1.53 1.67 1.67 2.38 2.38 1.64 1.64 1.67 1.67 1.64 1.64 2.29 2.29 1.90 1.90 3.03 1.47 1.68 Mean 1.22 1.18 1.15 1.12 1.11 1.10 1.10 1.10 1.09 1.09 1.09 1.09 K-means Mean 1.22 1.18 1.15 1.12 1.11 1.10 1.10 1.10 1.09 1.09 1.09 1.09 K-means STD 1.81 1.68 1.61 1.57 1.55 1.53 1.52 1.52 1.51 1.51 1.51 1.51 STD 1.81 1.68 1.61 1.57 1.55 1.53 1.52 1.52 1.51 1.51 1.51 1.51 Mean 2.14 1.30 1.29 1.26 1.17 1.25 1.48 1.35 1.32 1.27 1.13 1.36 NN-EM threshold Mean 2.14 1.30 1.29 1.26 1.17 1.25 1.48 1.35 1.32 1.27 1.13 1.36 NN-EM threshold STD 13.34 1.92 1.91 1.81 1.64 1.79 2.15 1.95 1.89 1.82 1.58 1.96 STD 13.34 1.92 1.91 1.81 1.64 1.79 2.15 1.95 1.89 1.82 1.58 1.96 The percentage The percentage of of success success (PS) (PS) for for each each methodology methodology is is plotted plotted in in Figure Figure 20. 20. Some Some important important points can points can be be remarked: remarked: • TheThe performance performanceof of NN NNwith manual with manualthreshold, threshold,NN NNwith withEMEMthreshold and K-means threshold and K-means for SNR >= >0=have SNR quite 0 have acceptable quite results, acceptable results,obtaining obtainingsuccess successrate ratepractically practicallyaround around 100% 100% of the correct detection. correct detection. • An An improvement improvement waswas observed observed for NN for the the NN that that has been has been trained trained with with EMinjust EM just SNRin = SNR = −2 −2 dB. dB. Figure 20. PS PS of of each each of the proposed methodologies. The proposals presented in this paper show a better performance than the original original methodology methodology (see Figure 20). Not only in PS, also, in the precision to detect the frequency edges (see Table Table 3). 3). 4.2. Signals 4.2. Signals Obtained Obtained from from the the Environment Environment The real The real signals signals mentioned mentioned in in Section Section 4.3 4.3 of of [10] [10] were were again again used used to to test test the the performance performance ofof different proposed methodologies. These signals were obtained from a whole band different proposed methodologies. These signals were obtained from a whole band varying from 0.6 varying from 0.6 GHz GHz to 2.6 to 2.6 GHz.GHz. TheThe percentage percentage of of occupation occupation inin thiswideband this widebandisishighlighted highlightedininFigure Figure 21. 21. Table Table 44 presents the specific bands measured and for which services are they assigned. presents the specific bands measured and for which services are they assigned. Sensors 2019, 19, 4715 17 of 22 17 of 22 Figure Figure 21. Percentage 21. Percentage ofofoccupation occupation for for aaweek weekininthethe frequency interval frequency [0.6–2.6] interval GHz presented [0.6–2.6] in GHz presented [10]. in [10]. Table Table 4. 4. Monitoredfrequency Monitored frequency bands bandspresented presentedin in [10]. [10]. Frequency Band Type of Communication Frequency Band Type of Communication [698–806] MHz Mobile and landline. [698–806] MHz [806–902] MHz Mobile Mobile andand landline.mobile. aeronautical [1.7–2]MHz [806–902] GHz Mobile and Mobile and aeronautical fixed. mobile. [2.3–2.5] GHz Radiolocation, amateur, mobile and fixed. [1.7–2] GHz Mobile and fixed. Aerial vehicles, radiolocation and, radio [2.6–2.8] GHz [2.3–2.5] GHz Radiolocation, amateur, navigation.mobile and fixed. [2.6–2.8] GHz Aerial vehicles, radiolocation and, radio navigation. 4.3. Results from the Real Signals The 4.3. Results realthe from signals Realobtained Signals from the environment are totally unknown, being this kind of scenarios a challenge regarding the right estimation of the set of frequency edges and power transmission of The PUs.real signals obtained Nevertheless, fromsignal after some the environment processing, itare totally unknown, is possible to infer thebeing this kind presence of scenarios of PUs. The methodology a challenge described regarding earlyestimation the right in this paper of and in the the set of previous frequency work presented edges in [10], and power it is examined transmission of PUs. with the ML techniques considering the different bands of Table 4. In the next, it Nevertheless, after some signal processing, it is possible to infer the presence of PUs. The methodology is shown a challenging signal for each band. The results of PU detection are plotted in Figures 22 described early in this paper and in the previous work presented in [10], it is examined with the ML to 26. The frequencies examined from 698 MHz to 806 MHz appear in Figure 22a. This signal presents techniques considering the different bands of Table 4. In the next, it is shown a challenging signal for a non-constant power transmission of PUs around 760–780 MHz. In practical terms, this kind of each band. The results of PU detection are plotted in Figures 22–26. 18 of 22 activity determines totally occupied this bandwidth. Thus, ML techniques and the original methodology detect frequency space as occupied as can be seen in Figure 22b. a) b) 22. (a) Figure 22. Figure (a)AAreal signal real obtained signal fromfrom obtained the frequency band [698–806] the frequency MHz. (b) band [698–806] The (b) MHz. result Theof applying result of the four proposed methodologies to the real signal in terms of spectrum occupancy. applying the four proposed methodologies to the real signal in terms of spectrum occupancy. The signals observed in the frequencies of 806 MHz to 910 MHz are shown in Figure 23. In this figure, most of the transmission signals are detected by all techniques. NN with manual threshold provides the best results. This technique detects a signal with low power that apparently corresponds to a PU. In contrary case, the original methodology does not detect changes on power and the bandwidth size detected is lower. The signals observed in the frequencies of 806 MHz to 910 MHz are shown in Figure 23. In this figure, most of the transmission signals are detected by all techniques. NN with manual threshold provides the best results. This technique detects a signal with low power that apparently corresponds to a PU. Sensors 2019,In 19,contrary 4715 case, the original methodology does not detect changes on power and 18 ofthe 22 bandwidth size detected is lower. a) b) 19 of 22 a) 23. (a) Figure 23. Figure (a)AAreal signal real obtained signal obtainedfromfrom the frequency band [806–902] the frequency MHz. (b) band [806–902] The (b) MHz. result Theof applying result of the four methodologies to the realtosignal in terms 19 of 22 applying the four methodologies the real signalofinspectrum occupancy. terms of spectrum occupancy. a) The frequencies measured from 1.7 GHz to 2 GHz are presented in Figure 24a. Here, in general, the occupied and empty bands are detected correctly by the ML techniques as can be seen in Figure 24b. However, b) in the range from 1.925 GHz to 2 GHz there are several transmissions. The NN with manual threshold techniques detects these changes in the occupation of the band. The frequency edges are not detected with high precision, but this technique is more sensitive to these changes. b) Figure 24. (a) A real signal obtained from the frequency band [1.7–2] GHz. (b) The result of applying the four methodologies to the real signal in terms of spectrum occupancy. Figure 25a shows the frequencies observed between 2.3 GHz and 2.5 GHz. In this case, the three techniques provide affordable results classifying the occupation of the signal as can be seen in Figure Figure Figure 24. (a) A 24. (a) A real real signal signal obtained obtained from from the the frequency frequency band band [1.7–2] [1.7–2] GHz. (b) The GHz. (b) The result result of of applying applying 25b. In thethis foursignal, all techniques methodologies show to the real good signal performance in terms in front of spectrum of PU transmission. occupancy. the four methodologies to the real signal in terms of spectrum occupancy. a) Figure 25a shows the frequencies observed between 2.3 GHz and 2.5 GHz. In this case, the three techniques provide affordable results classifying the occupation of the signal as can be seen in Figure 25b. In this signal, all techniques show good performance in front of PU transmission. a) b) b) Figure Figure 25. (a) A 25. (a) A real real signal signal obtained obtained from from the the frequency frequency band band [2.3–2.5] [2.3–2.5] GHz. (b) The GHz. (b) The result result of of applying applying the four methodologies to the real signal in terms of spectrum occupancy. the four methodologies to the real signal in terms of spectrum occupancy. In the case of the frequency range from 2.6 GHz to 2.8 GHz shown in Figure 26, all techniques provide good results detecting PUs. However, being K-means the one that shows the best detection Figure 25. (a) A real signal obtained from the frequency band [2.3–2.5] GHz. (b) The result of applying of frequency edges. the four methodologies to the real signal in terms of spectrum occupancy. Sensors 2019, 19, 4715 19 of 22 20 of 22 a) b) Figure 26. Figure (a)A 26. (a) Areal realsignal signal obtained obtained from from the the frequency frequency band band [2.6–2.8] [2.6–2.8] GHz. GHz. (b) (b) The The result result of of applying applying the four the four methodologies methodologies toto the the real real signal signal in in terms terms of of spectrum spectrum occupancy. occupancy. The frequencies examined from 698 MHz to 806 MHz appear in Figure 22a. This signal presents a 5. Conclusions non-constant power transmission of PUs around 760–780 MHz. In practical terms, this kind of activity In this totally determines work, occupied the frequency edges detection this bandwidth. Thus,phase, mentioned ML techniques andinthetheoriginal methodology proposed methodology in detect [10], was improved frequency based onasML space as occupied cantechniques. First, a22b. be seen in Figure NN with a manual threshold in the training phaseThewas implemented, allowing a more accurate signals observed in the frequencies of 806 MHz to 910 MHz classification of theareapproximation shown in Figure coefficients. 23. In this Similarly, applying EM to NN contributes to increase the effectiveness figure, most of the transmission signals are detected by all techniques. NN with manual threshold of the training phase. Besides this technique provides the bestavoids results.theThis intervention techniqueof another detects user with a signal to setlow a manual power that threshold of thecorresponds apparently NN training. to That is, we passed from a supervised technique to a fully automated algorithm. a PU. In contrary case, the original methodology does not detect changes on power and the bandwidth From theisresults size detected lower. of the phase of detection of frequency edges and PS to detect a PU presence correctly, we can The frequencies conclude measured thatfrom K-means is the 1.7 GHz to best 2 GHz ML aretechnique presentedanalyzed in Figurein24a. thisHere, work.inItgeneral, is also important to remark that an improvement in the detection, with signals the occupied and empty bands are detected correctly by the ML techniques as can be seen in Figure 24b. in a noisy environment, is observed. However, in Additionally, the range from this1.925 method GHzdoes to 2 not GHzrequire there are a great computational several transmissions. complexity, The NN with comparing manual with an NN that requires a more advanced hardware power. threshold techniques detects these changes in the occupation of the band. The frequency edges are not Throughout detected with high the differentbut precision, simulations, it hasisbeen this technique moreperceived sensitive tothat the changes. these precision does not depend only Figure on the 25ausedshows ML technique nor on the the frequencies chosenbetween observed method 2.3 of spectrum GHz andsensing, 2.5 GHz.it also depends In this on case, the the number of samples of the signal to be evaluated. Indeed, given that processed three techniques provide affordable results classifying the occupation of the signal as can be seen in elements are not directly the In Figure 25b. samples of theallspectrum this signal, techniques signal ( f ) but showXgood its approximation performance in front coefficients (obtained from of PU transmission. the multiresolution In the case of the analysis), frequency thatrange means the 2.6 from number GHz to of 2.8 points GHz ofshown the signalin Figure is reduced X( f ) 26, by 2 L all techniques provideLgood where results is the leveldetecting PUs. However, of decomposition in thebeing MRA. K-means Hence, theitone that shows the is noteworthy best that thedetection proposed of frequency edges.were tested on signals recovered from the work environment that only has 461 methodologies samples, which means that only 57 approximation coefficients will be obtained. Even though this 5. Conclusions number of samples is quite small for a multiband environment, the results of the methodology implemented In this work,in thisthe work frequencyare quite edgesgood. As part detection phase,of mentioned future work, in the methodology complete methodology proposed considering in [10], was improved based on ML techniques. First, a NN with manual threshold in theplatform. the ML techniques are thought to be implemented in a software defined radio training Also, phaseto getimplemented, was an accuracy description allowing a of proposed more accurate algorithms where classification of parameters as evacuation the approximation time coefficients. can be measured, Similarly, applying a cooperative cognitive radio EM to NN contributes network to increase theiseffectiveness going to be developed [27].phase. Besides of the training this technique avoids the intervention of another user to set a manual threshold of the NN training. Abbreviations That is, we passed from a supervised technique to a fully automated algorithm. CR From the results Cognitive radioof the phase of detection of frequency edges and PS to detect a PU presence correctly, we can conclude SDR Software-defined radiothat K-means is the best ML technique analyzed in this work. It is also important to remark DSA Dynamic spectrum access that an improvement in the detection, with signals in a noisy environment, is observed. SU Additionally, Secondary users this method does not require a great computational complexity, comparing with PU an NN that users Primary requires a more advanced hardware power. SS Spectrum sensing Sensors 2019, 19, 4715 20 of 22 Throughout the different simulations, it has been perceived that the precision does not depend only on the used ML technique nor on the chosen method of spectrum sensing, it also depends on the number of samples of the signal to be evaluated. Indeed, given that processed elements are not directly the samples of the spectrum signal X( f ) but its approximation coefficients (obtained from the multiresolution analysis), that means the number of points of the signal X( f ) is reduced by 2L where L is the level of decomposition in the MRA. Hence, it is noteworthy that the proposed methodologies were tested on signals recovered from the work environment that only has 461 samples, which means that only 57 approximation coefficients will be obtained. Even though this number of samples is quite small for a multiband environment, the results of the methodology implemented in this work are quite good. As part of future work, the complete methodology considering the ML techniques are thought to be implemented in a software defined radio platform. Also, to get an accuracy description of proposed algorithms where parameters as evacuation time can be measured, a cooperative cognitive radio network is going to be developed [27]. Author Contributions: A.P.-G., R.A.-G., and S.R.-B. conceived the original idea of how to use different machine learning technics purposes and designed the proposed methodologies. Y.M.-T. improved these proposals adding some ideas of how to obtain the data or how to do the simulations. Besides, Y.M.-T. performed all the experiments and simulations of this work. All authors wrote the article. Funding: This research received funding of the Mexican National Council of Science and Technology (CONACYT), Grant (no. 490180). Also, this work was supported by the Program for Professional Development Teacher (PRODEP). Acknowledgments: Authors are grateful to CONACYT, PRODEP and the Department of Electrical Engineering of UAM-I for providing the necessary support for this work. This work has been co-funded by the Spanish ministry of science through the project RTI2018-099880-B-C32 and with ERFD funds. Conflicts of Interest: The authors declare no conflict of interest regarding the publication of this manuscript. Abbreviations CR Cognitive radio SDR Software-defined radio DSA Dynamic spectrum access SU Secondary users PU Primary users SS Spectrum sensing MSS Multiband spectrum sensing WBSS Wide-band spectrum sensing MRA Multiresolution analysis HFD Higuchi fractal dimension ML Machine learning RL Reinforcement learning NN Artificial neural networks SVM Support vector machine CSS Cooperative spectrum sensing EM Expectation maximization NAC Normalized and interpolated approximated coefficients TW Test window STD Standard deviation PS Percentage of success References 1. Mitola, J.; Maguire, G.Q. Cognitive radio: Making software radios more personal. IEEE Pers. Commun. 1999, 6, 13–18. [CrossRef] 2. Haykin, S. Cognitive radio: Brain-empowered wireless communications. IEEE J. Sel. Areas Commun. 2005, 23, 201–220. [CrossRef] Sensors 2019, 19, 4715 21 of 22 3. Akyildiz, I.F.; Lee, W.-Y.; Vuran, M.C.; Mohanty, S. NeXt generation/dynamic spectrum access/cognitive radio wireless networks: A survey. Comput. Netw. 2006, 50, 2127–2159. [CrossRef] 4. Masonta, M.T.; Mzyece, M.; Ntlatlapa, N. Spectrum Decision in Cognitive Radio Networks: A Survey. IEEE Commun. Surv. Tutor. 2013, 15, 1088–1107. [CrossRef] 5. Arjoune, Y.; Kaabouch, N. A Comprehensive Survey on Spectrum Sensing in Cognitive Radio Networks: Recent Advances, New Challenges, and Future Research Directions. Sensors 2019, 19, 126. [CrossRef] [PubMed] 6. Hattab, G.; Ibnkahla, M. Multiband Spectrum Access: Great Promises for Future Cognitive Radio Networks. Proc. IEEE 2014, 102, 282–306. [CrossRef] 7. El-Khamy, S.E.; El-Mahallawy, M.S.; Youssef, E.N.S. Improved wideband spectrum sensing techniques using wavelet-based edge detection for cognitive radio. In Proceedings of the IEEE 2013 International Conference on Computing, Networking and Communications (ICNC), San Diego, CA, USA, 28–31 January 2013; pp. 418–423. 8. Kumar, A.; Saha, S.; Bhattacharya, R. Wavelet transform based novel edge detection algorithms for wideband spectrum sensing in CRNs. AEU Int. J. Electron. Commun. 2018, 84, 100–110. [CrossRef] 9. Diao, X.; Dong, Q.; Yang, Z.; Li, Y. Double-Threshold Cooperative Spectrum Sensing Algorithm Based on Sevcik Fractal Dimension. Algorithms 2017, 10, 96. [CrossRef] 10. Molina-Tenorio, Y.; Prieto-Guerrero, A.; Aguilar-Gonzalez, R. A Novel Multiband Spectrum Sensing Method Based on Wavelets and the Higuchi Fractal Dimension. Sensors 2019, 19, 1322. [CrossRef] [PubMed] 11. Ali, A.; Hamouda, W. Advances on Spectrum Sensing for Cognitive Radio Networks: Theory and Applications. IEEE Commun. Surv. Tutor. 2017, 19, 1277–1304. [CrossRef] 12. Zhou, X.; Sun, M.; Li, G.Y.; Juang, B.-H. Intelligent Wireless Communications Enabled by Cognitive Radio and Machine Learning. arXiv 2017, arXiv:1710.11240. 13. Bkassiny, M.; Li, Y.; Jayaweera, S.K. A Survey on Machine-Learning Techniques in Cognitive Radios. IEEE Commun. Surv. Tutor. 2013, 15, 1136–1159. [CrossRef] 14. Thilina, K.M.; Choi, K.W.; Saquib, N.; Hossain, E. Machine Learning Techniques for Cooperative Spectrum Sensing in Cognitive Radio Networks. IEEE J. Sel. Areas Commun. 2013, 31, 2209–2221. [CrossRef] 15. Oksanen, J.; Lunden, J.; Koivunen, V. Reinforcement learning method for energy efficient cooperative multiband spectrum sensing. In Proceedings of the 2010 IEEE International Workshop on Machine Learning for Signal Processing, Kittila, Finland, 29 August–1 September 2010; pp. 59–64. 16. Popoola, J.J.; Van Olst, R. Application of neural network for sensing primary radio signals in a cognitive radio environment. In Proceedings of the IEEE Africon’11, Livingstone, Zambia, 13–15 September 2011; pp. 1–6. 17. Shamsi, N.; Mousavinia, A.; Amirpour, H. A channel state prediction for multi-secondary users in a cognitive radio based on neural network. In Proceedings of the IEEE 2013 International Conference on Electronics, Computer and Computation (ICECCO), Ankara, Turkey, 7–9 November 2013; pp. 200–203. 18. Han, Z.; Zheng, R.; Poor, H.V. Repeated Auctions with Bayesian Nonparametric Learning for Spectrum Access in Cognitive Radio Networks. IEEE Trans. Wirel. Commun. 2011, 10, 890–900. [CrossRef] 19. Shah, H.A.; Koo, I. Reliable Machine Learning Based Spectrum Sensing in Cognitive Radio Networks. Wirel. Commun. Mob. Comput. 2018, 2018, 1–17. [CrossRef] 20. Azmat, F.; Chen, Y.; Stocks, N. Analysis of Spectrum Occupancy Using Machine Learning Algorithms. IEEE Trans. Veh. Technol. 2016, 65, 6853–6860. [CrossRef] 21. Shrestha, A.P.; Yoo, S.-J. Optimal Resource Allocation Using Support Vector Machine for Wireless Power Transfer in Cognitive Radio Networks. IEEE Trans. Veh. Technol. 2018, 67, 8525–8535. [CrossRef] 22. Sharma, V.; Bohara, V. Exploiting machine learning algorithms for cognitive radio. In Proceedings of the IEEE 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI), New Delhi, India, 24–27 September 2014; pp. 1554–1558. 23. Han, J.; Kamber, M. Data Mining: Concepts and Techniques, 3rd ed.; Elsevier: Burlington, MA, USA, 2012; ISBN 978-0-12-381479-1. 24. Jain, A.K. Data clustering: 50 years beyond K-means. Pattern Recognit. Lett. 2010, 31, 651–666. [CrossRef] 25. Meila, M. The uniqueness of a good optimum for K-means. In Proceedings of the 23rd International Conference on Machine Learning—ICML ’06, Pittsburgh, PA, USA, 25–29 June 2006; ACM Press: New York, NY, USA, 2006; pp. 625–632. Sensors 2019, 19, 4715 22 of 22 26. Dempster, A.P.; Laird, N.M.; Rubin, D.B. Maximum Likelihood from Incomplete Data via the EM Algorithm. J. R. Stat. Soc. 1977, 39, 1–22. [CrossRef] 27. Shrestha, A.P.; Yoo, S.-J. An energy efficient fair node selection for cooperative in-band and out-of-band spectrum sensing. Comput. Commun. 2018, 119, 83–93. [CrossRef] © 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).