# Introduction uditory information such as sound, speech, and music are processed in the brain through auditory pathways from the ear to the temporal lobe. Auditory signals are decoded in the brain, and interpreted. There are 3 mains stages of sound processing. When a person pays attention to a particular sound, this involves processing through a primary auditory pathway that starts as a reflex and passes from the cochlea to a sensory area of the temporal lobe called the auditory cortex. Each sound signal is decoded in the brain stem to its components such as time of duration, frequency and intensity. After two additional processing steps the brain localizes the sound source, or it knows from which direction the sound is coming. Once the sound is localized by the brain, the thalamus region of the brain is involved in producing a response through any other sensory area such as a motor response, or a vocal response. Source localization of sound from different directions has been addressed by researchers in several ways. Sound localization has been analyzed by different researchers. Virtual auditory stimuli was presented in 6 directions using headphones to human subjects, simultaneously recorded EEG data was classified using Support Vector Machines (SVMs) in I. Nambu et al. [1]. In S. Makeig et al. [2], Independent Component Analysis (ICA) was used to analyze Event Related Potentials for sound stimulus. Monaural and binaural auditory stimulus was classified using ERPs in A. Bednar et al. [3]. Interaural Time Difference (ITD) was used in Letawski [4] for auditory localization. However, the methods presented do not present a deeper comprehensive analysis of EEG responses to auditory stimulus, in as much as EEG is a stochastic signal, that varies in both time and frequency varying. In this paper, we present time frequency analysis as an alternate and viable method for identifying the source of sound by processing EEG responses to auditory stimulus. This paper is organized as follows. Section II presents a background on Electroencephalogram. Section III presents the time frequency analysis methods, Section IV presents the methodology for feature extraction and sound localization using the SVM classifier. Section V presents the results, and VI the conclusions. # II. EEG Data Collection with Auditory Stimulation Electroencephalogram (EEG) signals are electrical activity of the brain recorded using electrodes placed on the scalp using an EEG cap. The number of electrodes can vary from as few as 8 to 256. Each electrode provides a time series of voltage measurements at a particular sampling rate. The experiments for this paper were done in the Brain Computer Interface Lab (BCI Lab) at the University of Puerto Rico at Mayaguez (UPRM). The human subjects were college students with normal health. Informed consent was obtained from the participants according to an approved protocol by the institutional review board (IRB) of UPRM. The EEG equipment used to collect the data is the BrainAmp from BrainVission, LLC which has 32 channels in the Acticap arranged according to the 10-20 system of electrode placement. The acticap with the 32 electrodes is worn by the subject. Conducting gel is used to lower the impedance of electrodes making contact to the scalp. The impedance of the electrodes were adjusted to be below 10 kilo Ohms. The experiments were conducted in a quiet room, with only the subject and the investigator. A series of 16 sound stimuli were presented to the subject in right and left ear through wearing headphones. Two classes of 2 secs stimuli were applied randomly. The first is a pure tone of 3 kHz with 500 ms increasing tone, steady during 1 second, and then decreasing tone during 500 ms. The second stimulus was a burst of 3 kHz pure tone with durations of 100 ms ON, and 100 ms OFF for 10 trials. This give a total of 224 auditory stimuli (112 right/112 left). Each series of 16 stimuli takes around 2 minutes and the participant is allowed one minute to rest between trials. The sound stimuli were presented through a program written in Matlab. The National Instruments device was used to put markers in the EEG data as they were recorded simultaneously when auditory stimuli were presented. The results presented in this paper are from the analysis of EEG data collected from 3 different subjects. # Time-Frequency Analysis The 32 channel EEG data collected are preprocessed using band-pass filtering to remove low frequency noise, artifacts due to eye blinks and hardware induced artifacts. In order to extract meaningful features from signals for source identification, it is necessary to map data in overlapping feature spaces to a separable space by high dimensional feature mapping. This increases the dimensionality of the feature space, but the classes are easily separable in this space, and linear classifiers can be used to classify the data in this high dimensional space. In this project, we have mapped 1-D signal spaces to 2-D signal spaces through timefrequency methods (TFM). The group of signals taken from the electrodes on the scalp are in the space of continuous physical signals ( ) L ? (see Fig. 4). These In multiway signal-processing the selection of efficient and optimal methods for the processing of electroencephalographic signals is a problem addressed by many authors [5], [6], [7]. Fig. 5 shows the stages or levels of neural signal analysis presented in this paper. Time-frequency methods allow the observation of details in signals that would not be noticeable using a traditional Fourier transform. One of the problems of conversion to time-frequency spaces is that, according to the length of the input signal, the conversion can be time, and memory consuming. The two methods for time frequency analysis considered here are the Short Time Fourier Transform (STFT), and the Wavelet Transform (WT). CSTFT is defined as follows: , [ , ] [ ] N kn x v N N n S m k x n w m n W ? ? ? = ? ? ? ? Z (2) where, × ? Z Z . This ensures that the mapping is constant, independent of the length of the input signal because through, a different segmentation, it can be ensured that the conversion falls into the same signal space. The new signal space gives richer information. A special group of the STFT is the Gabor transform, a generalized version is given in Equation 3. This time-Figure 6 shows the EEG ERP responses to 2 direction auditory stimulation. The algorithm for feature extraction and classification using a support vector machine (SVM) classifier is shown in Figure 7. Time Frequency Method (TFM) is either the CSTFT or the CWT. The 32 channel EEG data were organized as tensors [9][10][11][12] and the time frequency methods were implemented in Python and visualized using MNE [13][14][15]. The EEG data of 112 trials is divided in to 56 for trials for training and 56 trials for testing. 40 random trial averaging was done for training and testing of the SVM classifier. The results for two subjects is shown in Table 1. # Figure 7: Flow chart of the classification algorithm Figure 8 shows the ERPs for 3 EEG channels that are from the frontal and temporal regions of the brain involved in processing of auditory stimulus. The time delay in these evoked potentials can be clearly seen. Similarly, Figures 9 and 10 shows the time delays in the evoked potentials for a 4 and 8 direction auditory stimulation, respectively. In this case, the occipital and parietal lobes are involved. This shows that as more complex sounds are presented, different neuronal pathways are activated in processing these auditory stimuli. Figure 11 shows the time frequency representation for the 3 features for the 2 direction auditory ERPs. Figure 12 # Results Table 1 shows very good accuracies for source localization when two directions auditory stimulus are presented. For the 4 direction case, in Table 2 it can be seen that the classifier has difficulty in identifying the South direction. The salient features from the CWT in each of the neuronal regions for multiple direction auditory stimulation can be seen from the time frequency representations. The CWT features performed well for 4 directions auditory stimulus localization. (W) R (E) L (W) 100% 0.10% R (E) 8% 92% L(W) R(E) L(W) 88% 12% R(E) 15% 85% L (W) R (E) L (W) 100% 0.0% R (E) 0.0% 100% As can be seen, results for 2 directions is close to 100%. The time delay plots in Figures 9 and 10 show that for more source directions, neuronal signals from the occipital and parietal regions have higher discriminatory power than frontal or temporal regions. # VI. # Conclusion Auditory processing in the brain was analyzed using time frequency analysis of EEG signals acquired from the brain using sound stimulus presentation. The results show that as number of source directions is increased, different regions of the brain are involved in processing the signals. This implies that as sound becomes more complex such as in speech, music, and language perception, higher intricate auditory pathways in the brain are involved in processing and decoded these sound patterns. The comparison of the time domain vs timefrequency domain factorization of EEG shows that increasing the dimensionality of the EEG signals, provides a better way to discriminate the ERP of auditory stimuli and localize sources. Apart from sound direction localization from EEG, it is evident that EEG can also be used as a neuroimaging modality for understanding and decoding sensory and motor functional pathways in the brain. This work can be extended to analyzing complex music, speech and language perception in the brain. 1![Figure 1: Pure tone and burst sound stimulus Apart from 2 directions of Left and Right for sound stimulus presentation, four direction stimuli were also presented to the subjects. For the four direction sound data presentation, we used sound localization using the head related transfer function. The HRTF is a novel technique to simulate direction of arrival of sounds. HRTF is also called as transfer function from the free field to a specific point of the ear canal. In mathematical terms the transfer functions for Left and Right are shown below: 0 0 ( , , , , ) ( , , , , ) / ( , ) ( , , , , ) ( , , , , ) / ( , ) L L L R R R H H r w P r w P r w H H r w P r w P r w ? ? ? ? ? ? ? ? ? ? ? ? = = = =](image-2.png "Figure 1 :") 2![Figures 2 and 3 show the directional sound stimulation implementations using HRTF.](image-3.png "Figures 2") 23![Figure 2: Two direction sound stimulus presentation](image-4.png "Figure 2 :Figure 3 :") ![sequences are the mapping from ( ) L ? to ( ) ? Z through an analog to digital process. The sampled/quantized version is computing using a window operator that transform ( ) ? Z to 2 ( ) N ? Z when an EEG experiment is conducted. These signals now are in 2 ( ) N ? Z and is the space of discrete finite signals with finite energy. EEG signals can be processed by a computer using timefrequency tools to map the signals from space . The increased dimensionality of the signal space reveals more information about the original signal. The mapping from while processing the EEG signal analysis in the transformed space.](image-5.png "") 4![Figure 4: A general approach for time-frequency signal analysis](image-6.png "Figure 4 :") 5![Figure 5: Stages of EEG signal analysis using time frequency methods a) Short Time Fourier Transform Analsys The STFT has been a widely-used timefrequency signal processing operator. The CSTFT has certain advantages over STFT that are mentioned below. I. Short Time Fourier Transform Definition Given a signal [ ] ( ) N x n ? ? Z](image-7.png "Figure 5 :") ![(time shift) and N k ? Z (frequency axis). The CSTFT makes a mapping from the space ( ) N ? Z to the space ( )](image-8.png "") 6![Figure 6: Time Domain Event-Related Potentials for Auditory Stimuli N=112 Right and Left](image-9.png "Figure 6 :") ![Figure8shows the ERPs for 3 EEG channels that are from the frontal and temporal regions of the brain involved in processing of auditory stimulus. The time delay in these evoked potentials can be clearly seen. Similarly, Figures9 and 10shows the time delays in the evoked potentials for a 4 and 8 direction auditory stimulation, respectively. In this case, the occipital and parietal lobes are involved. This shows that as more complex sounds are presented, different neuronal pathways are activated in processing these auditory stimuli.Figure11shows the time frequency representation for the 3 features for the 2 direction auditory ERPs. Figure12shows the topoplots for the EEG signals recorded during left and right direction auditory stimulation. Figures 13 to 15 shows the time frequency representation for the 3 features for the EEG signals evoked by auditory stimulation in 4 directions.](image-10.png "") 8![Figure 8: Time Delay Two Class, Three Features Evoked Potentials](image-11.png "Figure 8 :") 910![Figure 9: Time Delay Four Class, Three Features Evoked Potentials](image-12.png "Figure 9 :Figure 10 :") 11![Figure 11: Time Frequency Representations for the Three Features for Binary Classification](image-13.png "Figure 11 :") 12![Figure 12: Event Related Potentials Topomaps for Left and Right Events](image-14.png "Figure 12 :") 1314![Figure 13: Time Frequency Representations for Feature One in the Four Directions](image-15.png "Figure 13 :Figure 14 :") 15![Figure 15: Time Frequency Representations for Feature Three in the Four Directions](image-16.png "Figure 15 :") 1 2NESWN100%0%0%0%E0%82%0%18%S35%1%27%38%W0%4%0%79% © 2019 Global Journals frequency transform uses a window defined as a Gaussian function. ( , ) ( ) ## b) Continuous Wavelet Transform and Discrete Wavelet Transform Wavelet Transform is based on a group or class of translated and dilated functions called wavelets. The continuous wavelet functions are defined in Blu et al. [8] as: And the CWT is defined based on these wavelets. The CWT gives a time frequency representation in terms of delay and dilation. The CWT representation has advantages over the CSTFT at low frequencies. In EEG, the presence of information at low frequency is very common. Therefore, CWT is better than the CSTFT for EEG time-frequency analysis. IV. ## Feature Extraction and Classification * Estimating the intended sound direction of the user: toward an auditory brain computer interface using out of head sound localization MNambu MEbisawa SKogure HYano YHokari Wada Plos One 8 2 Feb. 2013 * Blind separation of auditory event-related brain responses into independent components SMakeig TPJung AJBell DGhahremani TJSejnowski Proc. Nat. Acad. Sci 94 Sept. 1997 * Different spatio-temporal electroencephalography features drive the successful decoding of binaural and monaural cues for sound localization ABednar FMBoland ECLalor European Journal of Neuroscience 45 2017 * Auditory spatial perception TRLetowski STLetowski ARL-TR-6016 2012 * Tensor decompositions for signal processing applications: From two-way to multiway component analysis ACichocki DMandic LDe Lathauwer GZhou QZhao CCaiafa HAPhan IEEE Signal Processing Magazine 32 2 March 2015 * A review of channel selection algorithms for EEG signal processing TAlotaiby FE AEl-Samie SAAlshebeili IAhmad 10.1186/s13634-015-0251-9 EURASIP Journal on Advances in Signal Processing 2015 1 66 2015 * Structure out of chaos: Functional brain network analysis with EEG, MEG, and functional MRI ECVan Straaten CJStam neural Networks in Psychiatry 2013 23 * TBlu JLebrun Linear Time-Frequency Analysis II: Wavelet-Type Representations. ISTE 2010 * 10.1002/9780470611203.ch4 Available * SVD based initialization: A head start for nonnegative matrix factorization CBoutsidis EGallopoulos Pattern Recognition 41 4 2008 * Nonnegative Matrix and Tensor Decomposition of EEG FCong 09 2016 * Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation ACichocki RZdunek A.-HPhan SAmari 2009 John Wiley Sons Ltd * Tensor decompositions and applications TGKolda BWBader SIAM Rev 51 3 Aug. 2009 * 10.1137/07070111X Available * MEG and EEG data analysis with MNE-Python AGramfort MLuessi ELarson DEngemann DStrohmeier CBrodbeck RGoj MJas TBrooks LParkkonen MHmlinen Frontiers in Neuroscience 7 267 2013 * 10.3389/fnins.2013.00267 * Scikit-learn: Machine learning in Python FPedregosa GVaroquaux AGramfort VMichel BThirion OGrisel MBlondel PPrettenhofer RWeiss VDubourg JVanderplas APassos DCournapeau MBrucher MPerrot EDuchesnay Journal of Machine Learning Research 12 2011 * Tensorly: Tensor learning in python JKossaifi YPanagakis AAnandkumar MPantic abs/1610.09555 CoRR 2018