ORIGINAL RESEARCH published: 31 May 2022 doi: 10.3389/fpsyg.2022.849083 Spanish Emotion Recognition Method Based on Cross-Cultural Perspective Lin Liang 1* and Shasha Wang 2 1 College of Foreign Languages and Cultures, Xiamen University, Xiamen, China, 2 Translation and International Studies Department, Universidad Autónoma de Barcelona, Bellaterra, Spain Linguistic communication is an important part of the cross-cultural perspective, and linguistic textual emotion recognition is a key massage in interpersonal communication. Spanish is the second largest language system in the world. The purpose of this paper is to identify the emotional features in Spanish texts. The improved BiLSTM framework is proposed. We select three widely used Spanish dictionaries as the datasets for our experiments, and then we finally obtain text sentiment classification results through text preprocessing, text emotion feature extraction, text topic detection, and emotion classification. We inserted the attention mechanism in the improved BiLSTM framework. It enables the shared feature encoder to obtain weighted representation results in the Edited by: extraction of emotion features, which enhances the generalization ability of the model for Xiaoqing Gu, Changzhou University, China text emotion feature recognition. Experimental results demonstrate that our approach Reviewed by: performs better for specialized Spanish dictionary datasets. In terms of emotion Zhiming Shi, recognition accuracy, the average value is as high as 76.21%. The overall performance Huaqiao University, China Hui Ouyang, outperforms current comparable machine learning methods and convolutional neural Wuhan University, China network methods. Hongru Zhao, Soochow University, China Keywords: emotion recognition, text processing, natural language processing, BiLSTM, Spanish, cross-cultural Khin Wee Lai, University of Malaya, Malaysia *Correspondence: INTRODUCTION Lin Liang lianglin@xmu.edu.cn Economic globalization has become a part of our daily life, and linguistic communication is an important part of the cross-cultural perspective. Proficiency in the semantic system of Spanish is a Specialty section: prerequisite for the rapid establishment of cross-cultural communication. It is a necessary skill for This article was submitted to cross-cultural personnel involved in diplomacy, foreign trade, and media communications. While Emotion Science, language learning is important, the emotions contained in language are the cultural manifestation a section of the journal of a nation. Science and technology have advanced rapidly and Spanish translation systems have Frontiers in Psychology become highly sophisticated. However, there are still deficiencies in dealing with the emotions Received: 05 January 2022 that language contains. Emotion is ubiquitous in our human society and it can be a good way Accepted: 25 February 2022 to explain our moods, feelings, and experiences. This is a major characteristic of our human race. Published: 31 May 2022 For people all over the world, there are commonalities in the expression of emotions, except for Citation: the use of facial expressions and body language. Textual language is like a big mountain that Liang L and Wang S (2022) Spanish Emotion Recognition Method separates people’s emotional expressions in different language systems. It is often manifested itself Based on Cross-Cultural Perspective. as a language barrier that affects cross-cultural emotional perception and expression from a cross- Front. Psychol. 13:849083. cultural perspective. Communication between different languages exists not only in the literal sense, doi: 10.3389/fpsyg.2022.849083 but also in the context, and more importantly, in the emotions implied by the text. By grasping Frontiers in Psychology | www.frontiersin.org 1 May 2022 | Volume 13 | Article 849083 Liang and Wang Deep Learning-Based Spanish Emotion Recognition the emotional baseline of a text, we can avoid various problems in has a bidirectional context learning mode, if the contextual cross-linguistic communication. There are three ways of dialogue information is not coherent, the learned features will be biased between people, the first way is locutionary, which refers to the and the prediction values will be affected. BERT can currently actual effect produced between the dialogue; the second way is perform language processing tasks not only in English but can illocutionary, which refers to the meaning beyond the words also be extended for multilingual versions. It has a certain between the words; the third way is perlocutionary, which refers foundation for Spanish processing, but when faced with Spanish to the effect produced after the dialogue is completed. In fact, text language text processing tasks, it cannot achieve the language emotion recognition has the same steps, the first step is to analyze processing effects of English (Pires et al., 2019). To address the literal meaning, the second step is to analyze the extra-verbal this issue, more and more researchers have begun to study the meaning, and finally to extract the emotional baseline of the text. multilingual system of BERT, and the processing of the BERT In recent years, researchers have used neural network framework in Spanish will be greatly improved in the future algorithms to automatically detect large amounts of texts and (Martin et al., 2019; Virtanen et al., 2019). perform emotional feature recognition. And a large number of Considering that a complete emotion recognition system industries with application potential have emerged (Strapparava, involves the fusion of expression, voice, and text, the research cost 2016). It is thus clear that emotion feature computing technology is huge. Therefore, this paper focuses on the emotion recognition has become the key to emotion recognition intelligent systems. of Spanish text. First, we use text pre-processing to convert Human emotions can be expressed through facial expressions, Spanish text into a suitable tensor; then we complete text emotion body movements, intonation, and speech systems. If computers feature extraction by the improved BiLSTM framework; then we want to capture and analyze these emotional characteristics, input all features to the topic detection layer to complete sentence they have to use sensors to acquire the above characteristics clustering and topic labeling; then we input the emotion classifier and convert them into computer language. Emotion recognition to classify the text emotion according to the weights, and finally, intelligence systems aim to enable computers to automatically we get the result of the emotion classification of the Spanish text. calculate emotional characteristics and predict specific human The remaining sections of this article are organized as emotional states and behaviors through factors such as speech, follows. Section Related Work discusses the work related to video, text, and muscle signals. These predictions are transmitted language emotion recognition. Section Materials and Methods as input to the secondary system, which analyzes and reacts to describes in detail the relevant principles and implementation these sentiments (Calvo and D’Mello, 2010). Due to the rapid process of the Spanish text emotion recognition method. Section development of the internet, various social media applications Experiments reports the experimental data set, evaluation have emerged, and people prefer to share their daily feelings indicators, and analysis of experimental results. Finally, and emotions on the internet. Therefore, the text will gradually Section Conclusion summarizes our research and reveals some become an important carrier for people’s emotional attachment. further research works. The text data on social media will become an important database for text emotion recognition (Hasan et al., 2019). The approach of natural language processing is gradually RELATED WORK changing. In the past, language processing methods simply connected and tracked the words of the text, and inferred a Nowadays, although English is an international language. representation of semantic features from the context. Current However, there are about 437 million people in the world who language processing approaches perform semantic segmentation speak Spanish as their mother tongue. Spanish is the second most based on context and then capture the segmented features spoken language system in the world, after Chinese. The Spanish- from the word level (McCann et al., 2017; Howard and Ruder, speaking regions of the world are shown in Figure 1. At the 2018). In the form of word expression, each word is expressed same time, Spanish is one of the six working languages of the as embedded and each embedding form contains contextual United Nations. Therefore, it is of extraordinary significance to semantic segmentation information. The complex features of understand the emotional messages contained in Spanish texts. lexemes are obtained by splitting words and finally modeling If one wants to fully understand the emotions in Spanish texts, based on the sequence of semantic features of lexemes, e.g., a one needs to start with the disciplines of Spanish sociology, word with multiple meanings. In the current context processing Spanish linguistics, Spanish history, Spanish philosophy, Spanish system, Google launched Bidirectional Encoder Representation ethics, and Spanish psychology. The emotion theory method from Transformers (BERT) (Devlin et al., 2018), which has proposed by Ekman is the basic theory of most current emotion demonstrated excellent language processing capabilities in recognition research (Yadollahi et al., 2017). In the research of a bidirectional context. BERT is a unique neural network emotion recognition, a series of models are gradually extended using Transformer Encoders (Vaswani et al., 2017). BERT and dedicated to the study of human emotion decomposition. abandons the traditional one-way language model shallow Anger, disgust, surprise, sadness, happiness, and fear are the main collocation approach and adopts the Masked Language Model recognition baselines for all emotion recognition research. From (MLM) bi-directional collocation approach, which can efficiently the perspective of the development of human history, the above compute and acquire features in bi-directional words natural six emotions are the basic human emotions, and their expressions language. However, BERT requires a certain degree of coherence are relatively straightforward, so they are easy to express in the between sentences and contexts. Because the BERT network work of language text recognition. Frontiers in Psychology | www.frontiersin.org 2 May 2022 | Volume 13 | Article 849083 Liang and Wang Deep Learning-Based Spanish Emotion Recognition FIGURE 1 | Spanish-speaking regions of the world. In the research of Spanish emotion recognition, feature learning methods have higher requirements for designing neural mining is one of the joint steps. Molina-González proposed networks. Firstly, the text emotion feature dictionary needs to a classification system, which is mainly used for the polarity be classified and then manually labeled with emotion tags. The classification of Spanish-specific terms and vocabulary (Molina- process of feature training is tedious, but the results of text González et al., 2015); Martínez-Cámara proposes a classification emotion feature recognition are still good. The method based on method based on a machine learning approach to classify social the text emotion dictionary has relatively high requirements for media texts with polarities. Experimental results demonstrate the construction of the data set, and it is more labor-intensive. the effectiveness of the method (Martinez-Camara et al., 2015). The advantage of this method lies in unsupervised training. For However, as a small language, Spanish has little research on a text database that has been subjected to emotional standards, language and text emotion recognition. Most of the emotion it can be directly used as the input of this method. The blended recognition studies for Spanish are based on the SEL dictionary, learning approach focuses more on the annotation of emotion which is a Spanish text emotion classification dictionary features of linguistic texts and requires a higher level for the established specifically for Spanish. Due to the limitations of the researcher. The researcher must have not only technical skills but Spanish emotion recognition dictionary, many researchers have also the ability to fully understand the language and culture. The tried to transfer the learning of emotion recognition models from method combines speech, image, and text factors for integrated English to Spanish. Redondo has improved on the current English emotion detection, which can guarantee the accuracy of emotion emotion recognition framework ANEW for emotion recognition recognition results to a great extent and is suitable for the overall in Spanish, and the improved method has higher recognition system application of emotion detection. The main problem of accuracy (Redondo et al., 2007). Plaza-del-Arco discovered the the keyword-based method is the degree of matching between similarities between the NRC strength dictionary (Mohammad, emotion features. At present, the performance evaluation is not 2017) and Spanish and optimized the dictionary to adapt to the as good as the previously mentioned method, so fewer researchers Spanish application (Plaza-del-Arco et al., 2018). In the literature will use this method. Generally speaking, this method is only used del Arco et al. (2018), the author optimized the WordNet- in some specific word emotion recognition situations. Affect method for classifying emotion recognition tasks for Deep learning methods have performed well in research Spanish, set the corresponding features at each difficulty level, on text emotion classification and are preferred by most and achieved excellent results in the final difficulty evaluation of researchers. At present, the application of deep learning emotion recognition. methods to text emotion classification research is based on the Currently, the most common methods used in research construction of a dataset, which in this case is a linguistic text on language and text emotion recognition are deep learning sentiment dictionary database. In the research of constructing and machine learning. Commonly used deep learning methods a data set, Rao et al. (2014) presented a neural network such as long and short-term memory (LSTM) (Hochreiter and approach supplemented with a pruning strategy to automate the Schmidhuber, 1997) and BiLSTN (Schuster and Paliwal, 1997). It construction of textual dictionary databases, and to automatically has been widely used in end-to-end text emotion classification extract emotion features of dictionary databases at the word level systems, and it is very effective in encoding text emotion and to automatically encode them. Strapparava (Strapparava and feature sequences. Deep learning methods tend to maintain Mihalcea, 2008) focused on the text emotion system for news relatively fast detection speed and detection accuracy. Machine headlines and constructed a handwritten text emotion lexicon, Frontiers in Psychology | www.frontiersin.org 3 May 2022 | Volume 13 | Article 849083 Liang and Wang Deep Learning-Based Spanish Emotion Recognition which achieved good results in fine-grained data evaluation. stages: (1) pre-processing; (2) text emotional feature learning Neviarouskaya (Neviarouskaya et al., 2007) focuses on text framework; (3) topic detection; (4) text emotion classifier. The emotion systems in virtual social environments. To achieve overall process is shown in Figure 2. text emotion recognition and visualization, rule-based deep learning methods have been properly applied and studied. Pre-processing Banhakawi (Bandhakavi et al., 2017) combined special terms in Text preprocessing is a key step in natural language processing. text data and manually annotated emotion text to extract emotion There are many differences between the characteristics of Spanish features of text from neural networks and used the MATCH corpus and English corpus. For example, Spanish can only use algorithm for matching to solve the problem of missing emotion the JIEBA library for word segmentation, while English word features. In the global semantic evaluation contest SemEval, segmentation only requires spaces. Due to the characteristics of most researchers who study text emotion recognition use shared the Spanish corpus, a special preprocessing was performed on data sets. Because sharing data sets can significantly save the Spanish dictionary dataset. The preprocessing is shown in the development time of text emotion recognition algorithms. Figure 3. Moreover, in studies where text datasets are shared, each text In the pre-processing program. First, we input raw data into word contains manually annotated a priori information, so the word segmentation operation module. The model’s accuracy the evaluation results of all the proposed methods are highly can be improved by using proper word segmentation, and it can informative. In the WASSA2017 conference, some researchers also assist the algorithm to build the correct semantic emotional have a great advantage in mining the emotional feature of model. Therefore, the word segmentation operation is a key step. the textual emotional lexicon (Mohammad and Bravo-Marquez, Next, it enters the Cleaning module to eliminate useless tags, 2017). However, although great results have been achieved in special symbols, and deactivated words in the dataset to reduce current research on English text emotion recognition, such as the probability of text emotion misidentification. Then it enters WNA (Strapparava and Valitutti, 2004) and LIWC (Boot et al., the Normalization module to ensure the good convergence of 2017). Yet, few models can be successfully transferred to Spanish the network. And it is input to the feature extraction module text emotion recognition. Zahiri proposed a two-stream LSTM method to obtain the deep connection of text emotional features (Zahiri and Choi, 2018). He accomplished lexeme classification by migration learning and integrated it with the BERT algorithm. Finally, excellent recognition accuracy was obtained in the text emotion dictionary. In the literature Jabreel and Moreno (2019), to transform the text emotion classification problem into a binary classification problem, the authors adopt a deep neural network approach to encode the text sequences and embed an attention mechanism unit and a batch normalization unit. Finally, the classification task is completed by ranking the corresponding emotion features of the matched texts. The experimental results demonstrate a correct classification rate of 96.3%. Chatterjee (Chatterjee et al., 2019) focuses on the study of emotion detection in English conversations. To improve the accuracy of emotion detection, he uses user conversations as the input of the model and divides the input into two sections. Each segment independently corresponds to a different LSTM layer, and the features acquired by each LSTM are different. The first layer of the LSTM sequentially encodes the semantic features to capture the semantic features of the dialogues. The second layer of LSTM traverses the sequence of emotion features to match the emotion word features in the dialogue. Finally, the two outputs are fused and matched, and then the conversation emotion detection results are output. MATERIALS AND METHODS Pipeline Overview We propose an improved BiLSTM framework to recognize the emotions of Spanish text. The overall implementation process of FIGURE 2 | Spanish text emotion recognition process. emotion recognition is mainly divided into the following four Frontiers in Psychology | www.frontiersin.org 4 May 2022 | Volume 13 | Article 849083 Liang and Wang Deep Learning-Based Spanish Emotion Recognition FIGURE 3 | Pre-processing process. FIGURE 4 | Improved bidirectional long short-term memory (BiLSTM) framework. to complete the emotion feature extraction of lexemes. N-gram In the improved BiLSTM model. To accelerate the language model and Word2vec distributed model are commonly convergence of the network and prevent overfitting, we connect used for feature processing units. Finally, we perform modeling. a Batch Normalization unit after each convolutional layer; then We chose a similarity algorithm and a classification algorithm connect a ReLU activation layer to reduce the computation; in the modeling. Through text pre-processing, the Spanish data and finally, connect a maximum pooling layer to reduce the set is converted into a tensor, which provides a reference for the number of parameters and remove redundant information. The hyperparameter selection of the later model, and improves the parameter configuration of each convolutional layer is shown in overall evaluation index of the model. Table 1. Assuming that the features in the input convolutional layer and finally output high-level semantic features are 1 × 74 × 128 Text Emotion Feature Learning Framework In the work of text emotion feature learning, we proposed TABLE 1 | Architecture of improved bidirectional long short-term the contextual bidirectional long short-term memory attention memory (BiLSTM) layer. network (we call it improved BiLSTM). For Spanish text emotion recognition, the network is organized into three layers: the input Type Output Convolution Stride Padding dimension kernel layer, the language-related emotion feature encoder layer, and the text emotion classification layer. The model structure is shown in Convolution 26 × 751 × 128 7×7 1×1 SAME Figure 4. Batch normalization 26 × 751 × 128 Among them, the language-related emotional feature encoder Non-linear activation 26 × 751 × 128 is made up of two layers of neural networks, the first layer is a two- Max pooling 13 × 370 × 128 2×2 2×2 SAME layer convolutional neural network (CNNs), and the second layer Convolution 1 × 370 × 128 13 × 7 1×1 VALID is a two-layer bidirectional long short-term memory (BiLSTM). Batch normalization 1 × 370 × 128 The objective is to learn the spatial distribution of text emotional Non-linear activation 1 × 370 × 128 features from the input sequence features. Max pooling 1 × 370 × 128 1×5 1×5 SAME Frontiers in Psychology | www.frontiersin.org 5 May 2022 | Volume 13 | Article 849083 Liang and Wang Deep Learning-Based Spanish Emotion Recognition dimensions, and then reshape them into 74 × 128-dimensional or a simple sentence. In some cases, it is also feasible to encode features, which are used in the BiLSTM network followed by the keywords or terms specifically. In the topic detection model we feature sequence M, as shown in the following Equation (1). designed, topic detection is based on the fact that each sentence or paragraph has an individual label, which is determined by the M = [m1 , m2 , ..., mt ] (1) clustering algorithm in preprocessing. In other words, text topic detection represents clustering from paragraph to sentence to Where t represents the length of the characteristic sequence M. phrase. It is further divided into different clusters. The subject LSTM network is an optimization of RNN network, the purpose objects mapped by each group of clusters are different. After is to solve the dependency problem of sequence. The LSTM unit completing sentence clustering, the next task is topic labeling. includes an input gate it , a forget gate ft , an output gate ot and a We use the hierarchical agglomerative clustering algorithm memory unit ct to update the hidden state ht , as shown below: (HAC) (Schütze et al., 2008) to complete the classification it = σ(Wi xt + Vi ht−1 + bi ) (2) task of Spanish sentences. According to the rules of the HAC algorithm, we need to obtain a text partition to cut into a hierarchical structure. Then, we used different thresholds to ft = σ(Wf xt + Vf ht−1 + bf ) (3) define the text hierarchy generated by these cuts and compared the text threshold with the cluster similarity value. According to the comparison result, the similarity is adjusted accordingly. ot = σ(Wo xt + Vo ht−1 + bo ) (4) Therefore, the sentence clustering process will be a cyclic difference compensation work. Until the cluster similarity is less ct = ft K ct−1 + it K tanh(Wc xt + Vc ht−1 + bc ) (5) than the threshold, the clustering is considered complete. To obtain the threshold value, we defined the standard cut point in K the hierarchical structure. According to the literature Shulcloper ht = ot tanh(ct ) (6) (2009), two evaluation indicators were proposed for validation. J The two evaluation indicators are as follows: (1) The average of Where is a kind of function which similar to the the similarity between all morphemes. (2) The average value of multiplication operate, V represents a matrix related to weight the maximum similarity between any morphemes. As shown in and b represents the learning vector. To improve the performance the following formula. of the model, the morphemes were trained on two LSTMs. The first one is a rightward shifting morpheme starting from the left sim (T1 , T2 ) side, and the next one is a reverse copy of the characters. The w ∈ {T1 } max sim (w, T2 ) ∗idf (w) P  outputs of the forward and reverse passes were combined in series 1 = w ∈ {T1 } idf (w) P before passing to the next layer. Finally, the activation function is 2 used to obtain the prediction results. Figure 5 depicts the LSTM ! w ∈ {T2 } max sim (w, T1 ) ∗idf (w) P network’s working principle. + (7) w ∈ {T2 } idf (w) P In the task of acquiring the feature sequence, to obtain a better prediction model, the contextual relationship needs to be considered at the same time. Two LSTM networks are jointly Where Ti represents a sentence, and w represents a word used, one is used for standard feature sequence learning, and contained in the sentence Ti . To compare the sentence similarity the other is used for reversed feature sequence learning. Finally, of the n-dimensional vector, the cosine similarity is also taken the obtained hidden state information is connected to synthesize into consideration. the final output. In the improved BiLSTM model, the temporal The topic label is a comment on the topic of each sentence relationship of the input intermediate sequence feature M is cluster, each sentence is clustered according to contextual key modeled by using a two-layer BiLSTM. The hidden vector phrases, and a simple description of the extracted topic type is representation of BiLSTM comes from the forward and reverse called annotation. The area covered by tags is smaller than the → ← LSTM, which are represented by h t and h t . The number of topic, so the division of tags under the same topic becomes a hidden layer nodes in challenge. The labels belonging to the clustering methods cannot h each layer i of LSTM is 128, and the final → ← represent the topics completely and correctly. For example, the hidden vector ht =σ h t , h t can be obtained by using non- sentence centroid method commonly used in sentence clustering linear transformation at the same time. The obtained output operations can only represent sentence features, not topics. characteristics ht of all time steps are used as the final shared Therefore, the later manual correction is particularly important. characteristics among different tasks. To figure out this issue, some researchers presented candidate topic models. All topics contained in each cluster are treated Topic Detection similarly as candidate topics, and TF-IDF weighted scores are Topic detection is an intermediate part of text emotion detection, set in the sentences to measure the relevance between sentences which can map texts to different emotion topics. In the process and phrases. Finally, the top three noun phrases are selected of theme detection, we combine the analysis methods of different as the topics of the sentence clusters for subsequent topic text units. The different text units include the entire document annotation work. Frontiers in Psychology | www.frontiersin.org 6 May 2022 | Volume 13 | Article 849083 Liang and Wang Deep Learning-Based Spanish Emotion Recognition Text Emotion Classifier When classifying all the hidden features generated by the BiLSTM model, these features have different responses to different classification tasks. Therefore, the introduction of relevant attention mechanisms allows the classifier to focus on different states in the sequence for different tasks, and to learn a feature representation that is more suitable for the current task. In this way, the accuracy of text emotional feature recognition is improved. In the improved BiLSTM model, the weighted sequence of sentiment features shared h by the text → ← i sentiment recognition task is represented by ht =σ h t , h t . The representation of this sequence references the sequence model of the attention mechanism. For the text emotion feature encoder, it is  more to extract the shared feature sequence ht = h1 , h2 , ..., ht , where t represents the length of the sequence. The attention mechanism first calculates the weight αet corresponding to the feature ht of each time series t, as expressed by Equation (8).  e exp f ht αt = Pt  (8) FIGURE 5 | Long and short term memory (LSTM) network. t=1 exp f ht Where f h = W T h, W represents the parameters that can  be trained. The output layer ce of the attention mechanism is the weighted sum of the output sequence, and its weighted expression is as follows. X t ce = αt ht (9) t=1 Finally, the high-level semantic features ce related to the text emotion are input to the next layer for emotion classification. The classifier of the multilingual emotion recognition task is composed of an emotion-related attention mechanism layer, a fully connected layer (FC), and an output layer. Among them, the FC layer has 128 nodes and uses the non-linear activation function ReLU. To avoid the overfitting issue, a random inactivation layer is added before the output layer, with an inactivation rate of 0.5. The output layer adopts softmax as the activation function. After activation, the emotion category corresponding to the text can be obtained. FIGURE 6 | Corpus size distribution of different data sets. EXPERIMENTS multi-language versions of text emotion data sets are all generated Datasets by Google Translate, and there is a certain gap in accuracy. So, we Our experiments mainly involve three data sets, which are NRC also chose the iSOL data set, the iSOL data set is a dictionary data Word-Emotion Association Lexicon (EmoLex) (Mohammad set composed of professional Spanish vocabulary and phrases. It and Turney, 2010), Improved Spanish Opinion Lexicon (iSOL) was manually proofread by professional Spanish linguists. The (Molina-González et al., 2013), Spanish Emotion Lexicon (SEL) application scenario of the iSOL data set is mainly the polarity (Sidorov et al., 2012). The above three corpus data sets have classification of vocabulary, which is only part of text emotion different corpus sizes, and their corpus size distribution is shown analysis and cannot cover other parts of text emotion recognition. in Figure 6. Therefore, this data set also has certain shortcomings. So, we also Among them, NRC is an emotional text data set launched by merged the third data set SEL. The SEL is also a professional Google, which is mainly developed for English. However, due to Spanish dictionary data set, all the texts are manually marked the increasing research on emotions in minor languages, NRC by annotators with different thresholds. Each phrase is measured has gradually developed a multilingual version. However, these by the standard Probability Factor of Affective (PFA), which Frontiers in Psychology | www.frontiersin.org 7 May 2022 | Volume 13 | Article 849083 Liang and Wang Deep Learning-Based Spanish Emotion Recognition corresponds to the emotional performance in different Spanish is improved by 12.57% compared to SVM. It can be demonstrated contexts. In the experimental verification work, we select the that neural network methods outperform machine learning above three data sets to evaluate the effect of text emotion methods in natural language processing. Among them, the RNN recognition, which fully guarantees the comprehensiveness of the and LSTM methods are inferior to our method for text emotion data set and the interpretability of the verification results. The recognition. It can be seen that the method of fusing CNN and number of each emotional phrase in the three data sets is shown BiLSTM in this paper outperforms the single neural network in Table 2. methods such as RNN and LSTM. In addition, the attention mechanism and multi-task learning structure introduced in this Evaluation Metrics paper enable the model to learn the salient features of different In the experimental verification process, we choose precision, text emotions and mitigate the adverse effects of unaffected recall, and F1 score to evaluate the performance of the method. factors. Meanwhile, the generalization ability and recognition Most classification and sequence labeling problems also adopt the ability of the model for different emotion features are improved, above three evaluation factors. The equation is as follows: and the accuracy of emotion recognition is also improved. In order to represent Valence and Arousal more precisely on 2 × precision × recall the level of emotional characteristics. Extend Valence to emotions F1 = (10) precision × recall of surprise and happiness, and Arousal to emotions of anger and TP sadness. In the previous step of experimental verification, it has Precision = (11) been proved that our method is better than the machine learning TP + FP method and the same type of neural network method. Therefore, TP the next step will further verify the emotion recognition effect Recall = (12) of this method on the two large Spanish-specific text emotion TP + FN dictionary data sets, iSOL, and SEL. The experimental results are In the above equation, TP (True Positive) is the number shown in Table 4. of properly-recognized word segments. FP (False Positive) Table 4 shows the emotion recognition effect of our method is the number of misrecognized word segmentation. FN in the three data sets of NRC, iSOL, and SEL. The experimental (False Negative) indicates the number of unrecognized results shows that the recognition effect of the NRC data set in word segmentation. the Spanish emotional dictionary is not as good as iSOL and SEL. Because NRC is a data set generated by Google Translate, there is Experimental Result a certain gap between the emotional feature label and the original We presented a test to verify the efficiency of the revised BiLSTM Spanish emotional feature. Our method is constructed based on approach, the recognition tasks were carried out at Valence and the original emotional features of Spanish, so the text emotion Arousal levels respectively, and SVM, RNN, and LSTM methods recognition effect is better in iSOL and SEL. were selected as the control experimental group. The NRC dataset To verify the average accuracy of emotion recognition on these was used for verification. The experimental results are shown in three data sets, we have added a new verification experiment. The Table 3. experimental results are shown in Table 5. Table 3 shows the average UAR rate of 70.52% for our method The experimental results in Table 5 verify our previous on the valence recognition task. The average performance of UAR conclusions again. Regarding specialized Spanish text emotion dictionary data sets, such as iSOL and SEL. The average accuracy of our proposed method can reach up to 76.21%, demonstrating TABLE 2 | The number of each emotional phrase in Spanish dictionary datasets. Datasets TABLE 4 | Text emotion recognition results of our method on the Spanish NRC iSOL SEL dictionary datasets. Train 10,217 7,302 1,848 NRC iSOL SEL Test 3,024 833 188 Total 13,241 8,135 2,036 P R F1 P R F1 P R F1 Surprise 0.64 0.65 0.68 0.74 0.68 0.69 0.77 0.63 0.69 TABLE 3 | Text emotion recognition results of different methods at the Valence Happiness 0.66 0.62 0.60 0.71 0.89 0.81 0.70 0.92 0.77 and Arousal level. Anger 0.64 0.71 0.69 0.75 0.71 0.72 0.78 0.76 0.77 Valence Arousal Sadness 0.68 0.63 0.62 0.75 0.69 0.72 0.78 0.64 0.69 IEMOCAP RECOLA Ave IEMOCAP RECOLA Ave TABLE 5 | The average recognition accuracy of our method on the Spanish SVM 59.13 56.77 57.95 63.01 58.04 60.03 dictionary datasets. RNN 65.92 63.32 64.82 68.06 60.51 64.29 NRC iSOL SEL LSTM 67.32 65.33 66.26 69.89 59.33 64.61 Ours 71.14 68.65 70.52 74.12 66.13 71.13 Average acc. 70.03 75.53 76.21 Frontiers in Psychology | www.frontiersin.org 8 May 2022 | Volume 13 | Article 849083 Liang and Wang Deep Learning-Based Spanish Emotion Recognition the effectiveness of our optimized BiLSTM method in Spanish DATA AVAILABILITY STATEMENT emotion recognition research. The original contributions presented in the study are included in the article/supplementary material, further inquiries can be CONCLUSION directed to the corresponding author/s. In this paper, we propose an improved BiLSTM framework for emotion recognition of Spanish texts. First, we convert the Spanish text into a suitable tensor by text pre-processing; ETHICS STATEMENT then we complete the text emotion feature extraction by a modified BiLSTM framework; then all features are The studies involving human participants were reviewed and input to the topic detection layer to complete sentence approved by Xiamen University. Written informed consent to clustering and topic labeling; then they are input to the participate in this study was provided by the participants’ legal emotion classifier to classify text emotion according to guardian/next of kin. the weights, and finally the text emotion classification results are output. Meanwhile, the attention mechanism is inserted in the improved BiLSTM framework. It enables the AUTHOR CONTRIBUTIONS shared feature encoder to obtain weighted representation results in the process of extracting emotion features, which Both authors designed the whole algorithm and experiments, enhances the generalization ability of the model for text contributed to the article, and approved the submitted version. emotion feature recognition. The validation of three major datasets demonstrates the effectiveness of our approach for Spanish text emotion recognition. For the specialized Spanish dictionary dataset, our method performs even better. FUNDING The average value in emotion recognition accuracy is up to 76.21%. The overall performance outperforms current This work was supported by Fujian Social Science Fund: A comparable machine learning methods and convolutional neural European Perspective on 16th Century Fujian and Taiwan: network methods. Martin de Rada’s Things of Taybin (1575) (FJ2021B115) and Considering that our method cannot perform good emotion Xiamen University Research Fund: Study of Translation- recognition for long Spanish texts. For the selection of the data oriented Terminology Work in Spanish and Chinese set, we will try to choose a special Spanish dictionary data set, so Context (ZK1105/0650). that the results will be more referential. In the next work, we will consider the use of a bidirectional recurrent neural network to enhance our approach to capture the global features of large texts. ACKNOWLEDGMENTS We will also upgrade our research to the Spanish voice and video level, and finally, build a complete Spanish emotion recognition We thank the reviewers whose comments and suggestions helped system to serve the internet industry and the robotics industry. improve this manuscript. REFERENCES Devlin, J., Chang, M. W., Lee, K., and Kristina, T. (2018). Bert: pre-training of deep bidirectional transformers for language understanding. arXiv [Preprint] Bandhakavi, A., Wiratunga, N., Padmanabhan, D., and Massie, S. (2017). arXiv:1810.04805, Lexicon based feature extraction for emotion text classification. Hasan, M., Rundensteiner, E., and Agu, E. (2019). Automatic emotion detection Pattern Recognit. Lett. 93, 133–142. doi: 10.1371/journal.pone.0 in text streams by analyzing Twitter data. Int. J. Data Sci. Anal. 7, 35–51. 194852 doi: 10.1007/s41060-018-0096-z Boot, P., Zijlstra, H., and Geenen, R. (2017). The Dutch translation of the linguistic Hochreiter, S., and Schmidhuber, J. (1997). Long short-term memory. Neural inquiry and word count (LIWC) 2007 dictionary. Dutch J. Appl. Linguist. 6, comput. 9, 1735–1780. doi: 10.1162/neco.1997.9.8.1735 65–76. doi: 10.1075/dujal.6.1.04boo Howard, J., and Ruder, S. (2018). Universal language model fine-tuning for text Calvo, R. A., and D’Mello, S. (2010). Affect detection: an interdisciplinary review of classification. arXiv [Preprint] arXiv:1801.06146, doi: 10.1016/j.jacr.2019.05. models, methods, and their applications. IEEE Trans. Affect. Comput. 1, 18–37. 007 doi: 10.3390/s19194079 Jabreel, M., and Moreno, A. (2019). A deep learning-based approach for multi-label Chatterjee, A., Gupta, U., Chinnakotla, M. K., Srikanth, R., Galley, M., and emotion classification in tweets. Appl. Sci. 9:1123. doi: 10.3390/app9061123 Agrawal, P. (2019). Understanding emotions in text using deep learning and Martin, L., Muller, B., Suárez, P. J. O., Dupont, Y., Romary, L., de la Clergerie, big data. Comput. Hum. Behav. 93, 309–317. doi: 10.1016/j.neunet.2019.06.010 E. V., et al. (2019). Camembert: a tasty french language model. arXiv [preprint] doi: 10.1016/j.chb.2018.12.029 arXiv:1911.03894. doi: 10.18653/v1/2020.acl-main.645 del Arco, F. M. P., Jiménez-Zafra, S. M., Martín-Valdivia, M. T., Martinez-Camara, E., Martín-Valdivia, M. T., Urena-Lopez, L. A., and Mitkov, R. and Ureˆ na-L’opez, L. A. (2018). “SINAI at SemEval-2018 task (2015). Polarity classification for Spanish tweets using the COST corpus. J. Inf. 1: emotion recognition in tweets,” in Proceedings of The 12th Sci. 41, 263–272. doi: 10.1177/0165551514566564 International Workshop on Semantic Evaluation, New Orleans, LA, McCann, B., Bradbury, J., Xiong, C., and Socher, R. (2017). Learned in translation: 128–132. contextualized word vectors. arXiv [Preprint] arXiv:1708.00107, Frontiers in Psychology | www.frontiersin.org 9 May 2022 | Volume 13 | Article 849083 Liang and Wang Deep Learning-Based Spanish Emotion Recognition Mohammad, S. M. (2017). Word affect intensities. arXiv [preprint] International Conference on Artificial Intelligence, (Heidelberg: Springer), 1–14. arXiv:1704.08798, doi: 10.3390/e23070859 Mohammad, S. M., and Bravo-Marquez, F. (2017). WASSA-2017 shared task on Strapparava, C. (2016). “Emotions and NLP: future directions,” in Proceedings of emotion intensity. arXiv [preprint] arXiv:1708.03700, doi: 10.18653/v1/W17- the 7th Workshop on Computational Approaches to Subjectivity, Sentiment and 5205 Social Media Analysis, San Diego, CA. doi: 10.18653/v1/W16-0430 Mohammad, S., and Turney, P. (2010). “Emotions evoked by common words and Strapparava, C., and Mihalcea, R. (2008). “Learning to identify emotions in text,” phrases: using mechanical turk to create an emotion lexicon,” in Proceedings of in Proceedings of the 2008 ACM Symposium on Applied Computing, Fortaleza, the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and 1556–1560. doi: 10.1145/1363686.1364052 Generation of Emotion in Text, Los Angeles, CA, 26–34. Strapparava, C., and Valitutti, A. (2004). “Wordnet affect: an affective extension Molina-González, M. D., Martínez-Cámara, E., Martín-Valdivia, M. T., and Ureña- of wordnet,” in Proceedings of the Fourth International Conference on Language López, L. A. A. (2015). Spanish semantic orientation approach to domain Resources and Evaluation Lrec, Vol. 4, Lisbon, 40. adaptation for polarity classification. Inf. Process. Manag. 51, 520–531. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., et al. Molina-González, M. D., Martínez-Cámara, E., Martín-Valdivia, M. T., and Perea- (2017). Attention is all you need. Adv. Neural Inf. Process. Syst. 30, 5998–6008. Ortega, J. M. (2013). Semantic orientation for polarity classification in Spanish Virtanen, A., Kanerva, J., Ilo, R., Luoma, J., Luotolahti, J., Salakoski, T., et al. reviews. Expert Syst. Appl. 40, 7250–7257. doi: 10.1016/j.eswa.2013.06.076 (2019). Multilingual is not enough: BERT for Finnish. arXiv [Preprint] Neviarouskaya, A., Prendinger, H., and Ishizuka, M. (2007). “Textual affect sensing arXiv:1912.07076, for sociable and expressive online communication,” in Proceedings of the 2nd Yadollahi, A., Shahraki, A. G., and Zaiane, O. R. (2017). Current state of text International Conference on Affective Computing and Intelligent Interaction, sentiment analysis from opinion to emotion mining. ACM Comput. Surv. (Berlin: Springer), 218–229. doi: 10.1007/978-3-540-74889-2_20 (CSUR) 50, 1–33. doi: 10.1145/3057270 Pires, T., Schlinger, E., and Garrette, D. (2019). How multilingual is multilingual Zahiri, S. M., and Choi, J. D. (2018). “Emotion detection on tv show transcripts BERT? arXiv [Preprint] arXiv:1906.01502, doi: 10.18653/v1/P19-1493 with sequence-based convolutional neural networks,” in Proceedings of the Plaza-del-Arco, F. M., Molina-González, M. D., Jiménez-Zafra, S. M., and Workshops at the 32nd AAAI Conference on Artificial Intelligence, New Orleans, Valdivia, M. T. M. (2018). Lexicon adaptation for Spanish emotion mining. LA. doi: 10.1155/2021/6694538 Procesamiento del Lenguaje Nat. 61, 117–124. Rao, Y., Lei, J., Wenyin, L., Li, Q., and Chen, M. (2014). Building emotional Conflict of Interest: The authors declare that the research was conducted in the dictionary for sentiment analysis of online news. World Wide Web 17, 723–742. absence of any commercial or financial relationships that could be construed as a doi: 10.1007/s11280-013-0221-9 potential conflict of interest. Redondo, J., Fraga, I., Padrón, I., and Comesaña, M. (2007). The Spanish adaptation of ANEW (affective norms for English words). Behav. Res. Methods 39, 600–605. Publisher’s Note: All claims expressed in this article are solely those of the authors doi: 10.3758/BF03193031 and do not necessarily represent those of their affiliated organizations, or those of Schuster, M., and Paliwal, K. K. (1997). Bidirectional recurrent neural networks. the publisher, the editors and the reviewers. Any product that may be evaluated in IEEE Trans. Signal Process. 45, 2673–2681. doi: 10.1109/78.650093 this article, or claim that may be made by its manufacturer, is not guaranteed or Schütze, H., Manning, C. D., and Raghavan, P. (2008). Introduction to endorsed by the publisher. Information Retrieval. Cambridge: Cambridge University Press. doi: 10.1017/ CBO9780511809071 Copyright © 2022 Liang and Wang. This is an open-access article distributed Shulcloper, J. R. (2009). Reconocimiento Lógico Combinatorio De Patrones: Teoría under the terms of the Creative Commons Attribution License (CC BY). The y Aplicaciones. Ph. D. Thesis. Santa Clara: Universidad Central de Las Villas use, distribution or reproduction in other forums is permitted, provided the ‘Marta Abreu’. original author(s) and the copyright owner(s) are credited and that the original Sidorov, G., Miranda-Jiménez, S., Viveros-Jiménez, F., Gelbukh, A., Castro- publication in this journal is cited, in accordance with accepted academic practice. Sánchez, N. A., Velasquez, F., et al. (2012). “Empirical study of machine learning No use, distribution or reproduction is permitted which does not comply with based approach for opinion mining in tweets,” in Proceedings of the Mexican these terms. Frontiers in Psychology | www.frontiersin.org 10 May 2022 | Volume 13 | Article 849083