JOURNAL OF MEDICAL INTERNET RESEARCH Ramírez-Cifuentes et al Original Paper Detection of Suicidal Ideation on Social Media: Multimodal, Relational, and Behavioral Analysis Diana Ramírez-Cifuentes1, MSc; Ana Freire1, PhD; Ricardo Baeza-Yates1, PhD; Joaquim Puntí2,3, PhD; Pilar Medina-Bravo4, PhD; Diego Alejandro Velazquez5, MSc; Josep Maria Gonfaus6, PhD; Jordi Gonzàlez5, PhD 1 Department of Information and Communication Technologies, Universitat Pompeu Fabra, Barcelona, Spain 2 Hospital de Día de Adolescentes, Servicio de Salud Mental, Consorci Corporació Sanitària Parc Taulí, Sabadell, Spain 3 Departamento de Psicología Clínica y de la Salud, Universitat Autònoma de Barcelona, Barcelona, Spain 4 Department of Communication, Universitat Pompeu Fabra, Barcelona, Spain 5 Computer Vision Center, Universitat Autònoma de Barcelona, Bellaterra (Barcelona), Spain 6 Visual Tagging Services, Bellaterra (Barcelona), Spain Corresponding Author: Diana Ramírez-Cifuentes, MSc Department of Information and Communication Technologies Universitat Pompeu Fabra Carrer de Tànger, 122-140 Barcelona, 08018 Spain Phone: 34 643294184 Email: diana.ramirez@upf.edu Abstract Background: Suicide risk assessment usually involves an interaction between doctors and patients. However, a significant number of people with mental disorders receive no treatment for their condition due to the limited access to mental health care facilities; the reduced availability of clinicians; the lack of awareness; and stigma, neglect, and discrimination surrounding mental disorders. In contrast, internet access and social media usage have increased significantly, providing experts and patients with a means of communication that may contribute to the development of methods to detect mental health issues among social media users. Objective: This paper aimed to describe an approach for the suicide risk assessment of Spanish-speaking users on social media. We aimed to explore behavioral, relational, and multimodal data extracted from multiple social platforms and develop machine learning models to detect users at risk. Methods: We characterized users based on their writings, posting patterns, relations with other users, and images posted. We also evaluated statistical and deep learning approaches to handle multimodal data for the detection of users with signs of suicidal ideation (suicidal ideation risk group). Our methods were evaluated over a dataset of 252 users annotated by clinicians. To evaluate the performance of our models, we distinguished 2 control groups: users who make use of suicide-related vocabulary (focused control group) and generic random users (generic control group). Results: We identified significant statistical differences between the textual and behavioral attributes of each of the control groups compared with the suicidal ideation risk group. At a 95% CI, when comparing the suicidal ideation risk group and the focused control group, the number of friends (P=.04) and median tweet length (P=.04) were significantly different. The median number of friends for a focused control user (median 578.5) was higher than that for a user at risk (median 372.0). Similarly, the median tweet length was higher for focused control users, with 16 words against 13 words of suicidal ideation risk users. Our findings also show that the combination of textual, visual, relational, and behavioral data outperforms the accuracy of using each modality separately. We defined text-based baseline models based on bag of words and word embeddings, which were outperformed by our models, obtaining an increase in accuracy of up to 8% when distinguishing users at risk from both types of control users. Conclusions: The types of attributes analyzed are significant for detecting users at risk, and their combination outperforms the results provided by generic, exclusively text-based baseline models. After evaluating the contribution of image-based predictive models, we believe that our results can be improved by enhancing the models based on textual and relational features. These methods can be extended and applied to different use cases related to other mental disorders. https://www.jmir.org/2020/7/e17758 J Med Internet Res 2020 | vol. 22 | iss. 7 | e17758 | p. 1 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Ramírez-Cifuentes et al (J Med Internet Res 2020;22(7):e17758) doi: 10.2196/17758 KEYWORDS social media; mental health; suicidal ideation; risk assessment; machine learning image-based approach for suicide risk assessment on social Introduction media at the user level. Background When we refer to suicide-related collections, to the extent of Mental disorders are a serious health issue worldwide. our knowledge, no previous publications have been reported According to the mortality data presented by the World Health on the generation of a suicide-related dataset built over Twitter Organization, the number of deaths because of suicide is where the annotation process involves expert clinicians labeling equivalent to a person dying every 40 seconds [1]. Considering tweets according to the presence of suicidal thoughts focused that the signs and symptoms of these disorders have been proven on two types of control groups. Furthermore, our annotation to be traceable on social media, scientists have started to work was performed at the user level after having processed and on the development of automated methods to detect signs and analyzed sequences of posts within a period of a year. This symptoms of these conditions [2-5] by addressing the allowed us to explore changes in time, as it has been proven to importance of early detection [6,7]. be useful for behavior analysis [23] and for the evaluation of early risk detection methods for depression [7,24]. State-of-the-art approaches for the detection of mental disorders in social media involve the use of machine learning techniques Objectives mainly based on textual features extracted from the posts [8-11]. This study addressed the detection of mental health issues on These models require the previous acquisition of annotated data, social media, with suicide risk assessment as our use case. Our which are extracted from a selected social platform. Considering main contributions are listed as follows: (1) we defined a this as a classification task, which manipulates sensitive data methodology to generate a reliable Twitter dataset for suicide [12], a proper annotation process is required to obtain a reliable risk assessment, which is also the first user-level dataset of this dataset. This has become one of the main challenges because type dedicated to posts in Spanish; (2) we presented a method diagnosing mental disorders involves the intervention of to obtain a subset of the user tweets related to a specific topic, specialized and experienced clinicians who follow strict in this case, suicidal ideation; (3) we generated models that screening proceedings [11]. explore the impact of not just relational and behavioral factors The methods used for analyzing user-generated data related to but also elements identified by specialists during consultations, suicide focus on tracking social networks at a post level, that which have been mapped to social networks; (4) we developed is, a tweet [13,14], or at a user level, that is, a sample of their image-based predictive models to detect suicidal ideation; (5) tweets or posts [15]. The latter is more related to risk assessment, we integrated the previous elements into a method that combines as more data from a single user can be explored. In terms of the multimodal data to build predictive models that address the type of information extracted and explored, state-of-the-art detection of mental health issues using cross-platform approaches perform an exhaustive analysis of textual information (Reddit, Instagram, and Twitter); and (6) we refined information contained in posts. This has been proven to be the evaluation process of predictive models for mental health relevant for screening and risk assessment tasks [16]. The issues by considering 2 different types of control groups within methods applied often consider bag of words (BoW) models, the social media context: users with posts that might not use topic models, lexicons, sentiment analysis tools, and readability terms related to mental conditions (generic control cases) and and syntactical analysis features [5,15,17]. The most recent users who make use of terms related to mental disorders work with deep learning approaches consists of exploring (focused control group). sequence models encoding vector representations of terms known as word embeddings [3,11,15]. Methods There are a few approaches that analyze features containing Data Collection behavioral and relational information. Colombo et al [18] In this section, we described the methodology designed to examined the connectivity and communication of users with generate a reliable Twitter dataset for suicide risk assessment. suicidal ideation on Twitter based on the evaluation of retweets, We selected Twitter as our main data source, as it has been whereas the work of De Choudhury et al [16] on Twitter defines proven to be suitable for analyzing mental disorders on social an egocentric social graph for depression detection. However, media [3,13,18,25-27], including suicidal ideation. We also these studies do not evaluate the performance of relational highlighted the following aspects that this platform offers for elements combined with image-based data. our research: (1) the possibility of selecting posts in multiple Regarding the use of visual information from posts shared in languages; (2) the availability of relational and behavioral social networks to address mental disorders, the closest factors; and (3) the provision of a set of chronologically ordered approaches to our goal are methods for personality prediction posts from each user, which allowed us to do a cumulative [19], and a few recent approaches address self-harm, depression, analysis of the data referring to suicidal ideation. and anxiety [20-22]. We believe that our study is the first https://www.jmir.org/2020/7/e17758 J Med Internet Res 2020 | vol. 22 | iss. 7 | e17758 | p. 2 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Ramírez-Cifuentes et al Combining Reddit and Twitter for Gathering We built a binary classifier distinguishing 2 classes: (1) suicidal Suicide-Related Data ideation–related tweet and (2) control tweet. To train the model, we chose as instances for the suicide tweet class the tweets of Before starting the crawling process on Twitter, we elaborated users tagged as suicidal ideation risk cases (513 tweets) and a list of suicide-related sentences to build a first filtering 346 phrases evaluated by the clinicians. For the control tweet approach to obtain tweets coming from users at risk. In doing class, we selected an equally proportional set of random tweets so, we started by collecting a sample of 500 titles of posts related to other topics, using Twitter’s Sample Tweets published in Reddit’s Suicide Watch forum [28]. These posts application programing interface (API) [30]. A BoW model were mostly written by users with suicidal ideations, so their with 1 to 5 grams was generated, and after applying principal titles can be considered to be suicide-related sentences. component analysis and logistic regression analysis, we achieved These phrases were then translated to Spanish and reviewed by F1=0.90, precision (Pr)=0.91, and recall (R)=0.89, with a 10-fold clinic psychologists. Some sentences were added, discarded, cross-validation procedure. This is considered as our short and/or adapted by the clinicians. We kept 454 phrases after the profile version classifier (SPVC). analysis by experts. Then, based on their relevance and link to The SPVC was applied to every tweet of the profile of all users suicide risk factors [29], a subset of 110 phrases were selected labeled as suicidal ideation risk and, for each user, we selected for use as search terms on Twitter. A total of 98,619 tweets the top 15 suicide-related tweets with the highest predicted containing these phrases were extracted for a year, that is, from probability values given by the SPVC. We considered these December 21, 2017, to December 21, 2018. These tweets tweets as the sample to be evaluated by 2 additional annotators: corresponded to 81,572 Twitter users, with 9559 users having a specialized clinician and a nonspecialized annotator. This more than one tweet matched with the search terms. At the same second annotator was given detailed instructions and information time, for all users, we extracted all their tweets posted within on risk factors related to suicide. The annotators at this stage the same search period. were asked to classify users into 2 categories: (1) suicidal Two-Level Annotation Process ideation risk or (2) control, now having more information about each user. Once the second annotation process was completed, First Level we only retained the positive cases (n=84) on which both In total, 2 labeling stages were followed to identify users with annotators agreed, corresponding to 89% (84/94) of all the cases suicidal ideation. As our intention was to follow a manual labeled as suicidal ideation by at least one of the annotators in labeling process done by clinicians, we selected a random this second stage. Thus, we ensured that the users labeled as sample of 1200 users among those who had at least two tweets suicidal ideation risk were classified under the agreement of 3 matching our search phrases. The user names were anonymized, annotators at different stages. and 3 tags were defined for labeling purposes: (1) control—defining users who on their tweets did not seem to We defined 2 different control groups with the same size as the manifest suicidal ideations, users who did not refer to their own suicidal ideation risk class: conditions, and users who were reporting news or opinions • Focused control group: users writing suicide-related regarding suicide; (2) suicidal ideation risk—labeling users keywords in a non–suicidal ideation risk context, that is, who, judging by their writings, seemed to present suicidal users who trivialize about suicide, news reports, and ideation signs; and (3) doubtful—dedicated to cases where information regarding the topic; or users who simply psychologists were not sure about labeling users within any of manifest their support or opinions to people at risk. the other categories. At the first labeling stage, a clinician Identifying these users is challenging for classification specializing in this topic was asked to classify users within these systems but is key in reducing false-positives. These users 3 categories based only on the tweets containing the were chosen at random among the users labeled as control suicide-related keywords. After the labeling process, 73.8% cases during the first annotation process. (885/1200) of users were classified as control cases, 9.6% • Generic control group: a set of Twitter users who might (115/1200) were classified as suicidal ideation risk cases, and not necessarily use terms related to suicide. These users 16.7% (200/1200) fell within the doubtful category. were selected randomly using the Sample Tweets API [30] Second Level: Short Profile Version and anonymized as had been done for the other groups. A second labeling process was followed for the users tagged as For both the control groups, the second annotation process was suicidal ideation risk cases. We analyzed more of their collected followed to discard possible cases of users at risk within these tweets to confirm their labels. Annotators noticed that there was samples. a high number of tweets that were not related to suicidal ideation Combining Multimodal Data for Detecting Suicide and even sometimes no tweets related to suicide were caught in the sample. To address this issue, we developed a classifier Risk at a tweet level to distinguish tweets containing signs of risk We proposed a method that given the profile of a user: (1) it from those that were not related at all with suicide. Thus, we uses a text-based model, described previously as the SPVC, could provide the second annotator a summarized version of a which selects a subset of relevant tweets related to suicidal user profile, which we call short profile version (SPV), that ideation. The set of tweets for which the SPVC provides a score contained mainly tweets related to suicide and its risk factors. over a given threshold is retained in SPV itself; (2) mostly from the outputted SPV, it extracts a set of relational, textual, https://www.jmir.org/2020/7/e17758 J Med Internet Res 2020 | vol. 22 | iss. 7 | e17758 | p. 3 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Ramírez-Cifuentes et al behavioral, lexical, statistical, suicidal ideation–related, and suicide risk factors (we referred to these features as image-based features from the content and metadata of the suicide-related lexicon features). Each of these types of features tweets; and (3) it builds and evaluates different predictive is described in the next subsections. models resulting from the combination of these features. Our Behavioral Features features are organized into 3 different groups: (1) BoW or n-grams and word embeddings as a representation of textual These features are based on the information extracted from the features; (2) a set of features known as social networks and metadata of tweets. Here, we measured the behavior of users psychological (SNPSY) features containing a group of relational, based on their activity within certain periods, which are defined behavioral, lexical, sentiment analysis, and statistical features, at different granularity levels. These features are detailed in in addition to a set of features that attempt to map to the social Table 1. media context certain signs and symptoms, which are usually The intention of the sleep time tweets ratio (STTR) and the considered by clinicians at the time of screening; and (3) an daytime tweets ratio (DTTR) features is to identify the image-based score. differences between control users and users at risk regarding Features Definition the periods of the day on which they post. Considering that our data collection is delimited by language but not by location, Generic Text-Based Features that the posting time provided for a tweet is in coordinated Bag of Words and N-Grams universal time and not the time of the user location, and that not enough information from our data was found to These are features that have been used to address similar tasks, automatically identify the location of all the users, we defined such as depression detection and eating disorders screening an approach to address this issue. As explained in Equation 1, [7,31]. These models represent terms or sequences of terms a day was divided into 8 fixed time slots of 3 hours each. (n-grams) based on their frequencies on the documents analyzed. Afterward, we assumed that an average user had at least around In our case, each user was represented by a document consisting 6 hours of sleep time, and within this 6-hour period, a smaller of the concatenation of the text of all their tweets. Afterward, number of tweets would be created compared with the rest of we used the Scikit-learn [32] Python library: TfIdfVectorizer to the day, so we counted the number of tweets (t) created within generate a tf.idf representation of 1 to 5 grams at the word level. each 3-hour time slot for all the tweets of the SPV of a user. A set of Spanish stop words were considered to build this Next, for each user, we calculated the sum of the number of representation [33]. These features are referred to as BoW tweets within each pair of continuous time slots and selected features in further sections. We also used ekphrasis [34] as a the minimum score obtained by all the pairs. We also assumed text preprocessing tool to replace generic tag elements such as that the first and last slots can be continuous. Finally, this value money, phone numbers, digits, hashtags, and emoticons. We was normalized according to the total number of tweets of the also removed the n-grams that appeared in less than 5% of the full profile of the user (T). This feature was considered as the documents to reduce the feature space. This is done considering STTR: that the features’ number is given by the size of the vocabulary of all the writings of the users, and in Twitter, we found cases in which terms are misspelled or elongated (ie, hellooooo instead of hello); therefore, we avoided having multiple representations for the same term assuming that each spelling mistake and The DTTR was given by the difference between 1 and the sleep elongation is different and less likely to be repeated over most time ratio: that is, DTTR=1−STTR. It is important to recall that of the documents. for the measurements that refer to a bigger granularity such as weekdays, weekends, and months, the impact of time difference Word Embeddings is not as big as for features based on day periods. Word embeddings are representations of textual terms as vectors of real numbers. Words that are semantically related have a Tweets Statistics similar representation over the vector space. Most of the recent This group refers to 5 types of features that correspond to predictive models dealing with textual features using deep statistical measures calculated from the tweets of users. We learning techniques make use of word embeddings to represent considered elements such as the number of tweets created and terms. The sequences of these representations are fed as inputs their length and the number of tweets that were retained for to train the predictive models. These types of representations each user at the SPV in relation to the total number of tweets have been recently used in state-of-the-art approaches to address posted. These features are described in Table 2. suicide risk assessment [3,11]. We made use of word Relational Features embeddings previously learned over a dataset with 2 million Spanish tweets [35]. These are informative features regarding the relationships and interactions between users. Elements such as the count of Social Networks and Psychological Features retweets and favorites received and given by the users can These consist of a group of features based on generic lexicons provide insight on the social support they receive, along with [36], statistics measured from the users’ writings, information information regarding the number of followers and users of interest for clinicians regarding the behavior of users in time, followed, as previously considered for depression screening the users’ social network (relational features) [16], and n-grams [16]. Table 3 describes the relational features extracted for our lexicons, which include terms referring to suicidal ideation or evaluation. https://www.jmir.org/2020/7/e17758 J Med Internet Res 2020 | vol. 22 | iss. 7 | e17758 | p. 4 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Ramírez-Cifuentes et al Table 1. Description of behavioral features. Feature Description Source Working week tweets count ratio Total number of tweets on weekdays (Monday to Friday) normalized by the total amount SPVa tweets of tweets Weekend tweets count ratio Total number of tweets on weekend days (Saturday and Sunday) normalized by the total SPV tweets amount of tweets Median time between tweets Median of the time (in seconds) that passes between the publication of each tweet SPV tweets Sleep time tweets ratio Ratio of tweets posted during the inferred sleep period of the user Full profile tweets Daytime tweets ratio Ratio of tweets posted during the period the user is usually awake Full profile tweets Normalized tweet count per quarter Number of tweets posted by the user within each quarter of the year, normalized by the total SPV tweets (4 features) amount of tweets generated by the user during the year a SPV: short profile version. Table 2. Description of features based on tweet statistics. Feature Description Source Suicide-related tweets ratio Ratio of tweets retained by the SPVCa over all the tweets of the full profile SPVb and full profile tweets Median SPVC score Median of the scores obtained by the tweets that are part of the SPV after applying the SPV tweets SPVC Median tweet length Median length of all the user tweets (word level) SPV tweets Number of SPV tweets Number of tweets SPV tweets Number of user tweets Number of tweets posted by the user since the creation of the account Tweet metadata a SPVC: short profile version classifier. b SPV: short profile version. Table 3. Description of relational features. Feature Description Source Followers number Number of followers Tweet metadata Friends number Number of accounts followed by the user Tweet metadata Favorites given Total number of favorites given by the user Tweet metadata Median favorites count Median of the favorites received by the user SPVa tweets Median retweets count Median of the retweets received by the user SPV tweets a SPV: short profile version. disdain, insomnia, and fear; and possible references to previous Lexicons and Suicide Risk Factors Vocabulary suicide attempts, experiencing racial or sexual discrimination, The use of lexicons has been proven to be successful for tasks eating disorders, substance abuse, bullying, lack of social dedicated to screen mental disorders [9]. For our approach, we support, and family and money issues, along with vocabulary counted the frequency of words belonging to all the categories that might imply that some sort of discrimination or abuse has of the Linguistic Inquiry and Word Count (LIWC) 2007 Spanish been suffered, that someone close has died from suicide, and dictionary [36,37] normalized by the size (in number of terms) even vocabulary regarding the lack of spiritual beliefs, as of the concatenated writings of the users. It is important to recall religion is considered to be a protective factor for screening that LIWC also contains categories that identify syntactical tasks [29]. The terms and phrases selected for these categories elements such as verbs, nouns, adverbs, and pronouns, among were based on manually mapping common terms and phrases others. To this dictionary, a group of other categories was added seen in a sample of tweets labeled as suicide related during the containing vocabulary and up to 3-gram phrases that could be first labeling process with the assessment of clinicians. These mapped to suicide-related terms and risk factors such as suicide features were calculated using SPV. methods; terms referring to self-injuries; explicit suicidal ideation references; self-loathing terms; words that might imply https://www.jmir.org/2020/7/e17758 J Med Internet Res 2020 | vol. 22 | iss. 7 | e17758 | p. 5 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Ramírez-Cifuentes et al Sentiment Analysis using Nesterov with a momentum of 0.99 on 2 GTX 1080 Ti. We obtained a score for each tweet in terms of its polarity. For We used dropout (50%) to avoid overfitting at the training stage. this purpose, we used senti-py [38], trained on Spanish texts Classification Tasks from different sources, including Twitter. It is based on a BoW model with an intermediate feature selection process. To obtain As we wanted to evaluate the change in the performance of a score per user, we calculated the median of the scores of all models that use 2 different types of control groups, one the tweets from the SPV. constituted by users who make use of vocabulary related to suicide (focused control) and another group of users who might Image-Based Feature not make use of these terms at all (generic control), we created We followed the methodology proposed in the study by experiments for comparing (1) users at risk versus focused Rodriguez et al [39], where a method for inferring the control users (task 1) and (2) users at risk versus generic control personality under the OCEAN model was presented. In this users (task 2). These were selected as our 2 supervised predictive sense, we created a classifier trained on images extracted from tasks. Our instances and their features for the predictive models Instagram using a subset of the phrases and keywords used in were previously defined following the process described in the the data collection process for Twitter. These images were Combining Multimodal Data for Detecting Suicide Risk section. considered suicide related, whereas a set of unrelated images Baselines was considered as our control cases. Afterward, this first model was applied to each of the images extracted from the users’ We defined as baselines 2 models exclusively based on generic tweets of our dataset. To create this model, we used 90,000 text representations. These models were generated using the images for training and 60,000 images for validation. To obtain previously extracted features and representations from the users: a single score per user (images user score), the average of the (1) full profile and (2) from their SPV. The first one is a BoW individual scores of the images of each user was considered as model trained with 1 to 5 grams, and the second one consists the user’s aggregated score. of a deep learning model defined by a CNN architecture that has been proven to be successful for text classification [45] and To define the image classifier, we used a convolutional neural has been used in a similar task that addresses suicide risk network (CNN) [40] because CNNs are especially suited for assessment on Reddit users [11]. image data. There are several variants of this type of network, but the most popular ones are based on the residual networks We adopted the approach of Shing et al [11] to define our introduced by He et al [41]. They used skip connections between user-level instances. Therefore, given a user represented by a layers that force the gradient to flow directly between set of sequential posts, we concatenated all these posts and convolutional blocks. This makes backpropagation much more represented each post as a concatenation of words, where each effective in deep architectures. word is represented by a vector (word embedding), as described in the Features Definition section. As in the study by The training process is performed by minimizing the Coppersmith et al [3] and as it has been proven successful on cross-entropy loss function through gradient descent. The similar tasks, we used a set of word embeddings previously problem when training CNNs is that they require many images learned on Twitter [35] to define the starting weights for our (often in the range of millions) to successfully extract relevant embedding layer and performed further fine-tuning to learn over features for the final classifier. As most of the time datasets are the training set and adapt the representations to the task domain. not large, networks are usually trained on a large dataset (eg, ImageNet [42]) and then fine-tuned on the target dataset. We considered the 2 models previously described as Fine-tuning refers to the process of using an already trained state-of-the-art approaches for the creation of generic and model and retraining it to fit a new distribution. This is much exclusively text-based models for the task, as it is one of the faster because the weights in the model are not randomly purposes of our work to analyze the contribution of the initialized, and one can often skip the training of the shallower additional feature types defined. We therefore defined 4 baseline layers, given the fact that they focus on detecting corners and models. Baselines 1 and 3 correspond to the BoW model edges, and thus the model converges faster. Note that the fully generated over the full profile tweets sample and the SPV, connected layer that acts as a classifier in the network must be respectively. Baselines 2 and 4 correspond to the deep learning trained from scratch because the target classes between datasets model built over the same data samples (full profile and SPV). vary. Classifiers For our experiments, we used 101-layer ResneXt [43], which With the intention of evaluating the individual contribution of is a residual architecture that uses grouped convolution. This the types of features defined, along with their combinations particular architecture uses convolution groups of size 32 with toward a classification/detection task, we considered 4 types of a dimensionality of 8 and a fully connected layer at the end that classification algorithms and a deep learning model. We performs the actual classification. The output of the CNN is a evaluated the performance of random forest, multilayer vector that holds the scores for each of the classes; in our case, perceptron, logistic regression, and support vector machines as there were only 2 classes. The network was trained on ImageNet, classifiers. For each feature combination approach, models were and we fine-tuned it on our Instagram images. We trained for built for all these classifiers using the Scikit-learn [32] library’s 8 epochs using stochastic gradient descent with warm restarts implementation, with a grid search for the best parameters. We [44] with a weight decay of 0.001 and a learning rate of 0.0001 used a CNN architecture for the embedding models. https://www.jmir.org/2020/7/e17758 J Med Internet Res 2020 | vol. 22 | iss. 7 | e17758 | p. 6 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Ramírez-Cifuentes et al Approaches for Combining Features along with the image user score. This approach corresponds to We evaluated several ways of combining our 3 main feature the Images+BoW+SNPSY model 1 with 3 attributes based on types defined: generic text-based features, SNPSY features, and the combination of the 3 independent models with all their the image-based feature (image user score). As can be seen in features. The second approach consists of using all the features Table 4, we first generated individual models using exclusively of the SNPSY type as attributes in addition to the BoW output all the features corresponding to the BoW model, the feature and the image user score, which lead to the definition embedding’s model, and the SNPSY model, with features mainly of Images+BoW+SNPSY model 2. It is necessary to recall that obtained from the users’ SPV, as described in Tables 1-3. the predicted probability scores from the BoW and SNPSY Afterward, we explored the combination of our different feature individual models that were used for some of the feature types using the BoW model to represent text-based features. combination approaches at the training stage correspond to the Our first approach involves combining the BoW features with outputs of the classifiers on the test folds during the the SNPSY features. In this case, given the large number of cross-validation process executed on the training set. This was BoW features and their sparsity, we opted to use the BoW done to avoid overfitting. model–predicted probabilities as values for a single feature, In addition to the combination approaches described, we created denoted as the BoW outputted feature, to be added to the SNPSY 2 other models over which we performed a feature selection set of features. This is described in Table 4 as the BoW+SNPSY procedure over all the feature types. We chose the features with model. Subsequently, we evaluated the combination of the BoW statistically significant differences among the suicide and control features with the image feature. For this case, we simply added groups to evaluate their contribution exclusively to a predictive to the BoW set of features the image user score as another model. We presented 2 models with features selected based on attribute; this combination is described by the Images+BoW the P values obtained after performing a Mann-Whitney U test model. Afterward, to combine the SNPSY features with the to compare the samples of each class. This is a feature selection image feature, we used the image user score as a new feature method that has been previously used in medical applications in addition to the SNPSY feature set, which is the [46]. In addition, we took into account the efficiency of this Images+SNPSY model. feature selection approach, given the large feature space Finally, to combine the 3 feature types, we defined 2 approaches. considered (Table 4). These models are defined as the selected The first approach is an ensemble model where we consider the features model 1 with the features where P<.05, when comparing outputs (predicted probability scores) of the BoW model (BoW the suicidal ideation risk and control groups; and the selected outputted feature) and SNPSY model (SNPSY outputted feature) features model 2, where P<.001. The number of features obtained for each model is also given in Table 4. Table 4. Models and features. Model Features Number of features Task 1 Task 2 BoWa model BoW features generated with the Tf.Idf vectorizer with 1- to 5-gram features 24,645 24,336 Embeddings model Word embeddings representations as input for a text-based convolutional neural network model 200 200 SNPSYb model SNPSY features=behavioral+relational+tweets statistics+lexicons+suicide risk factors vocabu- 112 112 lary+sentiment analysis features BoW+SNPSY model BoW outputted feature+SNPSY features 24,757 24,448 Images+BoW model Images user score+BoW features 24,646 24,337 Images+SNPSY model Images user score+SNPSY features 113 113 Images+BoW+SNPSY Ensemble model=images user score+BoW outputted feature+SNPSY outputted feature 24,758 24,449 model 1 Images+BoW+SNPSY SNPSY features+images user score+BoW outputted feature 114 114 model 2 Selected features model 1 Selected features from all the feature types with P<.05 5807 14,882 Selected features model 2 Selected features from all the feature types with P<.001 522 3250 a BoW: bag of words. b SNPSY: social networks and psychological features. a total of 1,214,474 tweets and 305,637 images, from which up Experimental Setup to 1000 images per user were selected for our experiments. We Dataset Description selected a balanced sample of 84 users presenting signs of Following the description of the Data Collection section, to suicidal ideation (users at risk), 84 focused control users, and evaluate our approach, we selected a sample of 252 users with 84 generic control users, who were classified within these groups https://www.jmir.org/2020/7/e17758 J Med Internet Res 2020 | vol. 22 | iss. 7 | e17758 | p. 7 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Ramírez-Cifuentes et al by clinicians after seeing samples of their posts. Table 5 shows is lower for users at risk than for the control and generic cases. the statistics regarding the users belonging to each of the defined We also saw that generic control users produce lower amounts groups. We can notice that the median tweet length (in words) of tweets compared with other types of users. Table 5. Full dataset labeled group statistics. Description Suicidal ideation risk group Focused control group Generic control group Number of users 84 84 84 Number of tweets 313,791 766,437 134,246 Median number of tweets per user 2797.5 2984 716 Median tweet length 11 19 14 Number of images 37,801 251,830 16,006 obtained by certain classifiers such as the CNNs were averaged Detection Tasks and Evaluation Framework results of multiple runs because of the randomness they can We considered 3 different aspects to analyze: (1) the utility of add. having defined the SPV, as we believed that this would allow us to focus on the topic we are analyzing by getting rid of the Results noise provided by tweets that make no reference to our subject of interest; (2) the individual and combined contribution of the Statistical Analysis different aspects we analyzed: textual, relational, behavioral, We performed an analysis of the features extracted to identify and image-based information; and (3) the change in the significant differences between the samples of users at risk and performance of models that use 2 different types of control our control groups. For each feature extracted for the groups groups, one constituted by users that make use of vocabulary analyzed, we conducted an independent 2-sample related to suicide (focused control) and another group of generic Mann-Whitney U test among the suicidal ideation group of users who might not make use of these terms at all. users and the different control groups. We also conducted this All posts from the full profile of the user were considered for test to compare both of our control groups (focused and generic baseline 1 and 2 models, whereas most of the features for our control groups). We performed a nonparametric test considering proposed models and combinations were extracted exclusively that our features do not follow a normal distribution and that using the SPV, except for some elements extracted from the there was no homogeneity of variance for most of them. user’s tweets metadata and features such as the STTR and When comparing the suicidal ideation risk and focused control DTTR, which required the usage of the posts from the full groups at the SNPSY features, we found significant differences profile. For each task, 70% of all the instances were retained with P<.001 among the following features: overall ratio, median for training, and the remaining 30% (around 25 users per class) time between tweets, verbs, verbs conjugated in singular of the were left for testing purposes as unseen cases. To keep balanced first person (“I”+verb), cognitive mechanisms, anxiety-related instances from each class, we used stratification for these sets. terms, usage of personal pronouns, usage of the pronoun “I,” In addition to these test sets, we also evaluated our best models negations, terms to express feelings, and coursing terms. over a sample of 200 users labeled as doubtful cases. This is Regarding suicide-related lexicons, the usage of suicide explicit done to verify if, as the human annotator, the models are capable terms, depression-related terms, self-loathing, substance abuse, of identifying most of these cases as users that are likely to be self-injuries, and terms expressing lack of social support also at risk. presented an important significance (P<.001). Regarding the The PowerTransformer class from Python’s Scikit-learn library features from the BoW model, after conducting the same test, was used to transform the feature values to a normal we found significant differences with P<.001 for n-grams such distribution-like representation using Yeo-Johnson’s [32]. To as I feel, sad, kill myself, cry/crying, depression, to die, horrible, choose the best classifier, a 10-fold cross-validation process anxiety, die, pills, among others. Considering all the features was followed over the training set with all the algorithms to used (24,758), a total of 522 features were significant for evaluate. Afterward, the ones with the best performance were distinguishing the groups according to these tests with P<.001. selected to perform a second 5-fold cross-validation along with Table 6 shows the medians and the distributions overlapping a grid search to find the most suitable parameters for the index [47] for both groups on a sample of relevant features. classifier chosen. When repeating the independent two-sample Mann-Whitney U We considered the Pr, R, F1 score (F1), accuracy, and area test to compare the suicidal ideation risk group with the generic under the receiver operating characteristics curve (AUC-ROC) control set of users regarding the SNPSY features, among the score denoted as AUC, which was the measure on which we ones with P<.001, we found the median classifier score, the based the parameter optimization of the grid search. The values number of tweets generated, and the median time between tweets for Pr, R, F1, and AUC corresponded to the suicidal ideation to be different among both groups (suicidal ideation risk vs risk class, as it is our main class of interest. We reported on generic control). We identified differences in discussion topics accuracy to analyze the performance of both classes. The results such as money and work, about which the generic control users https://www.jmir.org/2020/7/e17758 J Med Internet Res 2020 | vol. 22 | iss. 7 | e17758 | p. 8 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Ramírez-Cifuentes et al seem to discuss more, whereas the members of the suicidal STTR, generic control users obtained an STTR value of 0.02, ideation risk group use terms more related to health and whereas users at risk obtained an STTR value of 0.04, meaning biological aspects. As in the previous case, the use of that users at risk seemed to tweet more at night compared with self-references was higher in the suicidal ideation risk group. the generic and focused control users as well. Within the significant n-grams from the BoW model, we found The image scores were also significantly different according to terms such as feel, to die, songs, someone, cry/crying, anxiety, the test with P=.002 for the comparison between the suicidal life, breath, bad, and fear. This is shown in Table 7, which ideation risk and generic control groups, considering a 95% CI. displays the median value and overlapping index of the Curiously, for the comparison of the image scores between the distributions of the groups in terms of some of the attributes suicidal ideation risk group and the focused control group, the mentioned. Again, taking into account all the features used test scores were different, with P=.05. This can be explained (24,449), 3250 were significant for distinguishing between the by the fact that users providing information or news about suicidal ideation risk and the generic control group in terms of suicide make use of similar images, which characterize the this test with P<.001. condition, making it difficult to find a significant difference Regarding other features explored, considering a 95% CI, for only judging by pictures. As can be seen in Table 8, for both task 1 (suicidal ideation risk vs focused control), the number of the control groups and the suicidal ideation risk group, the friends (P=.04) and median tweet length (P=.04) were median image scores were slightly higher for the suicidal significantly different. For these cases, the median number of ideation risk group. friends for a focused control user (578.5) was higher than the Finally, to compare our control groups (focused and generic median number of friends at risk (372.0). The same was true control groups), we performed the same test (Mann-Whitney for the median tweet length, based on the SPV, which was higher U test) and found significant differences between some of these for focused control users with 16 words against 13 of the suicidal groups’ features (n=181) with P<.001. Among these features, ideation risk users. In addition, there were significant differences we found mainly suicide-related lexicons, such as suicide in the STTR (P=.049) and weekday count ratio (P=.01). Under methods, suicide explicit terms, bullying, discrimination, and the same CI, for task 2, the weekday count ratio (P=.001), the substance abuse–related terms. We also found differences STTR (P=.004), along with the number of followers (P=.05), (P<.001) in other textual, relational, and behavioral attributes, and the total amount of favorites given (P=.006) showed such as the number of tweets, number of friends, number of significant differences. In this sense, generic control users followers, median favorites and retweet counts, overall ratio, appeared to tweet more on weekdays (Monday to Friday) as polarity score, median time between tweets, and STTR, among well as focused control users, whereas the opposite behavior others. These differences confirmed our previous assumptions was found for suicidal ideation risk users. Regarding the median regarding the differences among the control groups. Table 6. Medians and Distribution Overlapping Index for some of the attributes with the most significant differences between the Suicidal ideation and Focused control groups. Attribute Suicidal ideation median Focused control median Overlapping index Anxiety 10.94 0 0.25 Coursing terms 21.52 7.68 0.43 To die (self-reference) 5.45 0 0.25 I feel 46.25 6.71 0.32 Self-loathing 0.03 0 0.35 Verb I 22.66 12.11 0.41 Table 7. Medians and Overlapping Index for some of the attributes with the most significant differences between the Suicidal ideation and Generic control groups. Attribute Suicidal ideation median Generic control median Overlapping index Median classifier score 0.72 0.65 0.46 To die 19.5 0 0.25 Number of user tweets 2076.5 453 0.38 Health 17.19 8.18 0.44 Work 35.46 49.59 0.44 I 41.32 9.60 0.23 https://www.jmir.org/2020/7/e17758 J Med Internet Res 2020 | vol. 22 | iss. 7 | e17758 | p. 9 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Ramírez-Cifuentes et al Table 8. Medians and Overlapping Index for the images score between the suicidal ideation, focused control and generic control classes. Attribute Group Median value Overlapping index Images score Suicidal ideation 0.24 0.64 Focused control 0.23 Suicidal ideation 0.24 0.52 Generic control 0.23 test sets. We reported the results for the best models, as Classification Task Results described in Table 4, along with the baselines. In this section, we reported the results of our experiments. Table 9 presents the evaluation measure results for each task on the Table 9. Predictive task results. Model Suicidal ideation versus focused control group Suicidal ideation versus generic control group Pra Rb F1c Acd AUCe Classifier Pr R F1 Ac AUC Classifier BoWf model—full profile (baseline 1) 0.78 0.81 0.79 0.78 0.81 MLPg 0.79 0.85 0.81 0.80 0.91 MLP Embeddings model—full profile (base- 0.76 0.81 0.79 0.77 0.82 CNN h 0.78 0.87 0.82 0.80 0.84 CNN line 2) BoW model—SPVi (baseline 3) 0.81 0.85 0.83 0.82 0.85 LRj 0.80 0.92k 0.86 0.84 0.89 MLP Embeddings model—SPV (baseline 4) 0.79 0.85 0.82 0.80 0.83 CNN 0.77 0.87 0.82 0.80 0.82 CNN SNPSYl model 0.85 0.85 0.85 0.84 0.86 SVMm 0.85 0.88 0.87 0.86 0.94 LR BoW+SNPSY model 0.82 0.88k 0.85 0.84 0.89 RFn 0.85 0.88 0.87 0.86 0.94 LR Images+BoW model 0.79 0.88k 0.84 0.82 0.86 MLP 0.82 0.88 0.85 0.84 0.90 LR Images+SNPSY model 0.88k 0.85 0.86k 0.86k 0.91 SVM 0.88 0.88 0.88k 0.88k 0.94 LR Images+BoW+SNPSY model 1 0.85 0.85 0.85 0.83 0.87 LR 0.85 0.92k 0.88k 0.88k 0.92 MLP Images+BoW+SNPSY model 2 0.88k 0.81 0.84 0.84 0.92k SVM 0.85 0.88 0.87 0.86 0.94 LR Selected features model 1 (P<.05) 0.85 0.85 0.85 0.84 0.90 MLP 0.91k 0.77 0.83 0.84 0.94 SVM Selected features model 2 (P<.001) 0.83 0.77 0.80 0.80 0.92k SVM 0.91k 0.81 0.86 0.86 0.95k SVM a Pr: precision. b R: recall. c F1: F1 score. d Ac: accuracy. e AUC: area under the curve. f BoW: bag of words. g MLP: multilayer perceptron. h CNN: convolutional neural network. i SPV: short profile version. j LR: logistic regression. k The best results for each of the evaluation measures. l SNPSY: Social networks and psychological features. m SVM: support vector machine. n RF: random forest. on both tasks. For these representations, tweets unrelated to the Short Profile Version Definition Results topic seem to introduce noise, as they generate a bigger feature As can be seen in Table 9, the definition of the SPVC is space. In contrast, setting a high decision threshold for the successful as the first filter for both the predictive tasks. Indeed, classifier implies reducing the vocabulary for the BoW model, the BoW models trained exclusively on the SPV (baselines 3 which might reduce the performance of the model with the test and 4) outperformed baselines 1 and 2 for most of the measures data. https://www.jmir.org/2020/7/e17758 J Med Internet Res 2020 | vol. 22 | iss. 7 | e17758 | p. 10 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Ramírez-Cifuentes et al Regarding the CNN embedding models trained exclusively on implied that the use of the SNPSY features alone allowed us to the SPV, we can see that the model of task 1 obtains slightly have a model with a reduced number of features that perform better results compared with the baseline 2 model, whereas the as well as the BoW model with thousands of features. results do not differ much for task 2. In general, we observed a Regarding the role of the images, we can see that when they are better performance with the SPV for BoW models. Therefore, individually combined either with the BoW features or the the combinations evaluated take into account these text-based SNPSY features, either the F score or the AUC score increases representations (BoW). minimally compared with baseline 3. As part of the experiments It is important to recall that for the focused control cases, after for this approach, it is necessary to mention that as some image applying the SPVC with a decision threshold over 0.5, 4 users scores were missing for a few users (up to 4 for each task), the were left without an SPV because any of their tweets obtained approach considered to address this issue was to replace the a predicted probability over the threshold. Considering that with scores by the mean of all the users except for the model where higher thresholds, more focused control and generic users could only a single score for each feature type was considered; for be lost for training our next classifier, this is the threshold we this case, the instances with missing values were removed. kept for our further experiments (0.5). However, these results In reference to the models with a set of selected features, we also showed that using SPVC reduces the number of control can notice that these models also outperform baselines 1 and 2 users with an SPV as the threshold value rises and that the in terms of F1, accuracy, and AUC. The selected features model definition of the SPV is useful for discarding users who do not 1 for both tasks outperformed baselines 3 and 4 on F1 and AUC. present tweets similar to those of the users at risk. Initially, we It should be noted that these models consider a reduced number found that focused control users were more easily discarded of features compared with the baseline models, and the than generic users. However, this could be explained by the fact Images+BoW model, as they attempted to reduce the overfitting that the control users discarded might correspond to informative that the usage of thousands of features might imply. accounts such as newspapers, which we assumed to make use of certain terms referring to suicide in a way that does not make It is important to mention that for the models evaluated; we did use of terms that imply a personal reference or opinion; not address the definition of specialized decision thresholds for therefore, the first classifier might find it easier to discard. In the classifiers. Therefore, for future work, this should be any case, this is a supposition as we did not have further access considered because false-negative predictions should be to the writings of users after the annotation. minimized, and a threshold should be defined to maximize the R value of the suicidal ideation class without leaving aside the Combining Model Results tradeoff with Pr. Regardless of this, we can see that most of our Regarding the methods considered for combining the types of models outperformed the baselines in terms of the AUC score, features extracted, we can observe that when these types are which implies that our models are capable of distinguishing evaluated independently from each other, each has a good between the classes successfully. In this sense, the definition accuracy, with the SNPSY model obtaining the best results. For of a better decision threshold can only contribute to improving the combinations reported in Table 4 for the suicidal ideation the performance of our models. risk versus focused control groups, we can observe that the models that use the 3 types of features do not significantly Comparative Results of Tasks improve the results obtained by the SNPSY model. However, When comparing the results of both tasks, we saw that the for Images+BoW+SNPSY model 2, we can see a 7% and 11% results obtained by the models to distinguish between users at increase in the AUC score compared with baseline 3 and risk from generic control users were not that different from baseline 1, respectively, for the suicidal ideation versus focused those trained over focused control users. However, we noticed control cases. The AUC difference of their ROC curves using always higher levels of certainty for the models trained to the Delong method was P=.04, which is statistically significant, compare users at risk and generic control users. This can be considering a 95% CI. observed when comparing the AUC scores, which are always higher for the models of task 2. In fact, for this task, we can see For task 2, the Images+BoW+SNPSY combination obtained that a high AUC score is already obtained by the baseline results that improved baseline 1 for the suicidal ideation versus models, and it does not improve significantly with other models. generic control task. For the Images+BoW+SNPSY model 1, This differs from task 1, where the feature combination is we noticed a 4% increase in accuracy compared with baseline relevant for improving the certainty of the models compared 3, and it increased to 8% compared with baseline 1. There was with the baseline. also an increase in the AUC value of up to 4% with the selected features model 2. We also noticed the same measured results Figure 1 shows the top 10 most correlated features with the between the SNPSY model, the BoW+SNPSY model, and the class for each task considering the features of Images+BoW+SNPSY model 1, implying that we might not Images+BoW+SNPSY model 2. The most correlated features improve the performance of the SNPSY model by adding other were given by textual elements such as the BoW model scores feature types. In fact, after conducting a Delong test to compare and lexicons. It is interesting to see that a behavioral feature as the ROC curves of these models with the baseline 1 model, we the median time between tweets is relevant for task 2. We can could not find significant differences, implying that their also notice that for both tasks, self-references are relevant and performance was not significantly different from the baseline that the usage of explicit suicide terms and health-related terms in terms of the AUC measure for this task. However, this also https://www.jmir.org/2020/7/e17758 J Med Internet Res 2020 | vol. 22 | iss. 7 | e17758 | p. 11 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Ramírez-Cifuentes et al is relevant for task 2, as generic control users are not We also evaluated the selected features model 2, as one of our characterized by the usage of terms related to suicide. models with the best results for AUC for both tasks, over a sample of 200 users who were initially labeled as doubtful cases. Referring to the features that were more predictive for the We evaluated 2 models, one trained with the data of task 1 models generated, we considered the random forest’s feature (selected features model 2—task 1) and another trained with importance function, which is based on its measure of impurity. the data of task 2 (selected features model 2—task 2). For the In this sense, we can see how much each feature decreases the first model, we predicted 65% of the doubtful cases as positive impurity. The more a feature decreases the impurity, the more (risk), whereas for the second model, 73% of the doubtful cases important is the feature. In this case, because random forest uses were found to be at risk. This indicates that our models detected multiple trees, the impurity decrease from each feature was signs of suicidal ideation in more than half of the doubtful users, averaged across all trees to determine the final importance of which is in concordance with the criteria of the first annotator. the variable. The most important features based on this approach, considering the features of Images+BoW+SNPSY model 2 is Finally, we evaluated the selected features model 2—Task 1 shown in Figure 2. For this case, we confirmed that for task 2, over a test set of suicidal ideation and generic control users to the usage of terms related to work and health is distinctive for evaluate the performance of this model over users who do not both classes, as mentioned in the Statistical Analysis section. use a suicide-related vocabulary. We obtained the following results: Pr=0.91, R=0.77, F1=0.83, accuracy=0.84, and For both approaches, we can see that the image scores do not AUC=0.95. These results showed that the model obtains better appear within the features more relevant for the tasks, implying results than generic control users in comparison with its that textual and behavioral features can be more relevant. performance over focused control users. Similarly, we evaluated Regardless of this, the scores given by certain feature the selected features model 2—task 2 over a test set of suicidal combinations showed that the inclusion of the image scores ideation and focused control users obtaining Pr=0.83, R=0.80, improves minimally the results of these predictive tasks. Some F1=0.82, accuracy=0.82, and AUC=0.91. The performance of of the images that most activated neurons at the image this model was worse than that of generic control users and was processing stage involve pictures of people crying and images consistent with the fact that distinguishing these 2 cases is much containing the term suicide written within them. harder. Figure 1. Features more correlated with the class to predict for both tasks: Suicidal ideation risk vs Focused control (left), and Suicidal ideation risk vs Generic control (right). Figure 2. Most predictive features for both tasks: Suicidal ideation risk vs Focused control (left), and Suicidal ideation risk vs Generic control (right). data to build statistical and deep learning–based predictive Discussion models. Our models consider a set of features based on BoW Principal Findings and n-grams, lexicons, relational, statistical, and behavioral information, in addition to an image analysis. To the best of our This study presented a methodology for suicide risk assessment knowledge, this is the first approach that addresses the on social media. We extracted information from multimodal https://www.jmir.org/2020/7/e17758 J Med Internet Res 2020 | vol. 22 | iss. 7 | e17758 | p. 12 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Ramírez-Cifuentes et al combination of all these types of features for suicide risk at risk are actually at risk because the annotation was performed assessment at the user level. Moreover, we highlighted the just from reading a few tweets. usefulness of discarding the noise of writings not related to the topic of study through the definition of an SPV, which Reproducibility and Ethical Concerns outperforms the baseline given by the analysis of the full profile The analysis of data provided by social networks to detect health of the user, with an increase in accuracy and F1. problems and assist clinicians is an open issue, not uncontroversial. The aim of our proposal, however, is to shed We also compared the performance of predictive methods light on the real capabilities of these systems in a specific trained on different control groups with the goal of making a theoretical application: suicide risk prevention. Before such more specialized classifier capable of distinguishing users at systems become available, a careful risk-benefit assessment risk from control cases, even when the discussed topic is similar. along with a proper analysis of applicable legal framework Better results are achieved in terms of AUC-ROC when using compliance and the potential threats to users’ privacy and civil generic control users instead of users who make use of suicidal liberties shall be conducted [12,26,49]. vocabulary. We also highlighted the importance of the interpretability of our features, considering elements that can On the reproducibility of this work, we should respect Twitter's be understood by clinicians and mapped to their screening policies on the distribution of the data collected through its API. practice. The results of our experiments showed that within the Taking into account the restrictions of sharing any information types of features analyzed, there are multiple significant features that can lead to identifying the users of our study [12], only the that may lead to the detection of risk situations, the most relevant calculated features of our experiments along with the code and ones are based on the identification of textual and behavioral parameters used on the classifiers will be available under request elements such as self-references, the number of tweets posted, with a proper explanation of the usage intended for the models. and the time that passes between each post (P<.001). Finally, it is also necessary to address the potential scenarios of misusage of tools based on these models. Guntuku et al [9] Text-based features were the most relevant for our model; mentioned hypothetical cases where these types of screening however, their combination with image-based scores, along tools can be used by employers or insurance companies against with relational and behavioral aspects, allowed us to obtain the interests of people with mental disorders; therefore, policies models that outperform the results provided by an exclusively against these applications shall be defined before their release. text-based model. Future Work Limitations By observing the performance of image-based predictive Our study presents some limitations, given mainly by the fact models, we believe that our results can be improved by that this is an observational study where there is no access to enhancing the contribution of the textual and relational features. the personal and medical information that is often considered This can be done by exploring other text representation methods in risk assessment studies. Limitations are given by the accuracy and analyzing deeply the network and interactions between of the methods applied to infer some of these aspects based on users. In addition, we believe that these methods can be extended the vocabulary and behavioral patterns of the users. For instance, and applied to different use cases related to mental disorders, the weekend tweets count ratio is because of the difference in such as depression, anxiety, or eating disorders. the posting time according to the user’s time zone and the sleeping time tweets ratio, for which we assume that the sleeping We believe that the approach presented in this study can evolve time is the period in a day when the user has less activity. into a real-time system that emits alerts when users at risk are found. It is necessary to recall that given that scenario, the idea A representativeness analysis regarding age, gender, and location of such a tool is to be the first filter to assist clinicians and does of the users analyzed was not performed, considering that this not intend by any means to replace their work. In fact, a readable type of information is not available on Twitter. However, the subset of tweets from the SPV, with the top k tweets ordered biases existing in our data samples, regarding these aspects, according to the SPVC, can be an output of the system for follow similar gender and age biases for suicides [48], as most clinicians to evaluate and proceed with future screening steps, Twitter users are male and middle aged [14]. In addition to this, if allowed. Indeed, legal and ethical issues for the deployment the fact that the lexicon used to extract data was obtained from of such systems should be analyzed before. However, we believe a different source such as Reddit might imply biases introduced that a tool of this kind can be the starting point for the by the type of vocabulary used in Reddit, along with the development of noninvasive interventions where specialists and limitations that the translation to Spanish might introduce even social media community members can contribute to the recovery if the terms and phrases used were thoughtfully inspected and and prevention of mental health issues and suicide. Again, the verified by specialized clinicians. appropriate protocols and procedures should be defined for this Finally, there are also limitations given by the nature of the instance. In particular, there should be a deep analysis of the users who post on Twitter, as they might differ from users at implications of a potential intervention taking into account the risk who do not choose to make their profiles public or even legal boundaries set for the treatment of personal data in this from users at risk who do not have a Twitter account. In context. addition, it is not guaranteed that the users annotated as users https://www.jmir.org/2020/7/e17758 J Med Internet Res 2020 | vol. 22 | iss. 7 | e17758 | p. 13 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Ramírez-Cifuentes et al Acknowledgments This study was supported by the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Program (MDM-2015-0502). The authors acknowledge the funding received by the Spanish project TIN2015-65464-R (MINECO/FEDER). The authors also acknowledge the support of NVIDIA Corporation for the donation of a Tesla K40 GPU and a GTX TITAN GPU, used for this research. Conflicts of Interest None declared. References 1. World Health Organization. Preventing Suicide: A Global Imperative. Geneva: World Health Organization; 2014. 2. Coppersmith G, Dredze M, Harman C, Hollingshead K, Mitchell M. CLPsych 2015 Shared Task: Depression and PTSD on Twitter. In: Proceedings of the 2nd Workshop on Computational Linguistics and Clinical Psychology. 2015 Presented at: CLPsych'15; June 5, 2015; Denver, Colorado. [doi: 10.3115/v1/w15-1204] 3. Coppersmith G, Leary R, Crutchley P, Fine A. Natural Language Processing of social media as screening for suicide risk. Biomed Inform Insights 2018;10:1178222618792860 [FREE Full text] [doi: 10.1177/1178222618792860] [Medline: 30158822] 4. De Choudhury M. Anorexia on Tumblr: A Characterization Study. In: Proceedings 5th International Conference on Digital Health. USA: ACM; 2015 Presented at: DH'15; May 18 - 20, 2015; Florence, Italy p. 43-50. [doi: 10.1145/2750511.2750515] 5. O'Dea B, Wan S, Batterham PJ, Calear AL, Paris C, Christensen H. Detecting suicidality on Twitter. Internet Interv 2015 May;2(2):183-188. [doi: 10.1016/j.invent.2015.03.005] 6. Losada DE, Crestani F, Parapar J. eRISK 2017: CLEF Lab on Early Risk Prediction on the Internet: Experimental Foundations. In: Proceedings of the International Conference of the Cross-Language Evaluation Forum for European Languages. 2017 Presented at: CLEF'17; September 11-14, 2017; Dublin, Ireland p. 343-360. [doi: 10.1007/978-3-319-98932-7_30] 7. Losada DE, Crestani F. A test collection for research on depression and language use. In: Fuhr N, Quaresma P, Gonçalves T, Larsen B, Balog K, Macdonald C, et al, editors. Experimental IR Meets Multilinguality, Multimodality, and Interaction. Cham: Springer; 2016:28-39. 8. De Choudhury M, Counts S, Horvitz E, Hoff A. Characterizing and Predicting Postpartum Depression from Shared Facebook Data. In: Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing. USA: ACM; 2014 Presented at: CSCW'14; February 15 - 19, 2014; Maryland, Baltimore, USA p. 626-638. [doi: 10.1145/2531602.2531675] 9. Guntuku SC, Yaden DB, Kern ML, Ungar LH, Eichstaedt JC. Detecting depression and mental illness on social media: an integrative review. Curr Opin Behav Sci 2017 Dec;18:43-49. [doi: 10.1016/j.cobeha.2017.07.005] 10. Reece AG, Reagan AJ, Lix KL, Dodds PS, Danforth CM, Langer EJ. Forecasting the onset and course of mental illness with Twitter data. Sci Rep 2017 Oct 11;7(1):13006 [FREE Full text] [doi: 10.1038/s41598-017-12961-9] [Medline: 29021528] 11. Shing HC, Nair S, Zirikly A, Friedenberg M, Daumé III H, Resnik P. Expert, Crowdsourced, and Machine Assessment of Suicide Risk via Online Postings. In: Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic.: Association for Computational Linguistics; 2018 Presented at: CLPsych'18; June 5, 2018; New Orleans, LA p. 25-36. [doi: 10.18653/v1/w18-0603] 12. Chancellor S, Birnbaum M, Caine E, Silenzio V, De Choudhury M. A Taxonomy of Ethical Tensions in Inferring Mental Health States from Social Media. In: Proceedings of the Conference on Fairness, Accountability, and Transparency. USA: ACM; 2019 Presented at: FAT*'19; January 29 - 31, 2019; GA, Atlanta, USA p. 79-88. [doi: 10.1145/3287560.3287587] 13. Abboute A, Boudjeriou Y, Entringer G, Azé J, Bringay S, Poncelet P. Mining Twitter for Suicide Prevention. In: Proceedings of the International Conference on Applications of Natural Language to Data Bases/Information Systems.: Springer International Publishing; 2014 Presented at: NLDB'14; June 18-20, 2014; Montpellier, France p. 250-253. [doi: 10.1007/978-3-319-07983-7_36] 14. Jashinsky J, Burton SH, Hanson CL, West J, Giraud-Carrier C, Barnes MD, et al. Tracking suicide risk factors through Twitter in the US. Crisis 2014;35(1):51-59. [doi: 10.1027/0227-5910/a000234] [Medline: 24121153] 15. Zirikly A, Resnik P, Uzuner Ö, Hollingshead K. CLPsych 2019 Shared Task: Predicting the Degree of Suicide Risk in Reddit Posts. In: Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology.: Association for Computational Linguistics; 2019 Presented at: CLPsych'19; June 6, 2019; Minneapolis, Minnesota p. 24-33. [doi: 10.18653/v1/w19-3003] 16. De Choudhury M, Gamon M, Counts S, Horvitz E. Predicting Depression via Social Media. In: Proceedings of the Seventh International AAAI Conference on Weblogs and Social Media. 2013 Presented at: ICWSM'13; July 8–11, 2013; Boston, Massachusetts p. 128-137. https://www.jmir.org/2020/7/e17758 J Med Internet Res 2020 | vol. 22 | iss. 7 | e17758 | p. 14 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Ramírez-Cifuentes et al 17. Desmet B, Hoste V. Recognising Suicidal Messages in Dutch Social Media. In: Ninth International Conference on Language Resources and Evaluation. 2014 Presented at: LREC'14; May 26-31, 2014; Reykjavik, Iceland p. 830-835. 18. Colombo GB, Burnap P, Hodorog A, Scourfield J. Analysing the connectivity and communication of suicidal users on Twitter. Comput Commun 2016 Jan 1;73(Pt B):291-300 [FREE Full text] [doi: 10.1016/j.comcom.2015.07.018] [Medline: 26973360] 19. Buraya K, Farseev A, Filchenkov A. Multi-View Personality Profiling Based on Longitudinal Data. In: Proceedings of the International Conference of the Cross-Language Evaluation Forum for European Languages. 2018 Presented at: CLEF'18; September 10-14, 2018; Avignon, France p. 15-27. [doi: 10.1007/978-3-319-98932-7_2] 20. Scherr S, Arendt F, Frissen T, Oramas J. Detecting Intentional Self-Harm on Instagram: Development, Testing, and Validation of an Automatic Image-Recognition Algorithm to Discover Cutting-Related Posts. Social Science Computer Review 2019 Mar 18. [doi: 10.1177/0894439319836389] 21. Reece AG, Danforth CM. Instagram photos reveal predictive markers of depression. EPJ Data Sci 2017;6(1):15. [doi: 10.1140/epjds/s13688-017-0110-z] 22. Guntuku C, Preotiuc-Pietro D, Eichstaedt J, Ungar L. What Twitter Profile and Posted Images Reveal about Depression and Anxiety. In: Proceedings of the 2019 International AAAI Conference on Web and Social Media. 2019 Presented at: ICWSM'19; June 12-14, 2019; Munich, Germany p. 236-246. 23. Leis A, Ronzano F, Mayer MA, Furlong LI, Sanz F. Detecting signs of depression in tweets in Spanish: behavioral and linguistic analysis. J Med Internet Res 2019 Jun 27;21(6):e14199 [FREE Full text] [doi: 10.2196/14199] [Medline: 31250832] 24. Cacheda F, Fernandez D, Novoa FJ, Carneiro V. Early detection of depression: social network analysis and random forest techniques. J Med Internet Res 2019 Jun 10;21(6):e12554 [FREE Full text] [doi: 10.2196/12554] [Medline: 31199323] 25. Ji S, Yu CP, Fung S, Pan S, Long G. Supervised learning for suicidal ideation detection in online user content. Complexity 2018 Sep 9;2018:1-10. [doi: 10.1155/2018/6157249] 26. Hswen Y, Naslund JA, Brownstein JS, Hawkins JB. Monitoring online discussions about suicide among Twitter users with schizophrenia: exploratory study. JMIR Ment Health 2018 Dec 13;5(4):e11483 [FREE Full text] [doi: 10.2196/11483] [Medline: 30545811] 27. Vioules MJ, Moulahi B, Aze J, Bringay S. Detection of suicide-related posts in Twitter data streams. IBM J Res Dev 2018;62(1):7:1-7:12. [doi: 10.1147/jrd.2017.2768678] 28. Reddit. Suicide Watch Subreddit URL: https://www.reddit.com/r/SuicideWatch/ [accessed 2017-12-01] 29. Fowler JC. Suicide risk assessment in clinical practice: pragmatic guidelines for imperfect assessments. Psychotherapy (Chic) 2012 Mar;49(1):81-90. [doi: 10.1037/a0026148] [Medline: 22369082] 30. Twitter Developer. Sample realtime Tweets URL: https://developer.twitter.com/en/docs/tweets/sample-realtime/api-reference/ get-statuses-sample [accessed 2019-01-02] 31. Ramírez-Cifuentes D, Mayans M, Freire A. Early Risk Detection of Anorexia on Social Media. In: Proceedings of the 2018 International Conference on Internet Science. In; 2018 Presented at: INSCI'18; October 24-26, 2018; St. Petersburg, Russia p. 3-14. [doi: 10.1007/978-3-030-01437-7_1] 32. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, et al. Scikit-learn: machine learning in Python. J Mach Learn Res 2011;12:2825-2830 [FREE Full text] 33. Savand A. The Python Package Index. 2014. stop-words 2018.7.23 URL: https://pypi.org/project/stop-words/ [accessed 2019-03-05] 34. Baziotis C, Pelekis N, Doulkeridis C. DataStories at SemEval-2017 Task 4: Deep LSTM with Attention for Message-level and Topic-based Sentiment Analysis. In: Proceedings of the 11th International Workshop on Semantic Evaluation. 2017 Presented at: SemEval'17; August 3-4, 2017; Vancouver, Canada p. 747-754. [doi: 10.18653/v1/s17-2126] 35. Deriu J, Lucchi A, De Luca V, Severyn A, Müller S, Cieliebak M, et al. Leveraging Large Amounts of Weakly Supervised Data for Multi-Language Sentiment Classification. In: Proceedings of the 26th International Conference on World Wide Web. 2017 Presented at: WWW'17; April 3 - 7, 2017; Perth, Australia p. 1045-1052. [doi: 10.1145/3038912.3052611] 36. Tausczik YR, Pennebaker JW. The psychological meaning of words: LIWC and computerized text analysis methods. J Lang Soc Psychol 2010;29(1):24-54. [doi: 10.1177/0261927X09351676] 37. Ramírez-Esparza N, Pennebaker J, García F, Suriá R. The Psychology of Words Usage: A Computer Guided Program to Analyze Texts in Spanish. Mexican Psychology Journal 2007;24(1):85-99 [FREE Full text] 38. Hofman E. GitHub. 2018. senti-py URL: https://github.com/aylliote/senti-py [accessed 2019-03-05] 39. Rodriguez P, Gonzàlez J, Gonfaus JM, Roca FX. Integrating vision and language in social networks for identifying visual patterns of personality traits. Int J Soc Sci Humanit 2019;9(1):6-12. [doi: 10.18178/ijssh.2019.v9.981] 40. Lecun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE 1998;86(11):2278-2324. [doi: 10.1109/5.726791] 41. He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016 Presented at: CVPR'16; June 27-30, 2016; Las Vegas, NV. [doi: 10.1109/cvpr.2016.90] 42. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM 2017 May 24;60(6):84-90. [doi: 10.1145/3065386] https://www.jmir.org/2020/7/e17758 J Med Internet Res 2020 | vol. 22 | iss. 7 | e17758 | p. 15 (page number not for citation purposes) XSL• FO RenderX JOURNAL OF MEDICAL INTERNET RESEARCH Ramírez-Cifuentes et al 43. Xie S, Girshick R, Dollár P, Tu Z, He K. Aggregated Residual Transformations for Deep Neural Networks. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017 Presented at: CVPR'17; July 21-26, 2017; Honolulu, HI p. 1492-1500. [doi: 10.1109/cvpr.2017.634] 44. Loshchilov I, Hutter F. SGDR: Stochastic Gradient Descent with Warm Restarts. In: Proceedings of the 2017 International Conference on Learning Representations. 2017 Presented at: ICLR'17; April 24 - 26, 2017; Toulon, France URL: https:/ /openreview.net/pdf?id=Skq89Scxx 45. Kim Y. Convolutional Neural Networks for Sentence Classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014 Presented at: EMNLP'14; October 25–29, 2014; Doha, Qatar p. 1746-1751. [doi: 10.3115/v1/d14-1181] 46. Pérez NP, López MA, Silva A, Ramos I. Improving the Mann-Whitney statistical test for feature selection: an approach in breast cancer diagnosis on mammography. Artif Intell Med 2015 Jan;63(1):19-31. [doi: 10.1016/j.artmed.2014.12.004] [Medline: 25555756] 47. Pastore M, Calcagnì A. Measuring distribution similarities between samples: a distribution-free overlapping index. Front Psychol 2019;10:1089 [FREE Full text] [doi: 10.3389/fpsyg.2019.01089] [Medline: 31231264] 48. World Health Organization. 2019. Suicide in the World: Global Health Estimates URL: https://www.who.int/ publications-detail/suicide-in-the-world [accessed 2020-05-27] 49. Zuluaga AF. Ethics and suicidal behaviors. Rev Colomb Psiquiatr 2001;30(4):369-382 [FREE Full text] Abbreviations API: application programing interface AUC: area under the receiver operating characteristics AUC-ROC: area under the receiver operating characteristics curve BoW: bag of words CNN: convolutional neural network DTTR: daytime tweets ratio LIWC: Linguistic Inquiry and Word Count SNPSY: social networks and psychological features SPV: short profile version SPVC: short profile version classifier STTR: sleep time tweets ratio Edited by G Eysenbach; submitted 15.01.20; peer-reviewed by M Conway, D Reidenberg; comments to author 13.02.20; revised version received 28.03.20; accepted 28.03.20; published 07.07.20 Please cite as: Ramírez-Cifuentes D, Freire A, Baeza-Yates R, Puntí J, Medina-Bravo P, Velazquez DA, Gonfaus JM, Gonzàlez J Detection of Suicidal Ideation on Social Media: Multimodal, Relational, and Behavioral Analysis J Med Internet Res 2020;22(7):e17758 URL: https://www.jmir.org/2020/7/e17758 doi: 10.2196/17758 PMID: 32673256 ©Diana Ramírez-Cifuentes, Ana Freire, Ricardo Baeza-Yates, Joaquim Puntí, Pilar Medina-Bravo, Diego Alejandro Velazquez, Josep Maria Gonfaus, Jordi Gonzàlez. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 07.07.2020. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included. https://www.jmir.org/2020/7/e17758 J Med Internet Res 2020 | vol. 22 | iss. 7 | e17758 | p. 16 (page number not for citation purposes) XSL• FO RenderX