Computational and Structural Biotechnology Journal 20 (2022) 6526–6533 journal homepage: www.elsevier.com/locate/csbj Mini review The dynamic landscape of peptide activity prediction Oriol Bárcenas a,1, Carlos Pintado-Grima a,1, Katarzyna Sidorczuk b, Felix Teufel c,d, Henrik Nielsen e, Salvador Ventura a,⇑, Michał Burdukiewicz a,f,⇑ a Autonomous University of Barcelona, Institute of Biotechnology and Biomedicine, Spain b University of Wrocław, Faculty of Biotechnology, Poland c University of Copenhagen, Copenhagen, Denmark d Novo Nordisk A/S, Digital Science and Innovation, Denmark e Technical University of Denmark, Denmark f Medical University of Białystok, Clinical Research Centre, Poland a r t i c l e i n f o a b s t r a c t Article history: Peptides are known to possess a plethora of beneficial properties and activities: antimicrobial, anticancer, Received 19 October 2022 anti-inflammatory or the ability to cross the blood–brain barrier are only a few examples of their func- Received in revised form 21 November 2022 tional diversity. For this reason, bioinformaticians are constantly developing and upgrading models to Accepted 21 November 2022 predict their activity in silico, generating a steadily increasing number of available tools. Although these Available online 24 November 2022 efforts have provided fruitful outcomes in the field, the vast and diverse amount of resources for peptide prediction can turn a simple prediction into an overwhelming searching process to find the optimal tool. Keywords: This minireview aims at providing a systematic and accessible analysis of the complex ecosystem of pep- Peptides Activity tide activity prediction, showcasing the variability of existing models for peptide assessment, their Prediction domain specialization and popularity. Moreover, we also assess the reproducibility of such bioinformatics Functional peptides tools and describe tendencies observed in their development. The list of tools is available under https:// Machine learning biogenies.info/peptide-prediction-list/. Deep learning Ó 2022 Published by Elsevier B.V. on behalf of Research Network of Computational and Structural Bio- Reproducibility technology. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/ licenses/by-nc-nd/4.0/). Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6527 2. Methods for data acquisition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6527 3. List of existing models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6528 4. Trends and associations in ML-based peptide predictors over time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6528 4.1. The vast diversity of functional activities derived from peptide predictors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6528 4.2. Emergence of deep architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6528 4.3. ML algorithm: what is the most common choice? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6528 4.4. Bioinformatics resources: web servers, code repositories and reproducibility issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6529 5. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6530 Funding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6531 CRediT authorship contribution statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6532 Declaration of Competing Interest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6532 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6532 Appendix A. Supplementary data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6532 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6532 ⇑ Corresponding authors at: Autonomous University of Barcelona, Institute of Biotechnology and Biomedicine, Spain. E-mail addresses: salvador.ventura@uab.cat (S. Ventura), michalburdukiewicz@gmail.com (M. Burdukiewicz). 1 Oriol Bárcenas and Carlos Pintado-Grima are joint first authors. https://doi.org/10.1016/j.csbj.2022.11.043 2001-0370/Ó 2022 Published by Elsevier B.V. on behalf of Research Network of Computational and Structural Biotechnology. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). O. Bárcenas, C. Pintado-Grima, K. Sidorczuk et al. Computational and Structural Biotechnology Journal 20 (2022) 6526–6533 1. Introduction heuristic algorithm for feature engineering. Thanks to their archi- tectural flexibility, deep models can learn from structured and In living organisms, peptides are short amino acid sequences unstructured data [31]. Irrespective of the method, predictive that accomplish a wide variety of biological functions [1–4]. Pep- models are created to provide useful tools to the scientific commu- tides are expressed either as mature products [5] or cleaved from nity to solve specific problems. In this sense, users are interested in natural precursor proteins. Considering their potential, researchers obtaining a wide range of high-quality predictions as quickly and have been long interested in the identification and functional char- efficiently as possible, providing valuable insight into their data. acterization of peptides with relevant activities. Furthermore, the method should be robust and stable so that users Several machine learning (ML) models have been developed to can accurately re-run computations. Web servers are usually the this end, providing new and relevant insights into the field [6]. preferable option to fulfill these requirements, but researchers However, considerable discrepancies in theoretical assumptions and private companies with privacy concerns may consider them can arise. For example, peptides are generally defined as short unsuitable when dealing with sensitive data. Consequently, devel- amino acidic segments, but their minimum and maximum lengths opers are also encouraged to provide standalone applications in is a matter of discussion. While some researchers have provided code repositories that can be employed independently by users evidence of dipeptide self-assembly into higher-order structures in local environments. Additionally, standalone tools can be easily [7,8], certain activities require longer peptide lengths [9]. A similar integrated with the local analytical pipelines and thus be more case applies to maximum lengths, with some studies considering appropriate for high-throughput studies. peptides above 100 residues [10,11]. Beyond theoretical assump- Considering all the benefits and general appeal of predictive tions, the goal behind model generation can also create significant models, an overwhelming number of bioinformatics tools have differences. While some tools specialize in predicting single- been developed in the last years to systematically predict and clas- activity peptides such as antimicrobial [12–17], anticancer [18– sify peptides. Such tools are based on different features and ML 21], antiparasitic [22] or antiviral [23], others intend to predict algorithms that aim at identifying specific activities. This seem- multiple functions overlapping the same peptide [24,25]. In some ingly positive fact sometimes turns against users as they may cases, these multiple classifications are faded by blurry definitions struggle to find the best suited predictor for their precise needs. such as ‘‘antimicrobial peptides” (AMPs). This category is often The problem is exacerbated when functional published tools found in the description of these tools and is sometimes inappro- drown among broken web servers that are no longer available priately used to refer exclusively to antibacterial peptides instead [32] or unreported code repositories [33] that lead to serious of more broad AMPs (which include e.g. antifungal and antiviral reproducibility issues. peptides), a trend that can lead to unclear results [12,26]. Although some overviews of these ML models have been pub- Beyond traditional peptide functionalities, new activities are lished [34], there is still a need for a systematic review of the cur- arising and expanding this field of study. The characterization rent state of the peptide prediction field. In this work, we inspect and identification of peptides for therapeutic use has been a topic the characteristics of 140 existing ML tools for peptide activity of interest for many researchers in the last years. In this sense, anti- identification and describe observed associations and tendencies. aggregating peptides have emerged as promising candidates for With this minireview, we expect readers to find an exhaustive clinical therapies. For example, amphipathic and cationic alpha- and integrated resource for conducting their peptide activity pre- helical peptidic scaffolds have been described to bind alpha- diction tasks successfully, as well as raising awareness about the synuclein toxic species found in Parkinson’s Disease (PD) with reproducibility crisis of many ML models for the scientific nanomolar affinity [27,28]. Strikingly, peptides with these features community. often exhibit a crosstalk between anti-amyloid, antimicrobial and antibiofilm activities [29]. This opens a new window in the search for therapeutic strategies to treat and diagnose PD and other 2. Methods for data acquisition related diseases at the molecular level. The emergence of new pep- tide entities and applications points at developing novel tools for Current software tools appearing in the tool list of the online predicting peptide activities with high confidence. supplementary website were screened using PubMed’s API and ML models are algorithms that can find patterns in provided Google Scholar looking for the keywords antimicrobial, anticancer, training data and make predictions on new and unseen datasets. antifungal, antiviral, antiparasitic, antitoxin, antiangiogenic, antibio- Therefore, the original dataset used for training such models is of film, antihypertensive, antiinflammatory, cell-penetrating peptides, tremendous importance, as it determines the quality of the blood–brain barrier peptides, chemotactic, quorum sensing, surface- obtained results. The blurry definitions of peptides with some binding or neuropeptides and peptide prediction in the title or the activities make the acquisition of high-quality positive data diffi- abstract of the publication until July 1st, 2022. The code provided cult. However, creating negative data is even more challenging as in repositories was not tested, as it is considered to be the same there are essentially no experimentally verified negative examples. that was provided during the review process. However, we did test The most common practice is generating negative data by sam- web servers both for the availability of the hosting site and the pling a large database for peptides that most likely do not possess functionality of the tool on October 14th, 2022. If web server links the intended activities, e.g., by searching UniProt for entries not are no longer working, they are considered non-active. On the annotated with specific keywords and restricting sequence length. other hand, tools that provide an active web server but do not pro- However, there are many ways to perform data sampling, and vide results when a model input is given are deemed as non- authors usually define their own sampling method. It leads to functional. It is important to note that this assessment of function- models that are meant to predict the same activity but possess dif- ality of the web server depends very highly on the date and hour of ferent areas of competence. Moreover, it results in biased compar- the accession and it could change dynamically. isons of the performance of such models [30]. To measure the popularity of tools, we obtained their citations Different strategies can be used to extract the information from from CrossRef on October 14th, 2022. Due to their nature, citations unstructured data, such as peptide primary sequences. The usage are noisier measures of popularity for more recent tools. However, of shallow models requires the conversion of unstructured sequen- they still at least partially reflect the general interest of the scien- tial data into the structured, tabular format by employing some tific community. 6527 O. Bárcenas, C. Pintado-Grima, K. Sidorczuk et al. Computational and Structural Biotechnology Journal 20 (2022) 6526–6533 The list of tools was manually curated from original publica- tides surpass them despite having much less available tools (3, tions to assess their reproducibility into golden, silver, or bronze 10 and 20, respectively). Thus, the available activities highlight categories as proposed by Heil et al. in 2021 [35]. We included the variability of the functional landscape of peptide prediction, an additional category (‘‘below bronze”) for tools that did not fulfill which is experiencing an evident rise in popularity. It is translated the minimum information required for bronze reproducibility: into a sixfold increase of available models developed in the last five data, models, and source code published and downloadable. years, from 34 in 2017 to 205 in 2022 (Fig. 1C). This year-to-year Besides these minimum requirements, if the model repository is model increase is also appreciated when considering non- well documented, key analysis details are suitably recorded and cumulative values of tools deployed per year (Supplementary dependencies can be set up in a single command (e.g. via docker, Fig. 2). The overwhelming amount of peptide tools is becoming a conda, snakemake or renv) the tool is granted a silver category. source of confusion and misunderstanding for many researchers, The gold category is only given for those methods that allow the as they struggle to find the optimal resource for their specific reproduction of the entire analysis with a single command (e.g. needs. via makefile, snakemake, drake or targets). This workflow is sum- marized in the Supplementary Fig. 1. 4.2. Emergence of deep architectures Given the revolution of deep learning (DL) in the proteomics 3. List of existing models field, it is also relevant to consider the differences between deep and shallow ML algorithms in the peptide activity prediction area. The PubMed and Google Scholar screening rendered a total of The benefit of the deep frameworks in the field of peptide property 140 tools for predicting different peptide functions published from predictions is not obvious as the peptide datasets are usually more 2009 to July 2022, which are freely available for all users as an limited than protein ones, especially considering the number of online supplementary website at https://biogenies.info/peptide- available sequences [38]. Still, DL models are rapidly gaining pop- prediction-list/. ularity, accounting for almost half of the published models in 2021, In this peptide prediction list, tools can be easily sorted accord- whereas no deep models were published before 2018 (Fig. 2A). ing to their functional prediction as detailed in the previous section When considering the number of citations of these models, some by keyword search. The overall availability of each method can be differences in the relative amount of citations for deep models in assessed by the presence of an active and functional web server, comparison with non-deep models can be observed (Fig. 2B). To code and training repositories and general reproducibility. There- quantify this observation, we introduce the citation score (C score) fore, each tool can be searched by name and publication DOI so and plot it against the year of publication (Fig. 2C). The C score pro- that users can identify all available bioinformatics resources linked vides an overview of the appeal of deep models, representing the to the tool according to their needs. relative number of citations of deep models vs non-deep models. To increase the accessibility of the online supplementary web- That is, a C score of 1 means that deep models have the same rel- site, tools with available web servers or repositories can be easily ative amount of citations as non-deep models. As an example, if accessed through the link provided in the corresponding columns. this value were 1.2, it would indicate that deep models are 20 % Additionally, all the information can be copied or downloaded as more cited than shallow models. With this score, it is clear that, csv, Excel or PDF files. except for those tools published in 2021, deep models are more appealing than non-deep models according to their relative num- 4. Trends and associations in ML-based peptide predictors over ber of citations. It highlights the popularity of deep models and time provides a plausible explanation for the explosion of deep models created in 2021. 4.1. The vast diversity of functional activities derived from peptide predictors 4.3. ML algorithm: what is the most common choice? One of the main questions that developers ask themselves when A key question when developing an ML algorithm relies on the developing a new predictor is what are the key points that differ- final classifier that will be selected for the prediction. Among all entiate average from successful and well-cited tools, beyond the ML classifiers used by the tools, shallow models represented by journal of publication. This is important, as bioinformaticians are support vector machines (SVM) and random forests (RF) are the constantly aiming to provide the best possible resources that most frequent, with 64 and 59 independent architectures, respec- would help researchers in their respective work fields, offering tively (Supplementary Fig. 3). They are closely followed by DL algo- solutions to major concerns. For example, pathologies affecting rithms (51) that have emerged as a common method to build ML the nervous system, such as Parkinson’s or Alzheimer’s disease, models either alone or in a combination of an ensemble. For the require dedicated software to screen for peptides with the ability sake of completeness, all independent ML classifiers from ensem- to cross the blood–brain barrier. Other cases include AMPs or anti- bles were included. Both RF and SVM, like all other shallow ML cancer peptides, which have been widely studied for their implica- algorithms, work with structured data. They are also quite popular tion for human health and disease and the development of novel in the ML field for being easy to train and tune, but they require therapeutics in the last decade [36,37]. This is exemplified by the converting unstructured sequential data to a tabular format using high number of tools predicting these activities (Fig. 1A). Anti- feature engineering. SVMs can perform both linear and non-linear cancer predictors are found to be the most prevalent with 42 tools, classification and regression but are often difficult to scale to large closely followed by the antimicrobial activity with 40, which datasets. In contrast, RF methods are less appropriate for regres- accounts for 20.5 % and 19.5 % of all annotated activities (205), sion but are able to learn how important each feature is to the pre- respectively. Nonetheless, activities with a lot of available tools diction [39]. do not have the highest median number of citations per year Although individual ML algorithms have been described to be (Fig. 1B). This could be a consequence of the high level of compe- highly accurate and precise in their predictions, some tools use tition between predictors in the most popular activities. In this the combination of different models to create an ensemble that sense, anticancer and AMPs have a median of around 5.5 citations can potentially improve the prediction capabilities of individual per year, whereas, for example, toxic, antifungal or antiviral pep- models. This includes deep and non-deep combinations that claim 6528 O. Bárcenas, C. Pintado-Grima, K. Sidorczuk et al. Computational and Structural Biotechnology Journal 20 (2022) 6526–6533 Fig. 1. Peptide prediction landscape by predictive activity, according to the number of publications per activity (A), the average number of citations per year (B) and the cumulative number of tools (C). In this plot, activities are counted individually. That is, if a tool has more than one predictive activity, each one is counted independently. Those activities with 10 or more published tools are presented in color (antibacterial, antimicrobial, antiinflammatory, cell-penetrating, anticancer, antifungal and antiviral). In these cases, the average number of citations is displayed as a boxplot to better show the distribution of the average citation differences between activities. These results indicate that the activities with the highest amount of models do not necessarily have the highest average citations. to be a viable strategy for obtaining more reliable classifications, might be driven by the recent assessment of reproducibility stan- for example by using features obtained by a DL algorithm to train dards for machine learning in the life sciences [35], here we only a shallow model [40]. analyze the reproducibility status of published tools from 2018 onwards (111 out of the 140 tools). We observed that tools that 4.4. Bioinformatics resources: web servers, code repositories and do not incorporate open web servers (57 out of 111) tend to have reproducibility issues a lower number of citations than those that include them (Fig. 3A). Nonetheless, having an active web server at the time of publication Once the purpose of the tool is defined and the corresponding is not enough. Published web servers tend to go offline due to the method has been validated, developers need to think of suitable associated maintenance costs and technical management required. platforms to allow users to run predictions. Web servers are usu- Among all 57 peptide tools with associated web servers, 24 were ally the priority option because they provide an easy framework no longer accessible or functional (42.1 %) on October 15th 2022. to run calculations without the necessity of technical expertise in It means that these tools have lost potential interactions with users programming or command-line management. that could be interested in running predictions using their algo- Aside from practical usability aspects, the tool should be repli- rithms. This is also reflected in the total number of citations, where cable given the source code and the data are provided. Interest- tools with non-active web servers obtain fewer citations than ingly, it is only recently that the authors have put more active predictors. emphasis on the reproducibility of their tools. Indeed, the first tool The most concerning observation, however, is related to the with at least a bronze level of reproducibility was published in reproducibility (or lack thereof) of ML methods (Fig. 3B). Of the 2018 (Supplementary Fig. 4). In an effort to balance this bias, which 111 tools, only 38 (34.2 %) contained the minimum information 6529 O. Bárcenas, C. Pintado-Grima, K. Sidorczuk et al. Computational and Structural Biotechnology Journal 20 (2022) 6526–6533 Fig. 2. Comparison between deep and non-deep models, according to the number of published models (A) and the number of citations (B). The citation score for the deep models (C) indicates for which years deep models were more cited than non-deep models (scores over 1). This score provides an accurate perspective on the attractiveness of DL against shallow tools. required to reproduce the ML algorithm independently: source ing the contribution weight of each feature to the final decision code, data, and models published and downloadable. Among them, limited their performance. Thanks to recent advances in ML and nine gold and four silver tools presented the highest possible stan- bioinformatics, an increasing number of tools based on ML have dard of reproducibility (Table 1). The rest fell into the ‘‘below been released, as it is becoming the standardized method to gener- bronze” category where, in the best-case scenario, only web ser- ate predictive algorithms. vers were provided to allow users to run predictions. Something The success of ML methods in predicting peptide properties has remarkable, however, is that web server availability is a much bet- led to the development of a vast amount of tools that intend to be ter indicator of the average number of citations per year than the useful resources for the scientific community. In this sense, reproducibility standard of these tools (Fig. 3C). This situation researchers generally prefer the development of user-friendly may be the root cause of the ML reproducibility crisis, as making web servers that are easy to use rather than complex code reposi- a reproducible model is time-consuming and does not render an tories that only people with some bioinformatics and program- apparent citation-wise benefit. Given the existing lag between ming background can run. Based on our analysis, it seems that the date of publication and when tools start receiving their first providing open-source code does not render any benefit citation- citations, it is also possible that the average number of citations wise, and as a consequence, researchers may opt to keep their orig- per year for very recent tools is underestimated compared to inal code for future research without losing significant impact. We longer-lived tools. believe this is the origin of the reproducibility crisis we have described in this minireview since only 27 % of all tools (38 out 5. Discussion of 140) present the minimum information required to reproduce the ML algorithm from scratch. Peptide property prediction has positioned itself as a key field of A similar tendency applies to DL algorithms, which have activity within bioinformatics in recent years. The ability of pep- become the standard ‘‘advanced” models for peptide property pre- tides to mediate different types of functional activities and interac- diction for some researchers. The ability of these algorithms to tions with other biomolecules offers innovative candidates to learn relevant features directly from the data led many experts target specific pathologies. At the molecular level, peptide therapy in data science and related disciplines to translate this knowledge emerges as a promising strategy to specifically target the patholog- into the peptide prediction field. Although DL was successfully ical agents underlying these diseases [28,41,42]. Early predictors applied to identify AMPs in metagenomics data [43], it is not clear used architectures different from ML, but the difficulty in identify- whether the success DL has seen in other fields of science has yet 6530 O. Bárcenas, C. Pintado-Grima, K. Sidorczuk et al. Computational and Structural Biotechnology Journal 20 (2022) 6526–6533 Fig. 3. Comparison of tools published from 2018 onwards according to the average number of citations per year, web server availability and their reproducibility standard. (A) The density chart of models according to the number of citations. Density colors represent the availability of web servers. (B) Bar chart representing the number of models fulfilling the given reproducibility standard. (C) The beeswarm chart of models. The y-axis represents the reproducibility standard and x-axis represents the average number of citations per year. The color of the point represents the availability of the web server. Table 1 correctness of their models. Here, biological domain knowledge Tools granted with the gold or silver standard of reproducibility classification as helps in the design of robust benchmarks for a fair evaluation of adapted from Heil et al. [35]. Most of them are designed to predict anticancer or the performance of proposed models. For example, it was reported antimicrobial activities, which account for the majority of peptide tools as observed in Fig. 1. that some AMP predictors yield higher probabilities of antimicro- bial activity for long peptides which makes little sense from the Tool name Activity prediction Reference biological point of view [10]. As the majority of the effort in the ACP-MHCNN Anticancer [20] field is directed toward proposing new tools and not evaluating CancerGram Anticancer [18] existing ones, we are still unaware of the similar shortcomings of DeepACP Anticancer [19] iACP-FSCM Anticancer [21] available solutions. amPEPpy 1.0 Antimicrobial [16] Altogether, we argue that there is a need for reorganization and AmpGram Antimicrobial [14] clarification in the peptide property prediction field. The redun- AMPlify Antimicrobial [17] dancy of specific tools and the difficulty of finding optimal bioin- PredAPP Antiparasitic [22] PPTPP Antiangiogenic; Antibacterial; [24] formatics resources keep adding complexity to the standard use Anticancer; Antiinflamatory; Antiviral; of these methods. To at least partially alleviate this issue, we Cell-penetrating; Quorum sensing; enhance our minireview with an online supplementary web Surface-binding resource that collects all available peptidic tools to scrutinize Ampir Antimicrobial [15] accessible models and help users find the best for their needs. Macrel Antimicrobial [13] MLBP Anticancer; Antihypertensive; [25] We aim to raise awareness among researchers and developers Antiinflamatory; Antimicrobial about the reproducibility crisis observed in peptide property pre- PreAntiCoV Antiviral [23] diction and encourage them to work together to promote open science and cooperation between researchers. Funding been translated to peptide property prediction. Indeed, some stud- C.P.-G was supported by the Secretariat of Universities and ies suggest that DL models so far have failed to improve perfor- Research of the Catalan Government and the European Social Fund mance over shallow models for AMP identification [38]. (2021 FI_B 00087). F.T. was funded in part by the Novo Nordisk An influx of new tools for developing deep and shallow models Foundation through the Center for Basic Machine Learning significantly reduced the entry barrier for peptide activity predic- Research in Life Science (NNF20OC0062606). K.S. was partially tion. Still, these tools can not replace researchers in assessing the supported by the National Science Centre grant (2018/31/N/ 6531 O. Bárcenas, C. Pintado-Grima, K. Sidorczuk et al. Computational and Structural Biotechnology Journal 20 (2022) 6526–6533 NZ2/01338). S.V. was funded by the Spanish Ministry of Science [11] Shi G, Kang X, Dong F, Liu Y, Zhu N, Hu Y, et al. DRAMP 3.0: an enhanced comprehensive data repository of antimicrobial peptides. Nucleic Acids Res and Innovation (PID2019-105017RB-I00) and by ICREA, ICREA- 2022;50:D488–96. https://doi.org/10.1093/nar/gkab651. Academia 2020. M. B. was supported by the Maria Zambrano grant [12] Veltri D, Kamath U, Shehu A. Deep learning improves antimicrobial peptide funded by the European Union-NextGenerationEU. recognition. Bioinformatics 2018;34:2740–7. https://doi.org/10.1093/ bioinformatics/bty179. [13] Santos-Júnior CD, Pan S, Zhao X-M, Coelho LP. Macrel: antimicrobial peptide CRediT authorship contribution statement screening in genomes and metagenomes. PeerJ 2020;8:e10555. https://doi. org/10.7717/peerj.10555. [14] Burdukiewicz M, Sidorczuk K, Rafacz D, Pietluch F, Chilimoniuk J, Rödiger S, Oriol Bárcenas: Data curation, Investigation, Validation, Visual- et al. Proteomic screening for prediction and design of antimicrobial peptides ization, Writing – original draft, Writing – review & editing. Carlos with AmpGram. Int J Mol Sci 2020;21:4310. https://doi.org/10.3390/ Pintado-Grima: Data curation, Investigation, Validation, Writing – ijms21124310. [15] Fingerhut LCHW, Miller DJ, Strugnell JM, Daly NL, Cooke IR. ampir: an R original draft, Writing – review & editing. Katarzyna Sidorczuk: package for fast genome-wide prediction of antimicrobial peptides. Data curation, Investigation, Methodology, Software, Writing – Bioinformatics 2020;36:5262–3. https://doi.org/10.1093/bioinformatics/ review & editing. Felix Teufel: Writing – review & editing. Henrik btaa653. [16] Lawrence TJ, Carper DL, Spangler MK, Carrell AA, Rush TA, Minter SJ, et al. Nielsen: Writing – review & editing. Salvador Ventura: Conceptu- amPEPpy 1.0: a portable and accurate antimicrobial peptide prediction tool. alization, Funding acquisition, Supervision, Writing – review & Bioinformatics 2020. https://doi.org/10.1093/bioinformatics/btaa917. editing. Michał Burdukiewicz: Conceptualization, Data curation, [17] Li C, Sutherland D, Hammond SA, Yang C, Taho F, Bergman L, et al. AMPlify: attentive deep learning model for discovery of novel antimicrobial peptides Methodology, Software, Visualization, Supervision, Writing – effective against WHO priority pathogens. BioRxiv 2020:155705. https://doi. review & editing. org/10.1101/2020.06.16.155705. [18] Burdukiewicz M, Sidorczuk K, Rafacz D, Pietluch F, Ba˛kała M, Słowik J, et al. CancerGram: an effective classifier for differentiating anticancer from Declaration of Competing Interest antimicrobial peptides. Pharmaceutics 2020;12:1045. https://doi.org/ 10.3390/pharmaceutics12111045. The authors declare that they have no known competing finan- [19] Yu L, Jing R, Liu F, Luo J, Li Y. DeepACP: A novel computational approach for accurate identification of anticancer peptides by deep learning algorithm. Mol cial interests or personal relationships that could have appeared Ther - Nucleic Acids 2020;22:862–70. https://doi.org/10.1016/j. to influence the work reported in this paper. omtn.2020.10.005. [20] Ahmed S, Muhammod R, Khan ZH, Adilina S, Sharma A, Shatabda S, et al. ACP- MHCNN: an accurate multi-headed deep-convolutional neural network to Acknowledgements predict anticancer peptides. Sci Rep 2021;11:23676. https://doi.org/10.1038/ s41598-021-02703-3. [21] Charoenkwan P, Chiangjong W, Lee VS, Nantasenamat C, Hasan MM, The authors want to thank Dominik Rafacz (Warsaw University Shoombuatong W. Improved prediction and characterization of anticancer of Technology) for his help in the online supplementary website activities of peptides using a novel flexible scoring card method. Sci Rep development and the anonymous reviewers for their valuable 2021;11:3017. https://doi.org/10.1038/s41598-021-82513-9. comments. [22] Zhang W, Xia E, Dai R, Tang W, Bin Y, Xia J. PredAPP: predicting anti-parasitic peptides with undersampling and ensemble approaches. Interdiscip Sci Comput Life Sci 2022;14:258–68. https://doi.org/10.1007/s12539-021- 00484-x. Appendix A. Supplementary data [23] Pang Y, Wang Z, Jhong J-H, Lee T-Y. Identifying anti-coronavirus peptides by incorporating different negative datasets and imbalanced learning Supplementary data to this article can be found online at strategies. Brief Bioinform 2021;22:1085–95. https://doi.org/10.1093/bib/ https://doi.org/10.1016/j.csbj.2022.11.043. bbaa423. [24] Zhang YP, Zou Q. PPTPP: a novel therapeutic peptide prediction method using physicochemical property encoding and adaptive feature representation References learning. Bioinformatics 2020;36:3982–7. https://doi.org/10.1093/ bioinformatics/btaa275. [25] Tang W, Dai R, Yan W, Zhang W, Bin Y, Xia E, et al. Identifying multi-functional [1] Ghosh C, Sarkar P, Issa R, Haldar J. Alternatives to conventional antibiotics in bioactive peptide functions using multi-label deep learning. Brief Bioinform the era of antimicrobial resistance. Trends Microbiol 2019;27:323–38. https:// 2022;23:bbab414. https://doi.org/10.1093/bib/bbab414. doi.org/10.1016/j.tim.2018.12.010. [26] Lin W, Xu D. Imbalanced multi-label learning for identifying antimicrobial [2] Zhou X, Smith QR, Liu X. Brain penetrating peptides and peptide–drug peptides and their functional types. Bioinformatics 2016;32:3745–52. https:// conjugates to overcome the blood–brain barrier and target CNS diseases. doi.org/10.1093/bioinformatics/btw560. WIREs Nanomed Nanobiotechnol 2021;13:e1695. https://doi.org/10.1002/ [27] Santos J, Gracia P, Navarro S, Peña-Díaz S, Pujols J, Cremades N, et al. wnan.1695. [3] Habault J, Poyet J-L. Recent advances in cell penetrating peptide-based a-Helical peptidic scaffolds to target a-synuclein toxic species with nanomolar affinity. Nat Commun 2021;12:3752. https://doi.org/10.1038/s41467-021- anticancer therapies. Molecules 2019;24:927. https://doi.org/ 24039-2. 10.3390/molecules24050927. [28] Santos J, Pallarès I, Ventura S. Is a cure for Parkinson’s disease hiding inside us? [4] Vandergriff A, Huang K, Shen D, Hu S, Hensley MT, Caranasos TG, et al. Trends Biochem Sci 2022;47:641–4. https://doi.org/10.1016/j. Targeting regenerative exosomes to myocardial infarction using cardiac tibs.2022.02.001. homing peptide. Theranostics 2018;8:1869–78. https://doi.org/10.7150/ [29] Santos J, Ventura S, Pallarès I. LL-37 and CsgC exemplify the crosstalk between thno.20524. anti-amyloid, antimicrobial, and anti-biofilm protein activities. Neural Regen [5] Saghatelian A, Couso JP. Discovery and characterization of smORF-encoded Res 2023;18:1027–8. https://doi.org/10.4103/1673-5374.355757. bioactive polypeptides. Nat Chem Biol 2015;11:909–16. https://doi.org/ [30] Sidorczuk K, Gagat P, Pietluch F, Kała J, Rafacz D, Ba˛kała L, et al. Benchmarks in 10.1038/nchembio.1964. antimicrobial peptide prediction are biased due to the selection of negative [6] Basith S, Manavalan B, Hwan Shin T, Lee G. Machine intelligence in peptide data. Brief Bioinform 2022;23:bbac343. https://doi.org/10.1093/bib/bbac343. therapeutics: A next-generation tool for rapid disease screening. Med Res Rev [31] Jurtz VI, Johansen AR, Nielsen M, Armenteros A, Juan J, Nielsen H, et al. An 2020;40:1276–314. https://doi.org/10.1002/med.21658. introduction to deep learning on biological sequence data: examples and [7] de Groot NS, Parella T, Aviles FX, Vendrell J, Ventura S. Ile-Phe dipeptide self- solutions. Bioinformatics 2017;33:3685–90. https://doi.org/10.1093/ assembly: clues to amyloid formation. Biophys J 2007;92:1732–41. https://doi. bioinformatics/btx531. org/10.1529/biophysj.106.096677. [32] Kern F, Fehlmann T, Keller A. On the lifetime of bioinformatics web services. [8] Gnanasekaran K, Korpanty J, Berger O, Hampu N, Halperin-Sternfeld M, Cohen- Nucleic Acids Res 2020;48:12523–33. https://doi.org/10.1093/nar/gkaa1125. Gerassi D, et al. Dipeptide nanostructure assembly and dynamics via in situ [33] Papin JA, Gabhann FM, Sauro HM, Nickerson D, Rampadarath A. Improving liquid-phase electron microscopy. ACS Nano 2021;15:16542–51. https://doi. reproducibility in computational biology research. PLOS Comput Biol 2020;16: org/10.1021/acsnano.1c06130. e1007881. https://doi.org/10.1371/journal.pcbi.1007881. [9] Clark S, Jowitt TA, Harris LK, Knight CG, Dobson CB. The lexicon of [34] Wang G, Vaisman II, van Hoek ML. Machine learning prediction of antimicrobial peptides: a complete set of arginine and tryptophan antimicrobial peptides. Methods Mol Biol Clifton NJ 2022;2405:1–37. sequences. Commun Biol 2021;4:1–14. https://doi.org/10.1038/s42003-021- https://doi.org/10.1007/978-1-0716-1855-4_1. 02137-7. [35] Heil BJ, Hoffman MM, Markowetz F, Lee S-I, Greene CS, Hicks SC. [10] Gabere MN, Noble WS. Empirical comparison of web-based antimicrobial Reproducibility standards for machine learning in the life sciences. Nat peptide prediction tools. Bioinformatics 2017;33:1921–9. https://doi.org/ Methods 2021;18:1132–5. https://doi.org/10.1038/s41592-021-01256-7. 10.1093/bioinformatics/btx081. 6532 O. Bárcenas, C. Pintado-Grima, K. Sidorczuk et al. Computational and Structural Biotechnology Journal 20 (2022) 6526–6533 [36] Gaspar D, Veiga AS, Castanho MARB. From antimicrobial to anticancer [40] Sharma R, Shrivastava S, Kumar Singh S, Kumar A, Saxena S, Kumar SR. peptides. A review. Front Microbiol 2013:4. https://doi.org/10.3389/ AniAMPpred: artificial intelligence guided discovery of novel antimicrobial fmicb.2013.00294. peptides in animal kingdom. Brief Bioinform 2021;22:bbab242. https://doi. [37] Deslouches B, Di YP. Antimicrobial peptides with selective antitumor org/10.1093/bib/bbab242. mechanisms: prospect for anticancer applications. Oncotarget [41] Craik DJ, Fairlie DP, Liras S, Price D. The future of peptide-based drugs. Chem 2017;8:46635–51. https://doi.org/10.18632/oncotarget.16743. Biol Drug Des 2013;81:136–47. https://doi.org/10.1111/cbdd.12055. [38] García-Jacas CR, Pinacho-Castellanos SA, García-González LA, Brizuela CA. Do [42] Xiao Y-F, Jie M-M, Li B-S, Hu C-J, Xie R, Tang B, et al. Peptide-based treatment: deep learning models make a difference in the identification of antimicrobial A promising cancer therapy. J Immunol Res 2015;2015:e761820. https://doi. peptides? Brief Bioinform 2022;23:bbac094. https://doi.org/10.1093/bib/ org/10.1155/2015/761820. bbac094. [43] Ma Y, Guo Z, Xia B, Zhang Y, Liu X, Yu Y, et al. Identification of antimicrobial [39] Greener JG, Kandathil SM, Moffat L, Jones DT. A guide to machine learning for peptides from the human gut microbiome using deep learning. Nat Biotechnol biologists. Nat Rev Mol Cell Biol 2022;23:40–55. https://doi.org/10.1038/ 2022;40:921–31. https://doi.org/10.1038/s41587-022-01226-0. s41580-021-00407-0. 6533