Expert Systems With Applications 237 (2024) 121593 Contents lists available at ScienceDirect Expert Systems With Applications journal homepage: www.elsevier.com/locate/eswa Multi-instance learning with application to the profiling of multi-victim homicides Rosario Delgado a ,∗, Héctor Sánchez-Delgado b a Department of Mathematics. Universitat Autònoma de Barcelona, Edifici C- Campus de la UAB. Av. de l’Eix Central s/n., 08193 Bellaterra (Cerdanyola del Vallès), Spain b Data Quality and Statistics Executive. Kantar, Carrer de Can Calders, 4, 08173, Sant Cugat del Vallès, Spain ARTICLE INFO ABSTRACT Keywords: Homicide involving multiple victims has a significant negative effect on society. Criminal profiling consists of Bayesian Network determining the traits of an unknown offender based on those of the crime and the victims, with a view to their Combination rule identification. To provide the most likely profile of the perpetrator of a multi-victim homicide, we propose a Criminal profile predictive model of supervised machine learning based on a Bayesian Network. Conventional classifiers can Ensemble generate the perpetrator’s profile according to the traits of each of the victims of the same homicide, but Multi-instance (MI) learning Multiple/mass homicide the profiles may differ from one another. To address this issue, we consider the Multi-Instance (MI) learning framework, in which the victims of the same incident form a bag, and each bag is associated with a unique label for each of the perpetrator’s features. We introduce the unanimity MI assumption in this domain, and accordingly allocate a label to the bag based on the labels and probabilities the Bayesian Network has assigned its instances, using a combination rule from those of the ensemble of classifiers. We apply this methodology to the Federal Bureau of Investigation (FBI) homicide database to compare three combination rules empirically in the validation process, as well as theoretically, using the one that ultimately proves to be the best to build the final model, which is then applied in some illustrative examples to achieve the criminal profile. 1. Introduction to be killed with malice aforethought, while manslaughter is the lesser form of homicide, which happens when death results even if there was Due to its brutality and the fear it instills, homicide is one of the no conscious desire to murder. Many U.S. states distinguish between most serious sorts of crime, being also one of the main causes of death two types of manslaughter: voluntary and involuntary, the first corre- in the United States of America (U.S.), following Fowler, Leavitt, Betz, sponding to the deliberate killing of another person during a sudden Yuan, and Dahlberg (2021). We must shed light and clarify the statistics altercation, when they were provoked and were unable to control the to comprehend the scope of homicides in the U.S. According to data situation, when they had some sort of diminished capacity due to a from the Federal Bureau of Investigation (FBI), there were 14,244 homicides on average per year from 2000 to 2021, with a rate of mental health issue, or when they had the sincere but ultimately irra- 4.58 per 100,000 inhabitants. To put it in a global context, following tional belief that using deadly force was necessary for self-defense. A the United Nations Office on Drugs and Crime, in 2017 the homicide person who kills another person unlawfully by criminal negligence, that rate per 100,000 inhabitants was much lower in the vast majority is, without intending to do so, is said to have committed involuntary of European countries, such as Spain, with 0.7, the United Kingdom manslaughter. with 1.2 or France, with 1.3. On the other hand, the countries of the In addition, we can divide homicides into four large groups, at- American continent have much higher rates in general. For example, tending to the number of victims per event or incident (time-place Costa Rica had 12.18 and Mexico, 25.71. So, we could say that on a combination), and the number of events per perpetrator: single, multi- global scale, the U.S. has a medium homicide incidence rate, although ple/mass, spree and serial. Single and multiple/mass homicides occur the harsh reality is that if we compare the U.S. with other developed in a single event, the first with a single victim, the multiple with 2 or 3 countries, its rate is significantly higher. victims, and the mass with more than 3. The difference between serial Murder and manslaughter are two distinct legal offenses that are murder and the multiple/mass homicide is that although in both cases included in ‘‘homicide’’, as explained in Newburn (2013). The distinc- tion between them is that murder implies that the victim was intended there are multiple victims, the latter occurs in a single act while serial ∗ Corresponding author. E-mail addresses: Rosario.Delgado@uab.cat (R. Delgado), Hector.Sanchez@kantar.com (H. Sánchez-Delgado). https://doi.org/10.1016/j.eswa.2023.121593 Received 6 April 2023; Received in revised form 18 August 2023; Accepted 10 September 2023 Available online 15 September 2023 0957-4174/© 2023 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by- nc-nd/4.0/). R. Delgado and H. Sánchez-Delgado Expert Systems With Applications 237 (2024) 121593 murder occurs at separate times and places, including a period of inac- capture the interdependence relationships between all the variables of tivity. This differentiation was established by FBI agent Robert Ressler the model, including the output variables, and that allows considering in 1974, who also coined the term ‘‘serial killer’’. Following Sutton and more than one as output variables to be predicted from the rest. Keatley (2021), the spree murder also kills different victims in different Multi-instance (MI) classification locations, but there is no cool-off period between them. Specifically, in The usual classifiers can provide a profile of the perpetrator ac- this work we are interested in developing a new methodology that can cording to the characteristics of each of the victims of the multi-victim be used to study multi-victim homicides (both, murders and voluntary homicide, but the profiles may differ from each other. This poses a ma- manslaughter, with multiple/mass victims), committed in the same jor problem as they have to give a single profile altogether. Somehow, time and place and registered as a single incident or event. We apply we have to get them to ‘‘agree’’ on what their murderer looks like, and this methodology to the case study of the FBI homicide database, which that is the challenge we face in this research. As a solution, we propose covers a period of 35 years, from 1980 to 2014, with data from all 51 to consider the victims of the same incident as forming a bag. In this states of the union. way, it could be considered that we move in the multi-instance (MI) Regardless of its incidence in terms of frequency or number of cases, classification environment described in Alpaydin, Cheplygina, Loog, and multi-victim homicide is a crime with a high impact on society that can Tax (2015), in which each bag is associated with a unique label (for be considered a persistent public health problem in the U.S. (see Fowler each of the output variables). et al., 2021). Hence, any instrument that can assist in identifying the Multi-instance (MI) learning was introduced by Dietterich, Lathrop, profile of the perpetrator of such a homicide will be valuable for both and Lozano-Pérez (1997) for drug activity prediction, and is generally law enforcement personnel responsible for upholding the law, and for applied in contexts where data is formed in terms of bags or sets of legislators who design targeted policies aiming at eliminating this social instances, these bags being the ones that carry the labels and not so the blight. individual instances. In this setting, the standard MI assumption (see Al- Criminal profiling paydin et al., 2015; Carbonneau, Cheplygina, Granger, & Gagnon, 2018; In general, criminal profiling is a technique that makes it possible Foulds & Frank, 2010), which assumes a binary classification task, to identify the traits of a criminal in relation to their crimes and states that the labels of the bags are determined by the disjunction of victims, the patterns of relationships between homicide victims and the labels of the instances in the manner described below: each instance their killer being of the utmost interest. It is frequently used to try to has a hidden class tag that can be either + or −, and + is assigned to a bag if and only if one or more instances that make up the bag are + . provide details about a serial killer that could aid in their identification The objective of the paper is to build a predictive model to assign and capture (Kocsis, 2006). But it can also be used for other types the labels at the bag level, with the particularity that it cannot be done of homicides, as we do in this work. We will not attempt to develop directly, but first the predictive model must be built at the instance a profile for a specific series of murders carried out by the same level and, later, the label will be assigned to each bag from the labels person. Instead, we will develop a predictive model that will give and probabilities assigned to the instances that compose it. For what investigators the most likely profile of the killer in the event of a multi- reason? Because each victim contributes an instance or case to the victim homicide. According to Fox (2022), homicides make up nearly database, providing information on each of the variables. That is, each a quarter of the crimes appeared in the offender profile publications, victim determines a case him/herself and cannot be considered as an demonstrating the interest in profiling this kind of offender, who is also incomplete part of a whole, as in the standard MI assumption. the criminal most typically portrayed in movies and television. Criminal profiling is generally based solely (or mainly) on the exper- Related works tise of investigators and forensic psychologists, and will then inevitably be subject to cultural biases or prejudices, inaccuracies, and misinter- In general, the standard MI assumption is useful in situations where pretations (see Kocsis, 2006; Palermo & Kocsis, 2004; Turvey, 2002), it is appropriate to describe an object by a set of parts, each of which although lately it is evolving towards a more rigorous process, adding is an instance carrying only a portion of the information needed for geographic analysis techniques, for example. To avoid the weaknesses classification. This is the case, for example, of visual recognition, where of this classic approach, machine learning methodologies can be used the image to be recognized is usually divided into small fragments, to build objective and quantitative decision making tools to support or in text categorization, where the instances correspond to the small profiling investigations, as in Brahan, Lam, Chan, and Leung (1998), paragraphs into which the text is divided. But even in these cases, the Gottschalk (2006) and Strano (2004). This is possible today by taking MI assumption is not guaranteed to hold and is generally too restrictive advantage of the opportunity to access the impressive computing power to handle real-life situations, as explained by Küçükasci and Baydogan of current computers, and criminal databases of solved crimes, such as in Küçükasci and Baydogan (2018), where encoding strategies are the one used in this work, which is the FBI database of solved homicides introduced to represent the bags using the frequency of the instances in the U.S. in each of the pieces into which the feature space is partitioned. Some Our objective is to develop a Machine Learning tool (a knowledge- other authors have relaxed this assumption by allowing other different based expert system) to predict certain characteristics of the perpetrator interactions between the labels of the instances and the label assigned from those of the homicide victims, learned from a real database, which to the bag they belong to. For example, Alpaydin et al. (2015) consider can help profile perpetrators of multi-victim homicides (considering that some of the instances can carry a label but it is not known which of that this type of homicide represents a challenge compared to single- the bag can, which forces the MI approach to be adapted to a situation victim homicides, as we will discuss later). Specifically, this tool will halfway between the pure instance-level approach and the bag-level be a probabilistic classifier that learns the dependency relationships classification. In addition, to address a problem of detecting failures in between these features and uses them to make the most plausible industrially manufactured entities, Graur, Maris, Potolea, Dinsoreanu, predictions for those that are unknown among them, which will be and Lemnaru (2018) introduce two approaches to solve MI learning those corresponding to the perpetrator of the crime. As the profile of problems in which the MI assumption is not met. We join these authors the aggressor consists of different characteristics (output variables) to in their heterodox vision of MI, but in a different situation from the one be predicted, that is, different variables or criteria according to which they consider, so we will have to make different premises, in line with to classify the perpetrator (age, race, . . . ), we have to predict the value Foulds and Frank (2010). These premises are grouped under the name of more than one target variable, which may or may not be correlated of unanimity MI assumption below. (multi-label classification). To do this, we use a Bayesian Network (BN On the other hand, there are different scientific papers in which a from now on), which is a probabilistic classifier that has the capacity to knowledge-based approach of the issue of criminal profiling is carried 2 R. Delgado and H. Sánchez-Delgado Expert Systems With Applications 237 (2024) 121593 out. What we will do now is briefly comment on some of the works individuals or elements. Each individual is identified with an carried out by other authors, without claiming to be exhaustive, to instance or case, and all those involved in the same event are highlight the differences with ours. The first works, to our knowledge, part of the corresponding bag. The conditions are: on the application of BN to criminal profiling are those of Baumgartner, Ferrari, and Salfati (2005) and Baumgartner, Ferrari, and Palermo (i) the instances cannot be considered as part of a whole (2008). In them, a BN is generated from a dataset of single-victim (each of them does not cover an aspect or part of the event homicides, but previously the collection of alternative structures to in which they are involved, but the total), consider is restricted using expert knowledge. The database is tiny (a (ii) each instance is assigned a label, but with the restriction few hundred cases) despite the big amount of variables (36 victim that the labels of all instances that make up a bag must characteristics plus 21 criminal traits), and contrast to the 𝐾-fold match, hence the term ‘‘unanimity’’ in its name. The same cross-validation technique we use, the validation is carried out by split- label is then assigned to the bag. validation, which prevents a statistical study to compare the suggested model with others. Despite this flaw, these works remain to be a 2. We built a predictive Machine Learning model leveraging the reference for the use of the BN methodology for profiling. This method- methodology of the combination rules of the ensembles of classi- ology has also been applied by Delgado, González, Sotoca, and Tibau fiers to predict the profile of a perpetrator involved in a multiple (2016, 2018) to the study of profiles of forest arsonists to understand homicide. With this approach, a single classifier is used to pre- their motivation, constructing archetypes that help identify the culprits, dict the characteristics of the perpetrator based on the attributes based on the data provided by the Spanish government. of the victims. To implement the classification following the Using homicide data recorded for one year in the National Incident- unanimity MI assumption introduced in the previous item, we Based Reporting System U.S. database, Yang and Olafsson (2011) con- present a predictive Machine Learning model that has been struct various machine learning classifiers, including Decision Trees, designed following these two steps: Random Forests, Support Vector Machines and Neural Networks, to (a) Classifier construction: since each instance has a label as- predict the relationship between murder victims and offenders. They signed to it, this step can and is done at the instance are limited to the context of single-label classification since they only level by learning a classifier as in single-instance learning, consider one variable in the murderer’s profile, which is the rela- tionship with the victim. Instead, we also forecast other aspects of bypassing that the instances are grouped into bags. the perpetrator (multi-label classification). Furthermore, they carry out (b) Prediction: Once the classifier is built, the prediction and their single-label multi-class classification by dividing it into several its validation procedure will be carried out at the bag simpler binary classification tasks. The point is that their solution level, since the objective is the labeling of the bags. To cannot always be properly translated to the original multi-class one. do this, we use the labels and probability distributions However, BNs allow multi-label multi-class classification to be carried assigned by the classifier to each of the instances that out naturally, without the need of any artifice. Finally, the authors also make up any new (unseen) bag, to assign it a label using use split-validation, which does not allow for a robust statistical study a suitable algorithm. This algorithm is a combination to adequately compare the four classifiers. rule (or scheme) specific to the ensemble of classifiers Another interesting study is presented in Fowler et al. (2021), (explained in Section 2.3 below). looking at multi-victim homicides in the U.S. With the help of bivariate Our approach is original, since focusing on multi-victim homi- statistical methods like chi-square tests and post-hoc pairwise compar- cides, it applies an innovative Supervised Machine Learning isons with Bonferroni corrections, as well as Exploratory Data Analysis, methodology that combines a probabilistic classifier (specifi- the authors looked at the characteristics of mass, multiple, and single- cally, a Bayesian Network, although others could be used as victim homicides. Their findings are based on a database of homicides well) at the instance level, with the MI learning approach com- from 35 of the 51 states, which covers a period of 15 years (from bined with ensembling. The details will be revealed throughout 2003 to 2017). Therefore, this study’s spatio-temporal window is visibly the work. narrower than ours. Yet, they have variables that we do not, such as 3. Application of the Predictive Model to the FBI homicide the incident’s location, which is a very valuable information. database: the predictive model developed is rigorously applied To wrap up this succinct overview, we note that Pecino-Latorre, to the FBI’s comprehensive database of criminal homicides. Pérez-Fuentes, and Patró-Hernández (2019) utilize the Classification This real-world application demonstrates the model’s ability and Regression Trees (CART) methodology to determine the charac- to analyze and predict homicidal offender profiles based on teristics of homicides using a database of 448 homicide cases in Spain. The authors develop a different model for each of the six target output victim-related features. By deploying the model with real data, variables they consider, thus missing the chance to improve the model’s the paper underscores its practicality and relevance within the predictive power by taking use of potential dependencies between domains of law enforcement and criminal profiling. them. In other words, they employ the simplest method for solving a In summary, the paper makes specialized assumptions for MI learn- multi-label classification –binary relevance– which ignores any potential ing, creates a new machine learning predictive model that employs relationship between labels. The Bayesian Networks methodology that a combination rule for the ensemble of classifiers, and validates the we have used in our work enables to not overlook these dependencies. model’s performance by applying it to the FBI homicide database. These contributions collectively advance the understanding and application of Paper contributions machine learning methodologies to predict perpetrator characteristics in multiple homicide situations. The main contributions of this paper are as follows: The organization of the rest of the paper is as follows: in Sec- 1. In line with the specific problem under investigation, which is tion 2 we present the data set, the Bayesian Networks as probabilistic criminal profiling of homicides with multiple victims, we present classifiers and the ensembling of combiners to be used in the sequel, two relevant conditions tailored to the context of MI learning as well as the validation procedure. Section 3 shows the results we that we have jointly called unanimity MI assumption. This have obtained, both for the Bayesian Network as predictive model, assumption is applicable to any problem domain corresponding and for the prediction at the bag level, comparing three combination to events identified with bags, each of them composed of several schemes through the different output variables, using the performance 3 R. Delgado and H. Sánchez-Delgado Expert Systems With Applications 237 (2024) 121593 Table 1 The 16 variables in the final database. (1) and (2) are used to exclude some cases and then removed (Fig. 1) while (3) is used to construct the new variable Bag, which is an auxiliary, and is then removed. Variables marked with (d) have been discretized, while those marked with (r) have been recoded. Variables in blue are those of the victim, variables of crime are in teal, and those of the perpetrator are in red, including their relationship to the victims. Variables in the original database Incorporated Removed Mantained Modified Record.ID State Victim.Age(d) Bag Agency.Code Year Victim.Race(r) Agency.Name Month Victim.Count(r) Agency.Type Victim.Sex Perpetrator.Age(d) City Victim.Ethnicity Perpetrator.Race(r) Crime.Solved(1) Perpetrator.Sex Perpetrator.Count(r) Crime.Type(2) Perpetrator.Ethnicity Weapon(r) Record.Source Relationship(r) Incident(3) metrics that we have introduced. We discuss our findings and conclude in Section 4, while the appendices include some tables (Appendix A) and figures (Appendix B). In Appendix C can be found the pseudo- code for the algorithms that implement the combination schemes and a comparison of their accuracy in the binary case. 2. Materials and methods 2.1. The data set Data source We got the FBI data through the kaggle repository.1 The most com- prehensive U.S. homicide database currently available, this database includes murders from the FBI’s Supplementary Homicide Report and covers a 35-year period, from 1980 to 2014 inclusive. It is compiled by the Murder Accountability Project, founded by Thomas Hargrove, to whom we wish to thank. Initially, the data included information on 24 variables for 638,454 cases (now and hereafter, a ‘‘case’’ or ‘‘instance’’ Fig. 1. Flowchart of the exclusion of victims from the initial database. To exclude is identified with a victim). unsolved cases we use the ‘‘Crime.Solved’’ variable in the original database, while to exclude victims of manslaughter by negligence we use the ‘‘Crime.Type’’ variable. Exclusion criteria Different criteria have been used to exclude cases and variables, variables, distinguishing between the homicides with multiple (2 or 3) until forming the final database with 36,269 cases and 16 variables. Re- garding the cases, Fig. 1 explains how the selection of those that make and massive (4 or more) victims. up the final database has been carried out. Regarding the variables, State: The 51 states of the union contribute cases to the database, with some have been eliminated, others have been modified and a new one California contributing the most (5300) and South Dakota the least has also been created and incorporated. We list them in Table 1 and (40), in absolute terms. give details in the Data description subsection below. Year: The data covers a 35-year period, from 1980 to 2014, peaking in Data description 1992 with 1307 victims and reaching the absolute minimum in 2014, Now we describe the variables in the database (Table 1) and, with just 844. in particular, we explain the preprocessing phase in which we have modified eight of them, six by recoding, and two by discretization. Month: The distribution of cases per months is quite balanced, varying We also explain the construction of the auxiliary variable Bag, which between 2722 in November and 3305 in January. has been incorporated into the final database. Table A.11 shows the relative frequencies (in percentages) of each category of the different Sex: The vast majority of the perpetrators are men (94.0% male, 6.0% female), with which the database is very unbalanced with respect to the variable Perpetrator.Sex, while as regards the victims, the 1 https://www.kaggle.com/datasets/murderaccountability/homicide- variable Victim.Sex is much more balanced, with 61.4% male, and reports?resource=download. 38.6% female. This fact evidences the sociological reality that women 4 R. Delgado and H. Sánchez-Delgado Expert Systems With Applications 237 (2024) 121593 continue to be victims much more frequently than perpetrators of involved, and has been recoded as binary with values 0 and 1, the serious crimes, especially homicides with various victims. latter being the category that groups any number of previous incidents greater than 0. Ethhicity and Race: The terms race and ethnicity are mutually in- dependent, according to the U.S. Office of Management and Budget Weapon: Our intention was to group some of the categories of this definition adhered to by the U.S. Census Bureau. Regardless of race, variable, corresponding to the means used by the perpetrator to commit ethnicity makes a distinction between people who claim to have ances- the homicide, by similarity, if they were rare. But due to its great tral origins in Spain or Latin America (Hispanic or Latino), and those variability, we have only done in these two cases: we have grouped in who do not (non-Hispanic).2 In 2020, 62 million of residents in the Long_hung both rifle and shotgun, and we have merged Drugs U.S., or 18.7% of the population, identified as Hispanic, of whom 12.5 and Poisson in a single category. Finally, we have 12 categories (plus million or 20.3% self-identified as White only3 down from the 2019 the Unknown). American Community Survey, when 38.3 million, or 65.5% of Latinos Relationship: Since it influences the type of aggressiveness, the rela- self-identified as White.4 In the datasets at hand, the vast majority of tionship between the perpetrator and the victims of an homicide has individuals categorized as Hispanic are also White. been considered as a cornerstone in the majority of studies on homi- Specifically, of the 3784 victims registered as Hispanic in our final cides, as in Yang and Olafsson (2011). In this work we have grouped database, 3723 are know to be White (98.4%), while of the 10,255 this variable into four categories: IPV (Intimate Partner Violence), non-Hispanics, 6348 are registered as White (61.9%). From the 1643 family (homicide committed by relatives, including in-laws, which Hispanic perpetrators, 1616 (98.4%) are recorded as White, and from cannot be considered IPV, such as, for example, son, daughter, father, the 4558 perpetrators known to be non-Hispanic, 2565 are recorded mother, sister, brother, . . . ), aquaintance (where the perpetrator is as White, representing 56.3%. However, for the reasons mentioned a friend, neighbor, employee or employer, . . . ) and stranger (com- above, we have kept ethnicity and race as two distinct variables, for both mitted by a person who does not know or is known by the victims). Our victims and perpetrator, rather than merging them into one. However, classification is slightly different from that of Yang and Olafsson (2011), our predictive model will capture the actual dependency between them. where the category Close to family includes other homicides in We have grouped race into three groups: White, Black (or African addition to IPV, which we have considered as a category by itself, American), and a third category, Native-Islander, which includes both due to its special social relevance and criminological interest, although Asian and Pacific Islander on the one hand, and Native American it only applies to the 12.3% of the perpetrators of our database. The and Alaska Native on the other (this grouping has been motivated rest of categories correspond to stranger (22.4%), family (25.9%) by the scarcity of individuals of each of these races in the database, and acquaintance (39.4%), the percentages corresponding to the which together represent 3.55% of the perpetrators, and 3.79% of the perpetrators for whom the relationship to the victims was known. victims). Bag: This auxiliary variable has been introduced as an essential tool Age: Regarding age, it has been discretized into ranges or intervals, to uniquely identify the cases (victims) of the same event, all of them according to a criterion of approximate equity between them, taking with the same perpetrator, assigning them a distinct number. There are into account that the frequency in the central intervals will naturally 15,862 different bags (referred to different incidents or perpetrators), be higher than in the extremes, and distinguishing the minority of legal with 4236 victims mass homicide victims killed in 858 incidents (av- age (18 years), but with some differences that we want to highlight erage of 4.9 victims/incident), and 32,033 multiple homicide victims between the authors and the victims. In both cases, ages 99 years killed in 15,004 incidents (average of 2.1 victims/incident). and over have been coded as ‘‘Unknown’’. The main difference is that we have also recoded as ‘‘Unknown’’ the ages of the perpetrators in 2.2. The probabilistic classifier case they are less than 10 in the original database. Although this is arbitrary, it has been motivated by the fact that some of the ages were As mentioned in the Introduction, we will use a probabilistic classi- suspiciously low, being 0 years in many cases, which suggests an error fier to predict each of the output variables for each of the victims that in data collection. Furthermore, due to the scarcity of cases in those are part of the same incident (bag), and apply a combination rule to ages, for the aggressor we have merged into a single category the obtain the joint prediction for the bag. Of the different methodologies intervals from 50 to 64 years, and 65 or more, which remain as different used in Machine Learning to learn classifiers from a database of solved intervals for the age of the victims. It is logical that the perpetrator’s cases, we consider probabilistic classifiers because they not only predict age moves towards the central age ranges with respect to that of the the label, but also estimate the probability distribution over the set of victims, having low frequencies in the lower range (10–17, 6.7%) and possible labels. As such a classifier we build a Bayesian Network with in the upper range (≥50, 8.6%). all the model variables except Bag, both input variables corresponding to the event: Year, Month, State and Weapon, and to the charac- Count: Despite their names, Perpetrator.Count and Victim. teristics of the victim: Sex, Age, Race, Ethnicity and Count, Count are variables with very different meanings in the FBI database. as well as the output, which are the characteristics of the murderer: Indeed, whereas Victim.Count is, for any victim, the total number Sex, Age, Race, Ethnicity, Count and Relationship (see of victims of the same incident discounting this victim, and has been Table 1). recoded as 1, 2 (multiple homicide) and >2 (mass homicide), for any The probabilistic relationships between the variables that affect a perpetrator, Perpetrator.Count appears to be, although it is not phenomenon – in this case, a multi-victim homicide – can be rep- perfectly clear from the database documentation provided by the FBI, resented graphically by BNs (Koller & Friedman, 2009), which are the number of previous incidents in which the perpetrator had been graphical models utilized for probabilistic inference. A standard BN is a model that depicts the joint probability distribution 𝑃 of a set 2 of random variables (discrete or categorical), which are represented Nancy L. Fisher (1996). Cultural and Ethnic Diversity: A Guide for graphically as nodes in a directed acyclic graph (DAG). The Markov Genetics Professionals. Johns Hopkins University Press. p. 19. ISBN 978-0- condition asserts that each node in the DAG is independent of those 8018-5346-3. https://books.google.es/books?id=mqXlA7e4VN8C&pg=PA19& redir_esc=y#v=onepage&q&f=false. nodes that are not its descendants, provided that the values taken by its 3 https://www.census.gov/data/tables/2020/dec/2020-redistricting- parents are known. Directed arcs between nodes entail conditional (not supplementary-tables.html. necessarily causal) dependencies that are governed by this condition. 4 https://www.census.gov/acs/www/data/data-tables-and-tools/data- The updating of probabilities with the BN from a specific piece of profiles/. evidence is referred to as Bayesian Inference; we compute a posteriori 5 R. Delgado and H. Sánchez-Delgado Expert Systems With Applications 237 (2024) 121593 Fig. 2. Scheme of classification differences at the instance and bag levels. 𝐶1 , … , 𝐶𝑛 are the 𝑛 different classifiers ensembled at the instance level, while 𝐶 is the only one classifier used at the bag level, whose predictions for the 𝑛 instances are ensembled. probabilities from the evidences and the a priori probabilities. Accord- 2. Ensemble Average EA: from the perspective of predictive power, it ing to the Maximum A Posteriori (MAP) criterion, the prediction of a is the most effective soft voting rule, choosing the label with the query variable given an evidence is its instantiation with the highest highest average of the predicted probabilities that the classifier a posteriori probability, and this probability is referred to as the confi- assigns for each instance of the bag. dence level (CL) of the prediction. We employ the Maximum Likelihood 3. Confidence Level based Majority Vote CL-MV: a semi-hard voting Estimation (MLE) approach to estimate the parameters, which are the rule halfway between MV and EA. Each label has a counter conditional probabilities of any variable to its parents, and the marginal that sums the predicted probability assigned to that class by the probabilities of the root nodes, which are the nodes without parents in classifier for all the instances in the bag for which that label is the DAG. With the restriction that no directed arcs are allowed from the predicted one (and 0 otherwise). CL-MV is just going with any input variable to any output variables, we learn the structure of the the prediction for the label that maximizes the counter. If 𝑛 is BN using the Hill-Climbing Score-Based Structure Learning algorithm, odd and the label predicted for each instance is right with a high which provides a pseudo-optimal DAG that (locally) maximizes the enough probability, this combination strategy performs better in Bayesian Information Criterion (BIC) score. For this we use the hc the binary classification setting (𝑟 = 2) when compared to MV, function from the R package bnlearn by Scutari (2010). Learning according to Delgado (2022), where in a sensitivity analysis, it and prediction algorithms have been implemented in the R language (R has also been compared with EA and found to be significantly Core Team, 2013), with the help of bnlearn. more resilient to probability estimation error. 2.3. Ensembling for the MI learning with the unanimity MI assumption Note that in the context of the unanimity MI assumption, it makes no sense to use trainable combination schemes that have additional parameters needing to be trained, such as those that give different Ensemble of classifiers (also known as ‘‘combined classifiers’’) is a weights to different predictions. This is due, in particular, to the fact method that has been widely applied in classification learning to create a new classifier by merging a set of base classifiers in accordance with that we do not know a priori what the number of instances forming a a rule or scheme. bag is, nor do we have a clear criteria to prioritize some instances over others, which would serve to assign them different weights. The algorithm that we propose here follows the same line, but instead of combining, given an instance, the predictions that different classifiers provide for it, what it does is use a single classifier and 2.4. Validation procedure combine the predictions that it provides for the different instances that make up each of the bags, with which the predictions at the bag level We choose to perform a 𝐾-fold cross-validation procedure with 𝐾 = are obtained, as schematically represented in Fig. 2. In fact, we will 10 (see Bishop) to validate and compare the different predictive models. use an ensemble algorithm to combine the predictions given by the In this way, we obtain a confusion matrix for any model and any classifier for the different instances that form a bag (the victims of the same incident/perpetrator), for any target variable. To do this, we 𝑘 = 1, … , 𝐾, and from them we can calculate the following measures must choose the combination scheme that we will apply. We consider of performance: accuracy and Matthews Correlation Coefficient (MCC), three different rules (more details can be found in Appendix C, see introduced in the binary case by Matthews (1975), and extended to the also Delgado (2022)): multi-class setting by Gorodkin (2004), for all the output variables. The Mean Absolute Error (MAE) (see Cardoso and Sousa (2011)) 1. Majority Vote MV: the most well-known, straightforward, and is applied for ordinal classification when Perpetrator.Age is pre- widely used hard voting rule, which selects the label with the dicted, which is a categorical variable of a quantitative nature that has highest number of votes from the predicted for each instance of been discretized into 5 intervals: [10, 18), [18, 25), [25, 35), [35, 50), ≥ the bag. 50 . 6 R. Delgado and H. Sánchez-Delgado Expert Systems With Applications 237 (2024) 121593 Fig. 3. DAG of the Bayesian Network used as classifier  for the prediction at the instance level, before the use of the combination schemes for the unanimity MI assumption. In blue the nodes corresponding to the output (perpetrator) variables. 3. Results Table 2 (Normalized to sum up 100) Freeman’s degree of centrality and measure of basic standard betweenness for the variables in the model. In bold, the three most influential 3.1. The Bayesian network variables and their values of centrality and betweenness. Variable Freeman’s centrality (%) Betweenness (%) In Fig. 3 we show the relationships between the characteristics of State 4.16667 0 the victim and the crime, on the one hand, and the perpetrator, on the Year 2.08333 0 other, captured by the DAG of the BN learned from the entire database, Month 0 0 at the instance level. Victim.Sex 4.16667 0 Victim.Race 6.250000 3.24074 We can use the arc.strength function from the bnlearn pack- Victim.Ethnicity 10.416667 22.22222 age to obtain a measure of the strength of the probabilistic relationships Victim.Age 8.33333 13.88889 expressed by the directed arcs of the BN, and obtain that all arcs have Victim.Count 4.16667 0 a very significant value of strength. We can establish which variables Weapon 4.16667 5.55556 Perpetrator.Sex 4.16667 0 play the main role in the model using measures of centrality and/or Perpetrator.Race 12.500000 24.07407 betweenness taken from the area of Network Analysis applied to the Perpetrator.Ethnicity 6.250000 3.24074 DAG in Fig. 3. Perpetrator.Age 8.33333 6.94444 Graph’s important (influential) nodes are identified using centrality Perpetrator.Count 6.350000 0 indicators in the fields of Graph Theory and Network Analysis, where Relationship 18.750000 20.83333 ‘‘importance’’ here is defined as a measure of node’s contribution to the cohesiveness of the network. We adopt the degree of centrality of Freeman (Freeman, 1977) and the basic standard betweenness measure the different combination rules for the ensemble used for the unanimity by Borgatti and Everett (2006) as indicators. Their values, which are MI assumption (CL-MV, MV and EA). The corresponding boxplots are calculated for each variable, are provided in Table 2. found in Figs. B.4–B.5. From a descriptive point of view, we can observe The variables highlighted in Table 2: Relationship, Victim. a clear advantage for the CL-MV ensemble combination scheme with all Ethnicity and Perpetrator.Race, followed by Victim.Age the metrics: accuracy, MAE and MCC. and Perpetrator.Age, act as gateways, and the arcs connecting The question now is whether these differences observed at a descrip- them as bridges through which information flows from one set of tive level are statistically significant or not. To figure it out we need variables in the model to another. to perform some statistical hypothesis testing. In this sense, Table 4 reports whether any of the combination schemes, the one registered 3.2. Predicting at the bag level in the row, is statistically significantly better (𝑝-value < 0.10) than the one registered in the column, according to the different metrics In Table 3 we record the average of the 𝐾 = 10 values obtained by used in the experiment. We use the Holm-Bonferroni correction for 𝐾-fold cross-validation for any of the metrics, the output variables and multiple comparisons to avoid detecting significant differences that are 7 R. Delgado and H. Sánchez-Delgado Expert Systems With Applications 237 (2024) 121593 Table 3 To evaluate the impact of various incident or victim factors on the Mean values of the metrics obtained by 𝐾-fold cross-validation for any of the output forecasted profile of the unknown offender, we have thus decided, in variables, using the different combination rules for the ensemble used in the unanimity MI assumption (CL-MV, MV and EA). By row, we indicate the order from best (1) to light of these results, to employ the final predictive model built using worst (3). the CL-MV combination scheme from the complete database. Mean accuracy An example of application CL-MV MV EA Just to give an example, imagining one particular incident out of Race 0.6161285 (1) 0.5476253 (3) 0.5551590 (2) the myriad of them that could occur in reality, let us consider that Ethnicity 0.5931035 (2) 0.5391634 (3) 0.6120940 (1) the incident is a mass homicide with 4 non-Hispanic victims, all under Age 0.3781464 (1) 0.3596979 (2) 0.2307187 (3) Count 0.5851616 (1) 0.5622917 (3) 0.5790462 (2) 10 years of age, a male (victim V2) and three females, who were killed Relationship 0.4398548 (2) 0.4189362 (3) 0.4434323 (1) with a firearm (State, Year and Month have not been included as Mean MAE input information about the incident). We carry out two sensitivity studies in this example, as an illustration of the potentiality of the CL-MV MV EA model to generate knowledge. Age 448.2 (1) 460.5 (2) 634.1 (3) Mean MCC Sensitivity study 1. Table 5 shows the predicted characteristics for CL-MV MV EA the perpetrator obtained with our predictive model, using CL-MV, with their associated confidence level, if the race of the victims were Race 0.3089283 (1) 0.21792450 (3) 0.27643789 (2) Ethnicity 0.2167372 (2) 0.0451777 (3) 0.25091453 (1) changed, one by one, from 4 White to 4 Black. Age 0.1343040 (1) 0.10409507 (2) 0.01568847 (3) We can observe how the predicted race for the perpetrator moves Count 0.1797634 (2) 0.13375927 (3) 0.18797883(1) accordingly from White to Black, how the predicted age decreases and Relationship 0.2286003 (1) 0.19496783 (3) 0.22395685 (2) how the predicted relationship goes from ‘‘family’’ to ‘‘acquaintance’’ (the sex of the perpetrator is always predicted to be male, and so has Table 4 been omitted, and the same goes for the variable Count, which is p-values for the comparison of the mean values of the metrics used, with the Holm- always predicted to be 0, and the variable Ethnicity, predicted to Bonferroni correction for multiple comparisons. Alternative hypothesis: the combination be non-Hispanic). Table 6 shows the confidence of the individual-level scheme of the row is better than that of the column. predictions for any of the incident victims, based on their race. Accuracy CL-MV MV EA Row > Column Algorithm 2 (Appendix C) with the CL-MV combination rule uses the information in Table 6 to derive the predicted perpetrator profile for the Race: 0.0017** Race: 0.0063** CL-MV Age: 0.016* Age: 2.5 × 10−7 *** entire incident that is reported in Table 5. For example, in Setting 1.1, Relationship: 0.066**** with respect to Relationship, three of the victims (V1, V3 and V4) MV Age: 1.7 × 10−6 *** predict Family with the same confidence level 0.4127566, while the EA Relationship: 0.079**** fourth predicts Stranger with 0.383445. Using Algorithm 2 with CL- MAE CL-MV MV EA MV to obtain the predicted label for the incident with the four victims, Row < Column we compare the sum of the confidence levels assigned to Family by CL-MV Age: 0.024* Age: 5.2 × 10−9 *** the victims: 3 × 0.4127566 = 1.2382698, with the corresponding sum MV Age: 1.4 × 10−7 *** assigned to Stranger: 0.383445, resulting in the prediction being MCC CL-MV MV EA Family, where 1.2382698/(1.2382698 + 0.383445) = 0.7635558 is Row > Column its confidence level, as recorded in Table 5. Race: 0.012* CL-MV Age: 0.00574** Age: 9.9 × 10−6 *** Sensitivity study 2. Another possibility would be to study the evolu- Relationship: 0.031* tion of the predicted profile of the perpetrator in the example, when MV Age: 0.00023*** the race of the four victims is White but their ethnicity changes, one by one, from non-Hispanic to Hispanic, as shown in Table 7. Now the * Only significant p-values are recorded: at 5%. predicted race of the perpetrator is always White, and then omitted. ** Only significant p-values are recorded: at 1%. Table 8 shows, analogously to Table 6, the confidence of the predictions *** Only significant p-values are recorded: 1h. **** Denotes a slight significance, at 10%. at the individual level for any of the victims of the incident, based on their ethnicity. In the sensitivity study 2, to predict Relationship in Setting 2.3 (Table 7), for example, we observe in Table 8 that V1 and V2 not truly significant, using the pairwise Wilcoxon signed-rank test or predict Stranger with confidence levels 0.2959600 and 0.4218292, the Student’s t-test to compare pairs of samples corresponding to the respectively, while the rest of the victims predict Family, with con- same run, as appropriate according to the Shapiro–Wilk normality test, fidence 0.4127566 each. To use CL-MV in Algorithm 2 (Appendix C), which has been performed previously. we compare the sum of the confidence levels assigned to Stranger, All in all, the CL-MV combination scheme is clearly the best of those which is 0.7177892, and the sum for Family, 0.8255132. The re- considered, since none of the others surpassed it with any of the target sulting prediction is the one corresponding to the largest, which is variables. More specifically, from Table 4 we can state that there are Family, with confidence level 0.8255132/(0.8255132 + 0.7177892) no significant results for the variables Count or Ethnicity, and that = 0.5349005. for the rest of output variables: We observe from Tables 5, 7 that the predicted race/ethnicity of the perpetrator matches that of the majority of the victims, with the confi- • Accuracy: CL-MV outperforms the other combination schemes for dence level increasing as the number of victims of that race/ethnicity the variables Race and Age, and outperforms MV for the variable does the same. The predicted age of the perpetrator tends to be lower if Relationship. the victims are Black than White, but if they are White, it is not affected • MAE (for the variable Age): CL-MV outperforms both MV and EA. by 𝑒thnicity. The predicted relationship changes from ‘‘family’’ to ‘‘ac- • MCC: CL-MV combination scheme shows to be better than MV for quaintance’’ if the victims’ majority race is Black, and to ‘‘stranger’’ if Race and Relationship, and better than both MV and EA for their ethnicity is Hispanic (of race White). We can see that depending Age. on the proportion of races and ethnicities in the group, this mass 8 R. Delgado and H. Sánchez-Delgado Expert Systems With Applications 237 (2024) 121593 Table 5 Sensitivity study 1: Example of predicted perpetrator profile using CL-MV, for an incident with four non- Hispanic victims (mass homicide), <10 years old, when the weapon is a firearm, as the race of the victims varies, one by one, from White to Black. Mass homicide. Sensitivity study 1 Victim Predicted perpetrator profile (incident) Race Race Age Relationship V1. White, V2. White White [25,35) Family Setting 1.1 V3. White, V4. White 1 1 0.7635558 V1. Black, V2. White White [25,35) Family Setting 1.2 V3. White, V4. White 0.7527732 0.7243044 0.5589648 V1. Black, V2. Black White [18,25) Family Setting 1.3 V3. White, V4. White 0.5123915 0.5520587 0.5737075 V1. Black, V2. Black Black [18,25) Acquaintance Setting 1.4 V3. Black, V4. White 0.7416863 0.7836611 0.4140512 V1. Black, V2. Black Black [18,25) Acquaintance Setting 1.5 V3. Black, V4. Black 1 1 0.699362 Table 6 Sensitivity study 1: Individual predictions for any of the four victims in the example (mass homicide), based on the victim’s race. In parentheses, the corresponding confidence level of the individual prediction. Mass homicide. Sensitivity study 1 Victim Race Individual predicted perpetrator profile for each victim Race Age Relationship White White (0.8932712) [25,35) (0.2883954) Family (0.4127566) V1 Black Black (0.8646841) [18,25) (0.3338198) Acquaintance (0.2679026) White White (0.8463076) [25,35) (0.3002172) Stranger (0.383445) V2 Black Black (0.8354481) [18,25) (0.3770374) Stranger (0.3454937) White White (0.8932712) [25,35) (0.2883954) Family (0.4127566) V3 Black Black (0.8646841) [18,25) (0.3338198) Acquaintance (0.2679026) White White (0.8932712) [25,35) (0.2883954) Family (0.4127566) V4 Black Black (0.8646841) [18,25) (0.3338198) Acquaintance (0.2679026) Table 7 Sensitivity study 2: Analogous to the example in Table 5 with four White victims (mass homicide) as their ethnicities vary, one by one, from non-Hispanic to Hispanic. Mass homicide. Sensitivity study 2 Victim Predicted perpetrator profile (incident) Ethnicity Ethnicity Age Relationship V1. non-Hispanic, V2. non-Hispanic Non-Hispanic [25,35) Family Setting 2.1 V3. non-Hispanic, V4. non-Hispanic 1 1 0.7635558 V1. Hispanic, V2. non-Hispanic Non-Hispanic [25,35) Family Setting 2.2 V3. non-Hispanic, V4. non-Hispanic 0.7807124 1 0.5485436 V1. Hispanic, V2. Hispanic Non-Hispanic [25,35) Family Setting 2.3 V3. non-Hispanic, V4. non-Hispanic 0.543309 1 0.5349005 V1. Hispanic, V2. Hispanic Hispanic [25,35) Stranger Setting 2.4 V3. Hispanic, V4. non-Hispanic 0.7158239 1 0.710652 V1. Hispanic, V2. Hispanic Hispanic [25,35) Stranger Setting 2.5 V3. Hispanic, V4. Hispanic 1 1 1 Table 8 Sensitivity study 2: Individual predictions for any of the four victims in the example (mass homicide), based on the ethnicity of the victim. In parentheses, the corresponding confidence level of the individual prediction. Mass homicide. Sensitivity study 2 Victim Etnicity Individual predicted perpetrator profile for each victim Etnicity Age Relationship Non-Hispanic Non-Hispanic (0.9290091) [25,35) (0.2883954) Family (0.4127566) V1 Hispanic Hispanic (0.7783224) [25,35) (0.2965019) Stranger (0.2959600) Non-Hispanic Non-Hispanic (0.9129823) [25,35) (0.3002172) Stranger (0.383445) V2 Hispanic Hispanic (0.7834779) [25,35) (0.3035398) Stranger (0.4218292) Non-Hispanic Non-Hispanic (0.9290091) [25,35) (0.2883954) Family (0.4127566) V3 Hispanic Hispanic (0.7783224) [25,35) (0.2965019) Stranger (0.2959600) Non-Hispanic Non-Hispanic (0.9290091) [25,35) (0.2883954) Family (0.4127566) V4 Hispanic Hispanic (0.7783224) [25,35) (0.2965019) Stranger (0.2959600) 9 R. Delgado and H. Sánchez-Delgado Expert Systems With Applications 237 (2024) 121593 Table 9 Comparing Tables 9–10 with Tables 5–7 we observe a higher pre- Example of predicted perpetrator profile using CL-MV, for an incident with three non- dicted age in the case of the multiple homicides, and that the predicted Hispanic victims (multiple homicide), <10 years old, when the weapon is a firearm, as the race of the victims varies, one by one, from White to Black. relationship between victims and perpetrator is always ‘‘family’’, re- Multiple homicide. Sensitivity study 1 gardless of the number of White/Black and non-Hispanic/Hispanic victims, unlike mass homicides, for which the prediction was ‘‘acquain- Victim Predicted perpetrator profile tance’’ when the number of Black victims is majority, or ‘‘stranger’’, Race Race Age Relationship when the majority of the victims were Hispanic. One of the two V1. White, V2. White White [35,50) Family scenarios mentioned above, scenario 1, could be used to frame this V3. White 1 0.6741045 0.7402884 multiple homicide. V1. Black, V2. White White [25,35) Family V3. White 0.6670902 0.6685955 0.7046505 4. Discussion and conclusions V1. Black, V2. Black Black [18,25) Family V3. White 0.655939 0.3615387 0.6756024 In this study, we describe a knowledge-based expert system de- V1. Black, V2. Black Black [25,35) Family signed to assist in profiling homicide offenders, that is, to forecast their V3. Black 1 0.6440647 0.6264575 attributes (sex, age, race, ethnicity, count and relationship to victims), when there are multiple victims in a single incident. This profile is Table 10 predicted based on the victim’s characteristics (age, sex, race, ethnicity) Analogous to the example in Table 9 with three White victims (multiple homicide) as and the incident’s details (state, year, month and weapon), and it can be their ethnicity varies, one by one, from non-Hispanic to Hispanic. crucial early in the investigation so that law enforcement investigators Multiple homicide. Sensitivity study 2 can more accurately identify the unknown perpetrator. Rather than Victim Predicted perpetrator profile replace criminologists’ capacity to profile suspects, smart software is Ethnicity Ethnicity Age Relationship intended to assist them in their goal of improving their effectiveness at V1. non-Hispanic, V2. non-Hispanic Non-Hispanic [35,50) Family preventing crime. V3. non-Hispanic 1 0.6741045 0.7402884 To address the issue of predicting the profile of the perpetrator V1. Hispanic, V2. non-Hispanic Non-Hispanic [25,35) Family of a multi-victim homicide, we adopt the MI learning paradigm. By V3. non-Hispanic 0.7057269 0.6579581 0.6998363 definition, neither the pure instance-level approach, where instances V1. Hispanic, V2. Hispanic Hispanic [25,35) Family are not considered as part of a bag for the classification task, nor the V3. non-Hispanic, 0.6245493 0.6593685 0.6809751 construction of a classifier exclusively at the bag-level without taking V1. Hispanic, V2. Hispanic Hispanic [25,35) Family instances into account, can be used to solve this problem. Rather, the V3. Hispanic 1 1 0.6239864 classifier, which in our case is a BN, must be built at the instance-level, and the labels and probabilities that have been assigned to each in- stance of the same bag must then be somehow combined to produce the homicide could fit into the two scenarios described in the Discussion tag for the bag as a whole. And this is where the combination schemes section in Fowler et al. (2021): homicides within the family, with a of the ensemble of classifiers methodology come into play, handling a large number of minor victims (scenario 1) and homicides in public MI problem that pure instance-level classifiers cannot address on their places such as schools, with most of the victims being young people own. and children, strangers or acquaintance to the perpetrator (scenario 2). The hybrid strategy known as CL-MV (semi-hard voting) has the best To quantify the association between the ethnic origin of the victims overall performance, according to our methodological comparison of and the perpetrator, for example, we can calculate from Table 7 the the three combination rules: the soft-voting majority vote MV, the hard- Odds Ratio (OR) in favor of the non-Hispanic prediction for the per- voting ensemble average EA, and the hybrid approach CL-MV. Indeed, petrator’s ethnic origin, when the number of non-Hispanic victims is CL-MV performs better than the other for all the variables for which reduced from 3 (Setting 2.2) to 2 (Setting 2.3): statistically significant differences are detected, with the performance metrics accuracy and MCC, in general, and MAE for the ordinal dis- 0.543309∕(1 − 0.543309) cretized variable Age. We also delved into the interpretability of the OR = = 0.3341546 ≈ 0.33 0.7807124∕(1 − 0.7807124) model, based on the DAG of the Bayesian Network from which it has that being less than 1 means that the reduction in the number of non- been built. The most ‘‘influential’’ variables, as determined by several Hispanic victims reduces the odds in favor of the non-Hispanic ethnicity measures of centrality and betweenness, were found to be: Relation- of the perpetrator. But it goes further, since OR quantifies this reduction ship, Victim.Ethnicity and Perpetrator.Race. The expert in odds, and tells us that it happens to be a third, approximately. Just system not only enables profiling but also generates knowledge that is one more example: from Table 5, the OR in favor of a relationship of provided objectively and quantitatively. For instance, we can infer from ‘‘acquaintance’’ when the number of Black victims goes from 3 (Setting the example presented in Tables 5–7 that the predicted relationship is 1.4) to 4 (Setting 1.5) is: more likely to be ‘‘acquaintance’’ if the victims are primarily Black, or ‘‘stranger’’ if they are predominantly Hispanic, as opposed to ‘‘family’’, 0.699362∕(1 − 0.699362) OR = = 3.29293 ≈ 3.3 if the victims are White and non-Hispanic. The Odds Ratio, which is 0.4140512∕(1 − 0.4140512) obtained from the probabilities estimated with the model, can be used which means that the increase in the number of Black victims raises to quickly determine the strength of the association between the input the odds in favor of ‘‘acquaintance’’, and quantifies this increase (more and the output variables. than triple). We believe that in some settings, such as multi-victim homicides, What would have happened if instead of 4 victims, the incident the unanimity MI assumption is an effective approach. We have re- had only 3 and was therefore a multiple homicide? Analogous to what vealed that the CL-MV criterion, as a rule to combine the labels and we did in the mass homicide example, we now assume 3 non-Hispanic probabilities assigned to the instances in the same bag, has produced victims in the same incident, all under the age of 10, one male (victim encouraging results when applied to the FBI homicide database in the V2) and two females, who were killed with a firearm. Tables 9–10 show U.S. The predictive model may self-adapt to other police databases, the predicted perpetrator characteristics and associated confidence lev- from other countries, or to other sorts of crimes with multiple victims, els for the entire incident in two sensitivity studies, as the race or the such as assaults, robberies, kidnappings, etc. because it was built with ethnicity of the victims changes. a Machine Learning methodology that learns from the database from 10 R. Delgado and H. Sánchez-Delgado Expert Systems With Applications 237 (2024) 121593 which it feeds. Although the topic of predicting the profile of the Table A.11 Relative frequencies (%) of the characteristics of the victim, of the perpetrator, and of perpetrator in a multi-victim homicide is a particular case, we are the weapon, depending on whether the homicide is multiple or massive. Note that for certain that this work will have more useful applications. For instance, the perpetrator, the age ranges 50–64 and ≥65 have been merged into ≥50, and that predicting the qualities of a teacher or school based on the traits of the the minimum age is 10. It is remarkable, though not unexpected, that while comprising students, who make up a bag. Perhaps those of the creator of a piece of merely about 6% of the offenders, women constitute over 37% of the victims. art when determining its authorship, or those of a sports coach based Characteristics of Victim Perpetrator on the characteristics of the team members, . . . Multiple Mass Multiple Mass Ultimately, this research may be valuable because we developed Female 37.54% 46.29% 6.07% 5.28% Sex an expert system that could be implemented to make it easier for law Male 62.46% 53.71% 93.93% 94.72% enforcement officers to use it for practical applications, such as early <10 9.21% 21.28% prevention of criminal recidivism, since aiding in the elaboration of 10–17 8.58% 12.16% 6.66% 7.10% 18–24 21.33% 15.81% 29.52% 28.29% criminal profiling will facilitate to identify and apprehend the culprits, Age 25–34 23.23% 17.84% 31.32% 30.21% preventing them from committing another homicide. We concur with 35–49 19.94% 18.28% 24.02% 24.68% the sentiment expressed in the Discussion of Fowler et al. (2021) in 50–64 10.81% 9.31% 8.48% 9.72% that preventing a mass homicide, especially if it involved a shooting, ≥65 6.90% 5.32% could have positive effects on preventing further crime, due to its Native-Islander 3.64% 5.00% 3.56% 3.45% Race Black 32.49% 28.12% 38.19% 37.70% potential contagion effect for almost two weeks (see also Towers, White 63.87% 66.88% 58.25% 58.85% Gomez-Lievano, Khan, Mubayi, and Castillo-Chavez (2015)). At the Hispanic 27.51% 21.82% 26.75% 24.45% same time, it also appears to be a solid starting point for upcoming Ethnicity Non-Hispanic 72.49% 78.18% 73.25% 75.55% criminological inquiries. Comparing the forecasted traits of the offender 1 81.20% 0.00% before and after the implementation of a legal measure that might have Count 2 18.80% 0.00% an effect on homicides might be instructive, for example. (victim) >2 0.00% 100.00% The local data-driven machine learning methodology for determin- Count 0 76.12% 76.53% ing a perpetrator’s profile in a multi-victim homicide described in this (perpetrator) 1 23.88% 23.47% paper, seems to be a useful and promising tool with significant crimi- Acquaintance 39.38% 39.88% nological applicability insofar, as it can assist criminologists to create Family 25.87% 25.87% Relationship IPV 12.25% 12.90% incident-tailored profiles, and authorities to better manage available Stranger 22.50% 21.35% resources. Weapon Multiple Mass Weapon Multiple Mass Blunt Object 4.47% 4.58% Firearm 6.63% 5.95% Declaration of competing interest Drowning 0.32% 0.40% Handgun 54.95% 54.98% Drugs-Poisson 0.32% 0.00% Knife 11.28% 10.41% Explosives 0.29% 0.34% Long hung 15.60% 17.39% The authors declare that they have no known competing finan- Fall 0.01% 0.06% Strangulation 0.97% 0.57% cial interests or personal relationships that could have appeared to Fire 4.12% 4.52% Suffocation 1.01% 0.80% influence the work reported in this paper. Data availability Appendix C. The combination schemes Data will be made available on request. For the sake of completeness, the pseudo-code of the algorithms for the implementation of the combination schemes MV, EA and CL-MV Acknowledgments are below (see Delgado (2022)). To predict an output variable at the bag level for a bag composed of 𝑛 instances 𝐸1 , … , 𝐸𝑛 , first, we need to The authors would like to express their gratitude to the reviewers obtain the prediction for each of the instances by Algorithm 1, whose who, with their comments, helped them to make an improved final output feeds the others. version of their work. We confirm that the manuscript has been written, read, and approved by all named authors and that there are no other Algorithm 1 Instance prediction persons who satisfied the criteria for authorship but are not listed. We Input instances 𝐸1 , … , 𝐸𝑛 forming a bag, classifier , output variable further confirm that the order of authors listed in the manuscript has classes 𝑦1 , … , 𝑦𝑟 been approved by all of us. Output the predicted classes 𝑦∗𝑗 for any instance 𝐸𝑗 , and the probability distributions {𝑝𝑗𝑘 , 𝑘 = 1, … , 𝑟}, 𝑗 = 1, … , 𝑛 Funding 1: for 𝑗 in 1 ∶ 𝑛 do 2: for 𝑘 in 1 ∶ 𝑟 do R. Delgado is supported by Ministerio de Ciencia e Innovación, 3: compute 𝑝𝑗𝑘 (probability assigned by  to 𝑦𝑘 , given evidence 𝐸𝑗 ) Gobierno de España, Spain, project ref. PID2021-123733NB-I00 4: 𝑗𝑚𝑎𝑥 = arg max𝑘=1,…,𝑟 𝑝𝑗𝑘 5: 𝑦∗𝑗 = 𝑦𝑗𝑚𝑎𝑥 (predicted class for instance 𝐸𝑗 with the MAP criterion) Appendix A. Data set: table of relative frequencies (in %) return {𝑦∗𝑗 , 𝑗 = 1, … , 𝑛} and {𝑝𝑗𝑘 , 𝑘 = 1, … , 𝑟, 𝑗 = 1, … , 𝑛} See Table A.11 Note that Algorithm 1 enables the prediction of any output variable for each instance using the Maximum A Posteriori (MAP) criterion. Appendix B. Additional figures This involves assigning the class with the maximum probability. Sub- sequently, Algorithm 2 presented below facilitates the combination See Figs. B.4 and B.5 of predictions acquired for individual instances within a bag, using a 11 R. Delgado and H. Sánchez-Delgado Expert Systems With Applications 237 (2024) 121593 Fig. B.4. Boxplots for the accuracy (the higher the better) of the ensembles at the bag level, using the different combination schemes: CL-MV, MV, and EA, for each of the output variables, from the data obtained in the validation procedure, and boxplot of MAE for the ordinal output variable Perpetrator.Age (the smaller the better). Consistent with Table 4, CL-MV shows greater accuracy compared to MV across Race, Age and Relationship variables, and compared to EA across Race and Age. Notably, neither MV nor EA outperforms CL-MV across any output variables. Concerning Age, CL-MV exhibits significantly lower MAE than EA and slightly lower than MV. specific combination rule. where combination ∈ {MV, EA, CL-MV} MV designates the bag’s class based on the most frequently chosen class ∑ and function 𝑔𝑘 in step 3 of Algorithm 2 is 1𝑛 𝑛𝑗=1 𝑝𝑗𝑘 if combination among the instances. Meanwhile, CL-MV assigns the bag’s prediction = EA, and otherwise, by maximizing the sum of probabilities assigned to each class by the classifier, but only considering the instances, out of 𝑛, that opted for ⎧0 if 𝑦(𝑘) = ∅ that particular class. ⎪ 𝑔𝑘 = ⎨#𝑦(𝑘) if 𝑦(𝑘) ≠ ∅, for 𝚌𝚘𝚖𝚋𝚒𝚗𝚊𝚝𝚒𝚘𝚗 = MV ⎪∑ Remark. MV offers the benefit that once we know the prediction ⎩ 𝑗∈𝑦(𝑘) 𝑝𝑗𝑘 if 𝑦(𝑘) ≠ ∅, for 𝚌𝚘𝚖𝚋𝚒𝚗𝚊𝚝𝚒𝚘𝚗 = CL-MV. of any of the instances, we do not need to maintain any other in- In words, EA selects the predicted class for the bag by maximizing formation, which is advantageous from a computational perspective the average (or sum) of the probabilities assigned to each class by the and saves storage space. EA is more computationally and storage- classifier across all 𝑛 instances comprising the bag. On the other hand, demanding because it must save and use every probability distribution 12 R. Delgado and H. Sánchez-Delgado Expert Systems With Applications 237 (2024) 121593 Fig. B.5. Boxplots for the MCC of the ensembles at the bag level, with combination schemes: CL-MV, MV and EA, for the output variables. The higher the better. As indicated by Table 4, CL-MV demonstrates greater MCC than MV for Race, Age and Relationship, and also outperforms EA for Age. Notably, neither MV nor EA outperforms CL-MV in terms of MCC across the output variables. value assigned by the classifier to the class prediction for any instance. the classifier provides the correct class label for any of the instances. Halfway between them since only uses the maximum of the probability Accuracy for the different combination schemes is (see Delgado, 2022): distribution for any instance, is CL-MV. ( ) ∑𝑛 𝑛 𝓁 𝐴𝑐𝑐MV = 𝑝 (1 − 𝑝)𝑛−𝓁 , 𝓁=⌊𝑛∕2⌋+1 𝓁 Comparing the combination schemes through accuracy in the bi- ( ) ∑ 𝑛 𝑛 𝓁 nary case 𝐴𝑐𝑐CL-MV = 𝑝 (1 − 𝑝)𝑛−𝓁 , 𝓁=⌊(1−𝑝) 𝑛⌋+1 𝓁 Following Delgado (2022) we compare in the binary case the ac- 𝐴𝑐𝑐EA = 𝑝 (independently of 𝑛, by construction), curacy of the different combination rules we are considering, with an odd number of instances forming the bag, if the predictions given by where ⌊𝑥⌋ denotes the integer part or floor of 𝑥. In Delgado (2022) the the classifier for any of the instances forming a bag are right or wrong following result is stated and proved, and we reproduce it here adapted independently of each other. Let 𝑝 ∈ (0, 1) denote the probability that to our unanimity MI assumption context. 13 R. Delgado and H. Sánchez-Delgado Expert Systems With Applications 237 (2024) 121593 Fig. C.6. Scheme of the statement of Proposition 2: comparing the accuracy of the different combination schemes in the binary case (𝑟 = 2) with 𝑛 = 3, for 𝑝 ∈ (0, 1). Algorithm 2 Combination scheme (Algorithms 1–3 Delgado, 2022) References Input the output of Algorithm 1 Alpaydin, E., Cheplygina, V., Loog, M., & Tax, D. M. J. (2015). Single- vs. Output the predicted class 𝑦∗combination for the bag multiple-instance classification. Pattern Recognition, 48, 2831–2838. 1: for 𝑘 in 1 ∶ 𝑟 do Baumgartner, K. C., Ferrari, S., & Palermo, G. (2008). Constructing Bayesian Networks 2: 𝑦(𝑘) = {𝑗 = 1, … , 𝑛 ∶ 𝑦∗𝑗 = 𝑦𝑘 } (indices of instances with predicted for criminal profiling from limited data. Knowledge-Based Systems, 21, 563–572. class 𝑦𝑘 ) Baumgartner, K. C., Ferrari, S., & Salfati, C. G. (2005). Bayesian network modeling of offender behavior for criminal profiling. In Proceedings of the 44th IEEE conference 3: 𝑔𝑘 on decision and control (pp. 2702–2709). Seville, Spain: http://dx.doi.org/10.1109/ 4: 𝓁 = arg max𝑘=1,…,𝑟 𝑔𝑘 (the class that maximizes function 𝑔) CDC.2005.1582571. 5: 𝑦∗ = 𝑦𝓁 Bishop, C. M. Series: Information science and statistics, Pattern recognition and machine combination learning. NY: Springer New York, ISBN: 978-0-387-31073-2, Hardcover. Published: return 𝑦∗combination 17 August 2006. Borgatti, S. P., & Everett, M. G. (2006). A graph-theoretic perspective on centrality. Social Networks, 28, 466–484. Brahan, J. W., Lam, K. P., Chan, H., & Leung, W. (1998). AICAMS: Artificial intelligence, Proposition 1 (Proposition 2 Delgado, 2022). In the binary case and with crime analysis and management system. Knowledge-Based Systems, 355–361. Carbonneau, M. A., Cheplygina, V., Granger, E., & Gagnon, G. (2018). Multiple instance an odd number of instances giving rise to a bag, 𝑛, we have that learning: A survey of problem characteristics and applications. Pattern Recognition, 77, 329–353. ⎧If 𝑝 ≤ 𝑛−1 , then 𝐴𝑐𝑐CL-MV < 𝐴𝑐𝑐MV , ⎪ 𝑛−1 2 𝑛 𝑛+1 Cardoso, J., & Sousa, R. (2011). Measuring the performance of ordinal classifica- ⎨If 2 𝑛 < 𝑝 ≤ 2𝑛 , then 𝐴𝑐𝑐CL-MV = 𝐴𝑐𝑐MV , tion. International Journal of Pattern Recognition and Artifical Intelligence, 25(8), ⎪ 𝑛+1 1173–1195. ⎩If 𝑝 > 2 𝑛 , then 𝐴𝑐𝑐CL-MV > 𝐴𝑐𝑐MV . Delgado, R. (2022). A semi-hard voting combination scheme to ensemble multi-class probabilistic classifiers. Applied Intelligence, 52, 3653–3677, https://doi-org.are.uab. It can be seen (Condorcet Jury Theorem, 1785, as reproduced in cat/10.1007/s10489-021-02447-7. Shapley & Grofman, 1984) that Delgado, R., González, J. L., Sotoca, A., & Tibau, X. A. (2016). A Bayesian Network profiler for wildfire arsonists. In P. Pardalos, P. Conca, G. Giuffrida, & G. Nicosia (a) if 𝑝 > 0.5, lim𝑛→∞ 𝐴𝑐𝑐𝑀𝑉 = 1 and it is monotonically increasing, (Eds.), Lecture notes in computer science: vol. 10122, Machine learning, optimization (b) if 𝑝 = 0.5, 𝐴𝑐𝑐𝑀𝑉 = 0.5 = 𝐴𝑐𝑐EA for all 𝑛, and big data. MOD 2016 (pp. 379–390). Cham: Springer, http://dx.doi.org/10.1007/ 978-3-319-51469-7_31. (c) if 𝑝 < 0.5, lim𝑛→∞ 𝐴𝑐𝑐𝑀𝑉 = 0 and it is monotonically decreasing, Delgado, R., González, J. L., Sotoca, A., & Tibau, X. A. (2018). Archetype of wildfire arsonists: An approach by using Bayesian Networks. In Janusz Szmyt (Ed.), Forest that is, while for 𝑝 = 0.5 MV and EA have the same accuracy, we can fire. IntechOpen, http://dx.doi.org/10.5772/intechopen.72615. expect MV to show improvement over EA only if 𝑝 > 0.5. Dietterich, T., Lathrop, R., & Lozano-Pérez, T. (1997). Solving the multiple instance In the particular case 𝑛 = 3, to be more specific in a concrete problem with axis-parallel rectangles. Artificial Intelligence, 89, 31–71. Foulds, J., & Frank, E. (2010). A review of multi-instance learning assumptions. The example, we have that Knowledge Engineering Review, 25(1), 1–25. Fowler, K. A., Leavitt, R. A., Betz, C. J., Yuan, K., & Dahlberg, L. L. (2021). 𝐴𝑐𝑐MV = 3 𝑝2 (1 − 𝑝) + 𝑝3 , Examining differences between mass, multiple and single-victim homicides to inform prevention: findings from the National Violent Death Reporting System. ⎧𝑝 3 if 𝑝 ≤ 1 ⎪ 3 Injury Epidemiology, 8, 49, 15 pages. 1 2 𝐴𝑐𝑐CL-MV = ⎨3 𝑝2 (1 − 𝑝) + 𝑝3 if <𝑝≤ Fox, B. (2022). Chapter 18 - Offender profiling: a review of the research and state 3 3 ⎪ 2 2 3 2 of the field. In Paulo Barbosa Marques, & Mauro Paulino (Eds.), Police psychology ⎩3 𝑝 (1 − 𝑝) + 3 𝑝 (1 − 𝑝) + 𝑝 if 𝑝 ≥ 3 . (pp. 381–394). Academic Press, ISBN: 9780128165447, http://dx.doi.org/10.1016/ B978-0-12-816544-7.00018-8. The following result establishes the comparison between them, and Freeman, L. C. (1977). A set of measures of centrality based upon betweenness. shows that CL-MV is better (it has greater accuracy) than MV, and both Sociometry, 40, 35–41. are better than EA, when the probability that each instance of the bag Gorodkin, J. (2004). Comparing two k-category assignments by a k-category correlation coefficient. Computational Biology and Chemistry, 28(5–6), 367–374. gives the correct label, 𝑝, is large enough. Its proof is a simple exercise Gottschalk, P. (2006). Stages of knowledge management systems in police and is therefore omitted, while a schematic of the result can be seen in investigations. Knowledge-Based Systems, 19, 381–387. Fig. C.6. Graur, D. O., Maris, R. A., Potolea, R., Dinsoreanu, M., & Lemnaru, C. (2018). Complex localization in the multiple instance learning context. In A. Appice, & et al. (Eds.), LNAI: vol. 10785, NFMCP 2017 (pp. 93–106). Springer International Publishing AG. Proposition 2. In the binary case and with 𝑛 = 3, we have that Kocsis, R. N. (2006). Criminal profiling: Principles and practice. Totowa, NJ: Humana Press. Koller, D., & Friedman, N. (2009). Adaptive computation and machine learn- ⎧If 𝑝 ≤ 1 , then 𝐴𝑐𝑐CL-MV < 𝐴𝑐𝑐MV , ing, Probabilistic graphical models: Principles and techniques. MIT Press, ISBN: ⎪ 1 3 2 9780262013192. CL-MV vs. MV ⎨If 3 < 𝑝 ≤ 3 , then 𝐴𝑐𝑐CL-MV = 𝐴𝑐𝑐MV , ⎪ Küçükasci, E. S., & Baydogan, M. G. (2018). Bag encoding strategies in multiple instance 2 ⎩If 𝑝 > 3 , then 𝐴𝑐𝑐CL-MV > 𝐴𝑐𝑐MV . learning problems. Information Sciences, 467, 559–578. Matthews, B. W. (1975). Comparison of the predicted and observed secondary structure ⎧ 1 ⎪If 𝑝 < 2 , then 𝐴𝑐𝑐MV < 𝐴𝑐𝑐EA , of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Structure, 405(2), ⎪ 1 442–451. MV vs. EA ⎨If 𝑝 = 2 , then 𝐴𝑐𝑐MV = 𝐴𝑐𝑐EA , Newburn, T. (2013). Criminology (2nd ed.). Oxon/New York: Routledge. ⎪ Palermo, G., & Kocsis, R. N. (2004). In Charles C. Thomas (Ed.), Offender profiling: An ⎪If 𝑝 > 12 , then 𝐴𝑐𝑐MV > 𝐴𝑐𝑐EA . introduction to the sociopsychological analysis of violent crime. Springfield, IL. ⎩ 14 R. Delgado and H. Sánchez-Delgado Expert Systems With Applications 237 (2024) 121593 Pecino-Latorre, M. M., Pérez-Fuentes, M. C., & Patró-Hernández, R. M. (2019). Homicide Sutton, M. R., & Keatley, D. (2021). Cooling-off periods and serial homicide: A profiles based on crime scene and victim characteristics. International Journal of case study approach to analysing behaviour between murders. Forensic Science Environmental Research and Public Health, 16, 3629. http://dx.doi.org/10.3390/ International: Mind and Law, 2, Article 100066. ijerph16193629, 13 pages. Towers, S., Gomez-Lievano, A., Khan, M., Mubayi, A., & Castillo-Chavez, C. (2015). R Core Team (2013). R: A language and environment for statistical computing. Vienna, Contagion in mass killings and school shootings. PLoS ONE, 10(7), Article Austria: R Foundation for Statistical Computing, http://www.R-project.org/. e0117259. Scutari, M. (2010). Learning Bayesian Networks with the bnlearn R Package. Journal Turvey, B. E. (2002). Criminal profiling: An introduction to behavioral evidence analysis of Statistical Software, 35(3), 1–22, http://www.jstatsoft.org/v35/i03/. (2nd ed.). Academic Press. Shapley, L., & Grofman, B. (1984). Optimizing group judgement accuracy in the Yang, R., & Olafsson, S. (2011). Classification for predicting offender affiliation with presence of interdependencies. Public Choice, 43, 329–343. murder victims. Expert Systems with Applications, 38, 13518–13526. Strano, M. (2004). A neural network applied to criminal psychological profiling: an Italian initiative. International Journal of Offender Therapy and Comparative Criminology, 48(4), 495–503. 15