Planetary and Space Science 238 (2023) 105802 Contents lists available at ScienceDirect Planetary and Space Science journal homepage: www.elsevier.com/locate/pss Deep machine learning for meteor monitoring: Advances with transfer learning and gradient-weighted class activation mapping Eloy Peña-Asensio a,b ,∗, Josep M. Trigo-Rodríguez b,c , Pau Grèbol-Tomàs b , David Regordosa-Avellana d , Albert Rimola a a Departament de Química, Universitat Autònoma de Barcelona, Carrer dels Til.lers, Bellaterra, 08193, Catalonia, Spain b Institut de Ciències de l’Espai (ICE, CSIC), Campus UAB, C/ de Can Magrans S/N, Cerdanyola del Vallès, 08193, Catalonia, Spain c Institut d’Estudis Espacials de Catalunya (IEEC), Carrer del Gran Capità, 2, Barcelona, 08034, Catalonia, Spain d Spanish Meteor Network (SPMN, CSIC), Campus UAB, C/ de Can Magrans S/N, Cerdanyola del Vallès, 08193, Catalonia, Spain ARTICLE INFO ABSTRACT Keywords: In recent decades, the use of optical detection systems for meteor studies has increased dramatically, resulting Meteorites in huge amounts of data being analyzed. Automated meteor detection tools are essential for studying the Meteors continuous meteoroid incoming flux, recovering fresh meteorites, and achieving a better understanding of Meteoroids our Solar System. Concerning meteor detection, distinguishing false positives between meteor and non-meteor Machine learning images has traditionally been performed by hand, which is significantly time-consuming. To address this issue, Convolutional neural networks Transfer learning we developed a fully automated pipeline that uses Convolutional Neural Networks (CNNs) to classify candidate meteor detections. Our new method is able to detect meteors even in images that contain static elements such as clouds, the Moon, and buildings. To accurately locate the meteor within each frame, we employ the Gradient-weighted Class Activation Mapping (Grad-CAM) technique. This method facilitates the identification of the region of interest by multiplying the activations from the last convolutional layer with the average of the gradients across the feature map of that layer. By combining these findings with the activation map derived from the first convolutional layer, we effectively pinpoint the most probable pixel location of the meteor. We trained and evaluated our model on a large dataset collected by the Spanish Meteor Network (SPMN) and achieved a precision of 98%. Our new methodology presented here has the potential to reduce the workload of meteor scientists and station operators and improve the accuracy of meteor tracking and classification. 1. Introduction spectra (Hughes, 1978; Koschny et al., 2017; Subasinghe et al., 2017; Drolshagen et al., 2021). Meteors, popularly known as shooting stars, particularly the most Traditionally, two classes of meteors are considered: those that are luminous ones called fireballs or bolides, are spectacular physical pro- expected to occur in a specific period of the year because they are cesses that have fascinated mankind for centuries (Trigo-Rodríguez, associated with meteoroid streams, and those that are sporadic and 2022). These dazzling streaks of light occur when a meteoroid enters have no discernible periodic pattern (Wiegert and Brown, 2004; Jopek the Earth’s atmosphere at hypersonic velocity, causing intense heating and Williams, 2013; Dumitru et al., 2017; Jenniskens, 2017; Vaubaillon through repeated collisions with air molecules (Ceplecha et al., 1998; et al., 2019; Peña-Asensio et al., 2022, 2023). Although showers exhibit Silber et al., 2018; Trigo-Rodríguez, 2019). Meteors are formed due regular activity, the sporadic meteors require constant sky monitoring to the extreme heat produced by the interaction with the gaseous to quantify the meteoroid flux and properties of the different sources environment and the rising atmospheric pressure which causes the (Trigo-Rodríguez and Blum, 2022). meteoroid to undergo rapid vaporization, a process known as ablation. Meteors can provide valuable information about the composition, This ablation leads to the formation of a luminous trail composed dynamics, and origin of our Solar System (Koschny et al., 2019). By of ionized gas and fragmented debris, which can be observed and recorded from the ground using optical devices. Current digital video analyzing the physical and chemical properties of meteorites, which imagery provides a sequential recording useful to obtain complete are pieces of meteoroids that survive their journey through the Earth’s light curves, high temporal and spatial resolution measurements, and atmosphere and land on the surface, scientists can gain insight into ∗ Corresponding author at: Departament de Química, Universitat Autònoma de Barcelona, Carrer dels Til.lers, Bellaterra, 08193, Catalonia, Spain. E-mail addresses: eloy.pena@uab.cat, eloy.peas@gmail.com (E. Peña-Asensio). https://doi.org/10.1016/j.pss.2023.105802 Received 24 May 2023; Received in revised form 4 October 2023; Accepted 25 October 2023 Available online 1 November 2023 0032-0633/© 2023 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). E. Peña-Asensio et al. Planetary and Space Science 238 (2023) 105802 the formation and evolution of comets and asteroids, as they provide Learning (ML) approaches can effectively enhance the accuracy and information about the age of the Solar System, the composition of the efficiency of automated pipelines, given the massive volume of data early solar nebula, and the processes that led to the formation of the generated by meteor networks. We present a fully automated pipeline planets (Bottke et al., 2002; Lauretta and McSween, 2006). leveraging Convolutional Neural Networks (CNNs) for meteor detection The growing interest in meteoritics has driven an increase in the and tracking using transfer learning and the Gradient-weighted Class number of video meteor detection networks around the world (Ce- Activation Mapping (Grad-CAM) technique. plecha, 1987; Ceplecha et al., 1998; Koten et al., 2019; Colas et al., 2020). Comprised of strategically placed stations equipped with cam- 2. Artificial intelligence for meteor detection eras and other sensors, these networks are designed to monitor atmo- spheric volumes with clear views of the sky, aiming to maximize the Advances in computer technology and hardware performance have number of recorded meteors within common observing fields. A note- fueled the remarkable progress of ML, particularly artificial neural worthy trend in recent years has been the rise of pro-am collaborations networks with multiple layers, which are classified as deep learning. in this field, involving professional scientists, amateur astronomers, Neural networks have become increasingly popular in various do- and citizen scientists working together to collect valuable data. These mains due to their exceptional performance in image classification and collaborations have significantly expanded the reach of meteor net- recognition. Among them, CNNs have gained popularity for their fault works, enabling the recording of events from diverse locations and tolerance and self-learning capabilities through multi-layer feedforward perspectives. networks with a convoluted structure (Gu et al., 2018). They can However, with the increasing number of detection stations, the handle complex environments and unclear background problems with accumulation of video and image data has also surged. Consequently, significantly better generalization ability compared to other methods. this data influx has created a bottleneck in processing and analysis, A typical CNN architecture includes an input layer, multiple convo- as traditional manual methods prove to be excessively time-consuming lutional layers, pooling layers, a fully connected layer, and an output and resource-intensive. To deal with these issues, many networks are layer. CNNs can be used for both supervised and unsupervised learning, embracing automation as a means to efficiently handle the significant and are applied in diverse fields such as computer vision, natural volumes of generated data (Molau, 2001; Spurný et al., 2007; Gural language processing, and others (Hastie et al., 2001). and Šegon, 2009; Brown et al., 2010; Gural, 2011; Weryk et al., 2013; In the context of meteor monitoring, current research efforts using Howie et al., 2017; Suk and Šimberová, 2017; Nikolic, 2019; Peña- ML focus on two main objectives. First, some studies concentrate on Asensio et al., 2021a,b; Vida et al., 2021). These automated approaches determining the presence of a meteor in a given event, with the allow meteor scientists to analyze and interpret meteor data faster and goal of distinguishing meteoric events from non-meteoric phenomena. more efficiently than ever before, helping to uncover new insights into meteor behavior and properties. Alternatively, other works rely on ML for meteor tracking, facilitating The detection of luminous sources moving in the sky is relatively the accurate localization and monitoring of meteoroids throughout easy to solve as the camera control software only needs to store and their bright atmospheric trajectory. The primary challenge is to effec- overwrite the last few minutes of recording, and in the event of a tively distinguish false positives caused by non-meteor objects such as sudden increase in illumination, permanently save this data. However, airplanes, birds, and insects, or atmospheric conditions (e.g. clouds). the trigger threshold must be carefully calibrated to avoid missing These innovative approaches are being increasingly employed within any meteors while minimizing the number of false positives. Defining meteor networks worldwide, including the Global Meteor Network this cut-off is complex as it can vary depending on a number of (GMN) (Gural, 2019), AllSky7 Fireball Network Germany (FNG1 ), Me- factors, including general lighting conditions, which need to be updated teorite Orbits Reconstruction by Optical Imaging (MOROI) (Nedelcu periodically to consider specific dusk and dawn illumination conditions et al., 2018), Canadian Automated Meteor Observatory (CAMO) (Weryk or the presence of the Moon. et al., 2013), Cameras for All-sky Meteor Surveillance (CAMS) (Jen- The pipelines that attempt to automate the detection and tracking niskens et al., 2011), EXOSS meteor network2 , and Meteor Automatic of meteors face a difficult task because meteors are virtually ran- Imager and Analyzer (MAIA) (Vítek et al., 2011). dom phenomena and can occur in a variety of ways due to impact In image classification, it is customary to use transfer learning geometry, variable velocity, size, shape, composition, viewing angle, techniques with pre-trained models of CNNs (Sennlaub et al., 2022; sky conditions (clouds or illumination), etc. In addition, meteors must Marsola and Lorena, 2019; Galindo and Lorena, 2018). These methods be distinguished from false positives caused by satellites, airplanes, allow inheriting the ability to detect objects from those pre-trained helicopters, drones, birds, lightning, or artificial light sources. The models, which need to be re-trained on meteors. With this methodol- combination of possible characteristics that meteors can exhibit makes ogy, optimal results are achieved with a smaller number of training it difficult to define fixed parameters that work in all cases. As a data compared to starting from an uninitialized model. As underlined result, many networks still rely on human experts to manually review in Galindo and Lorena (2018), uttermost results are achieved when the the footage and identify/classify meteors. However, human operators proper pre-trained dataset is selected. They compared the performance can occasionally make errors, particularly when artificial events cause of ImageNET and Fashion-MNIST (Xiao et al., 2017) with fine-tuning, confusion or ambiguity. However, there are also networks that use concluding that the latter is the most optimal as it is already trained to fully automated approaches based on traditional computer vision tech- work with black and white images. They also checked whether the CNN niques, such as image processing algorithms with fixed instructions, could distinguish meteors if the image was previously tweaked (slightly e.g. CAMS (Jenniskens et al., 2011), SonotaCo (SonotaCo, 2016), or zoomed, rotated, or flipped). The results showed that these transforma- EDMOND (Kornoš et al., 2014). Some of the detection pipelines cur- tions could produce unrealistic apparent trajectories and worsen the rently in use are MetRec, MeteorScan, and UFOCapture; an overview of classification. In order to solve these types of problems, Ganju et al. their capabilities is given in Molau and Gural (2005). These automated (2023) use a windowing technique to create new frames from existing approaches show a high percentage of events with suspicious calculated ones. With this, all meteor detection would have the same number results due to their reliance on fixed parameters that may not be of frames, easing the posterior analysis. In Cecil and Campbell-Brown appropriate for all scenarios (Hajdukova et al., 2020). (2020) is shown a comparison of different sets of combinations of Consequently, addressing the challenges of meteor monitoring re- quires the adoption of artificial intelligence techniques. In this paper, we delve into the utilization of new methodologies for meteor classifi- 1 https://allsky7.net/. 2 cation and processing tasks. Specifically, we investigate how Machine https://exoss.org/. 2 E. Peña-Asensio et al. Planetary and Space Science 238 (2023) 105802 Table 1 Summary of the main previous works on ML applied to meteor detection. For each contribution, we denote the meteor network source, the number of samples used to train the models, their goal, the transfer learning method used (if any), the different models compared, and the best F1 score and/or accuracy achieved (when provided). F1 score and accuracy have been normalized between 0 and 1. Article Meteor network # samples Goal Transfer learning Model F1 score Accuracy Ganju et al. (2023) CAMS 19,152 Detection BiLSTM – 0.91 0.89 Anghel et al. (2023) MOROI 9,858 Detection – CNN, GB and RF 0.989 0.998 Sennlaub et al. (2022) AllSky7 FNG 20,000 Detection ImageNET CNN: GRU and SVM – 0.991 Cecil and Campbell-Brown (2020) CAMO 50,745 Tracking No CNN: multi-layer and max pool 0.997 0.998 Gural (2019) CAMS 200,361 Tracking MeteorNet RNN and LSTM 0.974 0.981 Marsola and Lorena (2019) EXOSS 400 Detection ImageNET VGG16 – 0.8435 Vítek and Nasyrova (2019) MAIA – Tracking No DPT – – Galindo and Lorena (2018) EXOSS 1,660 Detection Fashion-MNIST – 0.94 0.960 different techniques used in image processing, such as convolutions and max-pools. Even though most CNN meteor detection algorithms have been trained to reach a satisfying prediction percentage (> 99%), particularly when considering large sample sizes of more than ∼10,000 events, some anomalies are still misclassified as meteors for small datasets. The next step in meteor detection algorithms is to consider the intrinsic properties of meteors on camera images to discard these misleading anomalies. Additionally, the main drawback of the current meteor tracking algorithms is the runtime required to analyze high-definition 1080p video images. However, they perform well when dealing with small, low-resolution video images. Fig. 1. A basic-block building block of ResNet-34. Beyond CNNs, other ML techniques are often used. It is the case of Recurrent Neural Networks (RNN), Gradient Boost (GB), or Ran- dom Forest (RF), which can also be used as complementary analysis 3. CNN, transfer learning, and Grad-CAM tools (Gural, 2019; Anghel et al., 2022). Temporal resolution can be introduced in the analysis by using other networks such as Long Our work aims to use ML techniques to achieve two main goals: Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Temporal detecting the presence of meteors in images and tracking the motion of Convolutional Network (TCN), Time Delay Neural Network (TDNN), meteors in the field of view. To achieve this, and based on the review of Support Vector Machine (SVM), or VGG16 (Siladi et al., 2015; Si- the scientific literature, we have decided to develop a CNN model that monyan and Zisserman, 2015; Sennlaub et al., 2022), supporting 16 classifies images into two groups, ‘‘Meteors’’ and ‘‘No-meteors’’, using layers. A particular additional technique is the discrete pulse trans- transfer learning. In addition, we implemented a novel application of form (DPT) (Rohwer and Laurie, 2006), in which image signals are Grad-CAM to track the coordinates of the meteor’s motion. decomposed in pulses. Vítek and Nasyrova (2019) introduced DPT to characterize the number of pulses related to the meteor compared to 3.1. Detection those of stars. Even though it is not specified if the results are better To build our model, we chose to use ResNet-34, which is a 34-layer than other works, they do underline that using DPT is faster than other pre-trained CNN from the Residual Network family (He et al., 2015). methods used in the MAIA data. This allowed us to quickly specialize the model for our specific use Sennlaub et al. (2022) also classified those false positives based case using transfer learning techniques with a small dataset and rapidly on their origin. They did not achieve solid statements but pointed inheriting all detection skills already learned by the network. ResNet- out similarities among false positive subgroups. As sketched in Le 34 mainly consists of an input layer, convolutional layers, residual Lan and Dinh (2021) a future algorithm to classify the false positives blocks, shortcut connections, downsampling, global average pooling, would include prior knowledge of each subgroup. These could also be and fully connected layers. One of the key elements of this network expanded to classify the re-entry of artificial space debris. They could is the residual building block, which is its infrastructure. As shown also include a cross-matching identification between different stations in Fig. 1, the residual building block consists of several convolutional as a method of improving the overall accuracy (Anghel et al., 2023). layers (Conv), batch normalizations (BN), a rectified linear unit (ReLU) Table 1 provides a comprehensive summary of the outcomes achie- activation function, and a link. This block is used for all 34 layers of ved in these works accompanied by the data source, the number of ResNet-34, as depicted in Table 2. The output of the residual block is samples, the technique used, and the results such as F1 score and given by the formula 𝑦 = 𝐹 (𝑥) + 𝑥, where 𝐹 is the residual function and 𝑥 and 𝑦 are the input and output of the residual function, respectively. accuracy. The F1 score is a performance metric that combines precision The entire residual network is composed of the first convolutional layer and recall to measure the accuracy of a classification model. Precision and multiple basic blocks, making it a highly effective and efficient represents the proportion of true positive predictions out of all positive deep learning architecture for image recognition tasks. predictions, while recall represents the proportion of true positive pre- For training and testing the model, we used a dataset of 982 images dictions out of all actual positive instances. The F1 score considers both of meteors detected by optical devices of the Spanish Meteor Network precision and recall, making it useful when the dataset is imbalanced (SPMN) network stations (Trigo-Rodríguez et al., 2006), along with or when false positives and false negatives have different consequences. 56,285 images without meteor detection collected over the year 2021, Accuracy is a common evaluation metric used to measure the overall particularly from the Pujalt observatory. To balance the two groups, correctness of a classification model. It calculates the proportion of we generated a dataset of 982 images with meteors and 1050 without correctly predicted instances (both true positives and true negatives) detections for training. To ensure reliable model performance, a portion out of the total number of instances in the dataset. of these images, specifically 20%, is allocated for validation, which is 3 E. Peña-Asensio et al. Planetary and Space Science 238 (2023) 105802 Table 2 The structure of ResNet-34. Layer name Output size 34-layer 7 × 7, 64, 𝑠𝑡𝑟𝑖𝑑𝑒2 Conv1 112 × 112 3 × 3 𝑚𝑎𝑥 𝑝𝑜𝑜𝑙, 𝑠𝑡𝑟𝑖𝑑𝑒2 [ ] 3 × 3, 64 Conv2_x 56 × 56 ×3 3 × 3, 64 [ ] 3 × 3, 128 Conv3_x 28 × 28 ×4 3 × 3, 128 [ ] 3 × 3, 256 Conv4_x 14 × 14 ×6 3 × 3, 256 [ ] Fig. 2. The general process of class activation mapping method. 3 × 3, 512 Conv5_x 7 × 7 ×3 Source: Adapted from Jiang et al. (2021). 3 × 3, 512 1 × 1 average pool, 1000-d fc, softmax To capture finer details for properly tracking the meteor motion, we focus on the activations of the initial convolutional layer, restricting utilized to evaluate the training progress. From this dataset, 300 images our attention to activations falling within the defined ROI delineated were specifically set aside for testing purposes, 150 from the meteor by the Grad-CAM. Within the CNN, the activation of the deepest layers class and 150 from the no-meteor. The test set serves as an independent offers a more detailed resolution map (56 × 56). However, these layers dataset to evaluate the final performance of the trained model after the are more prone to noise and less reliable in accurately identifying the completion of the training process. ROI related to meteor detection. To tackle these difficulties, we apply The dataset consisted of grayscale long exposure (30 s) images a noise reduction strategy by selectively retaining only the cells that that were pre-processed to enhance the meteor trail and remove static align with the cells from the higher precision but lower resolution visual elements from the background by subtracting consecutive im- Grad-CAM (7 × 7) with a non-zero value. By doing so, we filter out ages. This included converting the images to black and white, resizing noisy activations and focus on the cells that have a meaningful impact them to 400 × 400, and subtracting successive images to remove the on the meteor classification. Subsequently, we extract the cells with background. To facilitate the generalization of the model and reduce the maximum activation values from the refined high-resolution acti- overfitting, data augmentation techniques were used during the transfer vation map. By calculating the average position among these selected learning process. Specifically, each batch of images received by the cells, we are able to project a single point onto the original image. CNN during the 35 epochs of training was modified with geometric This refinement process significantly enhances the accuracy of meteor transformations such as randomly flipping, cropping, rotating, and detection by precisely pinpointing the location of meteors within the translating the images, or applying lighting modifications. frames generated by our model. Fig. 3 illustrates the described meteor 3.2. Tracking detection and tracking process. Note that we do not multiply the activations of the initial layer with The final layer of our CNN exhibits activations corresponding to the weights of the classification layer because the spatial and semantic neurons that are specifically triggered when a meteor is detected in gap between the initial convolutional layer and the classification layer an image. While such activations provide initial utility, leveraging limits the effectiveness of such an approach. Although Grad-CAM al- the subsequent classification layer’s weights in conjunction with these lows the analysis of any layer in the network, we observed that focusing activations further enhances their significance. It is important to note just on the activations from the initial convolutional layer within the that the classification layer possesses a comprehensive understanding of ROI calculated using Grad-CAM on the last layer yields good results. the meteor classification task, enabling it to assign appropriate weights to the activated neurons. Hence, certain previously activated neurons 4. Training and results may be deemed less influential in the final classification decision. This is the foundation of the so-called Class Activation Mapping (CAM) Fig. 4 shows the evolution during the training process of the loss technique (Zhou et al., 2015). function as a function of the batches processed, which is a metric The CAM technique is usually employed to generate a heatmap that measures the deviation between predicted and actual values. The for a given class (class ‘‘meteor’’ in our case). This technique involves primary goal during training is to minimize this function. The training capturing the activations from the last convolutional layer of the CNN set consists of the images used to train the CNN model, while the and multiplying them by the corresponding weights from the last fully validation set consists of a subset of 150 images randomly selected connected layer responsible for the classification task. By performing and reserved for evaluating the model’s performance and generalization this operation, the CAM technique effectively highlights the areas ability. By evaluating the loss function on both sets, we monitor the within the image that contribute the most to the meteor classification, progress of the model and detect any signs of overfitting or underfitting. that is, the Region of Interest (ROI). Fig. 2 illustrates the overall A batch refers to a group of images that are processed together procedures of this method. during each iteration of the training algorithm. The total number of However, in order to further enhance the performance of our model, batches processed can be calculated using the formula 𝐵𝑎𝑡𝑐ℎ𝑒𝑠 = 𝑁 ∗ we opted to incorporate an advanced variant of CAM known as Grad- (𝐼∕𝐵𝑆), where 𝑁 is the number of epochs, which is the number of CAM (Selvaraju et al., 2016). Grad-CAM builds upon the CAM method- times the entire training data set is passed through the network during ology by integrating gradient information instead of the weights of the training. 𝐼 denotes the total number of images in the training dataset, classification layer. This provides a more fine-grained localization of including both meteor and non-meteor images. Finally, 𝐵𝑆 refers to the important regions within an image. Grad-CAM computes the gradients batch size, which represents the number of images fed to the network of the target class of a specific layer with respect to the activations of in each training iteration. For this study, we used a batch size of 32. the same layer. By multiplying the activations from the last convolu- By substituting these values, we determined that a total of 1,515 tional layer with the average of the gradients across the feature map of batches were processed during the training process. This corresponds to that layer, Grad-CAM obtains the importance weights for the activation the maximum value observed in Fig. 4, indicating the completion of all maps. The weighted combination of the activation maps produces the batches. Understanding the relationship between the loss function, the final heatmap, which visually highlights the critical regions within the batches processed, and the progress of the training and validation sets input image for the classification of the target class. provides valuable insight into the training dynamics and performance 4 E. Peña-Asensio et al. Planetary and Space Science 238 (2023) 105802 Fig. 3. Illustrative diagram of the proposed process of meteor detection and tracking using CNN and Grad-CAM. The image exemplifies a video detection made by the SPMN station in Estepa, Seville operated by Antonio J. Robles. Fig. 4. Training and validation loss during model training. evaluation of the neural network model. As every batch comprises 32 Fig. 5. Confusion matrix of the trained model with normalized values in parentheses. images and is subjected to various random transformations, the overall training process, including data augmentation, incorporates a varied array of 48,480 distinct images. Table 3 compiles the evolution of different metrics during the training process. results show that our model achieved an accuracy of 0.96 on the test The next step involves defining the crucial hyperparameter for the dataset. This accuracy metric indicates the model’s ability to correctly training process: the learning rate. By evaluating the loss function classify meteor and non-meteor images with a high degree of accuracy. Furthermore, when specifically considering the meteor class, our model values across various learning rates, we can identify the region of achieved a precision of 0.98. sustained and substantial loss reduction, disregarding transient peaks To further analyze the performance of the model, a confusion and irregular drops as they do not represent reliable trends. Once matrix was constructed as it provides insight into the classification this region is identified, we pinpoint the midpoint of the steepest performance by showing the distribution of the predicted labels against descent line, which corresponds to the most significant loss reduction. the true labels. The confusion matrix for our two labels ‘‘Meteor’’ and This specific learning rate is then selected for the subsequent training ‘‘No-Meteor’’ is shown in Fig. 5. procedure. For our specific study, we determined a learning rate of In the confusion matrix, the rows correspond to the true labels, 0.003 to be the optimal choice. while the columns represent the predicted labels. The matrix values The training phase of our pipeline ends with an F1 score of 0.94, indicate the proportion of images belonging to each category. The indicating the high precision and recall achieved during the training confusion matrix shows that 47% of the images were correctly classified process. We then evaluated the performance of the trained model on as meteors, while 3.3% of the images were incorrectly classified as the test dataset to assess its generalization capabilities. The evaluation non-meteor images. Furthermore, 1% of the images were incorrectly 5 E. Peña-Asensio et al. Planetary and Space Science 238 (2023) 105802 Table 3 have resulted in slightly lower performance compared to some of the Metrics evolution during the training process, including for each epoch the error rate, models reported in Table 1. Despite this slight performance difference, accuracy, precision, recall, and F1 score. our pipeline still demonstrates a high level of accuracy and efficiency in Epoch Error Accuracy Precision F1 meteor detection and classification. The inclusion of data augmentation 0 0.245665 0.754335 0.763323 0.753592 techniques is crucial to promote better generalization and robustness in 1 0.210983 0.789017 0.788704 0.788804 2 0.205202 0.794798 0.798389 0.794714 the model, even though it may have had a slight impact on the overall 3 0.187861 0.812139 0.823898 0.811444 results compared to other models in the comparison. 4 0.127168 0.872832 0.872594 0.872726 We compared our results with those obtained by Galindo and Lorena 5 0.161850 0.838150 0.842479 0.838064 (2018) and Marsola and Lorena (2019), who used datasets of similar 6 0.104046 0.895954 0.895722 0.895867 size to ours. In this comparison, our methodology equals or outper- 7 0.138728 0.861272 0.875206 0.860709 8 0.086705 0.913295 0.913068 0.913222 forms existing approaches, demonstrating superior performance. Fur- 9 0.072254 0.927746 0.928404 0.927527 thermore, our pipeline incorporates the added complexity of meteor 10 0.063584 0.936416 0.936314 0.936382 tracking, which presents a significant challenge due to the smaller 11 0.049133 0.950867 0.950896 0.950775 luminous trace in each frame. Meteor tracking enables the computation 12 0.060694 0.939306 0.939599 0.939294 13 0.063584 0.936416 0.936896 0.936244 of the velocity curve, a key factor in both discriminating between 14 0.049133 0.950867 0.950896 0.950775 artificial and natural objects and in determining the heliocentric orbit 15 0.095376 0.904624 0.912476 0.904489 and the potential meteorite strewn field. Automating this process facil- 16 0.066474 0.933526 0.936153 0.933521 itates the extraction of orbital elements for hundreds or thousands of 17 0.049133 0.950867 0.950693 0.950834 meteors detected nightly, providing valuable insights into both the spo- 18 0.052023 0.947977 0.948137 0.947865 19 0.037572 0.962428 0.962805 0.962337 radic meteoroid background flux and the characteristics of meteoroid 20 0.052023 0.947977 0.948137 0.947865 streams. 21 0.043353 0.956647 0.956495 0.956586 In subsequent phases, we intend to enhance the pipeline by refining 22 0.052023 0.947977 0.948109 0.947961 the balance of false positives, encompassing a diverse spectrum of 23 0.046243 0.953757 0.953543 0.953719 24 0.037572 0.962428 0.962282 0.962375 potential false positive sources including satellites, planes, birds, bugs, 25 0.037572 0.962428 0.962805 0.962337 and other light sources in the training process. 26 0.049133 0.950867 0.950896 0.950775 27 0.034682 0.965318 0.965915 0.965224 5. Conclusions 28 0.043353 0.956647 0.956688 0.956566 29 0.049133 0.950867 0.950693 0.950834 30 0.049133 0.950867 0.950693 0.950834 The implementation of automated detection software has led to a 31 0.052023 0.947977 0.947761 0.947933 massive increase in the amount of data collected and reduced every 32 0.046243 0.953757 0.953551 0.953702 year by meteor networks. However, the need for human oversight to 33 0.046243 0.953757 0.953551 0.953702 filter out false positives and organize the records has created a bottle- 34 0.034682 0.965318 0.965525 0.965244 neck, and the traditional computer vision techniques implemented have limited performance due to the random and specific characteristics of each meteor event. We employed CNNs to address these challenges. classified as meteor images, while 49% were correctly identified as In our study, we used a dataset of 982 meteor images along with non-meteor images. The high accuracy achieved by the pipeline demon- 1,050 images without meteors detected by SPMN stations in 2021 to strates its robustness in accurately detecting and classifying meteor train a CNN model. A transfer learning technique was applied, and images. The low misclassification rates for both meteor and non-meteor Grad-CAM was used for accurate tracking. Our main results are as classes indicate the effectiveness of the trained CNNs in distinguishing follows: between these classes, then minimizing the required human time for (1) Our approach utilized ResNet-34, a deep learning architec- these time-consuming tasks. ture consisting of 34 pre-trained layers. By using pre-trained layers, It is worth noting that the proposed model shows the ability to we capitalized on the knowledge and representations gained from a generalize and accurately detect meteor trails even in color frames that large dataset during the initial training phase, resulting in improved have not undergone the previous frame subtraction process. This is model performance. In addition, data augmentation techniques were particularly noteworthy because these frames may contain stationary employed to facilitate the model’s ability to accurately generalize and elements such as clouds, the Moon, buildings, or other obstructions mitigate overfitting. The results achieved demonstrate a precision of that can cause interference and affect the accuracy of meteor detection. 98% for meteor classification. Despite these challenges, the model can effectively distinguish and (2) Grad-CAM was used to track the coordinates of the meteor identify meteor trails, providing robust and reliable results. This further within each image. This technique involved analyzing deeper layers of validates the model’s ability to operate under real-world conditions, the neural network, which have higher accuracy but lower resolution. making it a valuable tool for meteor scientists and enthusiasts alike. ROI information was extracted using the gradients of the last convo- However, this very capability in detecting meteor trails renders lutional layer and then combined with activation information from it susceptible to satellite misidentifications, as they often exhibit a the initial layer, which is characterized by higher resolution but lower similar trace when reflecting the sunlight. An instance of such incorrect accuracy. This fusion of information allowed the identification of the classification is illustrated in the top panel of Fig. 6 with a transit of most critical pixel corresponding to the position of the meteoroid. This a SpaceX Starlink satellite train, necessitating retraining the network technique allows for the precise localization of meteor positions within to encompass this new class. Conversely, the bottom panel shows an frames. undetected meteor, possibly due to its proximity to the full moon and (3) The high performance achieved by our pipeline underscores dimming by a cloud-covered field of view. its robustness in precisely detecting and classifying meteor images. We used data augmentation techniques during the training process, The success rate, even with a relatively small dataset, highlights the which is a common strategy that helps prevent overfitting and improves potential of our method to significantly reduce the workload of meteor the model’s ability to generalize well to unseen data. However, it is scientists and station operators involved in meteor data analysis. This worth noting that Xiao et al. (2017) suggested data augmentation could potential is further enhanced by one of the most notable advancements potentially degrade model performance in certain cases. Therefore, it is in our methodology: the novel use of Grad-CAM for meteor tracking in possible that our use of data augmentation in the training process may combination with the initial activation map. 6 E. Peña-Asensio et al. Planetary and Space Science 238 (2023) 105802 Fig. 6. Top panel: False positive of a SpaceX Starlink satellite track as it exhibits similar characteristics as a meteor trail. Recording obtained from the Alpicat SPMN station under the operation of Marc Corretgé-Gilart. Bottom panel: False negative of SPMN070523G superbolide recorded near the full moon with a cloudy sky. Recording obtained from Bartolo-Castelló SPMN station under the operation of Vicente Ibañez. In summary, our study highlights the significant potential of apply- Acknowledgments ing ML techniques to meteor monitoring. It illustrates the effectiveness of CNNs and transfer learning in reducing false positives and correctly This project has received funding from the European Research identifying meteors in images. By automating the meteor monitoring Council (ERC) under the European Union’s Horizon 2020 research and process, our pipeline increases the efficiency of meteor detection and innovation programme (grant agreement No. 865657) for the project tracking using the Grad-CAM technique. This, in turn, facilitates the ‘‘Quantum Chemistry on Interstellar Grains’’ (QUANTUMGRAIN). JMT- study of meteoroid fluxes, aids in population characterization, and R, EP-A, and PG-T acknowledge financial support from the project improves our ability to distinguish between meteorite-dropping events, PID2021-128062NB-I00 funded by MCIN/AEI/10.13039/5011000 thereby increasing fresh extraterrestrial material recovery rates. 11033. AR acknowledges financial support from the FEDER/Ministerio de Ciencia e Innovación – Agencia Estatal de Investigación (PID2021- CRediT authorship contribution statement 126427NB-I00, PI: AR). This work was also partially supported by the program Unidad de Excelencia María de Maeztu CEX2020-001058- Eloy Peña-Asensio: Conceptualization, Methodology, Writing. Josep M. We thank all SPMN station operators and photographers whose M. Trigo-Rodríguez: Supervision, Project administration. Pau Grèbol- continuous dedication has allowed to record incessantly the meteors of Tomàs: Investigation, Resources. David Regordosa-Avellana: Concep- the Iberian Peninsula, the Balearic Islands, and the Canary Islands. We tualization, Methodology, Software. Albert Rimola: Funding acquisi- express our gratitude to David Puiggròs-Figueras for his assistance in tion, Supervision. the CAM and Grad-CAM methodology. The following Python packages were extensively used in the study: Numpy, OpenCV, Matplotlib, PyTorch, Declaration of competing interest and fast.ai. The authors declare that they have no known competing finan- References cial interests or personal relationships that could have appeared to influence the work reported in this paper. Anghel, S., Nedelcu, D.-A., Birlan, M., Boaca, I., 2022. Machine learning methods applied to meteor detection filtering. In: European Planetary Science Congress. http://dx.doi.org/10.5194/epsc2022-1107, EPSC2022–1107. Data availability Anghel, S., Nedelcu, D.A., Birlan, M., Boaca, I., 2023. Single-station meteor detection filtering using machine learning on MOROI data. Mon. Not. R. Astron. Soc. 518 Data will be made available on request. (2), 2810–2824. http://dx.doi.org/10.1093/mnras/stac3229. 7 E. Peña-Asensio et al. Planetary and Space Science 238 (2023) 105802 Bottke, W.F., Cellino, A., Paolicchi, P., Binzel, R.P. (Eds.), 2002. Asteroids III. University Koschny, D., Drolshagen, E., Drolshagen, S., Kretschmer, J., Ott, T., Drolshagen, G., of Arizona Press, http://dx.doi.org/10.2307/j.ctv1v7zdn4. Poppe, B., 2017. Flux densities of meteoroids derived from optical double-station Brown, P., Weryk, R.J., Kohut, S., Edwards, W.N., Krzeminski, Z., 2010. Development of observations. Planet. Space Sci. 143, 230–237. http://dx.doi.org/10.1016/j.pss. an all-sky video meteor network in southern ontario, Canada the ASGARD system. 2016.12.007. WGN, J. Int. Meteor Organ. 38 (1), 25–30. Koschny, D., Soja, R.H., Engrand, C., Flynn, G.J., Lasue, J., Levasseur-Regourd, A.- Cecil, D., Campbell-Brown, M., 2020. The application of convolutional neural networks C., Malaspina, D., Nakamura, T., Poppe, A.R., Sterken, V.J., Trigo-Rodríguez, J.M., to the automation of a meteor detection pipeline. Planet. Space Sci. 186, 104920. 2019. Interplanetary dust, meteoroids, meteors and meteorites. Space Sci. Rev. 215 http://dx.doi.org/10.1016/j.pss.2020.104920. (4), 34. http://dx.doi.org/10.1007/s11214-019-0597-7. Ceplecha, Z., 1987. Geometric, dynamic, orbital and photometric data on meteoroids Koten, P., Rendtel, J., Shrbený, L., Gural, P., Borovička, J., Kozak, P., 2019. Meteors and from photographic fireball networks. Bull. Astron. Inst. Czech. 38, 222. meteor showers as observed by optical techniques. In: Ryabova, G.O., Asher, D.J., Ceplecha, Z., Borovička, J., Elford, W.G., Revelle, D.O., Hawkes, R.L., Porubčan, V., Campbell-Brown, M.J. (Eds.), Meteoroids: Sources of Meteors on Earth and beyond. Šimek, M., 1998. Meteor phenomena and bodies. Space Sci. Rev. 84, 327–471. p. 90. http://dx.doi.org/10.1023/A:1005069928850. Lauretta, D.S., McSween, H.Y. (Eds.), 2006. Meteorites and the Early Solar System II. Colas, F., Zanda, B., Bouley, S., Jeanne, S., Malgoyre, A., Birlan, M., Blanpain, C., University of Arizona Press, http://dx.doi.org/10.2307/j.ctv1v7zdmm. Gattacceca, J., Jorda, L., Lecubin, R., Zollo, A., 2020. FRIPON: A worldwide Le Lan, C., Dinh, L., 2021. Perfect Density Models Cannot Guarantee Anomaly De- network to track incoming meteoroids. Astron. Astrophys. 644, A53. http://dx. tection. Entropy 23 (12), 1690. http://dx.doi.org/10.3390/e23121690, URL https: doi.org/10.1051/0004-6361/202038649, arXiv:2012.00616. //www.mdpi.com/1099-4300/23/12/1690. Drolshagen, E., Ott, T., Koschny, D., Drolshagen, G., Vaubaillon, J., Colas, F., Zanda, B., Bouley, S., Jeanne, S., Malgoyre, A., Birlan, M., Vernazza, P., Gardiol, D., Marsola, T.C.L., Lorena, A.C., 2019. Meteor detection using deep convolutional neural Nedelcu, D.A., Rowe, J., Forcier, M., Trigo-Rodriguez, J.M., Peña-Asensio, E., networks. In: Anais do 14% Simpósio Brasileiro de Automação Inteligente. Lamy, H., Ferrière, L., Barghini, D., Carbognani, A., Di Martino, M., Rasetti, S., Molau, S., 2001. The AKM video meteor network. In: Warmbein, B. (Ed.), Meteoroids Valsecchi, G.B., Volpicelli, C.A., Di Carlo, M., Knapic, C., Pratesi, G., Riva, W., 2001 Conference. In: ESA Special Publication, vol. 495, pp. 315–318. Stirpe, G.M., Zorba, S., Hernandez, O., Grandchamps, A., Jehin, E., Jobin, M., Molau, S., Gural, P.S., 2005. A review of video meteor detection and analysis software. King, A., Sanchez-Lavega, A., Toni, A., Rimola, A., Poppe, B., 2021. Luminous WGN, J. Int. Meteor Organ. 33 (1), 15–20. efficiency based on FRIPON meteors and limitations of ablation models. Astron. Nedelcu, A.D., Birlan, M., Turcu, V., Badescu, O., Boaca, I., Gornea, A., Blagoi, O., Astrophys. 650, A159. http://dx.doi.org/10.1051/0004-6361/202040204. Danescu, C., Paraschiv, P., 2018. The MOROI network. Meteorites orbits recon- Dumitru, B.A., Birlan, M., Popescu, M., Nedelcu, D.A., 2017. Association between struction by optical imaging. In: ZAC 2018 - International Conference Outlook in meteor showers and asteroids using multivariate criteria. Astron. Astrophys. 607, Astronomy. p. 21. A5. http://dx.doi.org/10.1051/0004-6361/201730813. Nikolic, V., 2019. Automation of a video meteor network. In: Rudawska, R., Rendtel, J., Galindo, Y., Lorena, A.C., 2018. Deep Transfer Learning for Meteor Detection. In: Anais Powell, C., Lunsford, R., Verbeeck, C., Knofel, A. (Eds.), International Meteor Do XV Encontro Nacional de InteligÊncia Artificial E Computacional (ENIAC 2018). Conference, Pezinok-Modra, Slovakia. pp. 139–140. Sociedade Brasileira de Computação - SBC, São Paulo, pp. 528–537. http://dx.doi. Peña-Asensio, E., Trigo-Rodríguez, J.M., Gritsevich, M., Rimola, A., 2021a. Accurate 3D org/10.5753/eniac.2018.4445, URL http://portaldeconteudo.sbc.org.br/index.php/ fireball trajectory and orbit calculation using the 3D-FIRETOC automatic python eniac/article/view/4445. code. Mon. Not. R. Astron. Soc. 504 (4), 4829–4840. http://dx.doi.org/10.1093/ Ganju, S., Hatua, A., Jenniskens, P., Krishna, S., Ren, C., Ambardar, S., 2023. mnras/stab999, arXiv:2103.13758. AI-enhanced data processing and discovery crowd sourcing for meteor shower Peña-Asensio, E., Trigo-Rodríguez, J.M., Langbroek, M., Rimola, A., Robles, A.J., 2021b. mapping. http://dx.doi.org/10.48550/ARXIV.2308.02664, URL https://arxiv.org/ Using fireball networks to track more frequent reentries: Falcon 9 upper-stage abs/2308.02664. orbit determination from video recordings. Astrodynamics 5 (4), 347–358. http: Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., //dx.doi.org/10.1007/s42064-021-0112-2, arXiv:2109.01004. Cai, J., Chen, T., 2018. Recent advances in convolutional neural networks. Pattern Peña-Asensio, E., Trigo-Rodríguez, J.M., Rimola, A., 2022. Orbital characterization of Recognit. 77, 354–377. http://dx.doi.org/10.1016/j.patcog.2017.10.013. superbolides observed from space: Dynamical association with near-earth objects, Gural, P.S., 2011. The california all-sky meteor surveillance (CAMS) system. In: meteoroid streams, and identification of hyperbolic meteoroids. Astron. J. 164 (3), Proceedings of the International Meteor Conference, 29th IMC, Armagh, Northern 76. http://dx.doi.org/10.3847/1538-3881/ac75d2, arXiv:2206.03115. Ireland, 2010. pp. 28–31. Gural, P.S., 2019. Deep learning algorithms applied to the classification of video meteor Peña-Asensio, E., Trigo-Rodríguez, J.M., Rimola, A., Corretgé-Gilart, M., Koschny, D., detections. Mon. Not. R. Astron. Soc. 489 (4), 5109–5118. http://dx.doi.org/10. 2023. Identifying meteorite droppers among the population of bright ’sporadic’ 1093/mnras/stz2456. bolides imaged by the spanish meteor network during the spring of 2022. Mon. Not. Gural, P., Šegon, D., 2009. A new meteor detection processing approach for observa- R. Astron. Soc. 520 (4), 5173–5182. http://dx.doi.org/10.1093/mnras/stad102, tions collected by the Croatian meteor network (CMN). WGN, J. Int. Meteor Organ. arXiv:2301.03515. 37 (1), 28–32. Rohwer, C.H., Laurie, D.P., 2006. The discrete pulse transform. SIAM J. Math. Anal. Hajdukova, M., Sterken, V., Wiegert, P., Kornoš, L., 2020. The challenge of identifying 38, 1012–1034. interstellar meteors. Planet. Space Sci. 192, 105060. http://dx.doi.org/10.1016/j. Selvaraju, R.R., Das, A., Vedantam, R., Cogswell, M., Parikh, D., Batra, D., 2016. Grad- pss.2020.105060. CAM: Why did you say that?. http://dx.doi.org/10.48550/ARXIV.1611.07450, URL Hastie, T., Friedman, J., Tibshirani, R., 2001. The Elements of Statistical Learning. https://arxiv.org/abs/1611.07450. Springer, New York, http://dx.doi.org/10.1007/978-0-387-21606-5. Sennlaub, R., Hofmann, M., Hankey, M., Ennes, M., Müller, T., Kroll, P., Mäder, P., He, K., Zhang, X., Ren, S., Sun, J., 2015. Deep residual learning for image recognition. 2022. Object classification on video data of meteors and meteor-like phenomena: http://dx.doi.org/10.48550/arXiv.1512.03385, arXiv e-prints, arXiv:1512.03385, algorithm and data. Mon. Not. R. Astron. Soc. 516 (1), 811–823. http://dx.doi.org/ arXiv:1512.03385. 10.1093/mnras/stac1948, arXiv:2208.14914. Howie, R.M., Paxman, J., Bland, P.A., Towner, M.C., Cupak, M., Sansom, E.K., Siladi, E., Vida, D., Nyarko, K., 2015. Video meteor detection filtering using soft Devillepoix, H.A.R., 2017. How to build a continental scale fireball camera network. computing methods. In: International Meteor Conference Mistelbach, Austria. p. Exp. Astron. 43 (3), 237–266. http://dx.doi.org/10.1007/s10686-017-9532-7. 24. Hughes, D.W., 1978. Meteors. In: McDonnell, J.A.M. (Ed.), Cosmic Dust. pp. 123–185. Silber, E.A., Boslough, M., Hocking, W.K., Gritsevich, M., Whitaker, R.W., 2018. Physics Jenniskens, P., 2017. Meteor showers in review. Planet. Space Sci. 143, 116–124. of meteor generated shock waves in the earth’s atmosphere - A review. Adv. http://dx.doi.org/10.1016/j.pss.2017.01.008. Space Res. 62 (3), 489–532. http://dx.doi.org/10.1016/j.asr.2018.05.010, arXiv: Jenniskens, P., Gural, P.S., Dynneson, L., Grigsby, B.J., Newman, K.E., Borden, M., 1805.07842. Koop, M., Holman, D., 2011. CAMS: Cameras for allsky meteor surveillance to Simonyan, K., Zisserman, A., 2015. Very Deep Convolutional Networks for Large-Scale establish minor meteor showers. Icarus 216 (1), 40–61. http://dx.doi.org/10.1016/ Image Recognition. URL http://arxiv.org/abs/1409.1556, cs. j.icarus.2011.08.012. SonotaCo, 2016. Observation error propagation on video meteor orbit determination. Jenniskens, P., Gural, P., Dynneson, L., Grigsby, B., Newman, K., Borden, M., Koop, M., WGN, J. Int. Meteor Organ. 44 (2), 42–45. Holman, D., 2011. CAMS: Cameras for allsky meteor surveillance to establish minor meteor showers. Icarus 216 (1), 40–61. http://dx.doi.org/10.1016/j.icarus.2011.08. Spurný, P., Borovička, J., Shrbený, L., 2007. Automation of the czech part of the 012, URL https://www.sciencedirect.com/science/article/pii/S0019103511003290. European fireball network: equipment, methods and first results. In: Valsecchi, G.B., Jiang, P.-T., Zhang, C.-B., Hou, Q., Cheng, M.-M., Wei, Y., 2021. Layercam: Exploring Vokrouhlický, D., Milani, A. (Eds.), Near Earth Objects, Our Celestial Neigh- hierarchical class activation maps for localization. IEEE Trans. Image Process. 30, bors: Opportunity and Risk, Vol. 236. pp. 121–130. http://dx.doi.org/10.1017/ 5875–5888. http://dx.doi.org/10.1109/TIP.2021.3089943. S1743921307003146. Jopek, T.J., Williams, I.P., 2013. Stream and sporadic meteoroids associated with near- Subasinghe, D., Campbell-Brown, M., Stokan, E., 2017. Luminous efficiency estimates earth objects. Mon. Not. R. Astron. Soc. 430 (3), 2377–2389. http://dx.doi.org/10. of meteors -I. Uncertainty analysis. Planet. Space Sci. 143, 71–77. http://dx.doi. 1093/mnras/stt057. org/10.1016/j.pss.2016.12.009, arXiv:1704.08656. Kornoš, L., Koukal, J., Piffl, R., Tóth, J., 2014. EDMOND meteor database. In: Suk, T., Šimberová, S., 2017. Automated meteor detection by all-sky digital camera Gyssens, M., Roggemans, P., Zoladek, P. (Eds.), Proceedings of the International systems. Earth Moon Planets 120 (3), 189–215. http://dx.doi.org/10.1007/s11038- Meteor Conference, Poznan, Poland, 22-25 August 2013. pp. 23–25. 017-9511-z. 8 E. Peña-Asensio et al. Planetary and Space Science 238 (2023) 105802 Trigo-Rodríguez, J.M., 2019. The flux of meteoroids over time: meteor emission Vítek, S., Fliegel, K., Páta, P., Koten, P., 2011. MAIA: Technical development of spectroscopy and the delivery of volatiles and chondritic materials to earth. In: a novel system for video observations of meteors. Acta Polytech. 51 (1), http: Hypersonic Meteoroid Entry Physics. IOP Publishing, p. 4. http://dx.doi.org/10. //dx.doi.org/10.14311/1340. 1088/2053-2563/aae894ch4. Vítek, S., Nasyrova, M., 2019. Fast meteor tracking in noisy video sequences. Astron. Trigo-Rodríguez, J.M., 2022. Asteroid Impact Risk: Impact Hazard from Asteroids and Nachr. 340 (7), 646–651. http://dx.doi.org/10.1002/asna.201913670. Comets. Springer Nature, p. 126. Weryk, R.J., Campbell-Brown, M.D., Wiegert, P.A., Brown, P.G., Krzeminski, Z., Trigo-Rodríguez, J.M., Blum, J., 2022. Learning about comets from the study of mass Musci, R., 2013. The Canadian automated meteor observatory (CAMO): System distributions and fluxes of meteoroid streams. Mon. Not. R. Astron. Soc. 512 (2), overview. Icarus 225 (1), 614–622. http://dx.doi.org/10.1016/j.icarus.2013.04. 2277–2289. http://dx.doi.org/10.1093/mnras/stab2827, arXiv:2109.14428. 025. Trigo-Rodríguez, J.M., Llorca, J., Castro-Tirado, A.J., Ortiz, J.L., Docobo, J.A., Fabre- Weryk, R., Campbell-Brown, M., Wiegert, P., Brown, P., Krzeminski, Z., Musci, R., 2013. gat, J., 2006. The spanish fireball network. Astron. Geophys. 47 (2), 6.26–6.28. The Canadian automated meteor observatory (CAMO): System overview. Icarus 225 http://dx.doi.org/10.1111/j.1468-4004.2006.47626.x. (1), 614–622. http://dx.doi.org/10.1016/j.icarus.2013.04.025, URL https://www. Vaubaillon, J., Neslušan, L., Sekhar, A., Rudawska, R., Ryabova, G.O., 2019. From par- sciencedirect.com/science/article/pii/S0019103513001905. ent body to meteor shower: The dynamics of meteoroid streams. In: Ryabova, G.O., Wiegert, P., Brown, P., 2004. The problem of linking minor meteor showers to their Asher, D.J., Campbell-Brown, M.J. (Eds.), Meteoroids: Sources of Meteors on Earth parent bodies: initial considerations. Earth Moon Planets 95 (1–4), 19–26. http: and beyond. Cambridge University Press., p. 161. //dx.doi.org/10.1007/s11038-005-4342-8. Vida, D., Šegon, D., Gural, P.S., Brown, P.G., McIntyre, M.J.M., Dijkema, T.J., Xiao, H., Rasul, K., Vollgraf, R., 2017. Fashion-MNIST: A Novel Image Dataset Pavletić, L., Kukić, P., Mazur, M.J., Eschman, P., Roggemans, P., Merlak, A., for Benchmarking Machine Learning Algorithms. URL http://arxiv.org/abs/1708. Zubović, D., 2021. The global meteor network - methodology and first results. 07747, cs,stat. Mon. Not. R. Astron. Soc. 506 (4), 5046–5074. http://dx.doi.org/10.1093/mnras/ Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A., 2015. Learning deep fea- stab2008, arXiv:2107.12335. tures for discriminative localization. http://dx.doi.org/10.48550/arXiv.1512.04150, arXiv e-prints, arXiv:1512.04150, arXiv:1512.04150. 9