Bumblebees learn foraging routes royalsocietypublishing.org/journal/rsif through exploitation–exploration cycles Jackelyn M. Kembro1,2,3, Mathieu Lihoreau4, Joan Garriga3, Ernesto P. Raposo5 and Frederic Bartumeus3,6,7 Research 1 Universidad Nacional de Córdoba Facultad de Ciencias Exactas, Fı́sicas y Naturales, Instituto de Ciencia y Tecnologı́a de los Alimentos and Cátedra de Quı́mica Biológica, Córdoba, Argentina 2 Concejo de Invesigaciones Cientificas y Tecnologicas, Instituto de Investigaciones Biológicas y Tecnológicas, Córdoba, Argentina Cite this article: Kembro JM, Lihoreau M, 3 Centre d’Estudis Avançats de Blanes (CEAB-CSIC), Carrer Cala Sant Francesc 14, 17300 Blanes, Catalonia, Spain Garriga J, Raposo EP, Bartumeus F. 2019 4 Research Center on Animal Cognition (CRCA), Center for Integrative Biology (CBI); CNRS, University Paul Bumblebees learn foraging routes through Sabatier—Toulouse III, 31330 Toulouse, France 5 exploitation – exploration cycles. J. R. Soc. Laboratório de Fı́sica Teórica e Computacional, Departamento de Fı́sica, Universidade Federal de Pernambuco, 50670-901 Recife, Pernambuco, Brazil Interface 16: 20190103. 6 CREAF, Centre de Recerca Ecològica i Aplicacions Forestals, 08193 Bellaterra, Catalonia, Spain 7 http://dx.doi.org/10.1098/rsif.2019.0103 ICREA, Institut Català de Recerca i Estudis Avançats, 08010 Barcelona, Catalonia, Spain ML, 0000-0002-2463-2040; FB, 0000-0001-6908-3797 How animals explore and acquire knowledge from the environment is a key Received: 17 February 2019 question in movement ecology. For pollinators that feed on multiple small Accepted: 10 June 2019 replenishing nectar resources, the challenge is to learn efficient foraging routes while dynamically acquiring spatial information about new resource locations. Here, we use the behavioural mapping t-Stochastic Neighbouring Embedding algorithm and Shannon entropy to statistically analyse previously published sampling patterns of bumblebees feeding on artificial flowers in the field. We Subject Category: show that bumblebees modulate foraging excursions into distinctive behaviour- Life Sciences – Physics interface al strategies, characterizing the trade-off dynamics between (i) visiting and exploiting flowers close to the nest, (ii) searching for new routes and resources, Subject Areas: and (iii) exploiting learned flower visitation sequences. Experienced bees com- environmental science, computational biology bine these behavioural strategies even after they find an optimal route minimizing travel distances between flowers. This behavioural variability may Keywords: help balancing energy costs–benefits and facilitate rapid adaptation to changing trapline foraging, bumblebees, t-Stochastic environments and the integration of more profitable resources in their routes. Neighbouring Embedding, movement ecology, exploration –exploitation trade-off 1. Introduction Any search process, whether in space or mind, individual or collective, involves Author for correspondence: trade-offs between exploiting known opportunities and exploring for better Frederic Bartumeus options elsewhere [1–3]. The optimal balance between exploitation and explora- tion depends on the specificity of prior information and the quality of current e-mail: fbartu@ceab.csic.es observations [4]. How animals handle the exploitation–exploration trade-off for spatial decisions has broad implications for our understanding of sampling strat- egies and foraging behaviour. This question has received considerable theoretical interest [1–3] but has rarely been addressed experimentally due to the challenge of acquiring and analysing long-term individual movement data in the field. Some of the most intriguing animal foraging patterns can be observed in pollinators (orchid bees [5], bumblebees [6], honeybees [7], hummingbirds [8]) and some frugivorous vertebrates (monkeys [9], bats [10]) that exploit replenish- ing food resources scattered from a central nest or resting site. These animals often visit familiar feeding locations in predictable sequences known as ‘traplines’ [6]. A striking example is the case of bumblebees that collect nectar along traplines minimizing overall travel distances between multiple feeding locations (e.g. Electronic supplementary material is available flower patches, plants, trees) and the nest [11], a challenging optimization task analogous to the travelling salesman problem in graph theory [12]. online at https://dx.doi.org/10.6084/m9. figshare.c.4545515. & 2019 The Authors. Published by the Royal Society under the terms of the Creative Commons Attribution License http://creativecommons.org/licenses/by/4.0/, which permits unrestricted use, provided the original author and source are credited. Various heuristics have been proposed to explain this rout- sequences) in the dataset (see electronic supplementary 2 ing behaviour, from very simple rules of thumb (e.g. moving material, section S1). The data (previously published by Lihor- royalsocietypublishing.org/journal/rsif between nearest-neighbour flowers [13]) to more sophisticated eau et al. [23]) included all flower visitation sequences by focal learning algorithms (e.g. iterative improvement [14]). Only bumblebees using motion detection cameras placed on five recently have studies acknowledged the non-stationary dimen- artificial flowers arranged in a regular pentagon with 50 m sion of search processes [1], trapline foraging being a particular sides during a full day. Each flower contained a sucrose solution case where it has been shown that trial-and-error and stochas- reward equivalent to one-fifth of the crop of the bumblebee tic search rules are an essential part of route learning and (stomach) capacity and was refilled after each foraging bout stabilization [11,15,16]. Bumblebees may always maintain (foraging trip starting and ending at the nest entrance), in an exploratory behavioural component even after fixing move- order to encourage route development between all flowers. ment transitions between particular flowers [16] or a complete Herein, we analysed all foraging bouts with global statistical route [11]. Presumably this continuous exploratory strategy descriptors to build a behavioural landscape partitioned into allows for rapid adoption of new efficient routes following significant domains to identify elementary and repetitive environmental perturbation [17]. Bees are known to display types of foraging bouts. We then classified foraging bouts into J. R. Soc. Interface 16: 20190103 distinct spatial behaviours attributed to different navigational specific behavioural strategies, and characterized behavioural functions [18], such as learning flights [19,20] and orientation variability in terms of Shannon entropy and flight time prob- flights [19] to acquire view based memories of important ability distributions. Using this approach, our aims were locations (e.g. feeding sites and nest) on their first foraging (i) to identify exploration–exploitation behavioural strategies attempts, route-following to link familiar locations [20] and throughout the process of route formation and (ii) to investigate search loops when some of these locations are experimentally behavioural variability and its potential association with moved or when bees are displaced in new areas [21,22]. effective stochastic search and trial-and-error learning. While these movement patterns have been described in exper- imental scenarios specifically designed to study each strategy individually, how different behaviours are dynamically unfolded and used by bees during the process of route 2. Results formation is unknown. To understand how a bee combines these behavioural 2.1. Three main behavioural strategies characterize the strategies to sample flowers, and exploit vast, fragmented variability of foraging bouts and fast-changing foraging areas, we need to precisely measure Our characterization of the bumblebee flower visitation and characterize behavioural patterns. Part of the problem sequences defined a behavioural landscape delimited by three arises from the need to specify a key behavioural descriptor statistically significant domains (figure 1). Each domain rep- (e.g. transition probabilities between flowers, flight times, resented a dominant (and differentiated) movement mode, revisit frequencies) and finding the adequate spatio-temporal suggesting distinct behavioural strategies that we name: Near- scale of observation that accounts for meaningful behavioural nest visits (NNV), Route Development, and Traplining sequences. For instance, in the two studies that provide high- (figure 1 and table 1). The Route Development strategy resolution data on trapline formation by bumblebees, Lihoreau showed the largest and flattest area of the behavioural space et al. [23] quantified exploration based on the visual identifi- (figure 1), representing a highly variable behaviour. The cation of flight loops starting and ending at the same feeding Traplining strategy involved a much smaller area, with a location, whereas Woodgate et al. [16] considered flights in narrow and high peak (figure 1) indicative of highly repeatable which bees, starting at the vicinity of a flower, flew more and predictable behaviour. The NNV strategy resulted in an than 50 m from all previously discovered flowers. While both intermediate peak size and area coverage (figure 1), indicating metrics may ultimately reveal some tendency for exploration, both some level of variability and repeatability. There is the definition radically differs, affecting our views about how of course some relationship between the shape of the t-SNE animals produce and organize motor patterns. behavioural landscape and the observation window used to Unsupervised machine learning clustering methods (see characterize route development and learning processes by overview in the electronic supplementary material, section S1) bumblebees. However, here it is important to recall that a pre- offer the possibility to statistically map behavioural high- dominance (peak) in the t-SNE landscape depends on the dimensional spaces based on minimum assumptions [24]. relative similarities of the parametrized flower visitation pat- While this approach has been increasingly used to identify terns rather than on the frequency of the different patterns common behavioural principles across taxa, and describe the observed along the learning process. modularity and hierarchical organization of behaviour in lab- Skeleton diagrams illustrate the principal differences oratory set-ups using model organisms such as Caenorhabditis between the three behavioural strategies (figure 1). In the fora- elegans [1,25,26], Drosophila melanogaster [27,28] or zebrafish ging bouts classified as NNV, bumblebees visited flowers closer [29], it also holds considerable promises for analysing complex to the nest (figure 2a) in comparison to the other two strategies movement data from field studies, accounting for a much (figure 2b,c). These foraging bouts also showed lower levels more principled behavioural classification and avoiding of determinism (i.e. a measure of route repeatability [32], arbitrary binary descriptions. figure 2d), and larger mean turning angles (i.e. the mean of Here, we used an unsupervised l mapping method based on the absolute values of angles between three successively visited the t-Stochastic Neighbouring Embedding (t-SNE) algorithm flowers) in comparison to the other two strategies (figure 2e). In [24,27,30,31] to characterize the spatial behaviour of bumblebees NNV bouts, bees made seldom transition towards the flower (Bombus terrestris) foraging in semi-field conditions. The t-SNE furthest from the nest (table 1), and on average half of the tran- is a dimensionality reduction machine learning method suited sitions were performed towards nearest-neighbour flowers to identify repetitive multivariate patterns (or stereotypical (table 2). In the Route Development foraging bouts, bumblebees 3 3 royalsocietypublishing.org/journal/rsif 2 4 1 5 traplining route development N 3 2 4 J. R. Soc. Interface 16: 20190103 3 1 5 2 4 N NNV 1 5 N Figure 1. Behavioural landscape. Quantitative analysis of bumblebee foraging bouts in a large-scale set-up containing the nest-box (N) and 5 artificial flowers (1–5). Computation of the behavioural landscape of foraging bouts from seven bees based on a t-Stochastic Neighbouring Embedding (t-SNE) analysis. The heat map of t-SNE landscape reveals three behavioural strategies: NNV, Route Development and Traplining. The dominant behavioural mode characterizing each strategy is defined by a peak (black dot). The smaller the size of the clustered region and the larger the strength of the dominant peak, the less variable the foraging behaviour. The largest clustered area and the lowest peak correspond to the Route Development strategy, whereas the smallest clustered region and strongest dominant peak are associated with the Traplining strategy. Skeleton diagrams show representative foraging bouts (data from all bees pooled together) for each strategy. Only transitions between near flowers are represented for simplicity. White, dark grey and black double arrows represent transitions that occurred 0%, greater than or equal to 4% and greater than or equal to 7% of foraging bouts of each type, respectively. Grey numbers denote each flower. We used 11 variables for t-SNE analysis (table 1 and figure 2a): length of the flower visitation sequence, probability of immediate revisits to a flower, numbers of different flowers found and probability of visiting the flower furthest from the colony nest (flower 3), probability of symmetrical 2-flower transition types 3, 4 and 5 (elec- tronic supplementary material, figure S20), probability of 4-flower transition type 1 and 2, probability of 5-flower transition and the determinism index. (Online version in colour.) made more flower visits (table 1; electronic supplementary from the nest (figure 2a–c) can be considered an indicator of material, figure S17), travelled longer mean distances from the the travelling costs. In the NNV strategy, 91% of the foraging nest (figure 2b) and showed an intermediate mean turning bouts were characterized by a mean distance from nest of less angle (figure 2e) in comparison to the other two strategies. than the optimal 89 m (figure 2a, dotted black line), reflecting Bumblebees also made a larger number of nearest-neighbour a low-cost type of strategy. Accordingly, the Route Develop- transitions than the NNV strategy (table 2). Finally, in the ment strategy is the one showing a higher cost compared Traplining bouts, bumblebees performed almost exclusively with the other two, with 68% of the bouts at a mean distance near-neighbour transitions (table 2). Individuals showed a from nest beyond the optimal 89 m (figure 2b). high probability of performing three- or four-flower symmetri- Bumblebees spent more time on flowers in the Traplining cal transitions towards nearest neighbours (note that these were strategy than in the Route Development or NNV behavioural not performed in the other two behavioural strategies, table 1). strategies (table 2). They also showed a larger number of Consequently, bumblebees had significantly higher determin- different flowers visited and the lowest ratio between time ism and lower mean turning angles (figure 2d,e). Considering flying and time visiting a flower (table 2), a signature of that determinism was estimated for three consecutive foraging flower exploitation. In Route Development bouts, bumble- bouts, the high value observed for Traplining bouts indicated bees spent less time on the two flowers nearest to the nest repetition of the same pattern in consecutive foraging bouts, (flowers 1 and 5, figure 1) in comparison to the other two which is also reflected by the low variability observed in all strategies (table 2). In NNV bouts, bees showed a higher pro- the variables estimated (figure 2 and electronic supplementary portion of time on near-nest flowers relative to total bout time material, figure S17, tables S1 and S4). compared to the two other strategies (table 2). Interestingly, Bumblebees performing the Traplining strategy maintained bumblebees in NNV bouts showed a higher ratio between an average distance from the nest during flower visits of bout duration and the number of different flowers visited approximately 89 m in 78% of their foraging bouts (figure 2c), (cost of flying/reward) in comparison to the other strategies, which is equal to the average distance obtained during an opti- while intermediate values were observed in the Route mal sequence minimizing overall travel distances (N–1–2–3– Development strategy (table 2). These results suggest that 4–5–N or N–5–4–3–2–1–N, figure 1). The average distance bumblebees executing the NNV strategy explore their Table 1. Behavioural variables used in t-Stochastic Neighbouring Embedding (t-SNE) analysis for determining Near-nest visits (NNV), Route Development and 4 Traplining behavioural strategies. Note: Ten out of the 11 variables for t-SNE analysis are shown. The determinism index is depicted in electronic supplementary royalsocietypublishing.org/journal/rsif material, figure S2A. NNV Route Development Traplining p-value length of the flower visitation sequence 6a (5; 7) 8b (6; 13) 7b (7; 7) ,0.0001 a b c immediate revisits (%) 0 (0; 20) 14 (0; 24) 0 (0; 0) ,0.0001 number of flowers found 4a (3; 5) 5b (5; 5) 5b (5; 5) ,0.0001 probability of visiting farthest flower from the nest (%) 0a (0; 0) 20b (14; 27) 14c (14; 14) ,0.0001 a b b symmetrical 2-flower transition of type 3 (%) 0 (0; 0) 14 (8; 20) 17 (17; 17) ,0.0001 symmetrical 2-flower transition of type 4 (%) 0a (0; 0) 11b (0; 14) 17c (17; 17) ,0.0001 J. R. Soc. Interface 16: 20190103 symmetrical 2-flower transition of type 5 (%) 0a (0; 17) 0b (0; 8) 17c (17; 17) ,0.0001 4-flower transition type 1 (%) 0 (0; 0) 0 (0; 06) 25 (22; 25) — 4-flower transition type 2 (%) 0 (0; 0) 0 (0; 0) 25 (25; 25) — 5-flower transition (%) 0 (0; 0) 0 (0; 0) 25 (22, 25) — Median (Q1; Q3) a–c Strategies that do not share the same letter differ by p , 0.05 (Kruskal – Wallis). environment but preferentially exploit flowers near the nest. observation of longer flower visitation sequences (electronic However, we cannot exclude the possibility that bumblebees supplementary material, figure S14a) and higher number of also use exploration routes undetectable with flower visita- different flowers visited per bout (electronic supplementary tion sequence data, such as, for instance, long convoluted material, figure S14b) for Route Development bouts compared flight paths between visiting flowers. to NNV and Traplining bouts. At the beginning of the At the population level (i.e. when each of the foraging bouts experiment (when NNV bouts were prevalent), foraging of each of the seven bumblebees are pooled together), a clear bouts showed low entropy values (figure 2i,j ). These values time sequence is observed in the predominance of each of the increased at an intermediate stage of the experiment (when three strategies (figure 2h, see also arrows in figure 1), despite Route Development bouts were prevalent). At the end of the time not being made explicit in the t-SNE analysis. NNV experiment (when Traplining bouts were prevalent) Hft was bouts occurred predominately (approx. 70%, black line lower and Htrans was somewhat stabilized. Noteworthy, figure 2h) at the beginning of the learning process and decreased although calculations of Hfl and Htrans are based on different in frequency with experience. Traplining bouts started to variables, flight times and pairwise flower transitions, respect- appear after bout 10 (red line figure 2h). Interestingly, Route ively, a significant positive correlation exists between the two Development bouts co-occurred with the other two strategies, (y ¼ 1.175 þ 0.697x; r 2 ¼ 0.574). All in all, the entropic perspec- remaining relatively constant in percentage (between approx. tive suggests that NNV and Traplining strategies are more 20 and 50%, green line figure 2h) across time. At intermediate exploitative compared to the Route Development strategies times (bouts 15–22, figure 2h), the percentages of both NNV that shows a stronger exploratory component. The entropic and Traplining bouts were lower than the percentage of Route analysis is also concomitant with the fact that, in the t-SNE be- Development bouts. Thus, the percentage of Route Develop- havioural landscape (figure 1) the Route Development strategy ment bouts became relatively large compared with the other is characterized by a flatter and much larger area than the NNV two types of behavioural strategies at intermediate times. or Traplining strategy, suggesting the unfolding of a much richer behavioural repertoire and larger behavioural degrees of freedom at intermediate stages of the learning process. 2.2. Route development bouts lead to larger behavioural richness and entropic disorder 2.3. Exploitation–exploration dynamics is more For each behavioural strategy, we computed the Shannon information entropy (Materials and methods, equation (4.1)) complex at the individual than at the as an indicator of the level of uncertainty and the amount of population level degrees of freedom associated with flight time durations Although, on average, foraging patterns showed an ordered between flower visits (Hfl ) and pairwise transitions between sequence of behavioural strategies, each bumblebee did not flowers (Htrans). In general, Hft was significantly higher perform a completely ordered set of behavioural transitions. during Route Development bouts compared to NNV and Tra- On the contrary, a complex sequence of behaviours was plining bouts (figure 2c), which is consistent with the larger observed through time (figure 3a; electronic supplementary variability observed in flight times (electronic supplementary material, figures S15 and S18). For example, towards the end material, figure S13B) and the longer flower visitation of the experiment (after foraging bout 22) individual five per- sequences (electronic supplementary material, figure S13D). formed the sequence: Traplining (T)–Route Development Htrans also increased in Route Development bouts in compari- (R)–NNV (N)–T–R–T–T (figure 3a). Transitions through the son with NNV bouts, but not compared to Traplining t-SNE space are erratic (electronic supplementary material, bouts (figure 2d). These results are concordant with the figure S18), suggesting a non-deterministic learning process (a) NNV (b) route development (c) traplining 5 0.80 royalsocietypublishing.org/journal/rsif 0.75 probability 0.20 0.15 0.10 0.05 0 50 70 90 110 50 70 90 110 50 70 90 110 – – – d (m) d (m) d (m) J. R. Soc. Interface 16: 20190103 (d) (e) (f) (g) a 3.5 b 1.0 c 180 3.0 160 mean turning angle (°) 0.8 3.0 a b 140 a b b 2.5 determinism 0.6 120 Htrans Hfl 2.0 2.5 100 0.4 80 c 1.5 2.0 0.2 60 40 1.0 0 1.5 N R T N R T N R T N R T (h) NNV route dev traplining (i) (j) 2.8 100 2.0 80 2.6 bouts classified (%) 1.9 60 Htrans 1.8 2.4 Hfl – – 40 1.7 20 2.2 1.6 0 1.5 2.0 0 10 20 30 0 10 20 30 0 10 20 30 bout number bout number bout number Figure 2. Characterization of behavioural strategies and their unfolding across time. (a – c) Probability distribution of the mean distance to the nest (d) estimated as the mean distances expressed in metres between each flower visited during a foraging bout (including revisits) and the nest. Distinct distributions are observed in NNV, Route Development and Traplining behavioural strategies. Dotted black line represents an optimal trapline. (d – g) Boxplots show foraging bout variability of four variables for Near-nest visits (N), Route Development (R) and Traplining (T). (d) Determinism estimated for each period of three consecutive foraging bouts, and a minimum designated series length of three flowers, (e) mean turning angle (i.e. the mean of the absolute values of turning angles) and ( f,g) Shannon entropies (H ¼ 2Sp(x)log2p(x)), Hft where p is the relative flight duration (i.e. flight duration/duration of foraging bout), and Htrans where p is the probability of performing one of the 36 possible transitions between flowers and/or the nest. (d – g) Strategies that do not share the same letter differ by p , 0.05 (Tukey and Kramer (Nemenyi) test with Tukey-Dist approximation for independent samples). (h) Per cent of foraging bouts classified as belonging to each behavioural strategy as a function of the temporal sequence of bouts, computed over five foraging bouts overlapped running windows. Lines show a quadratic polynomial fit to data points of each foraging bout class. (i,j ) Median values of entropies Hft and Htrans for all seven bees, computed over five foraging bouts overlapped running windows. For illustrative purposes, we show quadratic polynomial fits to data points as lines. (Online version in colour.) and competing drives for exploration and exploitation, i.e. In addition to alternating between distinct behavioural strat- bumblebees balance the decision to continue exploiting close- egies, within a given strategy, individual bumblebees showed by flowers from the nest, performing an optimal trapline, or periodic fluctuations in entropy levels, both Htrans and Hfl seeking for other potential food sources. (figure 3a; electronic supplementary material, figure S15A). (a) 6 T royalsocietypublishing.org/journal/rsif R 2.6 N bee 1 0.8 T R 2.6 N bee 2 0.8 T R 2.6 behavioural strategies N bee 3 0.8 T Htrans (b) R 2.6 J. R. Soc. Interface 16: 20190103 N bee 4 0.8 T R 2.6 N bee 5 0.8 T R 2.6 N bee 6 0.8 T R 2.6 N bee 7 0.8 0 5 10 15 20 25 30 35 bout number (b) 4.6 Htrans (b + 1) 2.6 0.6 2.6 2.6 2.6 Htrans (b) Figure 3. Temporal evolution of behavioural strategies and entropy from naive to experienced bumblebees. (a) Time series of fluctuations between behavioural strategies Near-Nest Exploitation (N or NNV), Route Development (R) and Traplining (T) throughout route learning of each of the seven bumblebees evaluated. Grey squares represent the behavioural strategy (left axis) and lines represent the estimated entropy Htrans (right axis) of given bout. (b) Return plots of entropy Htrans estimated in two successive foraging bouts of the same behavioural strategy. Note the oscillatory nature of Htrans during NNV and Route Development bouts where higher values are often followed by lower entropies. Similar results are shown for Hfl in electronic supplementary material, figure S15. Moreover, NNV bout oscil- lations occurred mainly between 0.8 and 2.59, while between Route Development bouts a larger variability was observed mainly between 1.7 and 4.2. By contrast, during Traplining fairly constant values of Htrans were observed near the optimum value 2.6 (dotted lines). (Online version in colour.) Table 2. Key variables associated with behavioural strategies in bumblebees as determined by t-Stochastic Neighbouring Embedding (t-SNE) analysis (i.e. Near- nest visits (NNV), Route Development and Traplining). NNV Route Development Traplining p-value nearest-neighbour transitions (%) 54a (40; 67) 60a (44; 75) 100b (91; 100) ,0.0001 time visiting flower (s) 82a (52; 130) 110b (77: 149) 145c (108; 183) ,0.0001 per cent of bout time visiting flower (%) 20a (11; 33) 24a (12; 33) 40b (32; 49) ,0.0001 time visiting near-nest flowers (s) 45a (25; 79) 29b (22; 55) 44a (37; 72) 0.01 per cent of visits to near-nest flowers (%) 60a (50; 80) 33b (25; 50) 40b (40; 40) ,0.0001 b b a time flying/time visiting 5.5 (2.2; 11.7) 3.8 (2.2; 9.1) 2.0 (1.2; 4.4) ,0.0001 bout duration/number of different flowers 155a (102; 243) 114b (84; 197) 66c (57; 100) ,0.0001 Median (Q1; Q3) a–c Variables that do not share the same letter differ by p , 0.05 (Kruskal – Wallis test). Most evidently, in the NNV and Route Development strategies, Radar data analysis (electronic supplementary material, §3) 7 large entropies were frequently followed by low entropies, also reveals fine-grained behavioural changes by bumblebees royalsocietypublishing.org/journal/rsif suggesting that bumblebees alternated excursions where they when spreading out from flowers as experience progresses. visited large and diverse sets of flowers with much simplified This result reinforces the idea that different flower-visiting foraging bouts involving small and less diverse sets of flowers strategies are unfolded as bumblebees retain information (electronic supplementary material, figure S14A,B,D,E). about flower location and potential optimal routes. Our Although noisy, this oscillatory dynamics (electronic sup- approach shows that foraging bout variability is naturally plementary material, figure S19) was also evidenced in return grouped, according to bumblebee behavioural signatures, in plots (figure 4b and electronic supplementary material, figure three possible strategies. Our method signifies behavioural S15B), where the value of entropy estimated at a given foraging stereotypes and avoids unnecessary data pooling or arbitrarily bout (b þ 1) was plotted as a function of the previous foraging grouping often used in comparative behavioural analyses. In bout (b) of the same behavioural strategy. In the case of NNV the NNV strategy, bumblebees mainly performed strong turn- bouts, oscillations in Htrans occurred mainly in the range of 0.8 ing angles, as a way to restrict their initial foraging bouts in and 2.6 (left quadrant in figure 3b), while in Route Development space, and by revisiting nearby subsets of flowers, which J. R. Soc. Interface 16: 20190103 bouts a larger variability was observed and values ranged were known to be rewarding. This pattern may simply mainly between 1.7 and 4.2 (middle quadrant in figure 3b, emerge because bumblebees take time to find flowers and centred around the optimal value 2.59). By contrast, in Traplin- acquire knowledge of their spatial locations. Although in the ing bouts we observed almost constant values of Htrans close to NNV strategy resource exploitation near the nest is prevalent, the optimum value 2.59 (figure 3b). Similar results were some excursions to flowers far from the nest were also observed in return plots for Hfl. Nonetheless, for the later observed, consistent with Woodgate et al. [16]. The Route entropy Traplining bouts showed a much wider range of Development strategy showed larger behavioural degrees of values in the return plot compared to Htrans (electronic sup- freedom (i.e. t-SNE area), and a larger behavioural entropy in plementary material, figure S15B). Therefore, within each flower transitions and flight times between visits in compari- behavioural strategy bumblebees showed low–high variability son to the other two strategies. Therefore, while exploring, cycles in terms of both flower transitions and flight durations bumblebees are able to locate all the flowers and stabilize rel- between visits. Only in the Traplining strategy, flower evant routes at the cost of using more variable and transitions did not show this pattern as they were mostly fixed. suboptimal (long) paths. Albeit being potentially highly ener- Following [33], we extended our entropy analysis to study getically costly [34,35], a stronger variability in behavioural routine movement behaviour across time by estimating the strategies may facilitate trial-and-error learning and improve pth-order conditional entropy, Hp. Similar to the idea of deter- discovery rates, speeding up convergence to an optimal minism [32], ‘routine movement behaviour’ involves some route. After approximately 10 foraging bouts, the Traplining degree of regularity in the movement sequence pattern, which strategy emerged, involving repetitive and ordered visitation can be defined as the opposite of uncertainty. Our results (elec- sequences to all the flowers in the array. This strategy was tronic supplementary material, figure S21) are consistent with characterized by a strong and localized peak in the behavioural the other estimators used in this work (i.e. Shannon entropy, landscape (figure 1). determinism), the first-order conditional entropy values fluctu- Nearest-neighbour transition rules (i.e. systematic move- ate at the short-time scale, but decrease throughout route ment to the nearest unvisited target) have often been learning. The short-scale fluctuations suggest that bumblebees suggested as a relevant behavioural mechanism for route devel- alternate the exploitation of repeatable flower visit sequences opment and fixation by bees [16,36] (but see [11]). Indeed, in the with the exploration of new sequences, despite on average NNV and Route Development strategies, nearest-neighbour routine movement behaviour increasing steadily with time. In transitions represent about half of the transitions between two any case, the routine movement behaviour analysis should be flowers (see Symmetrical transitions in electronic supplemen- taken cautiously due to limitations in data structure and length tary material, table S4). This result is consistent with the (see caption at electronic supplementary material, figure S21). observations of Woodgate et al. [16] in a different array of flowers Finally, we also analysed the two-dimensional flight tra- where nearest-neighbour transitions always led to suboptimal jectories of bumblebees in some of the foraging bouts routes, which was not the case in our regular pentagonal array where individuals were also tracked with a harmonic radar of flowers. Therefore, experiments involving more complex (electronic supplementary material, §3). The results suggest spatial configurations of flowers in which nearest-neighbour that different flower-sampling behaviours are used through transitioning does not necessarily lead to short routes (e.g. time, as bumblebees gained experience on the experimental [11,16]) are needed to clarify the importance of the nearest- system (electronic supplementary material, §3). neighbour behaviour for route formation and optimization. At the population level, a clear temporal sequence from NNV to Route Development and Traplining strategies emerged, 3. Discussion with Route Development essentially co-occurring through time, We used an unsupervised mapping approach to characterize especially at intermediate training stages (figure 2h). At the indi- and analyse behavioural strategies of bumblebees while devel- vidual level, however, the strategy transition dynamics was less oping foraging routes between multiple resources in a large straightforward. For example, the NNV strategy, although pre- semi-field set-up and show that bumblebees develop routes dominating at the beginning, appeared recurrently throughout by alternating exploration (Route Development) and exploita- the route learning process as well, suggesting complex individ- tion (NNV and Traplining) cycles. Trial–error excursions ual decision-making (figure 3 and electronic supplementary (Route Development) were alternated with the exploitation of material, figure S15; see also the routine movement analysis in resources near the nest (NNV) and, in more advanced stages, electronic supplementary material, figure S21). The Traplining with the utilization of stable multi-location routes (Traplining). strategy started to appear after foraging bout number 10; ×10–3 8 (a) (b) 14 1.0 royalsocietypublishing.org/journal/rsif 12 10 0.8 8 6 4 0.6 costs (D̂) 2 0.4 1.0 0.2 0.5 ) ts (Dˆ 0 cos J. R. Soc. Interface 16: 20190103 0 0.5 0 0 0.2 0.4 0.6 0.8 1.0 benefits 1.0 (Ĥtr benefits (Ĥ trans) ans ) (c) (d) 1.0 bee 5 bee 4 0.8 0.6 costs (D̂) 0.4 0.2 0 0 0.2 0.4 0.6 0.8 1.0 0 0.2 0.4 0.6 0.8 1.0 benefits (Ĥ trans) benefits (Ĥ trans) (e) experimental set-up nature cost/benefit T0 T3 Toptimal T1 T2 T4 behavioural states Figure 4. Behavioural transition from naive to experienced bumblebees depicted in a cost –benefit landscape, and contextualized in simple and complex fitness landscapes for optimal routing. (a) The three behaviourally distinct strategies of figure 1 depicted in a cost – benefit landscape (distance from nest ^D versus explora- tory activity ^Htrans ). Near-Nest Exploitation (NNV, black open circles) represents low distances from nest and low exploratory activity, whereas Route Development (red open circles) represents large costs associated with being far from the nest and large exploratory activity relative to Traplining (green open circles). (b) Three- dimensional representation of a two-dimensional kernel density estimation of the population-level frequency of foraging bouts with a given amount of cost– benefit exposure (data of all bees pooled together). The peak represents optimized trapline foraging behaviour. Note also the smaller plateaus in the lower-left and upper-right quadrants, representing opposite behavioural strategies, surrounding optimal trapline foraging. (c,d) Examples of two-dimensional kernel density population-level representations with the individual sequences of foraging bouts for two different bees. (e) The different possible behavioural states an animal can adopt can be associated with a cost/benefit ratio. In this context, an optimal trapline minimizing travel distances can be associated with a local minimum of a cost/benefit. Attractors (local cost/benefit minimums) are constantly perturbed due to environmentally driven or induced behavioural variability, that may come from imperfect memory or motor control at some level, casting the forager out of the local minimum to another ( potentially better) minimum. Depending on the overall behavioural state landscape dynamics can be stable or unstable attractor points, so that bees may end up returning to specific spatial configuration (as is the case of the experimental set-up) or else never come back, a process endured by a never-ending exploratory process only constrained by the distance to the nest. (Online version in colour.) however, bumblebees tended to return to Route Development environments due to variability in resource replenishment rates 9 strategies during subsequent foraging bouts. This result is in [37,38], competition [39–41], changing weather conditions royalsocietypublishing.org/journal/rsif line with previous observations suggesting that once routes [42,43] and the presence of predators [44]. In such conditions, are stabilized, bumblebees often make further foraging bouts constant exploration might be crucial to track new options with alternative visitation sequences [11,16]. Imperfect learning and efficiently adjust foraging routes. Hence, traplines may and memory could favour these back-and-forth behavioural become unstable attractors, subject to constant perturbations shifts in experienced foragers. These cycles between exploitation both environmentally or behaviourally induced, caused either and exploration strategies could also reflect an inherent behav- by imperfect learning or by the need of new discoveries. Our ioural strategy, that is, the production of controlled motor results suggest that departures from specific traplines (local variability alternating the discovery of new flowers and flower minima) could rely on enhanced spreading dynamics out of sequences with the exploitation of routes already fixed [16,23]. particular flowers or traplines, resembling fast simulated There are considerable, unexplored connections between the annealing, as suggested in Lihoreau et al. [15]. current study and that of Bartumeus et al. [1] related to the fact Quantifying pollinator behaviour has historically been diffi- that search is essentially an organized, non-stationary stochastic cult [45] and long raised the need to develop metrics for J. R. Soc. Interface 16: 20190103 process, where exploration and exploitation processes of any describing and comparing their sophisticated spatial patterns kind must alternate. Provided that the environment is stable (see discussions in [6,32,46]). The use of unsupervised mapping enough for the animal to gain information about it, the learning methods on bumblebee data allowed us to quantify emerging process itself becomes a strong driver of behavioural change that flower-sampling strategies and individual behavioural tran- should be reflected in the motor output, through adjustments sitions (exploitation–exploration) during route learning, both between internal states, memory and discovery rates. from a bottom-up approach and at an unprecedented level To reinforce the idea that stochastic search does not rep- of detail. Although not entirely free of assumptions, this resent fully unstructured noise, we showed that the statistical methodology is statistically robust and better principled (well- signatures found for the Route Development strategy are not grounded on information theory) than previous approaches. obtained by adding a simply structured noise to the optimal Beyond research on pollinators, we believe that such approach route (electronic supplementary material, §4). As an example, will be instrumental for future quantitative research in the addition of Gaussian noise to optimized traplines changed movement and behavioural ecology, guided by powerful some of our behavioural descriptors according to expectation, assessments and comparisons of behavioural variability, both as it: (i) increased the variability between foraging bouts (e.g. structure and dynamics, in field conditions. entropy Htrans and number of flower visits; electronic sup- plementary material, figure S16C and S16D, respectively), (ii) decreased determinism, and (iii) increased the turning 4. Material and methods angles (electronic supplementary material, figure S16A and S16B). However, Gaussian-driven stochasticity could not 4.1. Subjects and study site The experimental set-up is described in detail by Lihoreau et al. [23]. explain (i) the anomalous diffusion observed in the local A bumblebee colony was installed in a flat, open area of mown pas- spreading out of the flowers, (ii) the large amount of flight dur- ture (51.8051498 N 0.3650808 W) free of natural flowers. The ations beyond the characteristic time scales associated with the entrance tube of the colony box was equipped with a series of experimental set-up, nor (iii) the wild heterogeneity (multiple gates in order to control the traffic of foragers. Bumblebees were scales) and scaling properties observed on flight time only allowed to leave the colony during the pre-training and train- distributions (electronic supplementary material, figure S13). ing periods. During pre-training, all foragers could freely explore To visualize the costs and benefits of the overall foraging the outside environment and collect sucrose solution (40% w/w) strategy of bumblebees, we parametrized each foraging bout from five artificial flowers arranged in a linear array (150 cm in terms of the average distance to the nest (cost related to the length) 50 m northwest of the nest entrance. Flowers were refilled ad libitum with 10 ml of sucrose solution to estimate the crop energy expenditure and error accumulation when flying) and capacity of each bumblebee. Pre-training typically lasted one day. the entropy associated with flower transitions (well correlated Regular foragers that made at least five foraging bouts in 1 h were with the Route Development activity and the benefit of finding selected for testing. During testing, only one forager was allowed new flowers or routes) relative to optimized traplining behav- to leave the colony. This focal bumblebee was observed foraging iour (figure 4). We found that NNV, Route Development and on flowers arranged in a regular pentagon (figure 1). The bumble- Traplining behavioural strategies can be mapped following a bee was therefore familiar with the outside environment but gradient from low cost–low benefit (NNV) to large cost– unfamiliar with the array of flowers. Each flower was at a distance large benefit scenarios (Route Development; figure 4a,b). Tran- of 50 m from two neighbour flowers, 80 m from the opposite flow- sitions from naive to more experienced foraging phases result in ers and 108 m from the most distant flower. These distances were very erratic paths in the depicted cost–benefit landscape, finally chosen to promote exploration. Given that B. terrestris workers are leading to Traplining (figure 4c,d). This reflects the presence of able to detect reflecting (non-self-luminant) targets from a back- ground subtending a visual angle of ca 38 [47] we assume that individual trade-offs and alternation of behaviourally distinct bees could detect our flowers from a maximal distance of 13 m sampling strategies, likely in part related to trial-and-error from any location in the experimental field. Each flower contained route learning and optimization. a sucrose reward equivalent to one-fifth of the crop capacity of If we consider route learning as a dynamical behavioural the focal bumblebee and was refilled after each foraging bout. system (figure 4e), the Traplining strategy would be the main Testing typically lasted one day. Each bumblebee was tested on attractor. In our simplified experimental context, where all flow- different days. ers were equally rewarding and replenished after each foraging bout, the optimal trapline is a stable attractor. In most natural 4.2. Flower-sampling sequences conditions, however, bumblebees may experience much more Seven individuals were trained in the pentagonal array until they complex cost–benefit (fitness) landscapes, with highly unstable found all flowers and visited them in the same sequence in at least three consecutive foraging bouts (6.7 + 1.0 (s.e.) hours of obser- In our case we estimated Htrans, where p was the probability 10 vation and 27.6 + 1.8 foraging bouts per bumblebee). During the of performing one of the 36 possible transitions between flowers tests, all departure and arrival times at the nest were recorded by and/or nest, and Hft, where p was the relative flight duration (i.e. royalsocietypublishing.org/journal/rsif the experimenter and were considered as all the times in which flight duration/duration of a foraging bout). the bee left and then returned to the colony during testing. Flower visits were recorded using motion-activated webcams — Determinism: metric for measuring sequence predictability (Logitech c250, Fremont, CA) mounted above each flower. Video by quantifying the number and length of recurrences and clips provided arrival and departure times from each flower, series of recurrences. In the case of trapline foraging, a recur- from which we reconstructed the flower visitation sequences in rence occurs whenever a forager revisits a resource [32]. Thus, each foraging bout (raw data available in Lihoreau et al. [23]; elec- a recurrent series occurs when sequence elements are tronic supplementary material, table S4). The following variables repeated in the same order (in either forward or reverse direc- were calculated: tions). Determinism is based on the proportion of recurrences (i.e. revisited behavioural sequences) that belong to a recur- — Length of the flower visitation sequence: number of flower rent series of a minimum designated length. It thus visits during a foraging bout. represents the proportion of revisited flowers that were vis- J. R. Soc. Interface 16: 20190103 — Flight duration (s): latency time between two successive visits ited in the same continuous order in multiple parts of the to a different or the same flower. visitation sequence. For a detailed description of the — Probability of visiting a given flower during a given foraging method applied to trapline foraging, refer to Ayers et al. bout. [32]. In order to assess recurrence of sequences in consecutive — Probability of revisiting the same flower during a foraging bouts (indicative of memory acquisition), we estimated deter- bout, including consecutive revisits to the same flower. minism every three consecutive bouts, and a minimum — Number of different flowers: number of different flowers designated length of three flowers was used. visited during a foraging bout (from 1 to 5). — Cumulative number of distinct flowers that the bee encoun- tered since the beginning of the experiment (flowers found). 4.3. Unsupervised behavioural mapping — Distance from nest (m): mean distance between each flower vis- We built our input dataset with 11 behavioural variables, ited during a foraging bout (including revisits) and the nest. namely: (1) the length of the flower visitation sequence, (2) the This was estimated taking into consideration that flower 1 per cent of immediate revisits to a flower, (3, 4, 5) the per cent was placed at 53.1 m, flower 2 at 105.7 m, flower 3 at of symmetrical 2-flower transitions (types 3, 4 and 5; electronic 126.2 cm, flower 4 at 104.4 and flower 5 at 53.4 cm from nest. supplementary material, figure S20A), (6,7) the per cent of both Thus, for an optimal trapline bout (minimizing overall travel types of 4-flower transition, (8) the per cent of 5-flower transition, distances) the mean distance from the nest during the bout is (9) the determinism index (a metric for detection of recurrent pat- approximately 89 m ((53.1 þ 105.7 þ 126.2 þ 104.4 þ 53.4)/5). terns in a behavioural sequence [36]), (10) the number of flowers — Trapline: the most common five-flower visitation sequence, found and (11) the probability of visiting flower 3 (the farthest excluding revisits, used by each bee [17]. flower from the colony nest, figure 1). — Probability of all 36 possible transitions between the five flow- We made an unsupervised mapping of the input dataset by ers, and between the flowers and the nest. Given the means of the three step mapping protocol described in Garriga & symmetrical distribution of the flowers, transitions were classi- Bartumeus [30], publicly available using the bigMap R-package fied according to equivalent path information (see graphical [49] and explained in the electronic supplementary material, representation in the electronic supplementary material, §1. This mapping protocol is mainly governed by a parameter figure S20). For example, a transition from the nest to flower 1 called ‘perplexity’ comparable with the number of nearest neigh- would be equivalent to the transition from the nest to flower bours that is employed in many manifold learners. To ensure the 5, given that both flowers are positioned at the same distance robustness of the results, we tested different values of perplexity from the nest and at the same relative angle, but in opposite (i.e. 30, 60, 90 with respect to a dataset size of N ¼ 193). For each direction. We also estimated 3-, 4- and 5-flower transition prob- test, we performed three different runs. For each run, we checked abilities. For example, 3-flower symmetrical transition type 1 all possible sets of input features from 2 (i.e. the first two princi- contains the transition from the nest to flower 1 followed by pal components) to 11 (i.e. the whole set of principal flower 2 (N– 1 – 2), and the transition from the nest to flower 5 components). After a thorough analysis of all the runs we ident- followed by flower 4 (N – 5 – 4). Four-flower symmetrical tran- ified a robust classification of the foraging bouts into three sition type 1 contains transitions N – 1 – 2– 3 and N – 5 – 4 – 3, clusters with a clear semantics, namely NNV, Route Develop- and type 2 the transitions 1 – 2 – 3 – 4 and 5 – 4 – 3 – 2. The 5- ment and Traplining. A detailed explanation of the procedure, flower symmetrical transition considers transitions N– 1 – 2 – including variable and parameter selection is provided in elec- 3 – 4 and N – 5 – 4 – 3 – 2. tronic supplementary material, S1. To highlight the biological — Turning angles: angles estimated from x- and y-position coor- significance and robustness of the behavioural strategies found dinates of three successively visited flowers as the difference in our study, we compared our results with another dimension- between the arrival direction and the departure direction at ality reduction method (i.e. non-negative matrix factorization) the middle flower [36]. Hence, 08 indicates a move straight discussed in the electronic supplementary material, S2. ahead, and 1808 a complete reversal in direction. Noteworthy, the five flowers were arranged so that choices of nearest 4.4. Statistical analyses neighbours were always consistent with choices of straightest We used the Kruskal and Wallis one-way analysis of variance by movements (i.e. lowest turning angle). Mean turning angle ranks to assess differences in foraging bout variables between be- was estimated as the mean of the absolute values of turning havioural strategies. We performed pairwise comparisons using angles of a foraging bout. Tukey and Kramer (Nemenyi) test with Tukey-Dist approxi- — Shannon entropy (H ): represents the uncertainty of a variable mation for independent samples test, using the functions [48]. For a probability p(x), where x is a variable capturing the kruskal.test and posthoc.kruskal.nemenyi.test in the PMCMR behavioural strategy of each bee, it is defined by R-package [50]. For graphical representations, we showed box- plots with median and 25% and 75% quantiles. X (X) ¼  p(x) log2 p(x): ð4:1Þ Data accessibility. This article has no additional data. Competing interests. We declare we have no competing interests. CONICET. M.L. was funded by the Centre National de la 11 Funding. This work was financially supported by grant no. Recherche Scientifique (CNRS) and the Agence Nationale de la Recherche (ANR-16-CE02-0002-01). E.P.R. thanks the support royalsocietypublishing.org/journal/rsif CGL2016-78156-C2-1-R, MINECO, Programa Estatal IþDþi, Retos de la Sociedad, in Spain. J.M.K. is a career member of from FACEPE and CNPq. References 1. Bartumeus F, Campos D, Ryu WS, Lloret-Cabot R, formation by bumblebees across spatial scales and 26. Stephens GJ, Johnson-Kerner B, Bialek W, Ryu WS. Mendez V, Catalan J. 2016 Foraging success under flower arrangements. PLoS Comput. Biol. 9, 2008 Dimensionality and dynamics in the behavior uncertainty: search tradeoffs and optimal space use. e1002938. (doi:10.1371/journal.pcbi.1002938) of C. elegans. PLoS Comput. Biol. 4, e1000028. Ecol. Lett. 19, 1299 –1313. (doi:10.1111/ele.12660) 15. Lihoreau M, Ings TC, Chittka L, Reynolds AM. 2016 (doi:10.1371/journal.pcbi.1000028) 2. Bartumeus F, Raposo EP, Viswanathan GM, da Luz Signatures of a globally optimal searching strategy 27. Berman GJ, Choi DM, Bialek W, Shaevitz JW. 2014 J. R. Soc. Interface 16: 20190103 MG. 2014 Stochastic optimal foraging: tuning in the three-dimensional foraging flights of Mapping the stereotyped behaviour of freely intensive and extensive dynamics in random bumblebees. Sci. Rep. 6, 30401. (doi:10.1038/ moving fruit flies. J. R. Soc. Interface 11, 20140672. searches. PLoS ONE 9, e106373. (doi:10.1371/ srep30401) (doi:10.1098/rsif.2014.0672) journal.pone.0106373) 16. Woodgate JL, Makinson JC, Lim KS, Reynolds AM, 28. Vogelstein JT, Park Y, Ohyama T, Kerr RA, Truman 3. Hills TT, Todd PM, Lazer D, Redish AD, Couzin ID. 2015 Chittka L. 2017 Continuous radar tracking illustrates JW, Priebe CE, Zlatic M. 2014 Discovery of brainwide Cognitive search research G. Exploration versus the development of multi-destination routes of neural-behavioral maps via multiscale unsupervised exploitation in space, mind, and society. Trends Cogn. bumblebees. Sci. Rep. 7, 17323. (doi:10.1038/ structure learning. Science 344, 386–392. (doi:10. Sci. 19, 46–54. (doi:10.1016/j.tics.2014.10.004) s41598-017-17553-1) 1126/science.1250298) 4. McNamara JM, Green RF, Olsson O. 2006 Bayes’ 17. Lihoreau M, Chittka L, Raine NE. 2010 Travel 29. Marques JC, Lackner S, Félix R, Orger MB. 2018 theorem and its application in animal behaviour. optimization by foraging bumblebees through Structure of the zebrafish locomotor repertoire Oikos 112, 243 –251. (doi:10.1111/j.0030-1299. readjustments of traplines after discovery of new revealed with unsupervised behavioral clustering. 2006.14228.x) feeding locations. Am. Nat. 176, 744 –757. (doi:10. Curr. Biol. 28, 181–195. (doi:10.1016/j.cub.2017. 5. Janzen DH. 1971 Euglossine bees as long-distance 1086/657042) 12.002) pollinators of tropical plants. Science 171, 18. Menzel R. 2011 Navigation and communication in 30. Garriga J, Bartumeus F. 2018 bigMap: big data 203–205. (doi:10.1126/science.171.3967.203) honeybees. In Animal thinking: contemporary issues mapping with parallelized t-SNE. (http://arxiv.org/ 6. Thomson JD, Slatirin M, Thomson BA. 1997 Trapline in comparative cognition (eds R Menzel, E Julia abs/1812.09869) foraging by bumble bees. II. Definition and Fischer), pp. 9 –21. New York, NY: MIT Press. 31. van der Maaten LJP, Hinton GE. 2008 Visualizing detection from sequence data. Behav. Ecol. 8, 19. Capaldi EA et al. 2000 Ontogeny of orientation high-dimensional data using t-sne. J. Mach. Learn. 199–210. (doi:10.1093/beheco/8.2.199) flight in the honeybee revealed by harmonic radar. Res. 9, 2579–2605. 7. Buatois A, Lihoreau M. 2016 Evidence of trapline Nature 403, 537–540. (doi:10.1038/35000564) 32. Ayers CA, Armsworth PR, Brosi BJ. 2015 foraging in honeybees. J. Exp. Biol. 219, 20. Collett M, Chittka L, Collett TS. 2013 Spatial Determinism as a statistical metric for ecologically 2426–2429. (doi:10.1242/jeb.143214) memory in insect navigation. Curr. Biol. 23, important recurrent behaviors with trapline foraging 8. Tello-Ramos MC, Hurly TA, Healy SD. 2015 R789 –R800. (doi:10.1016/j.cub.2013.07.020) as a case study. Behav. Ecol. Sociobiol. 69, Traplining in hummingbirds: flying short-distance 21. Reynolds AM, Smith AD, Menzel R, Greggers U, 1395– 1404. (doi:10.1007/s00265-015-1948-3) sequences among several locations. Behav. Ecol. 26, Reynolds DR, Riley JR. 2007 Displaced honey bees 33. Riotte-Lambert L, Benhamou S, Chamaillé-Jammes S. 812–819. (doi:10.1093/beheco/arv014) perform optimal scale-free search flights. Ecology 2017 From randomness to traplining: a framework for 9. Janson CH. 1998 Experimental evidence for route 88, 1955–1961. (doi:10.1890/06-1916.1) the study of routine movement behavior. Behav. Ecol. integration and strategic planning in wild capuchin 22. Riley JR, Greggers U, Smith AD, Reynolds DR, 28, 280–287. (doi:10.1093/beheco/arw154) monkey, Cebus paella. Anim. Behav. 55, Menzel R. 2005 The flight paths of 34. Heinrich B. 1975 Thermoregulation in bumblebees 1229–1243. (doi:10.1006/anbe.1997.0688) honeybees recruited by the waggle dance. II: energetics of warm-up and free flight. J. Comp. 10. Lemke TO. 1984 Foraging ecology of the long-nosed Nature 435, 205 – 207. (doi:10.1038/nature Physiol. A 96, 155 –166. (doi:10.1007/BF00706595) bat Glossophaga soricina with respect to food 03526) 35. Casey TM, May ML. 1985 Flight energetics of availability. Ecology 65, 538–548. (doi:10.2307/ 23. Lihoreau M, Raine NE, Reynolds AM, Stelzer RJ, Lim euglossine bees in relation to morphology and wing 1941416) KS, Smith AD, Osborne JL, Chittka L. 2012 Radar stroke frequency. J. Exp. Biol. 116, 271–289. 11. Lihoreau M, Chittka L, Comber SCL, Raine NE. 2012 tracking and motion-sensitive cameras on flowers 36. Ohashi K, Thomson JD, D’Souza D. 2007 Trapline Bees do not use nearest neighbour rules for reveal the development of pollinator multi- foraging by bumble bees: IV. Optimization of route optimization of multi-location routes. Biol. Lett. 8, destination routes over large spatial scales. PLoS geometry in the absence of competition. Behav. 13 –16. (doi:10.1098/rsbl.2011.0661) Biol. 10, e1001392. (doi:10.1371/journal.pbio. Ecol. 18, 1 –11. (doi:10.1093/beheco/arl053) 12. Applegate DL, Bixby RE, Chvátal V, Cook WJ. 2007 1001392) 37. Castellanos MC, Wilson P, Thomson JD. 2002 The traveling salesman problem: a computational 24. Todd JG, Kain JS, de Bivort BL. 2017 Systematic Dynamic nectar replenishment in flowers of study. Princeton Series in Applied Mathematics. exploration of unsupervised methods for mapping Penstemon (Scrophulariaceae). Am. J. Bot. 89, Princeton, NJ: Princeton University Press. behavior. Phys. Biol. 14, 015002. (doi:10.1088/ 111–118. (doi:10.3732/ajb.89.1.111) 13. Saleh N, Chittka L. 2007 Traplining in bumblebees 1478-3975/14/1/015002) 38. Luo EY, Ogilvie JE, Thomson JD. 2014 Stimulation of (Bombus impatiens): a foraging strategy’s ontogeny 25. Brown AE, Yemini EI, Grundy LJ, Jucikas T, flower nectar replenishment by removal: a survey of and the importance of spatial reference memory in Schafer WR. 2013 A dictionary of behavioral eleven animal-pollinated plant species. J. Pollinat. short-range foraging. Oecologia 151, 719–730. motifs reveals clusters of genes affecting Ecol. 12, 52 –62. (doi:10.26786/1920-7603(2014)2) (doi:10.1007/s00442-006-0607-9) Caenorhabditis elegans locomotion. Proc. Natl 39. Lihoreau M, Chittka L, Raine NE. 2016 Monitoring 14. Reynolds AM, Lihoreau M, Chittka L. 2013 A simple Acad. Sci. USA 110, 791 – 796. (doi:10.1073/pnas. flower visitation networks and interactions between iterative model accurately captures complex trapline 1211447110) pairs of bumble bees in a large outdoor flight cage. PLoS ONE 11, e0150844. (doi:10.1371/journal.pone. bumblebees in an environment with patchy light. interactions with spatial movement networks. 12 0150844) J. Exp. Biol. 215, 2173–2180. (doi:10.1242/jeb.065565) Ecol. Entomol. 42, 4 – 17. (doi:10.1111/een. royalsocietypublishing.org/journal/rsif 40. Makino TT, Sakai S. 2005 Does interaction between 43. Dyer AG, Whitney HM, Arnold SE, Glover BJ, Chittka 12446) bumblebees (Bombus ignitus) reduce their foraging L. 2006 Behavioural ecology: bees associate warmth 47. Kapustjansky A, Chittka L, Spaethe J. 2010 Bees use area? Bee-removal experiments in a net cage. with floral colour. Nature 442, 525. (doi:10.1038/ three-dimensional information to improve target Behav. Ecol. Sociobiol. 57, 617–622. (doi:10.1007/ 442525a) detection. Naturwissenschaften 97, 229–233. s00265-004-0877-3) 44. Ings TC, Chittka L. 2008 Speed-accuracy tradeoffs (doi:10.1007/s00114-009-0627-5) 41. Thomson JD, Peterson SC, Harder LD. 1987 and false alarms in bee responses to cryptic 48. Shannon CE. 1948 A mathematical theory of Response of traplining bumble bees to competition predators. Curr. Biol. 18, 1520– 1524. (doi:10.1016/ communication. Bell Syst. Tech. J. 27, 379–423, experiments: shifts in feeding location and j.cub.2008.07.074) 623–656. (doi:10.1002/j.1538-7305.1948.tb01338.x) efficiency. Oecologia 71, 295 –300. (doi:10.1007/ 45. Freeman RB. 1968 Charles Darwin on the routes of male 49. Garriga J, Bartumeus F. 2019 bigMap R-package. BF00377298) humble bees. Bull. Br. Mus. Nat. Hist. 3, 177–189. CRAN. 42. Arnold SE, Chittka L. 2012 Illumination preference, 46. Pasquaretta C, Jeanson R, Andalo C, Chittka L, 50. Pohlert T. 2014 The Pairwise Multiple Comparison of illumination constancy and colour discrimination by Lihoreau M. 2017 Analysing plant – pollinator Mean Ranks Package (PMCMR). J. R. Soc. Interface 16: 20190103