SYSTEMATIC REVIEW published: 29 March 2021 doi: 10.3389/fvets.2021.634338 A Systematic Review on Commercially Available and Validated Sensor Technologies for Welfare Assessment of Dairy Cattle Anna H. Stygar 1*, Yaneth Gómez 2 , Greta V. Berteselli 3 , Emanuela Dalla Costa 3 , Elisabetta Canali 3 , Jarkko K. Niemi 1 , Pol Llonch 2 and Matti Pastell 4 1 Bioeconomy and Environment, Natural Resources Institute Finland (Luke), Helsinki, Finland, 2 Department of Animal and Food Science, Universitat Autònoma de Barcelona, Cerdanyola del Vallès, Spain, 3 Dipartimento di Medicina Veterinaria, Università degli Studi di Milano, Milan, Italy, 4 Production Systems, Natural Resources Institute Finland (Luke), Helsinki, Finland In order to base welfare assessment of dairy cattle on real-time measurement, integration of valid and reliable precision livestock farming (PLF) technologies is needed. The aim of this study was to provide a systematic overview of externally validated and Edited by: commercially available PLF technologies, which could be used for sensor-based welfare Edward Narayan, assessment in dairy cattle. Following Preferred Reporting Items for Systematic Reviews The University of Queensland, Australia and Meta-Analyses (PRISMA) guidelines, a systematic literature review was conducted Reviewed by: to identify externally validated sensor technologies. Out of 1,111 publications initially Paulo Graziano, extracted from databases, only 42 studies describing 30 tools (including prototypes) State University of Campinas, Brazil met requirements for external validation. Moreover, through market search, 129 different Pierre Guy Marnet, Agrocampus Ouest, France retailed technologies with application for animal-based welfare assessment were *Correspondence: identified. In total, only 18 currently retailed sensors have been externally validated (14%). Anna H. Stygar The highest validation rate was found for systems based on accelerometers (30% of tools anna.stygar@luke.fi available on the market have validation records), while the lower rates were obtained Specialty section: for cameras (10%), load cells (8%), miscellaneous milk sensors (8%), and boluses This article was submitted to (7%). Validated traits concerned animal activity, feeding and drinking behavior, physical Animal Behavior and Welfare, condition, and health of animals. The majority of tools were validated on adult cows. a section of the journal Frontiers in Veterinary Science Non-active behavior (lying and standing) and rumination were the most often validated Received: 27 November 2020 for the high performance. Regarding active behavior (e.g., walking), lower performance of Accepted: 08 February 2021 tools was reported. Also, tools used for physical condition (e.g., body condition scoring) Published: 29 March 2021 and health evaluation (e.g., mastitis detection) were classified in lower performance Citation: Stygar AH, Gómez Y, Berteselli GV, group. The precision and accuracy of feeding and drinking assessment varied Dalla Costa E, Canali E, Niemi JK, depending on measured trait and used sensor. Regarding relevance for animal-based Llonch P and Pastell M (2021) A welfare assessment, several validated technologies had application for good health Systematic Review on Commercially Available and Validated Sensor (e.g., milk quality sensors) and good feeding (e.g., load cells, accelerometers). Technologies for Welfare Assessment Accelerometers-based systems have also practical relevance to assess good housing. of Dairy Cattle. Front. Vet. Sci. 8:634338. However, currently available PLF technologies have low potential to assess appropriate doi: 10.3389/fvets.2021.634338 behavior of dairy cows. To increase actors’ trust toward the PLF technology and prompt Frontiers in Veterinary Science | www.frontiersin.org 1 March 2021 | Volume 8 | Article 634338 Stygar et al. PLF for Dairy Welfare Assessment sensor-based welfare assessment, validation studies, especially in commercial herds, are needed. Future research should concentrate on developing and validating PLF technologies dedicated to the assessment of appropriate behavior and tools dedicated to monitoring the health and welfare in calves and heifers. Keywords: PLF, accelerometer, camera, milk sensor, scale, bolus, dairy cow, calf INTRODUCTION Moreover, PLF technologies could also be applied for monitoring animal welfare [e.g., (12)]. On the other hand, as demonstrated Recently introduced concept of One Welfare recognizes the by large-scale studies (13, 14) investment in sensor systems might interconnections among animal welfare, human well-being, and not necessarily lead to the economic gain for farmers. Therefore, the environment (1). Better understanding of the values of high the merits of each sensor system need to be assessed individually welfare standards can, among others, support food security, and the performance should to be verified before the promise of improve productivity, reduce antimicrobial use, and greenhouse improved management can be realized. gas emission [e.g., (2–4)]. Research groups and companies around the world has been Animal welfare is also a highly interesting topic for European engaged in developing new PLF sensors, however, not all PLF consumers (5, 6). This interest is seen in production statistics solutions developed in a lab environment can be successfully and consumer purchases decisions. Consumers are willing to pay implemented as commercial products on dairy farms. The reason a premium price for credence attributes of milk (7, 8), such as can be that some technologies will still be too expensive or organic, environmentally friendly, or high animal welfare (on will perform better in an experimental setting, where conditions average 28, 25, and 31% of premium). Moreover, consumers are controlled, and sample size is small, compared to the appreciate proactive approach to managing animal health and farming environment. Therefore, for successful assessment of welfare (9), and there is evidence that the animal-friendly on-farm welfare using PLF technology, it is essential to validate marketing strategies influence the uptake of products (10). this technology at the commercial level (external validation). Animal welfare friendly products can be identified through Furthermore, applying sensor-based welfare assessment for labeling. Most dairy welfare labeling schemes in Europe have labeling schemes or welfare support payments should be based requirements concerning resource-based welfare indicators such on widely available and validated technologies. as space allowance, provision of bedding and enrichments, The main goal of this review was to assess which welfare minimum transportation time, outdoor access, or absence aspects of cows’, heifers’, and calves’ husbandry can be addressed of mutilations (8). Recently, animal-based indicators have by available (and validated) technologies. To reach this goal, gained more attention, especially following the publication of commercially available and/or externally validated technologies Welfare Quality R (WQ R ) protocols and a few labeling schemes with potential use for animal-based welfare assessment in dairy highlighting animal-based measures have been introduced herds were first identified. Validated technologies were later during the past years (e.g., AENOR welfare certificate in Spain, grouped according to their performance. Finally, possible gaps Arla one farm milk in Finland, and ClassyFarm in Italy). between available and validated tools and needs for animal-based However, existing animal welfare assessment protocols show welfare assessment were identified based on the principles of the some inaccuracies as: (1) they are only applied at group level, WQ R protocol, including appropriate nutrition, housing, health, (2) are unable to continuously monitor animal welfare, and (3) and behavior. they rely on human judgments and decisions-making facilitating some degree of subjectivity on the assessment. Moreover, those protocols are not practical for detecting early-warning signals MATERIALS AND METHODS which could result in implementation of preventive measures. Abovementioned limitations could be, at least partially, solved by Market Availability Search application of precision livestock farming (PLF) technologies. A broad market research (using web Google search) on Different PLF techniques have been developed for monitoring commercially available PLF systems with potential application for dairy cattle production. Sudden change in the activity, feeding animal-based welfare assessment was conducted. This research and drinking, physical condition, and health of animals can be was done by exploring the assortment of technology providers detected by different sensors [e.g., radio-frequency identification that cover a wide range of sensors which could provide (RFID), accelerometers, load cells, and cameras]. Change in information on animal base indicators for welfare. The search behavior or in physical state may indicate problems related to criteria used included “dairy cow” and one of the following management (e.g., feeding system failure) or disease, as well terms describing sensors: (automatic drinker OR automatic as can signal specific physiological status such as estrus. PLF waterer), (automatic feeder), (activity sensor OR activity monitor), technologies can potentially add value to the farm management (RFID), (Global Positioning System OR GPS), (thermal camera), process by improving data processing, decision making, and (thermography), (mastitis sensor), (automatic mastitis detection), implementation of everyday herd management decisions (11). (somatic cells counter), (milk analyzer), (automatic weigh scale Frontiers in Veterinary Science | www.frontiersin.org 2 March 2021 | Volume 8 | Article 634338 Stygar et al. PLF for Dairy Welfare Assessment OR automatic weigh), (lameness sensor), (automatic lameness to assess the performance originating from the same animals or detection), (pressure mat OR force sensor), (body condition score herd/herds as used in the developing of the technology (16). sensor OR automatic body condition score), (body condition camera), (rumen bolus), (milking robot), and (accelerometer). Study Classification Also, search for calf automatic feeder was performed. As an This review includes papers presenting the higher standards of example, the following word combinations were used to look objective validation, which is external validation. Based on the for feeding equipment available on the market: “dairy cow” plus approach presented by Altman et al. (16) we have defined two “automatic feeder.” The first five pages (50 hits) from Google levels of external validation: search were scanned. Additionally, the availability of sensors 1. External self-validation was defined for studies where the was scanned using dedicated on-line marketplace for providers system was evaluated using a fully independent data set, that (https://www.agriexpo.online/). Search was performed between means data was collected from different herds not used for March and May 2020. Tools with exclusive use for reproduction system development. Research was conducted by the same (for estrus detection or calving alarms) were excluded from the scientist (at least one author involved in developing and final list. Information on sensor name, provider name, internet validation) or had been validated by at least one author link, sensor type (with attachment position to animal when representing a company providing a technology. applicable), aim, and country of origin (headquarters) for 129 2. External independent validation was defined for studies where technologies are provided in Supplementary Table A1. the system was evaluated using a fully independent data set, which means data was collected from different herds not used Literature Search and Exclusion Criteria for system building. Research was conducted by scientists not To explore technology limitation, a systematic literature search involved in technology development. based on Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology (15) was conducted. In order to determine the validation level, both origin of the Literature search was conducted through Web of Science and technology and validation location (herd) needed to be known. Scopus. Altogether, 147 different search terms were used. Technology was identified through commercial name or based on Each search included terms describing different phases in the studies describing building phase (for prototypes). Origin of the production cycle (“cow” OR “calves” OR “calf ” OR “heifer”) and validation herd was identified through information on location validation (“test” OR “assess∗ ” OR “evaluat∗ ” OR “validat∗ ”) as (country), and type (if a herd was commercial or research). We well as several exclusion terms: NOT (“review” OR “survey” have assumed that criteria of external validation were fulfilled if OR “beef ” OR “sheep” OR “goat∗ ” OR “hors∗ ” OR “buffalo” OR commercially available technology was validated in a commercial “steer∗ ” OR “ewe” OR “leg calf ” OR “muscle∗ ”). Additionally, herd or a research herd (different from the company/developer each search was supplemented with physiological and behavioral own research herd). For prototypes, the criteria of external term (e.g., feeding behavior), or sensor type (e.g., camera), or validation were fulfilled only if the scientific paper clearly the commercial name (e.g., CowView). For physiological and described where technology was validated, and validation place behavioral term as well as sensor types the following terms was different from the herd used for technology building (based were used: (“feeding behavior” OR “feeding behavior” AND on information from scientific publication describing building “monitoring”), (“monitoring feeding”), (“drinking behavior” phase). If both country and herd specifications (commercial or OR “drinking behavior” AND “monitoring”), (“vocalization”), research) could not be identified, then the study was excluded (“vision”), (“camera”), (“accelerometer∗ ”), (“temperature AND from this review (due to not enough information in materials and sensor”), (“mastitis AND sensor”), (“image analyses”), (“scale methods). However, papers stating that herds used for validation AND body weight”), (“pressure mat”), (“bolus”), (“indoor were different than those used for technology development AND position”), (“in-line”), (“tracking system”), (“RFID”), and (without mentioning location, for example due to privacy (“microphone”). The commercial names used in the search are concerns) were included into this review. presented in Supplementary Table A1, column A. The example search looked as follow: (“cow” OR “calves” Performance Measures for Validated Trials OR “calf ” OR “heifer”) AND (“test” OR “assess∗ ” OR “evaluat∗ ” In this review, we distinguished regression and classification OR “validat∗ ”) AND (“camera”) NOT (“review” OR “survey” OR measures for performance reporting. Regression measures, “beef ” OR “sheep” OR “goat∗ ” OR “hors∗ ” OR “buffalo” OR reflects the agreement between a continuous trait measured “steer∗ ” OR “ewe” OR “leg calf ” OR “muscle∗ ”). by validated technology (predictor) and the golden standard Only studies in peer-reviewed scientific journals published in (outcome). For example, the agreement between body weight English between January 2000 and May 2020 were included to measured by a conventional scale and partial scale attached this review. Since this review focuses on dairy production, all to a milk feeder. Regression measures can be presented using validation trials conducted on beef breeds or steers were omitted. any of the following measures including Pearson correlation Articles were excluded if not dealing with aspects directly related coefficient (r), Spearman’s rank correlation coefficient (rs), to the welfare of dairy cows (e.g., reproduction related problems coefficient of determination (R2 ), mean bias from the Bland– such as estrus detection, and environmental aspects such as Altman plots (B–A plots), significance tests for intercept and methane emission, etc.). We further excluded papers with only slope of linear regression (I/S), or concordance correlation internal validation, which was defined as validation data set used coefficient (CCC). Classification measures refers to the ability of Frontiers in Veterinary Science | www.frontiersin.org 3 March 2021 | Volume 8 | Article 634338 Stygar et al. PLF for Dairy Welfare Assessment a technology to predicting categorical outcomes e.g., locomotion 13 milking robots). GPS sensors were used in eight different score. Classification performance was usually reported using products offering the possibility to locate animal position. either area under the receiver operating characteristics curve Additionally, two systems using microphone, as well as one (AUC) or sensitivity (Se) and specificity (Sp) or Cohen’s kappa mobile app for body condition scoring was identified. All coefficient (κ). products based on accelerometers offered health alerts. Only In this review, we have distinguished tools validated for high one accelerometer-based product was dedicated for calves, the performance and lower performance. It was assumed that high remaining products were advertised for cows or heifers. Systems performance was reached when all indicators defined/selected based on load cells in combination with RFID were most by authors of studies fulfilled following criteria: r, rs, CCC, often used for managing and tracking the feeding program Se, and Sp, or AUC was >0.9, R2 and κ was >0.81, I/S did of individual animals. Also, few systems were used for body not differ significantly from 0 or 1, respectively, and B–A plot weight monitoring. Boluses were advertised as tools to measure included zero with the 95% interval of agreement. Criteria body temperature, pH, and rumen activity as well as for for high performance (precision and accuracy) were accepted animal identification. Among cameras, seven were dedicated similarly to those referred by studies assessing technology for body temperature monitoring (thermal cameras), two were performance (17–19). used for body condition scoring (BCS), and one camera for feeding monitoring. Assessment of Welfare Relevance Welfare Quality R is a scientifically rigorous animal welfare assessment protocol (20), which follows four animal welfare Peer-Reviewed Records on Technology principles (good housing, good feeding, good health, and Validation appropriate behavior). WQ R principles were used as a reference The literature search resulted in 1,111 titles, but after duplicate to classify indicators measured by technologies. In this review, removal and exclusion criteria throughout the review process, members of the ClearFarm project with expertise in animal 1,069 papers were omitted. A modified PRISMA flow diagram welfare were asked to evaluate the relevance of each indicator provides information on the number of excluded papers and measured by the PLF technologies listed in this review for reason for exclusion (Figure 1). A total of 42 articles satisfied assessing WQ R principles. Possible scores were: “relevant” the selection criteria, and 38 publications validated commercially and “not relevant.” For example, the panel was asked to available technologies. Moreover, we have identified four studies evaluate whether grazing time is relevant for the principles on prototype validation (Table 1). Only two papers validated of good feeding, good housing, good health, and appropriate more than one product, however several papers validated more behavior. Experts votes were categorized based on “relevant” than one indicator measured by the technology. The performance votes, so that all traits with more than 80% votes were of technologies with accelerometer sensors were the most often grouped in “very relevant” category, traits receiving from assessed (26 technology validation trials). Validation trials for above 20% up to 80% votes were in “moderately relevant” load cells (N = 6), bolus, and camera (four trials each), RFID category, and all traits with 20% or less votes were in “not (N = 3), microphone and viscosity sensor (two trials each), and relevant” category. conductivity and spectroscopy (1 trial each) were less frequent (Table 1). Regarding accelerometers, the precision and accuracy RESULTS of products offering different attachments to the animal were assessed in 11 sensors [leg (N = 5), collar (N = 3), ear (N = Technologies Commercially Available 2), and halter (N = 1)]. The most often validated technology The full list of commercially available technologies is presented originated from Itin+ Hoch GmbH, Liestal, Switzerland (six in the Supplementary Table A1. In total, 129 technologies were trials), Afimilk, Kibbutz Afikim, Israel (five trials), and Agis, found from 67 different providers located in 21 countries. The Harmelen, the Netherlands (five trials). Information on the United Kingdom, the Netherlands, and the United States, are study design (herd type, size, and location) for all qualified the leaders for providing technologies with potential use for papers is presented in Supplementary Table A2. In total, 28 animal-based welfare assessment. Technologies were grouped studies presented validation trials conducted on research farms. according to the used sensor. Accelerometer-based technologies The remaining studies (33%, N = 14) were conducted on and load cells were the largest group on the list (37 different commercial herds. The sample size used in validation trials products for each group) and constituted 57% of all found varied substantially. In general, the smallest sample size was tools. Commercially available accelerometers were offered with selected for experiments concerning cannulated cows [below different animal attachment solutions (collar, leg, ear, and halter), 10 animals for bolus validation, e.g., (52)], while the highest and some companies offered products with more than one sample size was selected for experiment testing performance attachment option. The collar was the most popular solution of online somatic cell count (SCC) estimation in automatic (65%, N = 24), while leg (30%, N = 10), ear (14%, N = 6), milking system [above 4,000 milking cows (43)]. When it comes and halter (3%, N = 1) were less frequent. We have identified to the geographical location of the herds, most technologies 14 boluses and 10 products using vision-based monitoring. were validated in the United States (11 studies) and Canada (5 Regarding milk quality, 25 sensor technologies (19% of a studies). The performance of tools was assessed using regression market share) for health monitoring were identified (including measures (27 papers), classification measures (7 papers), and Frontiers in Veterinary Science | www.frontiersin.org 4 March 2021 | Volume 8 | Article 634338 Stygar et al. PLF for Dairy Welfare Assessment FIGURE 1 | Modified Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram (15) with the systematic review search strategy and study selection. both measures (8 papers). Most of the reviewed papers were Performance of Technology Validated for classified as full independent validation, and only 33% (N = 14) Dairy Cows of reviewed papers were self-validated. Table 2 summarizes tools with available validation trials which could have practical application in welfare assessment for dairy cows. Validated animal-based traits concerned animal activity Validation Rate (walking, number of steps, lying, lying and standing, and According to the obtained results, only 18 commercially standing), feeding and drinking behavior (feeding time, presence available sensors listed in the Supplementary Table A1 have been at feeder, intake, grazing, rumination, drinking duration, externally validated (14%). The highest validation rate was found presence at a drinker, and water intake) physical condition, and for systems based on accelerometers (30% of tools available on health (locomotion score, BCS, rumen pH, body temperature, the market have validation records), while the lower rates were health disorder detection, and milk quality). obtained for cameras (10%), load cells (8%), miscellaneous milk Non-active behavior (lying, lying and standing, and standing) sensors (8%), and boluses (7%). as well as rumination and feeding time were the most often Frontiers in Veterinary Science | www.frontiersin.org 5 March 2021 | Volume 8 | Article 634338 Stygar et al. PLF for Dairy Welfare Assessment TABLE 1 | Summary of externally validated technologies with potential use for dairy welfare assessment. Technology name Technology provider No. of validation trials Used sensors and Independent Self-validationc for technology attachment positiona validationb provider AfiAct Pedometer Plus, Afimilk, Kibbutz Afikim, Israel 5 Accelerometer, leg (21–24) AfiTagII, Pedometer Plus (S.A.E.Afikim, Israel) AfiLab real-time milk analyzer Spectroscope (25) CowAlert IceQube, IceTag IceRobotics Ltd., Edinburgh, Scotland 4 Accelerometer, leg (21, 26–28) Track A Cow ENGS, Rosh Pina, Israel 1 Accelerometer, leg (21) MooMonitor+ Dairymaster, Tralee, Ireland 2 Accelerometer, collar (18, 29) HerdInsights Alanya Ltd., Cork, Ireland 1 Accelerometer, collar (30) Hi-Tag SCR Engineers Ltd., Netanya, Israel 2 Microphone, collar (31, 32) (currently Allflex) RumiWatch Itin+ Hoch GmbH, Liestal, Switzerland 7 Accelerometer and (34, 35) (29, 33, 36–38) pressure sensor, halter (29, 33–37), halter and leg (38) CowScout Leg GEA Farm Technologies, Bonen, 1 Accelerometer, leg (27) Germany CowManager SensOor Agis, Harmelen, Netherlands 5 Accelerometer, ear (21, 39–41) (17) The Smartbow Smartbow GmbH, Jutogasse, Austria 1 Accelerometer, ear (21) Lely activity Lely, Maassluis, the Netherlands 2 Accelerometer, collar (42) Lely- on-line California Viscosity meter (43) mastitis test GrowSafe GrowSafe Systems Ltd., Airdrie, AB, 1 RFID (neck collar), load (44) Canada cell Insentec Insentec, Marknesse, the Netherlands 1 RFID (ear), load cell (45) (now Hokofarm group) Intergado Intergado Ltd., Contagem, Minas 2 RFID (ear), load cell (46, 47) Gerais, Brazil Body Condition Scoring DeLaval International AB, Tumba, 2 Camera (48) Sweden Combi Load cell (49) eCow Farmer bolus eCow, Dekon, UK 2 Bolus (reticulum) (50, 51) KB 3/04 bolus Kahne Limited, New Zealand 1 Bolus (rumen sac) (52) Bella Ag Cattle Bella Ag LLC, USA 1 Bolus (reticulum) (53) Stepmetrix BouMatic, Madison, USA 1 Load cell (54) Optris Optris, Berlin, Germany 1 Camera (55) Prototypes—locomotion ns 2 Camera (19, 56) score IMAG model—prototype ns 1 Conductivity meter, (57) mastitis detection thermometer (milk temperature) Prototype-mastitis detection detection model (prototype) and online 1 Viscosity meter (58) cell counter, DeLaval International AB, Tumba, Sweden Prototype-activity ns 1 Accelerometer (59) a Sensor location is provided only for sensors attached to the animal. b Validated using independent data set (different animals and herd than for technology building) and co-authors were not involved in technology development. c Validated using independent data set (different animals and herd than for technology building) and was developed and validated by at least one the same co-author (based on the authorship of papers) or have been validated by at least one co-author representing a company providing a technology. validated attributes (20, 15, and 11 trials, respectively). There are of tools was reported. Regarding feeding and drinking, the several different commercially available technologies classified performance of the tool varied depending on measured with high performance for non-active behavior (Table 2). For traits and used sensor. Feeding time, which was monitored active behavior (walking, number of steps), lower performance using accelerometer-based sensors, was validated for lower Frontiers in Veterinary Science | www.frontiersin.org 6 March 2021 | Volume 8 | Article 634338 Stygar et al. PLF for Dairy Welfare Assessment TABLE 2 | Results of validation trials for dairy cows in respect to measured traits. Category Measured trait Validation Technologies validated for high performancea Technologies validated for lower place (farm) performanceb Activity Non-active behavior Research AfiAct Pedometer Plus (21), AfiTagII (22), CowAlert CowManager SensOor (40), Lely activity (42), (lying, lying and IceQube (21), CowManager SensOor (17, 39), Track MooMonitor+ (18), Pedometer Plus (23) standing) A Cow (21), RumiWatch (38) Commercial CowScout Leg (27), Ice Tag (26, 27), prototype (59) Standing—identified Research RumiWatch (38) Lely activity (42) as a separate behavior Commercial CowScout Leg (27), Ice Tag (26, 27) Active behavior Research CowManager SensOor (17, 39, 40), IceTag (walking, no. of steps) (28), Lely activity (42), RumiWatch (38), Commercial CowScout Leg (27), Ice Tag (26, 27) Feeding Feeding time Research CowManager SensOor (17, 21, 39, 40), and MooMonitor+ (18), RumiWatch [V0.7.0.0 (33) drinking and V0.7.4.5 34], Track A Cow (21) Commercial RumiWatch V0.7.3.2 (36) prototype (59), RumiWatch V0.7.2.0 (36) Presence at the Research GrowSafe (44), Insentec (45), Intergado (46) feederc Feed intake (kg) Research Insentec (45), Intergado (46) Grazing time Research MooMonitor+ (29), RumiWatch (29, 38) Rumination Research CowManager SensOor (17) CowManager SensOor (21, 39, 40), Lely MooMonitor+ (18, 29), Rumi Watch (29, 33–35, 38), activity (42) Smartbow (21), Hi-Tag (31) Commercial Prototype (59), Rumi Watch (36) Drinking time Research Rumi Watch [V0.7.0.0 (33) and V0.7.4.5 (34)] Commercial RumiWatch (36) Water intake Research Insentec (45) Presence at the Research Insentec (45) drinker Physical Locomotion score Commercial Prototype (19), prototype (56), Stepmetrix (54), condition Body condition Commercial DeLaval Body condition scoring (48) and scoring health Rumen pH Research eCow bolus (50, 51), KB 3/04 bolus (52) Body temperature Research KB 3/04 bolus (52), OPTRIS (55) Health disorderd Commercial HerdInsight (30) Milk qualitye Research AfiLab real-time milk analyzer (25) Mastitis detection Commercial Lely-on-line California mastitis test (43), prototype -IMAG model (57), prototype (58) a Allindicators defined/selected in validation trail (by authors of studies) were above high-performance threshold. High precision threshold was reached when Pearson correlation, Spearman’s rank correlation, concordance correlation coefficient, sensitivity, specificity, area under the curve (AUC) was higher than 0.9, regression coefficient and Kappa coefficient is higher than 0.81, significance tests for intercept and slope of linear regression did not differ significantly from 0 or 1, respectively, and Bland–Altman plot included zero with the 95% interval of agreement. b Any indicator validated with lower performance (below threshold defined above). c Animal identification and time. d e.g., mastitis or pneumonia. e Fat, lactose, and protein as indicator for mastitis. performance. Conversely, presence at the feeder and feed intake performance (19, 56)], and in general, none of reviewed (observed in feeding stations) and grazing time (monitored technologies was able to outperform the human observer. through accelerometer-based sensor) were evaluated for high Regarding BCS, the technology was reliable for dairy cattle with performance, but only in the research farm conditions. Drinking average body condition (scoring between 3.00 and 3.75 on the time was assessed using accelerometer-based tools, and the five-point scale) but did not score accurately for thinner or pressure sensor was evaluated for lower performance. All tools fatter cows. The only validated study on the accelerometer-based used for physical condition evaluation and health were classified system used for health alarms (30), reported a high number of under lower performance. Assessment of locomotion score false positives, but the true health disorders were alerted by the varied between presented tools [poor (54) or fair classification system before the farmer noticed them. Regarding technologies Frontiers in Veterinary Science | www.frontiersin.org 7 March 2021 | Volume 8 | Article 634338 Stygar et al. PLF for Dairy Welfare Assessment applied for monitoring milk quality and mastitis, real-time milk Regarding reason (1), altogether six studies reporting analyzers agreed moderately with SCC (43), protein, lactose, and validation trails were excluded from this review due to fat determined in the laboratory (25), while mastitis detection insufficient information provided about study design. models have acceptable results for sensitivity, specificity, and Reason (2), there might be lower scientific interest to validate error rates (57, 58). technologies that are not used for research experiments or are not yet integrated as data sources for other systems. For example, for many commercially available sensors based on Performance of Technology Validated for scales (like individual feed intake measurement), there are no Calves and Heifers validation trials available. However, the required precision for Table 3 summarizes the tools with available validation trials feeding monitoring tools (as well as the interest in validation) which could have practical application in welfare assessment might increase if the data from these tools, as in the example for young cattle (calves and heifers). Validated traits concerned from pig production (60), would be integrated into marketing active behavior (walking), non-active behavior (lying), feeding or health monitoring systems. Furthermore, the validation rate (time, presence at the feeder, and intake), rumination, drinking could be increased if technologies, similar as medical industry, (presence at the drinker and intake), body weight, and body receive specific certification [e.g., International Organization temperature. For calves and heifers, rumination and body for Standardization (ISO) standards]. Currently, devices and temperature were the most often validated traits (three and systems used for the purposes of official milk recording (e.g., two trials, respectively). Tools measuring active and non-active milk meters, samplers, and milk analyzers) need to meet the behavior (lying and walking), feeding and drinking behavior requirements specified in ISO standards and must be tested to (feed and water intake and presence at the drinker or feeder), achieve approval from The International Committee for Animal and body weight were validated for high performance. Feeding Recording (ICAR) (61). However, the data from the validation time, rumination, and body temperature were validated for process conducted by ICAR are not publicly available. This could lower performance. also explain a low number of validation records in peer-reviewed literature for milk recording devices and systems. Experts’ Assessment Reason (3), validation studies can be labor intensive and Answers from animal welfare experts concerning the relevance costly, due to the need to collect the reference data set. of the indicator in assessing good feeding, housing, health, For example, accelerometer-based systems are the most widely and appropriate behavior are summarized in Table 4. For available and validated among all PLF technologies. But, as good health, nine traits received “very relevant” evaluation demonstrated in this review, the majority of the accelerometer- (body temperature, BCS, lameness, mastitis, water consumption, based validation studies concerned behavioral monitoring and rumination, rumen pH, feed intake, and non-active behavior). only one validation study for the performance of accelerometer Regarding good feeding, seven traits were categorized as “very system for health monitoring was found. Validation of health relevant” (BCS, water consumption, rumination, rumen pH, monitoring technology requires obtaining reference data set grazing, feed intake, and feeding time). For good housing containing data on veterinary examinations and blood or milk evaluation, experts agreed on the usefulness of non-active samples to detect among others lameness, mastitis, ketosis, and behavior monitoring. While, for appropriate behavior, only pneumonia. The substantial costs needed for the reference data grazing monitoring was evaluated as “very relevant.” set might affect the number of available publications. Finally, for reason (4), it could be pondered, if the reason behind the relatively small number of validation studies is DISCUSSION due to reluctance in publishing negative results. Technology providers are involved in the validation process and altogether, Retailed and Validated PLF Technologies about one-third of all validation studies presented in this review for Welfare Assessment were classified as self-validation. Self-validation could raise the The aim of this review was to identify validated and/or question of conflict of interest in reporting negative results. commercially available technologies for measuring animal-based However, it is impossible to conclude how many of the negative welfare indicators in dairy cattle. Currently, farmers can select results were never published due to the conflict of interest. from at least 129 different sensors to monitor animal-based Certainly, technologies which are commercially available may indicators of health and welfare in dairy production. However, not all have been identified in this study. The search was there is still limited information on the performance of these conducted using internet websites in English, therefore all tools tools. According to our results, only 14% of commercially without English marketing material or presented in printed available sensors have external validation trials available, which company catalogs were omitted. The biggest producers will may thwart confidence on these technologies. have information provided in English, but smaller companies We identified four potential reasons for such a small number offering products for local markets or startups might not yet of validation trials: (1) insufficient reporting (2) low scientific have information available for international buyers. Therefore, interest for validating technology not for research (3) high cost a constructed list of retailed products is an approximation of and labor intensity of data collection (4) reluctance to publish the current market. Our goal was not to identify every single negative results. technology but to use this list to identify tendencies on the Frontiers in Veterinary Science | www.frontiersin.org 8 March 2021 | Volume 8 | Article 634338 Stygar et al. PLF for Dairy Welfare Assessment TABLE 3 | Results of validation trials for calves and heifers in respect to measured traits. Category Measured trait Validation Technologies validated Technologies validated for lower place (farm) for high performancea performanceb Activity Non-active behavior (lying time, lying Research AfiTag II (24) bouts) Active behavior (walking, no. of steps) Research AfiTag II (24) Feeding and drinking Feeding time Research CowManager SensOor (41) Presence at the feeder Commercial Intergado (47) Feed intake Commercial Intergado (47) Rumination Research CowManager SensOor (41), Hi-Tag (32), RumiWatch (37), Water intake Commercial Intergado (47) Presence at the drinker Commercial Intergado (47) Physical condition Body weight Research Combi DeLaval (49) Body temperature Research Bella Ag Cattle (53), OPTRIS (55) a All indicators defined/selected in validation trail (by authors of studies) were above high-performance threshold. High precision threshold was reached when Pearson correlation, Spearman’s rank correlation, concordance correlation coefficient, sensitivity, specificity, are under the curve, is higher than 0.9, regression coefficient and Kappa coefficient is higher than 0.81, significance tests for intercept and slope of linear regression did not differ significantly from 0 or 1, respectively, and Bland–Altman plot included zero with the 95% interval of agreement. b Any indicator validated with lower precision and or accuracy (threshold defined above). assessment. Applying sensor-based welfare assessment for TABLE 4 | Indicator evaluation for relevance in assessing good feeding, housing, health, and appropriate behaviora . labeling scheme or welfare subsidies should be based on widely available technologies. This review shows that reliable Indicator Good Good Good Appropriate technologies for monitoring welfare-related traits exist, however, feeding housing health behavior there are areas concerning sensors and algorithms which require further developments. For example, based on the presented Body temperature +− +− + – summary, it can be concluded that while recording behavior of Body condition scoring + – + – farm animals using machine-vision has shown great progress in Lameness – +− + – research (64, 65), it is only entering the commercial market, and Mastitis – +− + – external validation will be needed to confirm the performance. Water consumption + – + +− Furthermore, according to our results, the performance of Drinking duration +− – +− +− existing health and welfare monitoring systems was sporadically Rumination + +− + +− tested on young animals (heifers and calves). Validation studies Rumen pH + – + – with accelerometers based on collar were rare and only 14% Grazing time + +− +− + of validated traits for activity monitoring was obtained from Feeding intake + – + +− the collar devices. On the other hand, this was the most often Feeding time + – +− +− marketed attachment point for the accelerometer. Therefore, Active behavior – +− +− – further validation studies for collar-based systems are needed. Non-active behavior – + + +− To successfully assess welfare of young animals, more work on a Symbols +, +−, – refer to “very relevant,” “moderate,” and “not relevant” dedicated systems might be required. Further technological and evaluation, respectively. validation gaps regarding assessment of welfare will be discussed according to the principles defined in the WQ R protocol. market and set possible market constraints for developing sensor- Sensor-Based Welfare Assessment for based welfare assessment. One must also remember that not all Dairy Cows and Young Cattle—How Far validation studies available for a device were reported in this Are We? review. We have included only validation studies for attributes The concept of welfare has multidimensional nature, there is related to animal welfare, therefore, some validation studies for no one indicator that can be used to assess the welfare of performance of estrus detection on accelerometer-based devices an animal, but there are some indicators which are linked [e.g., (62)] or pregnancy detection from in-line analyzer [e.g., to several aspects of welfare. Quite often, welfare assessment (63)] were excluded. is performed using a combination of animal and resource- Precision livestock farming uses technology for real-time, based indicators (as in WQ R protocol, for instance) and the continuous monitoring of individual animals and/or groups of evaluation is performed by a human observer. Some of the animals, which provides an opportunity to improve welfare aspects which are evaluated using welfare protocols could be Frontiers in Veterinary Science | www.frontiersin.org 9 March 2021 | Volume 8 | Article 634338 Stygar et al. PLF for Dairy Welfare Assessment addressed by sensor-based technologies. Below, we will discuss Moreover, accelerometers together with noseband pressure the availability of technologies for the assessment of each sensors were used to measure drinking duration [e.g., (36)]. In welfare principle: the future, good feeding assessment could be further improved by integrating information from emerging technologies (such as Good Feeding video-based assessment of BCS). To fulfill the good feeding principle, animals should not suffer from prolonged hunger or thirst. For prolonged hunger, the WQ R protocol adopts animal-based indicator. Regarding thirst Good Housing criterion, only resource-based indicators are evaluated (20). In order to ensure good housing, animals should have thermal Therefore, PLF technologies can provide additional single- and resting comfort as well as enough space to move freely (20). animal level information for good feeding evaluation. For assessing comfort around resting, the WQ R protocol uses There were several attributes monitored by PLF technologies animal-based (e.g. time to needed lie down, animals colliding (BCS, rumination time, rumen pH, grazing time, feed and with housing equipment during lying down, animals lying water intake, and feeding time), which have high potential for partially or completely outside the lying area) and management- “good feeding” assessment. Some of the attributes (rumination based indicators (e.g., presence of tethering and access to outdoor time, rumen pH, grazing time, feed and water intake, and loafing area or pasture). Therefore, measuring the activities feeding time), when frequently monitored, can be used for of animals and the physical state using PLF technologies can designing early warning systems for disease detection and/or provide a more accurate assessment at an individual level. feeding system failures [e.g., (66)]. On the other hand, BCS, Regarding the evaluation of experts, non-active behavior (lying which assess the proportion of body fat, can have practical or standing still) has the highest potential to be used for the application for decision support systems (e.g., predicting the risk assessment of good housing. Allowing dairy cows adequate of cow developing ketosis or having reproduction problems) space and facilities to lie down is considered an important (67). The good feeding assessment might be hampered by the aspect for production as well as animal welfare (71). As commercial availability of technologies. Based on our search, recently reviewed, the lying time will depend on individual cow- only two providers offered a camera-based sensor used for based factors (reproductive status, age, and milk production), BCS monitoring. There is also a shortage of tools able to health status (lameness and mastitis), and the comfort of assess grazing time (only two technologies had validation studies housing facilities (72). For example, pasture-based cows are for grazing monitoring). Finally, measurements on individual characterized by longer, undisrupted lying times compared to feeding and drinking were performed at feeding stations, mostly cows kept in cubicles (73). Lameness can result in longer lying used for research (feeding experiments), and due to the high times while mastitis can reduce it (72). For this reason, to costs of equipment might have little relevance for commercial avoid confounding factors between animal health and housing application. Potentially, systems based on cameras can also conditions, an integration with other data sources, such as have application for feed availability and intake monitoring milking or breeding records, presence of lameness or mastitis is (68). However, these systems are still in development and necessary. Non-active and active behavior as well as grazing time only one commercial camera-based system for feed accessibility can be assessed using accelerometers. However, performance of monitoring was identified. There are several providers of boluses technologies varied in different farm conditions. For example, for rumen pH monitoring, but still, relatively little is known CowManager sensor was evaluated for high (17, 39) and lower on the performance of the detection models (with alarm- (40) performance in measuring lying behavior of cows. High based monitoring) for rumen pH monitoring. Additionally, performance was obtained in tie stall and free stall barn and short functional life of the pH boluses [around 40 days due lower performance for grazing cattle. These somehow varying to loss in accuracy of the electrode (69)], does not allow long performance results raise the question, if sensor systems should lasting individual-animal based assessment. Animal presence be adjusted (and also validated) for different environmental/ at the feeding trough or water bin, can be monitored using housing conditions. Cleanliness of udder, cleanliness of flank, RFID technologies [e.g., (46)], but available technologies have and cleanliness of upper and lower legs are other animal- been tested mostly in experimental farms, and examples with based indicators recorded in the WQ R protocol to assess commercial farm validation are rare. Increased competition the criterion of comfort around resting and consequently the among cattle at the feed bunk can be currently detected in principle of good housing (20). To the best knowledge of experimental settings [e.g., (70)] and can indicate shortage of authors, currently there are no available technologies able to food (decrease feeding time or dry matter intake). However, assess the cleanliness of animals. However, rapid development in there is a need for further validation studies on systems vision-based monitoring for automatic individual identification based on RFID for detecting food or water shortage at an [e.g., (74)] can prompt the development of algorithms capable individual level. of evaluating this welfare aspect. Thermal comfort can be According to our results, good feeding assessment based on assessed on an individual basis by application of invasive (e.g., animal indicators in commercial settings could be primarily boluses) and non-invasive sensors (thermal cameras). Both conducted using accelerometer technologies. Accelerometers- options are available on the market; however, there is a clear based systems are easily available and can assess rumination (with shortage of validation studies for monitoring systems based on high performance) and feeding time (with lower performance). those sensors. Frontiers in Veterinary Science | www.frontiersin.org 10 March 2021 | Volume 8 | Article 634338 Stygar et al. PLF for Dairy Welfare Assessment Good Health and positive emotional state (20). Based on the answers of For good health, animals should be free from physical injuries experts, it can be concluded that PLF technologies currently (like lameness and integument alterations) and disease and have a low potential to address appropriate behavior. With the should not suffer pain induced by inappropriate management exception of grazing behavior, none of the evaluated attributes or handling (20). As agreed by experts, several traits measured was evaluated as “very relevant” by all the experts. However, by PLF technologies have a potential application for assessment some of the attributes related to activity as well as feeding of good health (body temperature, BCS, lameness, mastitis, and drinking monitoring were evaluated as moderately relevant. water consumption, rumination, rumen pH, feed intake, and From all available technologies, only two tools were tested non-active behavior). The listed attributes can be categorized for evaluation of appropriate behavior (namely, for grazing as direct or indirect health indicators. Indirect indicators, such monitoring). There is, however, a substantial scientific interest as active and non-active behavior, rumen pH, and feeding in developing research tools aiming to address this welfare quantity, on its own does not indicate health status of an principle. For example, accelerometer-based tools were already animal, but changes in the behavior of animals possibly in applied to monitor social behaviors, such as discriminating combination with other data sources (e.g., lactation status spontaneous locomotor play (76) and licking/suckling (77) in and reproduction) can be processed to obtain early-warning dairy calves. Also, data from feeding and drinking stations signals for health problems (e.g., lameness, mastitis, and ketosis) were applied for monitoring social competition (70, 78, 79). In and potentially prevent them. Direct welfare indicators, like recent years, scientists pointed out the importance of positive the number of cows with increased SCC, brings knowledge emotions as key elements to ensuring good animal welfare (80). on health (if an animal is sick or not), and can be In experimental conditions, both ear postures (81) and nasal useful for operational decisions (e.g., antimicrobial treatment). temperature (82) have been proven to be useful measures of a Injuries, such as lameness, can be detected measuring animal change in emotional state of cows. For example, the drop in behavior (accelerometers), gait (load cells), posture (cameras), nasal temperatures of cows can be a result of the experience and increased body temperature (thermal camera). Though, of a positive, low arousal experience. However, further research the performance of accelerometer-based systems and thermal is needed to design systems able to monitor positive emotional cameras for lameness detection is unknown (we have not state in commercial settings. Also, there is a technological gap identified external validation studies for lameness detection concerning monitoring good human-animal interaction with no using those techniques), while the commercial availability and retailed technologies intended for this purpose. There is also performance of two remaining methods are still low. According very scarce information on any experimental techniques for to our knowledge, there are no commercially available systems measuring avoidance distance (which is used to assess good able to detect skin lesions; however, similar to cleanliness human-animal interaction) at the individual level (12). Human- evaluation, development in camera-based monitoring systems animal relationship could be automatically monitored using 3D could in the future allow identification of animals with cameras, which can capture the distance between a target and such problems. camera. However, application of vision technologies requires Assessment of good health (and especially presence of more research effort. diseases) should be based on integrating and analyzing data from different sources. There are commercially available examples of systems using multiple sensors (e.g., milking robots and Performance Results—Quality and activity collars) which provide data on milk production, Quantity of Validation Studies SCC, and animal behavior. System using both an automatic Validation studies are essential for further use of the tool milking system and an activity collar was presented by Elischer in scientific experiments as well as for welfare labeling or et al. (42). However, there are no external validation studies subsidy payments. Therefore, there should be more emphasis on the performance of these systems for disease detection. on the quantity and quality of conducting and reporting There are already examples of flexible models able to handle of validation studies for PLF technologies. The results of different sensor or non-sensor data for disease detection [e.g., validation studies are quite often presented as technical notes for mastitis prediction (75)] but the performance of these or short communications with rather limited space for detail tools need still to be tested in commercial settings. Even if description; however, this does not absolve authors from a technology is not able to provide highly accurate health presenting information necessary for readers to assess the risk data on individual level, it could still be useful to estimate of reporting bias. Based on this review, similar suggestions to herd level prevalence of health problems. Potential integration those presented by Hendriks et al. (83) on how to improve could concern milk sensor data, accelerometer data, load cells reporting can be made. The location of the trial (commercial vs. with RFID, boluses (for body temperature and rumen pH), experimental herd), criteria for animal selection (e.g., random cameras (for body temperature, gait, BCS), and microphones or based on a stage of lactation), building and management (cough detection). characteristics (e.g., floor type and grazing), and feeding system should be always reported, since results obtained in the different Appropriate Behavior production settings might not be comparable. Furthermore, Appropriate behavior concerns expression of social behavior, the validated tool, especially if not commercially available, expression of other behaviors, good human-animal relationship, should be described in enough detail for correct technology Frontiers in Veterinary Science | www.frontiersin.org 11 March 2021 | Volume 8 | Article 634338 Stygar et al. PLF for Dairy Welfare Assessment identification. Also, when possible, the software version should information which can assist in genetic evaluation [e.g., (86)]. be reported. Moreover, production data could be utilized for designing health In this review, we have grouped tools based on reported surveillance systems. For example, an attempt was made to performance measures. Threshold for high performance was use milk yield data to detect outbreaks of Bluetongue and selected based on available literature and represents very good Schmallenberg viruses (87). PLF technologies may provide agreement. Here, it should be noted that tools which did not evidence-based approach to the monitoring and surveillance of fulfill high performance criteria still have practical relevance. For animal welfare not only at the farm but also during transport example, online California Mastitis Test performed by milking or at slaughter (88). Already now, in some countries, there robots, agreed only moderately with laboratory measurements on are suggestions to base the certification system of livestock SCC. Even though this data was not very precise, nevertheless farming on real-time measurements and using animal behavior can be very useful for on-farm decision making, due to the as a criterion for quality labeling (89). This kind of policy high sampling frequency (43). Therefore, the practical relevance could increase transparency of the sector and could result in a of tools need to be assessed based on their objectives (84) wider selection of welfare friendly products. As demonstrated and judged one by one. The results from a single validation in a previous review, data routinely collected on the farm trial are not yet conclusive regarding the tool performance. (e.g., on milk yield, culling, and reproduction) and available Ideally, tools should be validated in different production in national data base, were associated with dairy cow welfare conditions (e.g., different countries and housing). According (90). Also, meat inspection data can have practical application to our results, some of the traits were validated multiple for welfare assessment (91). This review demonstrates that data times, by different research groups around the world. And as collected during on-farm monitoring has high potential to assess expected, some presented performance results were inconsistent different aspects of dairy cow welfare, and that currently available (as in the example of the CowManager sensor evaluated in technologies can provide animal-based welfare information. different housing conditions). Also, as seen from the example of However, for the data to be fully utilized for this purpose, there Supplementary Table A2, authors varied in reported statistics. is a need to develop new methodologies for data integration There seems to be no clear guideline on sufficient level of and processing. Data collected from various automatic recording information regarding performance which should be reported. technologies need to be processed and integrated into a single For example, for classification models, reports should not be outcome of animal welfare (which is easy to understand by based on presenting only sensitivity and specificity of the the consumer). This challenging task will be considered by model without information on selected thresholds for detection. the ClearFarm project, which aims to develop a platform to Instead, the performance of the classification model should be control animal welfare in pig and dairy farming. The integration preferably presented in receiver operating characteristic curve, of technologies for welfare, health surveillance, or breeding which is the overall performance indicator (85). Regarding evaluation will require access to a vast amount of PLF data from regression models, the adequate statistical tests are presented for different devices and different users. The utilization of these data example in Tedeschi (84). Including only Pearson correlation requires that data ownership rights, privacy, and confidentiality coefficient allows assessing precision of the tool, while nothing issues are resolved and agreed between the parties involved. is known about its inaccuracy (the systematic deviation from For example, for the EU markets, non-binding guidelines on the truth). Testing for tool performance is especially important data sharing from PLF technologies are available (92) and for technologies from which data will be post-processed and cover, among others, ownership, access, control, and privacy. used for building further algorithms. In this review, we have However, according to the recent review on digital agriculture, not removed or distinguished in result table studies which the area of data ownership regulations could receive more provided somehow limited information on the tool performance attention (93). Another challenge concerns data storage capacity (for example, only results on Pearson correlation). It could and strong computational power. However, there are already be possible to set additional exclusion criteria for papers efforts to design a set of industrial, large-scale high-performance selection; however, one must remember that even in the limited computing solutions to support the processing of very large PLF form, these studies provide some partial information about data sets from different users (94). the validity. DATA AVAILABILITY STATEMENT Application of Sensor-Based Technology for Welfare Assessment on Farms and The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be Beyond directed to the corresponding author/s. The primary goal of a sensor system is to improve animal management. Sensor systems provide information for decision making which may, among others, influence farm profitability AUTHOR CONTRIBUTIONS and animal health and welfare as well as have environmental impact (11). However, potential application of sensor systems can MP, JN, AS, EC, and PL conceived the presented idea. AS defined go beyond a single farm level. There are studies demonstrating literature search. AS, GB, and EDC carried out literature search. that data routinely recorded from milking robots provide YG defined and performed company search. AS wrote the paper Frontiers in Veterinary Science | www.frontiersin.org 12 March 2021 | Volume 8 | Article 634338 Stygar et al. PLF for Dairy Welfare Assessment with input from all the co-authors. All authors contributed to the ACKNOWLEDGMENTS article and approved the submitted version. We would like to thank members of ClearFarm consortium for FUNDING their valuable comments. This study was conducted within the ClearFarm project which aim is to co-design, develop, and validate a software SUPPLEMENTARY MATERIAL platform powered by PLF technologies to provide animal welfare information. This project has received funding from The Supplementary Material for this article can be found the European Union’s Horizon 2020 research and innovation online at: https://www.frontiersin.org/articles/10.3389/fvets. program under grant agreement No. 862919. 2021.634338/full#supplementary-material REFERENCES 16. Altman DG, Vergouwe Y, Royston P, Moons KGM. Prognosis and prognostic research: validating a prognostic model. BMJ. (2009) 338:b605. 1. Pinillos RG, Appleby MC, Manteca X, Scott-Park F, Smith C, Velarde A. One doi: 10.1136/bmj.b605 welfare – a platform for improving human and animal welfare. Vet Rec. (2016) 17. Bikker JP, van Laar H, Rump P, Doorenbos J, van Meurs K, Griffioen GM, 179:412–3. doi: 10.1136/vr.i5470 et al. Technical note: evaluation of an ear-attached movement sensor to 2. Stygar AH, Chantziaras I, Toppari I, Maes D, Niemi JK. High biosecurity record cow feeding behavior and activity. J Dairy Sci. (2014) 97:2974–9. and welfare standards in fattening pig farms are associated with reduced doi: 10.3168/jds.2013-7560 antimicrobial use. Animal. (2020) 14:1–9. doi: 10.1017/S1751731120000828 18. Grinter LN, Campler MR, Costa JHC. Technical note: validation of a 3. von Keyserlingk MAG, Rushen J, de Passillé AM, Weary DM. Invited review: behavior-monitoring collar’s precision and accuracy to measure rumination, the welfare of dairy cattle—Key concepts and the role of science. J Dairy Sci. feeding, and resting time of lactating dairy cows. J Dairy Sci. (2019) 102:3487– (2009) 92:4101–11. doi: 10.3168/jds.2009-2326 94. doi: 10.3168/jds.2018-15563 4. Llonch P, Somarriba M, Duthie C-A, Haskell MJ, Rooke JA, Troy S, 19. Van Hertem T, Tello AS, Viazzi S, Steensels M, Bahr C, Romanini et al. Association of temperament and acute stress responsiveness with CEB, et al. Implementation of an automatic 3D vision monitor for dairy productivity, feed efficiency, and methane emissions in beef cattle: an cow locomotion in a commercial farm. Biosyst Eng. (2018) 173:166–75. observational study. Front Vet Sci. (2016) 3:43. doi: 10.3389/fvets.2016.00043 doi: 10.1016/j.biosystemseng.2017.08.011 5. European Commission. Attitudes of EU Citizens Towards Animal Welfare, 20. Welfare Quality R . Welfare Quality R Assessment Protocol for Cattle. Welfare Report; Special Eurobarometer 442. Brussels: European Commission (2016). Quality R Consortium, Lelystad (2009). Available online at: http://www. 6. European Commission. Online Consultation on the Future of Europe. welfarequalitynetwork.net/media/1088/cattle_protocol_without_veal_calves. Second Interim Report. (2019). Available online at: https://ec.europa.eu/ pdf (accessed September 18, 2020). commission/sites/beta-political/files/online-consultation-report-april-2019_ 21. Borchers MR, Chang YM, Tsai IC, Wadsworth BA, Bewley JM. A validation of en.pdf (accessed September 18, 2020). technologies monitoring dairy cow feeding, ruminating, and lying behaviors. 7. Yang W, Renwick A. Consumer willingness to pay price premiums for J Dairy Sci. (2016) 99:7458–66. doi: 10.3168/jds.2015-10843 credence attributes of livestock products – a meta-analysis. J Agric Econ. 22. Henriksen JC, Munksgaard L. Validation of AfiTagII, a device for (2019) 70:618–39. doi: 10.1111/1477-9552.12323 automatic measuring of lying behaviour in Holstein and Jersey 8. Heinola K, Latvala T, Raussi S, Kauppinen T, Niemi J. Kuluttajanäkökulmia cows on two different bedding materials. Animal. (2019) 13:617–21. eläinten hyvinvointimerkin kehittämiseen. SMST. (2020) 38:1–12. doi: 10.1017/S1751731118001623 doi: 10.33354/smst.89454 23. Mattachini G, Antler A, Riva E, Arbel A, Provolo G. Automated measurement 9. Clark B, Panzone LA, Stewart GB, Kyriazakis I, Niemi JK, Latvala T, et al. of lying behavior for monitoring the comfort and welfare of lactating dairy Consumer attitudes towards production diseases in intensive production cows. Livestock Sci. (2013) 158:145–50. doi: 10.1016/j.livsci.2013.10.014 systems. PLoS ONE. (2019) 14:e0210432. doi: 10.1371/journal.pone.0210432 24. Swartz TH, McGilliard ML, Petersson-Wolfe CS. Technical note: the use of an 10. Riemsdijk L, Ingenbleek PTM, Veen G, Trijp HCM. Positioning strategies for accelerometer for measuring step activity and lying behaviors in dairy calves. animal-friendly products: a social dilemma approach. J Consum Aff. (2020) J Dairy Sci. (2016) 99:9109–13. doi: 10.3168/jds.2016-11297 54:100–29. doi: 10.1111/joca.12240 25. Kaniyamattam K, De Vries A. Agreement between milk fat, protein, and 11. Rojo-Gimeno C, van der Voort M, Niemi JK, Lauwers L, Kristensen AR, lactose observations collected from the Dairy Herd Improvement Association Wauters E. Assessment of the value of information of precision livestock (DHIA) and a real-time milk analyzer. J Dairy Sci. (2014) 97:2896–908. farming: a conceptual framework. NJAS Wageningen J Life Sci. (2019) 90– doi: 10.3168/jds.2013-7690 1:100311. doi: 10.1016/j.njas.2019.100311 26. Mattachini G, Riva E, Bisaglia C, Pompe JCAM, Provolo G. Methodology for 12. Maroto Molina F, Pérez Marín CC, Molina Moreno L, Agüera Buendía EI, quantifying the behavioral activity of dairy cows in freestall barns. J Anim Sci. Pérez Marín DC. Welfare Quality R for dairy cows: towards a sensor-based (2013) 91:4899–907. doi: 10.2527/jas.2012-5554 assessment. J Dairy Res. (2020) 87:28–33. doi: 10.1017/S002202992000045X 27. Nielsen PP, Fontana I, Sloth KH, Guarino M, Blokhuis H. Technical note: 13. Steeneveld W, Hogeveen H, Oude Lansink AGJM. Economic consequences validation and comparison of 2 commercially available activity loggers. J Dairy of investing in sensor systems on dairy farms. Comput Electron Agric. (2015) Sci. (2018) 101:5449–53. doi: 10.3168/jds.2017-13784 119:33–9. doi: 10.1016/j.compag.2015.10.006 28. Shepley E, Berthelot M, Vasseur E. Validation of the ability of a 3D pedometer 14. Steeneveld W, Vernooij JCM, Hogeveen H. Effect of sensor systems for cow to accurately determine the number of steps taken by dairy cows when housed management on milk production, somatic cell count, and reproduction. J in tie-stalls. Agriculture. (2017) 7:53. doi: 10.3390/agriculture7070053 Dairy Sci. (2015) 98:3896–05. doi: 10.3168/jds.2014-9101 29. Werner J, Umstatter C, Leso L, Kennedy E, Geoghegan A, Shalloo L, et al. 15. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred Evaluation and application potential of an accelerometer-based collar device reporting items for systematic reviews and meta-analyses: the PRISMA for measuring grazing behavior of dairy cows. Animal. (2019) 13:2070–9. statement. PLoS Med. (2009) 6:e1000097. doi: 10.1371/journal.pmed.1000097 doi: 10.1017/S1751731118003658 Frontiers in Veterinary Science | www.frontiersin.org 13 March 2021 | Volume 8 | Article 634338 Stygar et al. PLF for Dairy Welfare Assessment 30. Brassel J, Rohrssen F, Failing K, Wehrend A. Automated detection of health 48. Mullins IL, Truman CM, Campler MR, Bewley JM, Costa JHC. Validation of a disorders in lactating dairy cattle on pasture: a preliminary study. Polish J Vet commercial automated body condition scoring system on a commercial dairy Sci. (2019) 22:761−7. doi: 10.24425/pjvs.2019.131406 farm. Animals. (2019) 9:287. doi: 10.3390/ani9060287 31. Schirmann K, von Keyserlingk MAG, Weary DM, Veira DM, Heuwieser W. 49. Cantor MC, Pertuisel CH, Costa JHC. Technical note: estimating body weight Technical note: validation of a system for monitoring rumination in dairy of dairy calves with a partial-weight scale attached to an automated milk cows. J Dairy Sci. (2009) 92:6052–5. doi: 10.3168/jds.2009-2361 feeder. J Dairy Sci. (2020) 103:1914–9. doi: 10.3168/jds.2019-16918 32. Burfeind O, Schirmann K, von Keyserlingk MAG, Veira DM, Weary DM, 50. Neubauer V, Humer E, Kroeger I, Braid T, Wagner M, Zebeli Q. Differences Heuwieser W. Technical note: evaluation of a system for monitoring between pH of indwelling sensors and the pH of fluid and solid phase in the rumination in heifers and calves. J Dairy Sci. (2011) 94:426–30. rumen of dairy cows fed varying concentrate levels. J Anim Physiol Anim Nutr. doi: 10.3168/jds.2010-3239 (2018) 102:343–9. doi: 10.1111/jpn.12675 33. Ruuska S, Kajava S, Mughal M, Zehner N, Mononen J. Validation of 51. Falk M, Münger A, Dohme-Meier F. Technical note: a comparison of a pressure sensor-based system for measuring eating, rumination and reticular and ruminal pH monitored continuously with 2 measurement drinking behaviour of dairy cattle. Appl Anim Behav Sci. (2016) 174:19–23. systems at different weeks of early lactation. J Dairy Sci. (2016) 99:1951–5. doi: 10.1016/j.applanim.2015.11.005 doi: 10.3168/jds.2015-9725 34. Steinmetz M, von Soosten D, Hummel J, Meyer U, Daenicke S. Validation of 52. Lohoelter M, Rehage R, Meyer U, Lebzien P, Rehage J, Daenicke S. Evaluation the RumiWatch Converter V0.7.4.5 classification accuracy for the automatic of a device for continuous measurement of rumen pH and temperature monitoring of behavioural characteristics in dairy cows. Arch Anim Nutr. considering localization of measurement and dietary concentrate proportion. (2020) 74:164–72. doi: 10.1080/1745039X.2020.1721260 Landbauforschung. (2013) 63:61–8. doi: 10.3220/LBF_2013_61-68 35. Kroeger I, Humer E, Neubauer V, Kraft N, Ertl P, Zebeli Q. Validation 53. Knauer WA, Godden SM, McDonald N. Technical note: preliminary of a noseband sensor system for monitoring ruminating activity in evaluation of an automated indwelling rumen temperature bolus cows under different feeding regimens. Livestock Sci. (2016) 193:118–22. measurement system to detect pyrexia in preweaned dairy calves. J Dairy Sci. doi: 10.1016/j.livsci.2016.10.007 (2016) 99:9925–30. doi: 10.3168/jds.2015-10770 36. Zehner N, Umstatter C, Niederhauser JJ, Schick M. System specification and 54. Bicalho RC, Cheong SH, Cramer G, Guard CL. Association between a visual validation of a noseband pressure sensor for measurement of ruminating and and an automated locomotion score in lactating Holstein cows. J Dairy Sci. eating behavior in stable-fed cows. Comput Electron Agric. (2017) 136:31–41. (2007) 90:3294–300. doi: 10.3168/jds.2007-0076 doi: 10.1016/j.compag.2017.02.021 55. Hoffmann G, Schmidt M, Ammon C, Rose-Meierhoefer S, Burfeind O, 37. Eslamizad M, Tuemmler L-M, Derno M, Hoch M, Kuhla B. Technical Heuwieser W, et al. Monitoring the body temperature of cows and calves using note: development of a pressure sensor-based system for measuring video recordings from an infrared thermography camera. Vet Res Commun. rumination time in pre-weaned dairy calves. J Anim Sci. (2018) 96:4483–9. (2013) 37:91–9. doi: 10.1007/s11259-012-9549-3 doi: 10.1093/jas/sky337 56. Schlageter-Tello A, Van Hertem T, Bokkers EAM, Viazzi S, Bahr C, 38. Werner J, Leso L, Umstatter C, Niederhauser J, Kennedy E, Geoghegan Lokhorst K. Performance of human observers and an automatic 3- A, et al. Evaluation of the RumiWatchSystem for measuring dimensional computer-vision-based locomotion scoring method to detect grazing behaviour of cows. J Neurosci Methods. (2018) 300:138–46. lameness and hoof lesions in dairy cows. J Dairy Sci. (2018) 101:6322–35. doi: 10.1016/j.jneumeth.2017.08.022 doi: 10.3168/jds.2017-13768 39. Zambelis A, Wolfe T, Vasseur E. Technical note: validation of an ear-tag 57. de Mol RM, Ouweltjes W, Kroeze GH, Hendriks M. Detection of estrus and accelerometer to identify feeding and activity behaviors of tiestall-housed mastitis: field performance of a model. Appl Eng Agric. (2001) 17:399–407. dairy cattle. J Dairy Sci. (2019) 102:4536–40. doi: 10.3168/jds.2018-15766 doi: 10.13031/2013.6201 40. Pereira GM, Heins BJ, Endres MI. Technical note: validation of an 58. Sorensen LP, Bjerring M, Lovendahl P. Monitoring individual cow udder ear-tag accelerometer sensor to determine rumination, eating, and health in automated milking systems using online somatic cell counts. J Dairy activity behaviors of grazing dairy cattle. J Dairy Sci. (2018) 101:2492–5. Sci. (2016) 99:608–20. doi: 10.3168/jds.2014-8823 doi: 10.3168/jds.2016-12534 59. Tamura T, Okubo Y, Deguchi Y, Koshikawa S, Takahashi M, Chida Y, 41. Reynolds MA, Borchers MR, Davidson JA, Bradley CM, Bewley JM. Technical et al. Dairy cattle behavior classifications based on decision tree learning note: an evaluation of technology-recorded rumination and feeding behaviors using 3-axis neck-mounted accelerometers. Anim Sci J. (2019) 90:589–96. in dairy heifers. J Dairy Sci. (2019) 102:6555–8. doi: 10.3168/jds.2018-15635 doi: 10.1111/asj.13184 42. Elischer MF, Arceo ME, Karcher EL, Siegford JM. Validating the accuracy 60. Stygar AH, Kristensen AR. Detecting abnormalities in pigs’ growth – of activity and rumination monitor data from dairy cows housed in a a dynamic linear model with diurnal growth pattern for identified pasture-based automatic milking system. J Dairy Sci. (2013) 96:6412–22. and unidentified pigs. Comput Electron Agric. (2018) 155:180–9. doi: 10.3168/jds.2013-6790 doi: 10.1016/j.compag.2018.10.004 43. Deng Z, Hogeveen H, Lam TJGM, van der Tol R, Koop G. Performance of 61. ICAR. The Global Standard for Livestock Data. Section 11 -Guidelines for online somatic cell count estimation in automatic milking systems. Front Vet Testing, Approval and Checking of Milk Recording Devices. (2020). Available Sci. (2020) 7:221. doi: 10.3389/fvets.2020.00221 online at: https://www.icar.org/Guidelines/11-Milk-Recording-Devices.pdf 44. DeVries TJ, von Keyserlingk MAG, Weary DM, Beauchemin KA. Technical 62. Schweinzer V, Gusterer E, Kanz P, Krieger S, Suess D, Lidauer L, note: validation of a system for monitoring feeding behavior of dairy et al. Evaluation of an ear-attached accelerometer for detecting estrus cows. J Dairy Sci. (2003) 86:3571–4. doi: 10.3168/jds.S0022-0302(03) events in indoor housed dairy cows. Theriogenology. (2019) 130:19–25. 73962-9 doi: 10.1016/j.theriogenology.2019.02.038 45. Chapinal N, Veira DM, Weary DM, von Keyserlingk MAG. Technical 63. Bruinjé TC, Ambrose DJ. Technical note: validation of an automated in-line note: validation of a system for monitoring individual feeding and drinking milk progesterone analysis system to diagnose pregnancy in dairy cattle. J behavior and intake in group-housed cattle. J Dairy Sci. (2007) 90:5732–6. Dairy Sci. (2019) 102:3615–21. doi: 10.3168/jds.2018-15692 doi: 10.3168/jds.2007-0331 64. Guzhva O, Ardo H, Nilsson M, Herlin A, Tufvesson L. Now you see me: 46. Chizzotti ML, Machado FS, Valente EEL, Pereira LGR, Campos MM, Tomich convolutional neural network based tracker for dairy cows. Front Robot Ai. TR, et al. Technical note: validation of a system for monitoring individual (2018) 5:107. doi: 10.3389/frobt.2018.00107 feeding behavior and individual feed intake in dairy cattle. J Dairy Sci. (2015) 65. Wurtz K, Camerlink I, D’Eath RB, Fernández AP, Norton T, Steibel J, 98:3438–42. doi: 10.3168/jds.2014-8925 et al. Recording behaviour of indoor-housed farm animals automatically 47. Oliveira BR, Ribas MN, Machado FS, Lima JAM, Cavalcanti LFL, Chizzotti using machine vision technology: a systematic review. PLoS ONE. (2019) ML, et al. Validation of a system for monitoring individual feeding and 14:e0226669. doi: 10.1371/journal.pone.0226669 drinking behaviour and intake in young cattle. Animal. (2018) 12:634–9. 66. Schirmann K, Weary DM, Heuwieser W, Chapinal N, Cerri RLA, von doi: 10.1017/S1751731117002002 Keyserlingk MAG. Short communication: rumination and feeding behaviors Frontiers in Veterinary Science | www.frontiersin.org 14 March 2021 | Volume 8 | Article 634338 Stygar et al. PLF for Dairy Welfare Assessment differ between healthy and sick dairy cows during the transition period. J wearable accelerometers in research to monitor lying behaviors of dairy cows. Dairy Sci. (2016) 99:9917–24. doi: 10.3168/jds.2015-10548 J Dairy Sci. (2020) 103:12140–57. doi: 10.3168/jds.2019-17887 67. Roche JR, Friggens NC, Kay JK, Fisher MW, Stafford KJ, Berry DP. 84. Tedeschi LO. Assessment of the adequacy of mathematical models. Agric Syst. Invited review: body condition score and its association with dairy (2006) 89:225–47. doi: 10.1016/j.agsy.2005.11.004 cow productivity, health, and welfare. J Dairy Sci. (2009) 92:5769–801. 85. Dominiak KN, Kristensen AR. Prioritizing alarms from sensor-based doi: 10.3168/jds.2009-2431 detection models in livestock production - a review on model performance 68. Bloch V, Levit H, Halachmi I. Assessing the potential of photogrammetry and alarm reducing methods. Comput Electron Agric. (2017) 133:46–67. to monitor feed intake of dairy cows. J Dairy Res. (2019) 86:34–9. doi: 10.1016/j.compag.2016.12.008 doi: 10.1017/S0022029918000882 86. Heringstad B, Kjøren Bugten H. Genetic evaluations based on data from 69. Phillips N, Mottram T, Poppi D, Mayer D, McGowan MR. Continuous automatic milking systems. In: 39th ICAR Session, May 19-23. Berlin (2014). monitoring of ruminal pH using wireless telemetry. Anim Prod Sci. (2010) 87. Veldhuis A, Brouwer-Middelesch H, Marceau A, Madouasse A, Van der Stede 50:72. doi: 10.1071/AN09027 Y, Fourichon C, et al. Application of syndromic surveillance on routinely 70. Huzzey JM, Weary DM, Tiau BYF, von Keyserlingk MAG. Short collected cattle reproduction and milk production data for the early detection communication: automatic detection of social competition using an electronic of outbreaks of Bluetongue and Schmallenberg viruses. Prev Vet Med. (2016) feeding system. J Dairy Sci. (2014) 97:2953–8. doi: 10.3168/jds.2013-7434 124:15–24. doi: 10.1016/j.prevetmed.2015.12.006 71. Haley DB, Rushen J, de Passillé AM. Behavioural indicators of cow comfort: 88. Buller H, Blokhuis H, Lokhorst K, Silberberg M, Veissier I. Animal activity and resting behaviour of dairy cows in two types of housing. Can J welfare management in a digital world. Animals. (2020) 10:1779. Anim Sci. (2000) 80:257–63. doi: 10.4141/A99-084 doi: 10.3390/ani10101779 72. Tucker CB, Jensen MB, de Passillé AM, Hänninen L, Rushen J. Invited review: 89. Council on Animal Affairs (RDA). Digitisation of the Livestock Farming Sector. lying time and the welfare of dairy cows. J Dairy Sci. (2020) 104:20–46. (2019). Available online at: https://english.rda.nl/publications/publications/ doi: 10.3168/jds.2019-18074 2020/02/13/digitisation-of-the-livestock-farming-sector (accessed January 4, 73. Olmos G, Boyle L, Hanlon A, Patton J, Murphy JJ, Mee JF. Hoof 2020). disorders, locomotion ability and lying times of cubicle-housed 90. de Vries M, Bokkers EAM, Dijkstra T, van Schaik G, de Boer IJM. Invited compared to pasture-based dairy cows. Livestock Sci. (2009) 125:199–207. review: associations between variables of routine herd data and dairy doi: 10.1016/j.livsci.2009.04.009 cattle welfare indicators. J Dairy Sci. (2011) 94:3213–28. doi: 10.3168/jds. 74. Li W, Ji Z, Wang L, Sun C, Yang X. Automatic individual identification of 2011-4169 Holstein dairy cows using tailhead images. Comput Electron Agric. (2017) 91. Nielsen S, Denwood M, Forkman B, Houe H. Selection of meat inspection data 142:622–31. doi: 10.1016/j.compag.2017.10.029 for an animal welfare index in cattle and pigs in Denmark. Animals. (2017) 75. Jensen DB, Hogeveen H, De Vries A. Bayesian integration of sensor 7:94. doi: 10.3390/ani7120094 information and a multivariate dynamic linear model for prediction of dairy 92. COPA-COGECA, CEMA. EU Code of Conduct on Agricultural Data Sharing cow mastitis. J Dairy Sci. (2016) 99:7344–61. doi: 10.3168/jds.2015-10060 by Contractual Agreement. (2018). Available online at: https://www.fefac.eu/ 76. Groessbacher V, Buckova K, Lawrence AB, Spinka M, Winckler C. files/81630.pdf (accessed February 4, 2020). Discriminating spontaneous locomotor play of dairy calves using 93. Klerkx L, Jakku E, Labarthe P. A review of social science on digital accelerometers. J Dairy Sci. (2020) 103:1866–73. doi: 10.3168/jds.2019-17005 agriculture, smart farming and agriculture 4.0: new contributions and a 77. Roland L, Schweinzer V, Kanz P, Sattlecker G, Kickinger F, Lidauer L, et al. future research agenda. NJAS Wageningen J Life Sci. (2019) 90–1:100315. Evaluation of a triaxial accelerometer for monitoring selected behaviors in doi: 10.1016/j.njas.2019.100315 dairy calves. J Dairy Sci. (2018) 101:10421–7. doi: 10.3168/jds.2018-14720 94. Perakis K, Lampathaki F, Nikas K, Georgiou Y, Marko O, Maselyne 78. Foris B, Thompson AJ, von Keyserlingk MAG, Melzer N, Weary J. CYBELE – fostering precision agriculture & livestock farming DM. Automatic detection of feeding- and drinking-related agonistic through secure access to large-scale HPC enabled virtual industrial behavior and dominance in dairy cows. J Dairy Sci. (2019) 102:9176–86. experimentation environments fostering scalable big data analytics. doi: 10.3168/jds.2019-16697 Comput Netw. (2020) 168:107035. doi: 10.1016/j.comnet.2019. 79. McDonald P, von Keyserlingk MAG, Weary DM. Technical note: using an 107035 electronic drinker to monitor competition in dairy cows. J Dairy Sci. (2019) 102:3495–500. doi: 10.3168/jds.2018-15585 Conflict of Interest: The authors declare that the research was conducted in the 80. Mellor DJ. Animal emotions, behaviour and the promotion of positive welfare absence of any commercial or financial relationships that could be construed as a states. N Z Vet J. (2012) 60:1–8. doi: 10.1080/00480169.2011.619047 potential conflict of interest. 81. Proctor HS, Carder G. Can ear postures reliably measure the positive emotional state of cows? Appl Anim Behav Sci. (2014) 161:20–7. Copyright © 2021 Stygar, Gómez, Berteselli, Dalla Costa, Canali, Niemi, Llonch and doi: 10.1016/j.applanim.2014.09.015 Pastell. This is an open-access article distributed under the terms of the Creative 82. Proctor HS, Carder G. Nasal temperatures in dairy cows are Commons Attribution License (CC BY). The use, distribution or reproduction in influenced by positive emotional state. Physiol Behav. (2015) 138:340–4. other forums is permitted, provided the original author(s) and the copyright owner(s) doi: 10.1016/j.physbeh.2014.11.011 are credited and that the original publication in this journal is cited, in accordance 83. Hendriks SJ, Phyn CVC, Huzzey JM, Mueller KR, Turner S-A, Donaghy DJ, with accepted academic practice. No use, distribution or reproduction is permitted et al. Graduate student literature review: evaluating the appropriate use of which does not comply with these terms. Frontiers in Veterinary Science | www.frontiersin.org 15 March 2021 | Volume 8 | Article 634338