A systematic review on commercially available and validated sensor technologies for welfare assessment of dairy cattle Stygar1*A.H., Gómez3 Y., Berteselli4 G.V., Dalla Costa4 E., Canali4 E., Niemi1 J.K., Llonch3 P, Pastell2 M. 1 Natural Resources Institute Finland (Luke), Bioeconomy and Environment, Latokartanonkaari 9, 00790 Helsinki (Finland) 2 Natural Resources Institute Finland (Luke), Production systems, Latokartanonkaari 9, 00790 Helsinki (Finland) 3 Universitat Autònoma de Barcelona, Department of Animal and Food Science, Campus UAB, 08193 Cerdanyola del Vallès, Barcelona (Spain) 4 Università degli Studi di Milano, Dipartimento di Medicina Veterinaria, (Italy) * Correspondence: Anna Helena Stygar anna.stygar@luke.fi Keywords: PLF, accelerometer, camera, milk sensor, scale, bolus, dairy cow, calf Abstract In order to base welfare assessment of dairy cattle on real-time measurement, integration of valid and reliable precision livestock farming (PLF) technologies is needed. The aim of this study was to provide a systematic overview of externally validated and commercially available PLF technologies which could be used for sensor-based welfare assessment in dairy cattle. Following PRISMA guidelines, a systematic literature review was conducted to identify externally validated sensor technologies. Out of 1,111 publications initially extracted from data bases only 42 studies describing 30 tools (including prototypes) met requirements for external validation. Moreover, through market search, 129 different retailed technologies with application for animal-based welfare assessment were identified. In total, only 18 currently retailed sensors have been externally validated (14%). The highest validation rate was found for systems based on accelerometers (30% of tools available on the market have validation records), while the lower rates were obtained for cameras (10%), load cells (8%), miscellaneous milk sensors (8%) and boluses (7%). Validated traits concerned animal activity, feeding and drinking behavior, physical condition and health of animals. Majority of tools was validated on adult cows. Non-active behavior (lying and standing) and rumination were the most often validated for the high performance. Regarding active behavior (e.g. walking), lower performance of tools was reported. Also, tools use for physical condition (e.g. body condition scoring) and health evaluation (e.g. mastitis detection) were classified in lower performance group. The precision and accuracy of feeding and drinking assessment varied depending on measured trait and used sensor. Regarding relevance for animal-based welfare assessment, several validated technologies had application for good health (e.g. milk quality sensors) and good feeding (e.g. load cells, accelerometers). Accelerometers-based systems have also practical relevance to assess good housing. However, currently available PLF technologies have low potential to assess appropriate behavior of dairy cows. To increase actors’ trust towards the PLF technology and prompt sensor-based welfare assessment, validation studies, especially in commercial herds, are needed. Future research should concentrate on developing and validating PLF technologies dedicated for assessment of appropriate behavior and tools dedicated for monitoring health and welfare in calves and heifers. 1 Introduction Recently introduced concept of One Welfare recognises the interconnections between animal welfare, human wellbeing and the environment (1). Better understanding of the values of high welfare standards can, among others, support food security, improve productivity, reduced antimicrobial use and greenhouse gas emission (e.g. (2–4)). Animal welfare is also a highly interesting topic for the European consumers (5,6). This interest is seen in production statistics and consumer purchases decisions. Consumers are willing to pay a price premium for credence attributes of milk (7,8), such as organic, environmentally friendly or high animal welfare (on average 28, 25 and 31% of premium). Moreover, consumers appreciate proactive approach to manage animal health and welfare (9) and there is evidence that the animal-friendly marketing strategies influences the uptake of products (10). Animal welfare friendly products can be identified through labelling. Most dairy welfare labelling schemes in Europe have requirements concerning resource-based welfare indicators such as space allowance, provision of bedding and enrichments, minimum transportation time, outdoor access or absence of mutilations (8). Recently, animal-based indicators have gained more attention, especially following the publication of Welfare Quality (WQ®) protocols and a few labelling schemes highlighting animal-based measures have been introduced during the past years (e.g. AENOR welfare certificate in Spain, Arla one farm milk in Finland, Classyfarm in Italy). However, existing animal welfare assessment protocols show some inaccuracies as: 1) they are only applied at group level, 2) are unable to continuously monitor animal welfare and 3) they rely on human judgments and decisions- making facilitating some degree of subjectivity on the assessment. Moreover, those protocols are not practical for detecting early-warning signals which could result in implementation of preventive measures. Above mentioned limitations could be, at least partially, solved by application of precision livestock farming (PLF) technologies. Different PLF techniques have been developed for monitoring dairy cattle production. Sudden change in animals’ activity, feeding and drinking, physical condition and health can be detected by different sensors [e.g. radio-frequency identification (RFID), accelerometers, load cells, cameras]. Change in behaviour or in physical state may indicate problems related to management (e.g. feeding system failure) or disease, as well as can signal specific physiological status such as oestrus. PLF technologies can potentially add value to the farm management process by improving data processing, decision making and implementation of everyday herd management decisions (11). Moreover, PLF could also 2 be applied for monitoring animal welfare (e.g. (12)). On the other hand, as demonstrated by large-scale studies (13,14) investment in sensor systems might not necessarily lead to the economic gain for farmers. Therefore, the merits of each sensor system need to be assessed individually and the performance should to be verified before the promise of improved management can be realized. Research groups and companies around the world has been engaged in developing new PLF sensors, however, not all PLF solutions developed in a lab environment can be successfully implemented as commercial products on dairy farms. The reason can be that some technologies will still be too expensive or will perform better in an experimental setting, where conditions are controlled, and sample size is small, compared to the farming environment. Therefore, for successful assessment of on-farm welfare using PLF technology, it is essential to validate this technology at commercial level (external validation). Furthermore, applying sensor-based welfare assessment for labelling schemes or welfare support payments should be based on widely available and validated technologies. The main goal of this review was to assess which welfare aspects of cows’, heifers’ and calves’ husbandry can be addressed by available (and validated) technologies. To reach this goal commercially available and/or externally validated technologies with potential use for animal-based welfare assessment in dairy herds were first identified. Validated technologies were later grouped according to their performance. Finally, possible gaps between available and validated tools and needs for animal- based welfare assessment were identified based on the principles of the WQ® protocol, including appropriate nutrition, housing, health and behaviour. 2 Material and methods Market availability search A broad market research (using web Google search) on commercially available PLF systems with potential application for animal-based welfare assessment was conducted. This research was done by exploring the assortment of technology providers that cover a wide range of sensors which could provide information on animal base indicators for welfare. The search criteria used included “dairy cow” and one of the following terms describing sensors: (automatic drinker OR automatic waterer), (automatic feeder), (activity sensor OR activity monitor), (RFID), (GPS), (thermal camera),(thermography), (mastitis sensor), (automatic mastitis detection), (somatic cells counter), (milk analyzer), (automatic weigh scale OR automatic weigh), (lameness sensor), (automatic lameness detection), (pressure mat OR force sensor), (body condition score sensor OR automatic body condition score), (body condition camera), (rumen bolus), (milking robot), (accelerometer). Also search for calf automatic feeder was performed. As an example, the following word combination was used to look for feeding equipment available on the market: “dairy cow” plus “automatic feeder”. The first 5 pages (50 hits) from Google search were scanned. Additionally, the availability of sensors was scanned using dedicated on-line marketplace for providers (https://www.agriexpo.online/). Search was performed between March and May 2020. Tools with exclusive use for reproduction (for oestrus detection or calving alarms) were excluded from the final list. Information on sensor name, provider name, internet link, sensor type (with attachment position to animal when applicable), aim, and country of origin (headquarters) for 129 technologies were provided in Supplementary Material 1. Literature search and exclusion criteria 3 To explore technology limitation, a systematic literature search based on Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology (15) was conducted. Literature search was conducted through Web of Science and Scopus. Altogether, 147 different search terms were used. Each search included terms describing different phases in the production cycle (“cow” OR “calves” OR “calf” OR “heifer”) and validation (“test” OR “assess*” OR “evaluat*” OR “validat*”) as well as several exclusion terms: NOT (“review” OR “survey” OR “beef” OR “sheep” OR “goat*” OR “hors*” OR “buffalo” OR “steer*” OR “ewe” OR "leg calf" OR “muscle*”). Additionally, each search was supplemented with physiological and behavioural term (e.g. feeding behaviour), or sensor type (e.g. camera) or the commercial name (e.g. CowView). For physiological and behavioural term as well as sensor types following terms were used: (“feeding behaviour” OR “feeding behavior” AND “monitoring”), (“monitoring feeding”), (“drinking behaviour” OR “drinking behavior” AND “monitoring”), (“vocalization”), (“vision”), (“camera”), (“accelerometer*”) , (“temperature AND sensor”), (“mastitis AND sensor”), (“image analyses”), (“scale AND body weight”), (“pressure mat”), (“bolus”), (“indoor AND position”), (“in- line”),(“tracking system”), (“RFID”), (“microphone”). The commercial names used in the search are presented in Appendix table 1, column A. The example search looked as follow: (“cow” OR “calves” OR “calf” OR “heifer”) AND (“test” OR “assess*” OR “evaluat*” OR “validat*”) AND (“camera”) NOT (“review” OR “survey” OR “beef” OR “sheep” OR “goat*” OR “hors*” OR “buffalo” OR “steer*” OR “ewe” OR "leg calf" OR “muscle*”). Only studies in peer-reviewed scientific journals published in English between January 2000 and May 2020 were included to this review. Since this review focuses on dairy production, all validation trials conducted on beef breeds or steers were omitted. Articles were excluded if not dealing with aspects directly related to welfare of dairy cows (e.g. reproduction related problems such as estrus detection, environmental aspects such as methane emission, etc.). We further excluded papers with only internal validation, which was defined as validation dataset used to assess the performance originating from the same animals or herd/herds as used in the developing of the technology (16). Study classification This review includes papers presenting the higher standards of objective validation, which is external validation. Based on the approach presented by Altman et al., (16) we have defined two levels of external validation: 1. External self-validation was defined for studies where the system was evaluated using a fully independent data set, that means data was collected from different herds not used for system development. Research was conducted by the same scientist (at least one author involved in developing and validation) or had been validated by at least one author representing a company providing a technology. 2. External independent validation was defined for studies where the system was evaluated using a fully independent data set, that means data was collected from different herds not used for system building. Research was conducted by scientists not involved in technology development. In order to determine the validation level, both origin of the technology and validation location (herd) needed to be known. Technology was identified through commercial name or based on studies describing building phase (for prototypes). Origin of the validation herd was identified through 4 information on location (country), and type (if a herd was commercial or research). We have assumed that criteria of external validation were fulfilled if commercially available technology was validated in a commercial herd or a research herd (different from the company/developer own research herd). For prototypes, the criteria of external validation were fulfilled only if the scientific paper clearly described where technology was validated, and validation place was different from the herd used for technology building (based on information from scientific publication describing building phase). If both country and herd specification (commercial or research) could not be identified, then study was excluded from this review (due to not enough information in material and methods). However, papers stating that herds used for validation where different than those used for technology development (without mentioning location, for example due to privacy concerns) were included into this review. Performance measures for validated trials In this review, we distinguished regression and classification measures for performance reporting. Regression measures, reflects the agreement between a continuous trait measured by validated technology (predictor) and the golden standard (outcome). For example, the agreement between body weight measured by conventional scale and partial scale attached to milk feeder. Regression measures can be presented using any of the following measures including Pearson correlation coefficient (r), or Spearman’s rank correlation coefficient (rs), or coefficient of determination (R2), or mean bias from the Bland-Altman plots (B-A plots), or significance tests for intercept and slope of linear regression (I/S) or concordance correlation coefficient (CCC). Classification measures refers to the ability of a technology to predicting categorical outcomes e.g. locomotion score. Classification performance was usually reported using either area under the receiver operating characteristics curve (AUC) or sensitivity (Se) and specificity (Sp) or Cohen's kappa coefficient (κ). In this review, we have distinguished tools validated for high performance and lower performance. It was assumed that high performance was reached when all indicators defined/selected by authors of studies fulfilled following criteria: r, rs, CCC, Se and Sp or AUC was greater than 0.9, R2 and κ was greater than 0.81, I/S did not differ significantly from 0 or 1, respectively and B-A plot included zero with the 95% interval of agreement. Criteria for high performance (precision and accuracy) were accepted similarly to those referred by studies assessing technology performance (17–19). Assessment of welfare relevance WQ® is a scientifically rigorous animal welfare assessment protocol (20) which follows four animal welfare principles (good housing, good feeding, good health and appropriate behaviour). WQ® principles were used as a reference to classify indicators measured by technologies. In this review, members of the ClearFarm project with expertise in animal welfare were asked to evaluate the relevance of each indicator measured by PLF technology listed in this review for assessing WQ® principle. Possible scores were: “relevant” and “not relevant”. For example, the panel was asked to evaluate whether grazing time is relevant for the principles of good feeding, good housing, good health and appropriate behaviour. Experts votes were categorized based on “relevant” votes, so that all traits with more than 80% votes were grouped in “very relevant” category, traits receiving from above 20% up to 80% votes were in “moderately relevant” category and all traits with 20% or less votes were in “not relevant” category. 3 Results Technologies commercially available 5 The full list of commercially available technologies is presented in the Supplementary Material 1. In total, 129 technologies were found from 70 different providers located in 23 countries. The United Kingdom, the Netherlands and the United States, are the leaders for providing technologies with potential use for animal-based welfare assessment. Technologies were grouped according to the used sensor. Accelerometer-based technologies and load cells were the largest group on the list (37 different products for each group) and constituted 57% of all found tools. Commercially available accelerometers were offered with different animal attachment solutions (collar, leg, ear, halter) and some companies offered products with more than one attachment option. Collar was the most popular solution (65%, N=24), while leg (30%, N=10), ear (14%, N=6) and halter (3%, N=1) were less frequent. We have identified 14 bolus and 10 products using vision-based monitoring. Regarding milk quality, 25 sensor technologies (19% of a market share) for health monitoring were identified (including 13 milking robots). GPS sensors were used in eight different products offering the possibility to locate animal position. Additionally, two systems using microphone, as well as one mobile app for body condition scoring was identified. All products based on accelerometers offered health alerts. Only one accelerometer-based product was dedicated for calves, remaining were advertised for cows or heifers. Systems based on load cells in combination with radio-frequency identification (RFID), were most often used for managing and tracking the feeding program of individual animals. Also, few systems were used for body weight monitoring. Boluses were advertised as tools to measure body temperature, pH, rumen activity as well as for animal identification. Among cameras, seven were dedicated for body temperature monitoring (thermal cameras), two were used for body condition scoring (BCS), and one camera for feeding monitoring. Peer-reviewed records on technology validation The literature search resulted in 1,111 titles, but after duplicate removal and exclusion criteria throughout the review process 1,069 papers were omitted. A modified PRISMA flow diagram provides information on the number of excluded papers and reason for exclusion (Figure 1). A total of 42 articles satisfied the selection criteria. Of the selected 42 publications, most validated commercially available technologies (26 products originating from 20 companies). Moreover, we have identified 4 studies on prototype validation (Table 1). Only 2 papers validated more than 1 product, however several papers validated more than one indicator measured by the technology. The performance of technologies with accelerometer sensors were the most often assessed (26 technology validation trials). Validation trials for load cells (N=6), bolus and camera (4 trials each), RFID (N=3), microphone and viscosity sensor (2 trials each), conductivity and spectroscopy (1 trial each) were less frequent (Table 1). Regarding accelerometers, the precision and accuracy of products offering different attachments to the animal were assessed in 11 sensors (leg (N=5), collar (N=3), ear (N=2) and halter (N=1)). The most often validated technology originated from Itin+ Hoch GmbH, Liestal, Switzerland (6 trials), Afimilk, Kibbutz Afikim, Israel (5 trials) and Agis, Harmelen, Netherlands (5 trials). Information on the study design (herd type, size, location) for all qualified papers is presented in Supplementary Material 2. In total, 28 studies presented validation trials conducted on research farms. The remaining studies (33%, N= 14) were conducted on commercial herds. The sample size used in validation trials varied substantially. In general, the smallest sample size was selected for experiments concerning cannulated cows (below 10 animals for bolus validation, e.g. (21)), while the highest sample size was selected for experiment testing performance of online somatic cell count estimation in automatic milking system (above 4,000 milking cows (22)). When it comes to herds’ geographical location, most technologies were validated in the United States (11 studies), and Canada (5 studies). The performance of tools was assessed using regression measures (27 papers), classification measures (7 papers) and both measures (8 papers). Most of the reviewed papers were classified as full independent validation, and only 31% (N=13) of reviewed papers were self-validated. 6 Validation rate According to obtained results, only 18 commercially available sensors listed in the Supplementary Material 1 have been externally validated (14%). The highest validation rate was found for systems based on accelerometers (30% of tools available on the market have validation records), while the lower rates were obtained for cameras (10%), load cells (8%), miscellaneous milk sensors (8%) and boluses (7%). Performance of technology validated for dairy cows Table 2 summarized tools with available validation trials which could have practical application in welfare assessment for dairy cows. Validated animal-based traits concerned animal activity, (walking, number of steps, lying, lying and standing, standing), feeding and drinking behavior (feeding time, presence at feeder, intake, grazing, rumination, drinking duration, presence at a drinker, water intake) physical condition and health (locomotion score, BCS, rumen pH, body temperature, health disorder detection, milk quality). Non-active behavior (lying, lying and standing, standing) as well as rumination and feeding time were the most often validated attributes (20, 15 and 11 trials respectively). There are several different commercially available technologies classified with high performance for non-active behavior (Table 2). For active behavior (walking, number of steps) lower performance of tools was reported. Regarding feeding and drinking, the performance of the tool varied depending on measured traits and used sensor. Feeding time, which was monitored using accelerometer-based sensors, was validated for lower performance. Conversely, presence at the feeder and feed intake (observed in feeding stations) as well as grazing time (monitored through accelerometer-based sensor) were evaluated for high performance, but only in the research farm conditions. Drinking time, assessed using accelerometer-based tools and the pressure sensor was evaluated for lower performance. All tools used for physical condition evaluation and health were classified for lower performance. Assessment of locomotion score varied between presented tools (poor (23) or fair classification performance (19,24)), and in general none of reviewed technologies was able to overperform human observer. Regarding BCS, the technology was reliable for dairy cattle with average body condition (scoring between 3.00 and 3.75 in 5-point scale) but did not score accurately for thinner or fatter cows. The only validated study on the accelerometer- based system used for health alarms (25), reported a high number of false positives, but the true health disorders were alerted by the system before the farmer noticed them. Regarding technologies applied for monitoring milk quality and mastitis, real-time milk analyzers agreed moderately with somatic cell count (SCC) (22), protein, lactose and fat determined in the laboratory (26). While mastitis detection models have acceptable results for sensitivity, specificity and error rates (27,28). Performance of technology validated for calves and heifers Table 3 summarized tools with available validation trials which could have practical application in welfare assessment for young cattle (calves and heifers). Validated traits concerned active behavior (walking), non-active behavior (lying), feeding (time, presence at the feeder, intake), rumination, drinking (presence at the drinker, intake), body weight and body temperature. For calves and heifers, rumination and body temperature were the most often validated traits (3 and 2 trials, respectively). Tools measuring active and non-active behavior (lying, walking), feeding and drinking behavior (feed and water intake, presence at the drinker or feeder) and body weight were validated for high performance. Feeding time, rumination and body temperature were validated for lower performance. Experts assessment 7 Answers from animal welfare experts concerning the relevance of the indicator in assessing good feeding, housing, health and appropriate behavior were summarized in Table 4. For good health, 9 traits received “very relevant” evaluation (body temperature, BCS, lameness, mastitis, water consumption, rumination, rumen pH, feed intake, non-active behavior). Regarding good feeding, 7 traits were categorized as “very relevant” (BCS, water consumption, rumination, rumen pH, grazing, feed intake, feeding time). For good housing evaluation, experts agreed on the usefulness of non-active behavior monitoring. While, for appropriate behavior only grazing monitoring was evaluated as “very relevant”. 4 Discussion 4.1 Retailed and validated PLF technologies for welfare assessment The aim of this review was to identify validated and/or commercially available technologies for measuring animal-based welfare indicators in dairy cattle. Currently, farmers can select from at least 129 different sensors to monitor animal-based indicators of health and welfare in dairy production. However, there is still limited information on the performance of these tools. According to our results, only 14% of commercially available sensors have external validation trials available, which may thwart confidence on these technologies. We identified 4 potential reasons for such a small number of validation trials: 1) insufficient reporting 2) low scientific interest for validating technology not for research 3) high cost and labor intensity of data collection 4) reluctancy to publish negative results. Regarding reason 1) altogether, 6 studies reporting validation trails were excluded from this review due to insufficient information provided about study design. Reason 2) there might be lower scientific interest to validate technologies which are not used for research experiments or are not yet integrated as data sources for other systems. For example, for many commercially available sensors based on scales (like individual feed intake measurement) there are no validation trials available. However, the required precision for feeding monitoring tools (as well as the interest in validation) might increase if the data from these tools, as in the example from pig production (29), would be integrated into marketing or health monitoring systems. Furthermore, the validation rate could be increased if technologies, similar as medical industry, receive specific certification (e.g. ISO standards). Currently, devices and systems used for purposes of official milk recording (e.g. milk meters, samplers, milk analyzers) need to meet the requirements specified in ISO standards and must be tested to achieve approval from The International Committee for Animal Recording (ICAR) (30). However, the data from the validation process conducted by ICAR is not publicly available. This could also explain a low number of validation records in peer-reviewed literature for milk recoding devices and systems. Reason 3) validation studies can be labor intensive and costly, due to the need to collect the reference data set. For example, accelerometer-based systems are the most widely available and validated among all PLF technologies. But, as demonstrated in this review, the majority of the accelerometer-based validation studies concerned behavioral monitoring and only one validation study for the performance of accelerometer system for health monitoring was found. Validation of health monitoring technology requires obtaining reference data set containing data on veterinary examinations and blood or milk samples to detect among others lameness, mastitis, ketosis, pneumonia. The substantial costs needed for the reference data set might affect the number of available publications. 8 Finally, for point 4), it could be pondered, if the reason behind the relatively small number of validation studies is due to reluctance in publishing negative results. Technology providers are involved in the validation process and altogether, about one third of all validation studies presented in this review were classified as self-validation. Self-validation could raise the question of conflict of interest in reporting negative results. However, it is impossible to conclude how many of the negative results were never published due to the conflict of interest. Certainly, technologies which are commercially available may not all have been identified in this study. The search was conducted using internet websites in English, therefore all tools without English marketing material or presented in printed company catalogues were omitted. The biggest producers will have information provided in English, but smaller companies offering products for local markets or start-ups might not yet have information available for international buyers. Therefore, constructed list of retailed products is an approximation of the current market. Our goal was not to identify every single technology but to use this list to identify tendencies on the market and set possible market constraints for developing sensor-based welfare assessment. One must also remember that not all validation studies available for a device were reported in this review. We have included only validation studies for attributes related to animal welfare, therefore, some validation studies for performance of estrus detection on accelerometer-based devices (e.g. (31)), or pregnancy detection from in-line analyzer (e.g. (32)) were excluded. PLF uses technology for real-time, continuous monitoring of individual animals and/or groups of animals, which provides an opportunity to improve welfare assessment. Applying sensor-based welfare assessment for labelling scheme or welfare subsidies should be based on widely available technologies. This review shows that reliable technologies for monitoring welfare-related traits exist, however there are areas concerning sensors and algorithms which require further developments. For example, based on the presented summary, it can be concluded that while recording behavior of farm animals using machine-vision has shown great progress in research (33,34), it is only entering the commercial market and external validation will be needed to confirm the performance. Furthermore, according to our results, the performance of existing health and welfare monitoring systems was sporadically tested on young animals (heifers and calves). Validation studies with accelerometers based on collar were rare and only 14% of validated traits for activity monitoring were obtained from collar devices. On the other hand, this was the most often marketed attachment point for the accelerometer. Therefore, further validation studies for collar-based systems are needed. To successfully assess welfare of young animals, more work on dedicated systems might be required. Further technological and validation gaps regarding assessment of welfare will be discussed according to principles defined in WQ® protocol. 4.2 Sensor-based welfare assessment for dairy cows and young cattle – how far are we? The concept of welfare has multidimensional nature, there is no one indicator that can be used to assess the welfare of an animal, but there are some indicators which are linked to several aspects of welfare. Quite often, welfare assessment is performed using a combination of animal and resource-based indicators (as in WQ® protocol for instance) and the evaluation is performed by a human observer. Some of the aspects which are evaluated using welfare protocols could be addressed by sensor-based technologies. Below, we will discuss the availability of technologies for assessment of each welfare principle: 4.2.1 Good feeding 9 To fulfill good feeding principle, animals should not suffer from prolonged hunger or thirst. For prolonged hunger, WQ® protocol adopts animal-based indicator. Regarding thirst criterium, only resource-based indicators are evaluated (20). Therefore, PLF technologies can provide additional single-animal level information for good feeding evaluation. There were several attributes monitored by PLF technologies (BCS, rumination time, rumen pH, grazing time, feed and water intake as well as feeding time) which have high potential for “good feeding” assessment. Some of the attributes (rumination time, rumen pH, grazing time, feed and water intake as well as feeding time), when frequently monitored, can be used for designing early warning systems for disease detection and/or feeding system failures (e.g. (35)). On the other hand, BCS which assess the proportion of body fat, can have practical application for decision support systems (e.g. predicting the risk of cow developing ketosis or having reproduction problems) (36). The good feeding assessment might be hampered by the commercial availability of technologies. Based on our search, only two providers offered a camera-based sensor used for BCS monitoring. There is also a shortage of tools able to assess grazing time (only 2 technologies had validation studies for grazing monitoring). Finally, measurements on individual feeding and drinking were performed at feeding stations, mostly used for research (feeding experiments), and due to high costs of equipment might have little relevance for commercial application. Potentially, systems based on cameras can also have application for feed availability and intake monitoring (37). However, these systems are still in development and only one commercial camera-based system for feed accessibility monitoring was identified. There are several providers of boluses for rumen pH monitoring, but still, relatively little is known on the performance of the detection models (with alarm-based monitoring) for rumen pH monitoring. Additionally, short functional life of the pH boluses (around 40 days due to loss in accuracy of the electrode (38)), does not allow long lasting individual-animal based assessment. Animal presence at the feeding trough or water bin, can be monitored using RFID technologies (e.g.(39)), but available technologies have been tested mostly in experimental farms, and examples with commercial farm validation are rare. Increased competition among cattle at the feed bunk can be currently detected in experimental settings e.g. (40) and can indicate shortage of food (decrease feeding time or dry matter intake). However, there is a need for further validation studies on systems based on RFID for detecting food or water shortage at an individual level. Based on presented results, good feeding assessment based on animal indicators in commercial settings could primarily be based on accelerometer technologies. Accelerometers-based systems are easily available and can assess rumination (with high performance) and feeding time (with less performance). Moreover, accelerometers together with noseband pressure sensors were used to measure drinking duration (e.g. 24). In the future, good feeding assessment could be further improved by integrating information from emerging technologies (such as video-based assessment of BCS). 4.2.2. Good housing In order to ensure good housing, animals should have thermal and resting comfort as well as enough space to move freely (20). For assessing comfort around resting, WQ® protocol is using animal-based (e.g. time to needed lie down, animals colliding with housing equipment during lying down, animals lining partially or completely outside the lining area) and management-based indicators (e.g. presence of tethering and access to outdoor loafing area or pasture). Therefore, measuring animals’ activities and physical state using PLF, can provide a more accurate assessment at an individual level. Regarding the experts’ evaluation, non-active behavior (lying or standing still) has the highest potential to be used for the assessment of good housing. Allowing dairy cows adequate space and facilities to lie down is considered an important aspect for the production as well as the animal welfare (42). As recently 10 reviewed, the lying time will depend on individual cow-based factors (reproductive status, age, and milk production), health status (lameness and mastitis) as well as the comfort of housing facilities (43). For example, pasture-based cows are characterized by longer, undisrupted lying times compared to cows kept in cubicles (44). Lameness can result in longer lying times while mastitis can reduce it (45). For this reason, to avoid confounding factors between animal health and housing conditions, an integration with other data sources, such as milking or breeding records, presence of lameness or mastitis is necessary. Non-active and active behavior as well as grazing time can be assessed using accelerometers. However, performance of technologies varied in different farm conditions. For example, CowManager sensor was evaluated for high (17,46) and lower (47) performance in measuring lying behavior of cows. High performance was obtained in tie stall and free stall barn and lower performance for grazing cattle. These somehow varying performance results raise the question, if sensor systems should be adjusted (and also validated) for different environmental/ housing conditions. Cleanliness of udder, cleanliness of flank, upper and lower legs are other animal-based indicators recorded in the WQ® protocol to assess the criterion of comfort around resting and consequently the principle of good housing (20). To the best knowledge of authors, currently there are no available technologies able to assess cleanliness of animals. However, rapid development in vision-based monitoring for automatic individual identification (e.g. (48)) can prompt development of algorithms able to evaluate this welfare aspect. Thermal comfort can be assessed on an individual basis by application of invasive (e.g. boluses) and non-invasive sensors (thermal cameras). Both options are available on the market, however there is a clear shortage of validation studies for monitoring systems based on those sensors. 4.2.3 Good health For good health, animals should be free from physical injuries (like lameness and integument alterations) and disease and should not suffer pain induced by inappropriate management or handling (20). As agreed by experts, several traits measured by PLF technologies have potential application for assessment of good health (body temperature, BCS, lameness, mastitis, water consumption, rumination, rumen pH, feed intake, non-active behavior). The listed attributes can be categorized as direct or indirect health indicators. Indirect indicators, such as active and non-active behavior, rumen pH, feeding quantity, on its own does not indicate health status of an animal, but changes in animals’ behavior possibly in combination with other data sources (e.g. lactation status, reproduction) can be processed to obtain early-warning signals for health problems (e.g. lameness, mastitis, ketosis) and potentially prevent them. Direct welfare indicators, like the number of cows with increased somatic cell count (SCC), brings knowledge on health (if an animal is sick or not), and can be useful for operational decisions (e.g. antimicrobial treatment). Injuries, such as lameness, can be detected measuring animal behavior (accelerometers), gait (load cells), posture (cameras) and increased body temperature (thermal camera). Though, the performance of accelerometer-based systems and thermal cameras for lameness detection is unknown (we were have not identified external validation studies for lameness detection using those techniques), while the commercial availability and performance of two remaining methods is still low. According to our knowledge, there are no commercially available systems able to detect skin lesions, however similar to cleanliness evaluation, development in camera- based monitoring systems could in the future allow identification of animals with such problems. Assessment of good health (and especially presence of diseases) should be based on integrating and analyzing data from different sources. There are commercially available examples of systems using multiple sensors (e.g. milking robots and activity collars) which provide data on milk production, SCC and animal behavior. System using both an automatic milking system and an activity collar was presented by Elischer et al. (49). However, there are no external validation studies on the performance 11 of these systems for disease detection. There are already examples of flexible models able to handle different sensor or non-sensor data for the disease detection (e.g. for mastitis prediction (50)) but the performance of these tools need still to be tested in commercial settings. Even if a technology is not able to provide highly accurate health data on individual level it could still be useful to estimate herd level prevalence of health problems. Potential integration could concern milk sensor data, accelerometer data, load cells with RFID, boluses (for body temperature and rumen pH), cameras (for body temperature, gait, BCS) and microphones (cough detection). 4.2.4 Appropriate behavior Appropriate behavior concerns expression of social behaviour, expression of other behaviours, good human-animal relationship, and positive emotional state (20). Based on experts’ answers, it can be concluded that PLF technologies currently have low potential to address appropriate behavior. With the exception of grazing behavior, none of the evaluated attributes was evaluated as “very relevant” by all of the experts. However, some of the attributes related to activity as well as feeding and drinking monitoring were evaluated as moderately relevant. From all available technologies, only two tools were tested for evaluation of appropriate behaviour (namely for grazing monitoring). There is, however, a substantial scientific interest in developing research tools aiming to address this welfare principle. For example, accelerometer based tools were already applied to monitor social behaviors, such as discriminating spontaneous locomotor play (51) and licking/suckling (52) in dairy calves. Also, data from feeding and drinking stations were applied for monitoring social competition (40,53,54). In recent years, scientists pointed out the importance of positive emotions as key elements to ensuring good animal welfare (55). In experimental conditions, both ear postures (56) and nasal temperature (57) have been proven to be useful measures of a change in emotional state of cows. For example, the drop in nasal temperatures of cows can be a result of the experience of a positive, low arousal experience. However, further research is needed to design systems able to monitor positive emotional state in commercial settings. Also, there is a technological gap concerning monitoring good human- animal interaction with no retailed technologies intended for this purpose. There is also very scarce information on any experimental techniques for measuring avoidance distance (which is used to assess good human-animal interaction) at individual level (12). Human-animal relationship could be automatically monitored using 3D cameras, which can capture the distance between a target and camera. However, application of vision technologies requires more research effort. 4.3 Performance results - quality and quantity of validation studies Validation studies are essential for further use of the tool in scientific experiments as well as for welfare labelling or subsidy payments. Therefore, there should be more emphasize on the quantity and quality of conducting and reporting of validation studies for PLF technologies. The results of validation studies are quite often presented as technical notes or short communications with rather limited space for detail description; however, this does not absolve authors from presenting information necessary for readers to assess the risk of reporting bias. Based on this review, similar suggestions to those presented by Henriks et al., (58) on how to improve reporting can be made. The location of the trial (commercial versus experimental herd), criteria for animal selection (e.g. random or based on a stage of lactation), building and management characteristics (e.g. floor type, grazing), feeding system should be always reported, since results obtained in the different production settings might not be comparable. Furthermore, the validated tool, especially if not commercially available, should be described in enough details for correct technology identification. Also, when possible, the software version should be reported. 12 In this review we have grouped tools based on reported performance measures. Threshold for high performance was selected based on available literature and represents very good agreement. Here it should be noted that tools which did not fulfill high performance criteria still have practical relevance. For example, online California Mastitis Test performed by milking robots, agreed only moderately with laboratory measurements on SCC. Even though this data was not very precise, nevertheless can be very useful for on farm decision making, due to the high sampling frequency (22). Therefore, the practical relevance of tools need to be assessed based on their objectives (59) and judged one by one. The results from a single validation trial are not yet conclusive regarding the tool performance. Ideally, tools should be validated in different production conditions (e.g. different countries, housing). According to our results, some of the traits were validated multiple times, by different research groups around the World. And as expected, some presented performance results were inconsistent (as in example of CowManager sensor evaluated in different housing conditions). Also, as seen from the example of Appendix table 2, authors varied in reported statistics. There seems to be no clear guideline on sufficient level of information regarding performance which should be reported. For example, for classification models, reports should not be based on presenting only sensitivity and specificity of the model without information on selected thresholds for detection. Instead, the performance of the classification model should be preferably presented in receiver operating characteristic curve which is the overall performance indicator (60). Regarding regression models, the adequate statistical tests are presented for example in Tedeschi (59). Including only Pearson correlation coefficient allows to assess precision of the tool, while nothing is known about its inaccuracy (the systematic deviation from the truth). Testing for tool performance is especially important for technologies from which data will be post-processed and used for building further algorithms. In this review, we have not removed or distinguished in result table studies which provided somehow limited information on the tool performance (for example only results on Pearson correlation). It could be possible to set additional exclusion criteria for papers selection; however, one must remember that even in the limited form, these studies provide some partial information about the validity. 4.4 Application of sensor-based technology for welfare assessment on farms and beyond The primary goal of a sensor system is to improve animal management. Sensor systems provide information for decision making which may, among others, influence farm profitability, animal health and welfare as well as have environmental impact (11). However, potential application of sensor systems can go beyond a single farm level. There are studies demonstrating that data routinely recorded from milking robots provide information which can assist in genetic evaluation (e.g. (61)). Moreover, production data could be utilised for designing health surveillance systems. For example, an attempt was made to use milk yield data to detect outbreaks of Bluetongue and Schmallenberg viruses (62). PLF technology may provide evidence-based approach to the monitoring and surveillance of animal welfare not only at the farm but also during transport or at slaughter (63). Already now, in some countries there are suggestions to base the certification system of livestock farming on real-time measurements and using animal behaviour as a criterion for quality labelling (64). This kind of policy could increase transparency of the sector and could result in a wider selection of welfare friendly products. As demonstrated in a previous review, data routinely collected on the farm (e.g. on milk yield, culling, and reproduction) and available in national data based, were associated with dairy cow welfare (65). Also, meat inspection data can have practical application for welfare assessment (66). This review demonstrates that data collected during on-farm monitoring has high potential to assess different aspects of dairy cow welfare, and that currently available technologies can provide animal- based welfare information. However, for the data to be fully utilized for this purpose, there is a need to develop new methodologies for data integration and processing. Data collected from various automatic recording technologies need to be processed and integrated into a single outcome of animal 13 welfare (which is easy to understand by the consumer). This challenging task will be considered out by the ClearFarm project which aims to develop a platform to control animal welfare in pig and dairy farming. The integration of technologies for welfare, health surveillance or breeding evaluation will require access to a vast amount of PLF data from different devices and different users. The utilization of these data require that data ownership rights, privacy and confidentiality issues are resolved and agreed between the parties involved. For example, for the EU markets, non-binding guidelines on data sharing from PLF technologies are available (67) and cover, among others, ownership, access, control, privacy. However, according to the recent review on digital agriculture, the area of data ownership regulations could receive more attention (68). Another challenge, concerns data storage capacity and strong computational power. However, there are already efforts to design a set of industrial, large-scale high-performance computing solutions to support the processing of very large PLF datasets from different users (69). References 1. Pinillos RG, Appleby MC, Manteca X, Scott-Park F, Smith C, Velarde A. One Welfare – a platform for improving human and animal welfare. Veterinary Record (2016) 179:412–413. doi:10.1136/vr.i5470 2. Stygar AH, Chantziaras I, Toppari I, Maes D, Niemi JK. High biosecurity and welfare standards in fattening pig farms are associated with reduced antimicrobial use. Animal (2020)1–9. doi:10.1017/S1751731120000828 3. von Keyserlingk MAG, Rushen J, de Passillé AM, Weary DM. Invited review: The welfare of dairy cattle—Key concepts and the role of science. Journal of Dairy Science (2009) 92:4101– 4111. doi:10.3168/jds.2009-2326 4. Llonch P, Somarriba M, Duthie C-A, Haskell MJ, Rooke JA, Troy S, Roehe R, Turner SP. Association of Temperament and Acute Stress Responsiveness with Productivity, Feed Efficiency, and Methane Emissions in Beef Cattle: An Observational Study. Front Vet Sci (2016) 3: doi:10.3389/fvets.2016.00043 5. European Commission. Attitudes of EU Citizens towards Animal Welfare, Report; Special Eurobarometer 442; European Commission: Brussels, Belgium. (2016) Available at: file:///C:/Users/03110737/AppData/Local/Temp/ebs_442_en.pdf [Accessed September 18, 2020] 6. European Commission. Online consultation on the Future of Europe. Second interim report. (2019) Available at: https://ec.europa.eu/commission/sites/beta-political/files/online- consultation-report-april-2019_en.pdf [Accessed September 18, 2020] 7. Yang W, Renwick A. Consumer Willingness to Pay Price Premiums for Credence Attributes of Livestock Products – A Meta‐Analysis. J Agric Econ (2019) 70:618–639. doi:10.1111/1477- 9552.12323 8. Heinola K, Latvala T, Raussi S, Kauppinen T, Niemi J. Kuluttajanäkökulmia eläinten hyvinvointimerkin kehittämiseen. SMST (2020) doi:10.33354/smst.89454 14 9. Clark B, Panzone LA, Stewart GB, Kyriazakis I, Niemi JK, Latvala T, Tranter R, Jones P, Frewer LJ. Consumer attitudes towards production diseases in intensive production systems. PLoS ONE (2019) 14:e0210432. doi:10.1371/journal.pone.0210432 10. Riemsdijk L, Ingenbleek PTM, Veen G, Trijp HCM. Positioning Strategies for Animal‐Friendly Products: A Social Dilemma Approach. J Consum Aff (2020) 54:100–129. doi:10.1111/joca.12240 11. Rojo-Gimeno C, van der Voort M, Niemi JK, Lauwers L, Kristensen AR, Wauters E. Assessment of the value of information of precision livestock farming: A conceptual framework. NJAS - Wageningen Journal of Life Sciences (2019) 90–91:100311. doi:10.1016/j.njas.2019.100311 12. Maroto Molina F, Pérez Marín CC, Molina Moreno L, Agüera Buendía EI, Pérez Marín DC. Welfare Quality ® for dairy cows: towards a sensor-based assessment. Journal of Dairy Research (2020) 87:28–33. doi:10.1017/S002202992000045X 13. Steeneveld W, Hogeveen H, Oude Lansink AGJM. Economic consequences of investing in sensor systems on dairy farms. Computers and Electronics in Agriculture (2015) 119:33–39. doi:10.1016/j.compag.2015.10.006 14. Steeneveld W, Vernooij JCM, Hogeveen H. Effect of sensor systems for cow management on milk production, somatic cell count, and reproduction. Journal of Dairy Science (2015) 98:3896– 3905. doi:10.3168/jds.2014-9101 15. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med (2009) 6:e1000097. doi:10.1371/journal.pmed.1000097 16. Altman DG, Vergouwe Y, Royston P, Moons KGM. Prognosis and prognostic research: validating a prognostic model. BMJ (2009) 338:b605–b605. doi:10.1136/bmj.b605 17. Bikker JP, van Laar H, Rump P, Doorenbos J, van Meurs K, Griffioen GM, Dijkstra J. Technical note: Evaluation of an ear-attached movement sensor to record cow feeding behavior and activity. Journal of Dairy Science (2014) 97:2974–2979. doi:10.3168/jds.2013-7560 18. Grinter LN, Campler MR, Costa JHC. Technical note: Validation of a behavior-monitoring collar’s precision and accuracy to measure rumination, feeding, and resting time of lactating dairy cows. Journal of Dairy Science (2019) 102:3487–3494. doi:10.3168/jds.2018-15563 19. Van Hertem T, Tello AS, Viazzi S, Steensels M, Bahr C, Romanini CEB, Lokhorst K, Maltz E, Halachmi I, Berckmans D. Implementation of an automatic 3D vision monitor for dairy cow locomotion in a commercial farm. Biosystems Engineering (2018) 173:166–175. doi:10.1016/j.biosystemseng.2017.08.011 20. Welfare Quality®. Welfare Quality® assessment protocol for cattle. Welfare Quality® Consortium, Lelystad, Netherlands. (2009) Available at: http://www.welfarequalitynetwork.net/media/1088/cattle_protocol_without_veal_calves.pdf [Accessed September 18, 2020] 15 21. Lohoelter M, Rehage R, Meyer U, Lebzien P, Rehage J, Daenicke S. Evaluation of a device for continuous measurement of rumen pH and temperature considering localization of measurement and dietary concentrate proportion. Landbauforschung (2013) 63:61–68. doi:10.3220/LBF_2013_61-68 22. Deng Z, Hogeveen H, Lam TJGM, van der Tol R, Koop G. Performance of Online Somatic Cell Count Estimation in Automatic Milking Systems. Frontiers in Veterinary Science (2020) 7:221. doi:10.3389/fvets.2020.00221 23. Bicalho RC, Cheong SH, Cramer G, Guard CL. Association Between a Visual and an Automated Locomotion Score in Lactating Holstein Cows. Journal of Dairy Science (2007) 90:3294–3300. doi:10.3168/jds.2007-0076 24. Schlageter-Tello A, Van Hertem T, Bokkers EAM, Viazzi S, Bahr C, Lokhorst K. Performance of human observers and an automatic 3-dimensional computer-vision-based locomotion scoring method to detect lameness and hoof lesions in dairy cows. Journal of Dairy Science (2018) 101:6322–6335. doi:10.3168/jds.2017-13768 25. Brassel J, Rohrssen F, Failing K, Wehrend A. Automated detection of health disorders in lactating dairy cattle on pasture: a preliminary study. Polish Journal of Veterinary Sciences (2019) 22:761– 767. doi:10.24425/pjvs.2019.131406 26. Kaniyamattam K, De Vries A. Agreement between milk fat, protein, and lactose observations collected from the Dairy Herd Improvement Association (DHIA) and a real-time milk analyzer. Journal of Dairy Science (2014) 97:2896–2908. doi:10.3168/jds.2013-7690 27. Sorensen LP, Bjerring M, Lovendahl P. Monitoring individual cow udder health in automated milking systems using online somatic cell counts. Journal of Dairy Science (2016) 99:608–620. doi:10.3168/jds.2014-8823 28. de Mol RM, Ouweltjes W, Kroeze GH, Hendriks M. Detection of estrus and mastitis: Field performance of a model. Applied Engineering in Agriculture (2001) 17:399–407. 29. Stygar AH, Kristensen AR. Detecting abnormalities in pigs’ growth – A dynamic linear model with diurnal growth pattern for identified and unidentified pigs. Computers and Electronics in Agriculture (2018) 155:180–189. doi:10.1016/j.compag.2018.10.004 30. ICAR. The global standard for livestock data. Section 11 -Guidelines for Testing, Approval and Checking of Milk Recording Devices. https://www.icar.org/Guidelines/11-Milk-Recording- Devices.pdf. (2020) 31. Schweinzer V, Gusterer E, Kanz P, Krieger S, Suess D, Lidauer L, Berger A, Kickinger F, Oehlschuster M, Auer W, et al. Evaluation of an ear-attached accelerometer for detecting estrus events in indoor housed dairy cows. Theriogenology (2019) 130:19–25. doi:10.1016/j.theriogenology.2019.02.038 32. Bruinjé TC, Ambrose DJ. Technical note: Validation of an automated in-line milk progesterone analysis system to diagnose pregnancy in dairy cattle. Journal of Dairy Science (2019) 102:3615– 3621. doi:10.3168/jds.2018-15692 16 33. Guzhva O, Ardo H, Nilsson M, Herlin A, Tufvesson L. Now You See Me: Convolutional Neural Network Based Tracker for Dairy Cows. Frontiers in Robotics and Ai (2018) 5:107. doi:10.3389/frobt.2018.00107 34. Wurtz K, Camerlink I, D’Eath RB, Fernández AP, Norton T, Steibel J, Siegford J. Recording behaviour of indoor-housed farm animals automatically using machine vision technology: A systematic review. PLoS ONE (2019) 14:e0226669. doi:10.1371/journal.pone.0226669 35. Schirmann K, Weary DM, Heuwieser W, Chapinal N, Cerri RLA, von Keyserlingk MAG. Short communication: Rumination and feeding behaviors differ between healthy and sick dairy cows during the transition period. Journal of Dairy Science (2016) 99:9917–9924. doi:10.3168/jds.2015-10548 36. Roche JR, Friggens NC, Kay JK, Fisher MW, Stafford KJ, Berry DP. Invited review: Body condition score and its association with dairy cow productivity, health, and welfare. Journal of Dairy Science (2009) 92:5769–5801. doi:10.3168/jds.2009-2431 37. Bloch V, Levit H, Halachmi I. Assessing the potential of photogrammetry to monitor feed intake of dairy cows. Journal of Dairy Research (2019) 86:34–39. doi:10.1017/S0022029918000882 38. Phillips N, Mottram T, Poppi D, Mayer D, McGowan MR. Continuous monitoring of ruminal pH using wireless telemetry. Anim Prod Sci (2010) 50:72. doi:10.1071/AN09027 39. Chizzotti ML, Machado FS, Valente EEL, Pereira LGR, Campos MM, Tomich TR, Coelho SG, Ribas MN. Technical note: Validation of a system for monitoring individual feeding behavior and individual feed intake in dairy cattle. Journal of Dairy Science (2015) 98:3438–3442. doi:10.3168/jds.2014-8925 40. Huzzey JM, Weary DM, Tiau BYF, von Keyserlingk MAG. Short communication: Automatic detection of social competition using an electronic feeding system. Journal of Dairy Science (2014) 97:2953–2958. doi:10.3168/jds.2013-7434 41. Ruuska S, Kajava S, Mughal M, Zehner N, Mononen J. Validation of a pressure sensor-based system for measuring eating, rumination and drinking behaviour of dairy cattle. Applied Animal Behaviour Science (2016) 174:19–23. doi:10.1016/j.applanim.2015.11.005 42. Haley DB, Rushen J, Passillé AM de. Behavioural indicators of cow comfort: activity and resting behaviour of dairy cows in two types of housing. Can J Anim Sci (2000) 80:257–263. doi:10.4141/A99-084 43. Tucker CB, Jensen MB, de Passillé AM, Hänninen L, Rushen J. Invited review: Lying time and the welfare of dairy cows. Journal of Dairy Science (2020)S0022030220308857. doi:10.3168/jds.2019-18074 44. Olmos G, Boyle L, Hanlon A, Patton J, Murphy JJ, Mee JF. Hoof disorders, locomotion ability and lying times of cubicle-housed compared to pasture-based dairy cows. Livestock Science (2009) 125:199–207. doi:10.1016/j.livsci.2009.04.009 45. Tucker C, Jensen MB, de Passille AM, Hänninen L, Rushen J. Lying time and the welfare of dairy cows: a review. Journal of Dairy Science (2020) 17 46. Zambelis A, Wolfe T, Vasseur E. Technical note: Validation of an ear-tag accelerometer to identify feeding and activity behaviors of tiestall-housed dairy cattle. Journal of Dairy Science (2019) 102:4536–4540. doi:10.3168/jds.2018-15766 47. Pereira GM, Heins BJ, Endres MI. Technical note: Validation of an ear-tag accelerometer sensor to determine rumination, eating, and activity behaviors of grazing dairy cattle. Journal of Dairy Science (2018) 101:2492–2495. doi:10.3168/jds.2016-12534 48. Li W, Ji Z, Wang L, Sun C, Yang X. Automatic individual identification of Holstein dairy cows using tailhead images. Computers and Electronics in Agriculture (2017) 142:622–631. doi:10.1016/j.compag.2017.10.029 49. Elischer MF, Arceo ME, Karcher EL, Siegford JM. Validating the accuracy of activity and rumination monitor data from dairy cows housed in a pasture-based automatic milking system. Journal of Dairy Science (2013) 96:6412–6422. doi:10.3168/jds.2013-6790 50. Jensen DB, Hogeveen H, De Vries A. Bayesian integration of sensor information and a multivariate dynamic linear model for prediction of dairy cow mastitis. Journal of Dairy Science (2016) 99:7344–7361. doi:10.3168/jds.2015-10060 51. Groessbacher V, Buckova K, Lawrence AB, Spinka M, Winckler C. Discriminating spontaneous locomotor play of dairy calves using accelerometers. Journal of Dairy Science (2020) 103:1866– 1873. doi:10.3168/jds.2019-17005 52. Roland L, Schweinzer V, Kanz P, Sattlecker G, Kickinger F, Lidauer L, Berger A, Auer W, Mayer J, Sturm V, et al. Evaluation of a triaxial accelerometer for monitoring selected behaviors in dairy calves. Journal of Dairy Science (2018) 101:10421–10427. doi:10.3168/jds.2018-14720 53. Foris B, Thompson AJ, von Keyserlingk MAG, Melzer N, Weary DM. Automatic detection of feeding- and drinking-related agonistic behavior and dominance in dairy cows. Journal of Dairy Science (2019) 102:9176–9186. doi:10.3168/jds.2019-16697 54. McDonald P, von Keyserlingk MAG, Weary DM. Technical note: Using an electronic drinker to monitor competition in dairy cows. Journal of Dairy Science (2019) 102:3495–3500. doi:10.3168/jds.2018-15585 55. Mellor DJ. Animal emotions, behaviour and the promotion of positive welfare states. N Z Vet J (2012) 60:1–8. doi:10.1080/00480169.2011.619047 56. Proctor HS, Carder G. Can ear postures reliably measure the positive emotional state of cows? Applied Animal Behaviour Science (2014) 161:20–27. doi:10.1016/j.applanim.2014.09.015 57. Proctor HS, Carder G. Nasal temperatures in dairy cows are influenced by positive emotional state. Physiol Behav (2015) 138:340–344. doi:10.1016/j.physbeh.2014.11.011 58. Hendriks SJ, Phyn CVC, Huzzey JM, Mueller KR, Turner S-A, Donaghy DJ, Roche JR. Graduate Student Literature Review: Evaluating the appropriate use of wearable accelerometers in research to monitor lying behaviors of dairy cows. Journal of Dairy Science (2020)S0022030220308183. doi:10.3168/jds.2019-17887 18 59. Tedeschi LO. Assessment of the adequacy of mathematical models. Agricultural Systems (2006) 89:225–247. doi:10.1016/j.agsy.2005.11.004 60. Dominiak KN, Kristensen AR. Prioritizing alarms from sensor-based detection models in livestock production - A review on model performance and alarm reducing methods. Computers and Electronics in Agriculture (2017) 133:46–67. doi:10.1016/j.compag.2016.12.008 61. Heringstad B, Kjøren Bugten H. Genetic evaluations based on data from automatic milking systems. 39th ICAR Session, May 19-23 Berlin, Germany. (2014) 62. Veldhuis A, Brouwer-Middelesch H, Marceau A, Madouasse A, Van der Stede Y, Fourichon C, Welby S, Wever P, van Schaik G. Application of syndromic surveillance on routinely collected cattle reproduction and milk production data for the early detection of outbreaks of Bluetongue and Schmallenberg viruses. Preventive Veterinary Medicine (2016) 124:15–24. doi:10.1016/j.prevetmed.2015.12.006 63. Buller H, Blokhuis H, Lokhorst K, Silberberg M, Veissier I. Animal Welfare Management in a Digital World. Animals (2020) 10:1779. doi:10.3390/ani10101779 64. Council on Animal Affairs (RDA). Digitisation of the Livestock Farming Sector. (2019) Available at: https://english.rda.nl/publications/publications/2020/02/13/digitisation-of-the- livestock-farming-sector [Accessed January 4, 2020] 65. de Vries M, Bokkers EAM, Dijkstra T, van Schaik G, de Boer IJM. Invited review: Associations between variables of routine herd data and dairy cattle welfare indicators. Journal of Dairy Science (2011) 94:3213–3228. doi:10.3168/jds.2011-4169 66. Søren Nielsen, Matthew Denwood, Björn Forkman, Hans Houe. Selection of Meat Inspection Data for an Animal Welfare Index in Cattle and Pigs in Denmark. Animals (2017) 7:94. doi:10.3390/ani7120094 67. COPA-COGECA, CEMA. EU Code of conduct on agricultural data sharing by contractual agreement,. (2018) Available at: https://www.fefac.eu/files/81630.pdf [Accessed February 4, 2020] 68. Klerkx L, Jakku E, Labarthe P. A review of social science on digital agriculture, smart farming and agriculture 4.0: New contributions and a future research agenda. NJAS - Wageningen Journal of Life Sciences (2019) 90–91:100315. doi:10.1016/j.njas.2019.100315 69. Perakis K, Lampathaki F, Nikas K, Georgiou Y, Marko O, Maselyne J. CYBELE – Fostering precision agriculture & livestock farming through secure access to large-scale HPC enabled virtual industrial experimentation environments fostering scalable big data analytics. Computer Networks (2020) 168:107035. doi:10.1016/j.comnet.2019.107035 70. Borchers MR, Chang YM, Tsai IC, Wadsworth BA, Bewley JM. A validation of technologies monitoring dairy cow feeding, ruminating, and lying behaviors. Journal of Dairy Science (2016) 99:7458–7466. doi:10.3168/jds.2015-10843 19 71. Henriksen JC, Munksgaard L. Validation of AfiTagII, a device for automatic measuring of lying behaviour in Holstein and Jersey cows on two different bedding materials. Animal (2019) 13:617– 621. doi:10.1017/S1751731118001623 72. Mattachini G, Antler A, Riva E, Arbel A, Provolo G. Automated measurement of lying behavior for monitoring the comfort and welfare of lactating dairy cows. Livestock Science (2013) 158:145–150. doi:10.1016/j.livsci.2013.10.014 73. Swartz TH, McGilliard ML, Petersson-Wolfe CS. Technical note: The use of an accelerometer for measuring step activity and lying behaviors in dairy calves. Journal of Dairy Science (2016) 99:9109–9113. doi:10.3168/jds.2016-11297 74. Mattachini G, Riva E, Bisaglia C, Pompe JCAM, Provolo G. Methodology for quantifying the behavioral activity of dairy cows in freestall barns. Journal of Animal Science (2013) 91:4899– 4907. doi:10.2527/jas.2012-5554 75. Nielsen PP, Fontana I, Sloth KH, Guarino M, Blokhuis H. Technical note: Validation and comparison of 2 commercially available activity loggers. Journal of Dairy Science (2018) 101:5449–5453. doi:10.3168/jds.2017-13784 76. Shepley E, Berthelot M, Vasseur E. Validation of the Ability of a 3D Pedometer to Accurately Determine the Number of Steps Taken by Dairy Cows When Housed in Tie-Stalls. Agriculture (2017) 7:53. doi:10.3390/agriculture7070053 77. Werner J, Umstatter C, Leso L, Kennedy E, Geoghegan A, Shalloo L, Schick M, O’Brien B. Evaluation and application potential of an accelerometer-based collar device for measuring grazing behavior of dairy cows. Animal (2019) 13:2070–2079. doi:10.1017/S1751731118003658 78. Schirmann K, von Keyserlingk MAG, Weary DM, Veira DM, Heuwieser W. Technical note: Validation of a system for monitoring rumination in dairy cows. Journal of Dairy Science (2009) 92:6052–6055. doi:10.3168/jds.2009-2361 79. Burfeind O, Schirmann K, von Keyserlingk MAG, Veira DM, Weary DM, Heuwieser W. Technical note: Evaluation of a system for monitoring rumination in heifers and calves. Journal of Dairy Science (2011) 94:426–430. doi:10.3168/jds.2010-3239 80. Steinmetz M, von Soosten D, Hummel J, Meyer U, Daenicke S. Validation of the RumiWatch Converter V0.7.4.5 classification accuracy for the automatic monitoring of behavioural characteristics in dairy cows. Archives of Animal Nutrition (2020) 74:164–172. doi:10.1080/1745039X.2020.1721260 81. Kroeger I, Humer E, Neubauer V, Kraft N, Ertl P, Zebeli Q. Validation of a noseband sensor system for monitoring ruminating activity in cows under different feeding regimens. Livestock Science (2016) 193:118–122. doi:10.1016/j.livsci.2016.10.007 82. Zehner N, Umstatter C, Niederhauser JJ, Schick M. System specification and validation of a noseband pressure sensor for measurement of ruminating and eating behavior in stable-fed cows. Computers and Electronics in Agriculture (2017) 136:31–41. doi:10.1016/j.compag.2017.02.021 20 83. Eslamizad M, Tuemmler L-M, Derno M, Hoch M, Kuhla B. TECHNICAL NOTE: Development of a pressure sensor-based system for measuring rumination time in pre-weaned dairy calves. Journal of Animal Science (2018) 96:4483–4489. doi:10.1093/jas/sky337 84. Werner J, Leso L, Umstatter C, Niederhauser J, Kennedy E, Geoghegan A, Shalloo L, Schick M, O’Brien B. Evaluation of the RumiWatchSystem for measuring grazing behaviour of cows. Journal of Neuroscience Methods (2018) 300:138–146. doi:10.1016/j.jneumeth.2017.08.022 85. Reynolds MA, Borchers MR, Davidson JA, Bradley CM, Bewley JM. Technical note: An evaluation of technology-recorded rumination and feeding behaviors in dairy heifers. Journal of Dairy Science (2019) 102:6555–6558. doi:10.3168/jds.2018-15635 86. DeVries TJ, von Keyserlingk MAG, Weary DM, Beauchemin KA. Technical note: Validation of a system for monitoring feeding behavior of dairy cows. Journal of Dairy Science (2003) 86:3571–3574. doi:10.3168/jds.S0022-0302(03)73962-9 87. Chapinal N, Veira DM, Weary DM, von Keyserlingk MAG. Technical note: Validation of a system for monitoring individual feeding and drinking behavior and intake in group-housed cattle. Journal of Dairy Science (2007) 90:5732–5736. doi:10.3168/jds.2007-0331 88. Oliveira BR, Ribas MN, Machado FS, Lima JAM, Cavalcanti LFL, Chizzotti ML, Coelho SG. Validation of a system for monitoring individual feeding and drinking behaviour and intake in young cattle. Animal (2018) 12:634–639. doi:10.1017/S1751731117002002 89. Mullins IL, Truman CM, Campler MR, Bewley JM, Costa JHC. Validation of a Commercial Automated Body Condition Scoring System on a Commercial Dairy Farm. Animals (2019) 9: doi:10.3390/ani9060287 90. Cantor MC, Pertuisel CH, Costa JHC. Technical note: Estimating body weight of dairy calves with a partial-weight scale attached to an automated milk feeder. Journal of Dairy Science (2020) 103:1914–1919. doi:10.3168/jds.2019-16918 91. Neubauer V, Humer E, Kroeger I, Braid T, Wagner M, Zebeli Q. Differences between pH of indwelling sensors and the pH of fluid and solid phase in the rumen of dairy cows fed varying concentrate levels. Journal of Animal Physiology and Animal Nutrition (2018) 102:343–349. doi:10.1111/jpn.12675 92. Falk M, Münger A, Dohme-Meier F. Technical note: A comparison of reticular and ruminal pH monitored continuously with 2 measurement systems at different weeks of early lactation. Journal of Dairy Science (2016) 99:1951–1955. doi:10.3168/jds.2015-9725 93. Knauer WA, Godden SM, McDonald N. Technical note: Preliminary evaluation of an automated indwelling rumen temperature bolus measurement system to detect pyrexia in preweaned dairy calves. Journal of Dairy Science (2016) 99:9925–9930. doi:10.3168/jds.2015-10770 94. Hoffmann G, Schmidt M, Ammon C, Rose-Meierhoefer S, Burfeind O, Heuwieser W, Berg W. Monitoring the body temperature of cows and calves using video recordings from an infrared thermography camera. Veterinary Research Communications (2013) 37:91–99. doi:10.1007/s11259-012-9549-3 21 95. Tamura T, Okubo Y, Deguchi Y, Koshikawa S, Takahashi M, Chida Y, Okada K. Dairy cattle behavior classifications based on decision tree learning using 3-axis neck-mounted accelerometers. Animal Science Journal (2019) 90:589–596. doi:10.1111/asj.13184 5 Figures Fig 1. Modified PRISMA flow diagram (15) with the systematic review search strategy and study selection 6 Tables Table 1. Summary of externally validated technologies with potential use for dairy welfare assessment. Nr of validation Used sensors and Technology Independent Self- Technology provider trials for attachment name validation2 validation3 technology position1 provider AfiAct Afimilk, Kibbutz Afikim, Accelerometer, leg (70–73) Pedometer Plus, Israel AfiTagII, (S.A.E.Afikim,Israel) Pedometer Plus 5 AfiLab real- Spectroscope (26) time milk analyzer 22 CowAlert IceRobotics Ltd., Accelerometer, leg (70,74–76) IceQube, Edinburgh, Scotland 4 IceTag Track A Cow ENGS, Rosh Pina, Israel 1 Accelerometer, leg (70) MooMonitor+ Dairymaster, Tralee, Accelerometer, (18,77) 1 Ireland collar HerdInsights Alanya Ltd., Cork, Ireland Accelerometer, (25) 1 collar Hi-Tag SCR Engineers Ltd., Microphone, collar (78,79) Netanya, Israel (currently 2 Allflex) RumiWatch Itin+ Hoch GmbH, Liestal, Accelerometer and (18) (41,77,82– Switzerland pressure sensor, 84) 7 halter (41,77,80– 83), halter and leg (84) CowScout Leg GEA Farm Technologies, Accelerometer, leg (75) 1 Bonen, Germany CowManager Agis, Harmelen, Accelerometer, ear (46,47,70,85) (17) 5 SensOor Netherlands The Smartbow Smartbow GmbH, Accelerometer, ear (70) Jutogasse, Austria Lely activity Lely, Maassluis, the Accelerometer, (49) Netherlands collar Lely- on-line 2 Viscosity meter (22) California mastitis test GrowSafe GrowSafe Systems Ltd., RFID (neck collar), (86) 1 Airdrie, AB, Canada load cell Insentec Insentec, Marknesse, RFID (ear), load (87) the Netherlands (now 1 cell Hokofarm group) Intergado Intergado Ltd., Contagem, RFID (ear), load (39,88) Minas Gerais, Brazil 2 cell Body Condition DeLaval International AB, Camera (89) Scoring Tumba, Sweden 2 Combi Load cell (90) eCow Farmer eCow, Dekon, UK Bolus (reticulum) (91,92) 2 bolus KB 3/04 bolus Kahne Limited, New Bolus (rumen sac) (21) 1 Zealand Bella Ag Cattle Bella Ag LLC, USA 1 Bolus (reticulum) (93) Stepmetrix BouMatic, Madison, USA 1 Load cell (23) Optris Optris, Berlin, Germany 1 Camera (94) Prototypes – ns Camera (19,24) locomotion 2 score 23 IMAG model – ns Conductivity (28) prototype meter, 1 mastitis thermometer (milk detection temperature) Prototype- detection model Viscosity meter (27) mastitis (prototype) and online cell detection counter, DeLaval 1 International AB, Tumba, Sweden Prototype- ns Accelerometer (95) 1 activity 1 sensor location is provided only for sensors attached to the animal 2 validated using independent data set (different animals and herd than for technology building) and co-authors were not involved in technology development 3 validated using independent data set (different animals and herd than for technology building) and was developed and validated by at least one the same co-author (based on the authorship of papers) or have been validated by at least one co-author representing a company providing a technology Table 2. Results of validation trials for dairy cows in respect to measured traits Category Measured trait Validation Technologies validated for Technologies validated for place (farm) high performance1 lower performance2 Activity Non-active Research AfiAct Pedometer Plus (70), CowManager SensOor (47), behavior AfiTagII (71), CowAlert Lely activity (49), (lying, lying IceQube (70), CowManager MooMonitor+ (18), Pedometer and standing) SensOor (17,46), Track A Plus (72) Cow (70), RumiWatch (84) Commercial CowScout Leg (75), Ice Tag (74,75), prototype (95) Standing - Research RumiWatch (84) Lely activity (49) identified as a Commercial CowScout Leg (75), Ice Tag separate (74,75) behavior Active Research CowManager SensOor behavior (17,46,47), IceTag (76), Lely (walking, nr of activity (49), RumiWatch (84), steps) Commercial CowScout Leg (75), Ice Tag (74,75) Feeding Feeding time Research CowManager SensOor and (17,46,47,70), MooMonitor+ drinking (18), RumiWatch (V0.7.0.0 24 (41) and V0.7.4.5 (80)), Track A Cow (70) Commercial RumiWatch V0.7.3.2 (82) prototype (95), RumiWatch V0.7.2.0 (82) Presence at the Research GrowSafe (86), Insentec feeder3 (87), Intergado (39) Feed intake Research Insentec (87), Intergado (39) (kg) Grazing time Research MooMonitor+ (77), RumiWatch (84) Rumination Research CowManager SensOor (17) Hi-Tag (78), CowManager MooMonitor+ (18,77), SensOor (46,47,70), Lely Rumi Watch (41,80,81,84), activity (49) Smartbow (70) Commercial prototype(95), Rumi Watch (82) Drinking time Research Rumi Watch (V0.7.0.0 (41) and V0.7.4.5 (80)) Commercial RumiWatch (82) Water intake Research Insentec (87) Presence at the Research Insentec (87) drinker Physical Locomotion Commercial prototype (19) , prototype (24), condition score Stepmetrix (23), and Body condition Commercial DeLaval Body condition health scoring scoring (89) Rumen pH Research eCow bolus (91,92), KB 3/04 bolus (21) Body Research KB 3/04 bolus (21), OPTRIS temperature (94) Health Commercial HerdInsight (25) disorder4 Milk quality5 Research AfiLab real-time milk analyzer (26) Mastitis Commercial Lely- on-line California detection mastitis test (22), prototype - IMAG model (28), prototype (27) 1 all indicators defined/selected in validation trail (by authors of studies) were above high-performance threshold. High precision threshold was reached when Pearson correlation, Spearman’s rank correlation, concordance correlation coefficient, sensitivity, specificity, area under the curve was higher than 0.9, regression coefficient and Kappa coefficient is higher than 0.81, significance tests for 25 intercept and slope of linear regression did not differ significantly from 0 or 1, respectively and Bland- Altman plot included zero with the 95% interval of agreement. 2 any indicator validated with lower performance (below threshold defined above) 3 animal identification and time 4 e.g. mastitis or pneumonia 5 fat, lactose, protein as indicator for mastitis Table 3. Results of validation trials for calves and heifers in respect to measured traits Category Measured trait Validation Technologies Technologies validated for place (farm) validated for high lower performance2 performance1 Activity Non-active behavior Research AfiTag II (73) (lying time, lying bouts) Active behavior Research AfiTag II (73) (walking, nr of steps) Feeding Feeding time Research CowManager SensOor (85) and Presence at the feeder Commercial Intergado (88) drinking Feed intake Commercial Intergado (88) Rumination Research CowManager SensOor (85), Hi-Tag (79), RumiWatch (83), Water intake Commercial Intergado (88) Presence at the drinker Commercial Intergado (88) Physical Body weight Research Combi DeLaval (90) condition Body temperature Research Bella Ag Cattle (93), OPTRIS (94) 1 all indicators defined/selected in validation trail (by authors of studies) were above high-performance threshold. High precision threshold was reached when Pearson correlation, Spearman’s rank correlation, concordance correlation coefficient, sensitivity, specificity, are under the curve, is higher than 0.9, regression coefficient and Kappa coefficient is higher than 0.81, significance tests for intercept and slope of linear regression did not differ significantly from 0 or 1, respectively and Bland- Altman plot included zero with the 95% interval of agreement. 2 any indicator validated with lower precision and or accuracy (threshold defined above) Table 4. Indicator evaluation for relevance in assessing good feeding, housing, health and appropriate behavior1 Indicator Good feeding Good housing Good health Appropriate behavior Body temperature +- +- + - Body condition scoring + - + - 26 Lameness - +- + - Mastitis - +- + - Water consumption + - + +- Drinking duration +- - +- +- Rumination + +- + +- Rumen pH + - + - Grazing time + +- +- + Feeding intake + - + +- Feeding time + - +- +- Active behavior - +- +- - Non-active behavior - + + +- 1 symbols +, +-, - refer to “very relevant”, “moderate” and “not relevant” evaluation, respectively. 7 Conflict of Interest The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. 8 Author Contributions M.P., J.N, A.S., E.C. and P.L conceived of the presented idea. A.S. defined literature search. A.S., G.B. and E.D.C carried out literature search. Y.G. defined and performed company search. A.S. wrote paper with input from all coauthors. 9 Funding This study was conducted within the ClearFarm project which aim is to co-design, develop and validate a software platform powered by PLF technologies to provide animal welfare information. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 862919. 10 Acknowledgments We would like to thank members of ClearFarm consortium for their valuable comments. 27