Data mining to combine sensor information to improve oestrus detection
•Download as PPTX, PDF•
1 like•450 views
Presented at the Smart Agrimatics Conference in 2014, in Paris, France
Recommended
Recommended
More Related Content
What's hot
What's hot (11)
Similar to Data mining to combine sensor information to improve oestrus detection
Similar to Data mining to combine sensor information to improve oestrus detection (20)
Recently uploaded
Recently uploaded (20)
Data mining to combine sensor information to improve oestrus detection
- 1. Modelling in the SDF project Case: improve heat detection by combining sensor data Claudia Kamphuis, Wageningen University Kirsten Huijps, CRV Pieter Hogewerf, Wageningen Livestock Research
- 2. Content - The Why and What of Smart Dairy Farming - Fertility and heat detection - Where to from here
- 3. SDF: Why & What Improve a cow’s productive lifetime by putting the famer and the cow in the centre Three processes that are difficult to manage but that will increase productive lifetime Collaboration between commercial companies, research centres, universities and FARMERS Develop decision support models, management tools and advisory products that contribute to an increased productive lifetime
- 4. SDF: Fertility Project - Importance of fertility - A cow needs a calf to produce milk - Goal to get cows pregnant fast - Key drivers: detection and insemination - Current challenges - Time consuming - Recording - Increased herd sizes
- 5. SDF: Fertility Project - Automated heat detection: a success story- Automated heat detection: a success story...when it all works
- 6. SDF: Fertility Project GOAL Improve automated heat detection by using all available data Provide insemination advice (bull information, timing) Generate follow-up lists Generate calving attentions
- 7. Data collection in the field Two commercial farms in ‘Friesland’ • 250 cows • 2.5 fte • 450 cows • 4.5 fte
- 8. 820-8-2015 Farm 1 Yield, Weight, Feed intake every milking (24/7) Activity (alerts), Rumination every 2h Collection since June 2013 - Manual recording of heat observations - Continuous automatic recording of sensor data Activity (alerts) every hour Yield every day Activity, Eating, Feeding, Ruminating, every hour Lactation, calving dates, every 4wks
- 9. Difficulties in data collection
- 10. Action Farm 1 Farm 2 Combining sensor data streams (2h blocks) 617,000 788,000 Adding observed heat events 398 events 203 cows 477 events 236 cows Develop predictive variables from all raw sensor data 60 105 Develop Logitboost model 398 events 398 GS+ 20000 GS- 121 events 441 GS+ 11,032 GS- Validate model using April 2014 63 events 63 GS+ 111,186 GS- 24 events 72 GS+ 22,371 GS- Mining a predictive model
- 11. Preliminary results Model GS positive Sensitivity (%) False positives per day One sensor 63 44 2.5 Combined 63 75 2.8 Model GS positive Sensitivity (%) False positives per day One sensor 24 92 2.2 Combined 24 96 1.1
- 12. Heat tab List of cows that were detected as ‘in heat’ Where to from here? From modelling to near real-time 4 times a day Per individual cow best insemination time bull advice time left for insemination
- 13. Where to from here Near real-time on Farm 2 Fine-tune current models 200 farmers across The Netherlands in 2015 X different combinations of sensing systems X number of new detection models Scaling up
- 14. -Smart Dairy Farming: why and what -Importance of (automated) heat detection -Improve detection by combining sensor data -SDF-Model appears to have potential -SDF-Model is running near real-time ‘as-we-speak’ -Future work -Near real-time Farm 2 -Fine-tuning models -Scaling up Where to from here
Editor's Notes
- Thank you, Kees, for your introduction and I would like to thank you for attending this presentation.
- In the next 15 minutes I will address three topics: Firstly, I will briefly discuss the why and what of Smart Dairy Farming Secondly, I’ll discuss the case of fertility and heat detection in dairy cattle and what SDF has done in this specific project area. Finally, I’ll provide a brief glance into the future and discuss where we have to go
- So, let me start with smart dairy farming. And this is going to be a very brief introduction as Mathijs Vonder will discuss this project in more detail later this morning. The goal of SDF is to improve the cow’s productive lifetime of a current three lactations to 5 in 2015. SDF wants to reach this goal by putting the farmer and the individual cow in a central space. To increase the cow’s productive lifetime, SDF focusses on three key processes that are difficult to manage for the farmer: These processes are Rearing of young stock, the future of the dairy farm Transition period, which are the 6 wks around calving And fertility, or in other words, getting cows pregnant again. SDF wants to improve these key processes by extensive collaboration between commercial companies that are involved in sensor development, breeding, feeding, accountancy, the dairy industry, research centres, universities and most importantly, farmers and by collecting field data from existing and prototype sensors at 8 dairy farms. The data will be used to develop decision-support models, practical management tools and advisory products together to help farmer better managing the three key processes leading to an increase in the cows productive lifetime. As I mentioned, one of the three processes SDF focusses on is fertility. And that is the case that I will talk about in the remainder of this presentation.
- Right, SDF and the fertility project. It’s important for you to realize why fertility of dairy cows is so important for a farmer. Fertility is all about getting a cow pregnant because it requires a calf for the cow to produce milk. It is, thus, important to get cows pregnant again after calving and to establish this pregnancy as soon as possible. The key drivers for this goal are detection and insemination. A farmer needs to know when a cow is ‘in heat’ or ready to inseminate and then the insemination has to be done at the right timing. This sounds easier than it is in practice as cows not becoming pregnant is one of the main reasons for farmers to cull a cow. What are the challenges then, why is it so difficult? Several aspects play a role in this. First of all, detecting cows that are in heat is a time-consuming task. Perhaps you have seen this situation before while you’re were driving and walking and this is the sign that farmers are looking for to see a cow in heat. The advice to farmers is to look for this situation at least three times a day for 15 minutes. Once the farmers sees this, he needs to know which cow it is that is STANDING, that’s the cow that is in heat. This may be easier with smaller herd sizes where farmers know the cow just by looking at her but with increasing herd sizes this task becomes more difficult because.....can you tell who’s in heat now?
- Given the difficulties to identify cows in heat, automated heat detection has been developed. This means that sensors are, usually, placed on the leg or on the neck collars of cows that, also usually, measure the cow’s activity. We know that, when a cow’s in heat that she’s more active than normal and thus this behavioural change is used as indicator for in heat. The fact that farmers are adopting the technology shows that it works, so after the system provides an alert, the farmer has to confirm whether the alert was correct. When confirmed, the farmer needs to select the right bull and order semen to inseminate the cow. It’s important to follow-up on her to check whether the insemination was successful and to confirm that the cow can expect a calf in 9 months. And this is the success story when everything and every aspects works as it should....but The general complaint is that these sensors generate too many false alerts and they don’t find every cow that is in heat. Around 80/90% SN is reached but with false alerts on a daily basis. The challenge is thus to filter out these false alerts. Also, inseminating a cow is not easy. That has to be done correctly but also at the right time. Detecting a cow in heat does not mean that you have to inseminate her straight away but you have to wait a couple of hours to increase the likelihood of a successful insemination. And finally, you have to keep record of which cow to follow-up and when after insemination. These are all issues that makes the fertility of dairy cows complex and a difficult to manage process, which requires a lot of skills from farmers.
- The goal of SDF for this fertility project is thus to Improve automated heat detection, to provide insemination advice, to generate follow-up lists and to generate calving attentions. Or in other words, go from a situation where farmers need to combine heaps of different information sources, which is difficult, to one simple and clear dashboard that provides all the information and required actions in a blink of an eye. How is this done?
- We started to collect field data on two dairy farms located in the North of Holland. These are larger farms, with one farm milking 450 cows conventionally, that is by hand twice a day. The other farm milks 250 cows robotically or automatically. Two different farms, two different situations and two scenarios with different sources of information.
- Data collection started in June last year and involved the manual recording of every cow that was observed to be in heat by the dairy farmer. It also involved the continuous and automated collection of sensor data. This differed slightly per farm. On farm 1, we had an automated heat detection system of which we collected the heat alerts and raw activity data. This was done for all cows and was measured every hour. We also had milk yield measured on a daily basis. Since november 2013, all cows had also an ear-sensor measuring the minutes of each hour that the cow spend on being active, eating, feeding and ruminating. Finally, we had some cow-data coming that was measured during milk recording which happened every 4 weeks. So different sensors, measured with different time intervals. On farm two, we also had the ear sensor and data from milk recording. However, the ear-sensor was placed in 25 cows only. Additionally we had data on milk yield, the cow’s weight and concentrate intake every milking. This information varied within and between cows as cows can be milked 24/7 whenever they please. Finally, this second farm also had an automated heat detection system of which we collected the alerts, but also the raw activity and rumination activity that was measured every 2 h. So slightly more data sources on this farm, all with their own time interval, and with the additional complexity that not all the data is collected for all cows in the herd.
- Now, collecting data always introduces complexity and it never ever goes smoothly straight away. A similar thing happened here. We had difficulties with crashing hardware and software, we lost sensors due to one way or another, we had the huge challenge to combine different systems that all have their own recorded data in their own specific time frequency and we had datasets where it was unclear whether there was a difference in meaning between blanc cells and those with ‘missing values’ and the decision we had to make what to do with this incomplete data.
- So, in a nutshell the actions we did: We combined all the sensor data streams into 2h time blocks and removed periods with for example known data logging errors. This means that we left 600,000 records for farm1 Then we had to add the observed heats. For farm 1 we had 398 heat observations from 203 cows. From all the raw sensor data, we developed 60 predictive variables The next step was to develop a heat detection model and we used an algorithm called LogitBoost to do this. This algorithm is used within the area of data mining I will not go into detail here, but for this model we had to label the records into gold standard positive and negative records. For farm 1 we had 398 heat observations, so 398 gold standard positive records and because of this large amount of heat events, we took a random sample of 20000 gold standard negative records from the pool of gold standard negative records. We then validated the model using ALL data from April 2014 which included 63 heat events, so 63 cows were in heat, and we had 63 gold standard positive records and over 111,000 gold standard negative records Then the situation for farm two, which was slightly different. We had almost 800,000 useful records since June 2013. There were 236 cows observed in heat, but remember that this farmer only recorded ‘morning’ or ‘afternoon’ so we have more gold standard positive events due to this recording than actual cows in heat. Because this farm had more sensor data sources, we developed 105 predictive variables. We then limited ourselves to only those cows which had all sensors available, and remember that I mentioned before that just 25 cows of that herd had that sensor in the ear. Therefore, the model on this farm was trained in 121 heat events, resulting in 441 gold standard positive records. Because the number of heat events was much lower than on farm 1, we didn’t take a random sample from the pool of gold standard negative records but we simply used all records in that time period, resulting in 11000 gold standard negatives. This model was also evaluated on all data from April 2014, that is 24 cows that were in heat, resulting in 72 gold standard positive records and just over 22000 gold standard negative records.
- And what did we see from this model development in which we combined all the sensor data available on the farm to predict heat? 44% out of the 63 heat events in april were found by the current detection model on farm 1. That same model generated, on average, 2.5 false alerts each day. The SDF model, which combined all the sensor data that is available on that farm, had a SN of 75%, and increase of 30 percent points with a similar amount of false alerts each day. In my view, a huge improvement. The second farm had 24 heat events in April. The current heat detection model identified 92% of these events and generated 2.2 false positive alerts per day. The SDF model, which combined all the sensor data on this farm was able to find slightly more heat events but most importantly, it reduced the number of falser alerts by 50%. Given the fact that the common complaing of farmers is the fact that these systems generate too many false alerts, I think SDF was able to improve heat detection on this farm too. So this is where we are at with this fertility project. So...where to from here
- Well, I’ve showed this picture earlier during this presentation and explained that SDF wanted to have a simple dashboard that uses an improved heat detection model to generate clear actions for farmers to improve fertility and to have that happy face? I believe we’re at this point right now. Modelling is of course not reality, so the challenge now is to get the model working real-time. And that is what we’re currently working on. The model is actually running as we speak on the first farm, and it provides output 4 times a day on this dashboard. So we have an improved detection model that lists all cows in heat And per individual cow, the dashboard provides information regarding the best insemination time, which sire best matches this cow and the amount of time left to inseminate this cow. Happy face.
- But this is not the only challenge we face. We still have to let the model run real-time on the second farm and we also have to spend some more time in fine-tuning the models. After that, the biggest challenge we face is scaling up. SDF has the goal to have this principle running on 200 commercial farms across the Netherlands in 2015. With this increasing in volume, we have to think about how to handle the different combinations of available sensors on-farm and the challenge that introduces with model development. Do we develop an x number of models? Or is there another solution?
- So, in summary I discussed three topics today, I briefly introduced the smart dairy farming project and Mathijs will discuss this in more detail during the last presentation I informed you on the importance of heat detection and how SDF aimed at improving this detection by combining different sources of information. The model we developed seem to have potential given the validation results and one model is running on farm 1 as we speak in real-time The challenges we face now is to fine-tune some aspects of the current models as we have and of course, scaling up this procedure for 200 farms. I hope you enjoyed this presentation and thank you for listening!