A look into Proagrica's use of HPCC Systems' open source big data analytics platform for precision agriculture.

1. Using big data to help feed the world
Jeff Bradshaw - CTO

2. Who are Proagrica
Proagrica, the global agricultural division of RELX, drives growth and improves efficiency by
delivering high-value insight and data, critical tools and advanced technology solutions

3. Agriculture is at the Centre of Global Change

4. How could data help?

5. Precision Agri: Our Data Landscape / Assets
Vast amounts of data spread across the Agricultural landscape. Proagrica is consolidating, organising and enhancing this data
to help drive value across the entire industry, from the farm gate all the way to the super market shelf
Farm Machinery
Every piece of equipment on the farm is
now generating data and wants to be
precise
Agronomist
Providing farm advice, shape files and
data to farmers
Manufacturers & Distributors
Adaptris manages supply chain
connectivity between MFRS and their
Distributors
Weather Data
Global current and historical weather and
soil moisture data at sub-field level
Farm Management Information
Systems (FMIS)
A wide spectrum of tools used by
Farmers all generating data
Satellites / Drones
Ability to identify yield and crop issues
from space / drones
Sensors
Ground and animal sensors measuring
everything from animal fertility to soil
moisture
Soil
Global soil type horizons

6. 570 million Farms, 25 million Tractors, 50 billion chickens, 1
billion sheep, 1 billion pigs, 80 million turkeys, 1.5 billion cows
in the world !
100% of them with passports in the UK vs 36% of the US
populaIon….
…and Big Brother / Data is here, for Animals at least as
they are all being monitored / reporIng data

7. Source: aea.co.uk

8. An overview of the approach

9. ProAgrica HPCC Platform
DATA
STORA
GE,
ENTITY
LINKIN
G &
ETL
LAYER
DATA
SCIENCE
LAYER
THE
WORLD
INDEX SEARC
H
REF DATA APIs
GEOLOCATION
INFO
SOIL TYPES
WEB API
vIPER
DATA
VIZ
PRODU
CT
Farm Management Systems
IoT / Machinery

10. What does it deliver?
▶ Global insight through fully integrated data, Data As A
Service and a range of Analytics tools
▶ An agile, scalable, resilient and secure platform that can
consume data from any source, consolidate, enrich and
expose global agricultural data from everything soil to
animals and all the way to satellites
▶ Precision Ag covering the full Ag value chain from Mfr,
through Agronomist, CO-OP, Farmer and Distributor
▶ A range of Analytics solutions focused on Pesticides,
Herbicides, Fertilizers, Seeds, Cattle, Milk, etc. that provide
insight at market, region, farm, field and sub-field levels
▶ Enabling the industry to increase yield and profitability
whilst reducing inputs and improving environmental impact

11. There is a lot of detail……..

12. We can present it differently to manage risk

13. What does it deliver?
▶ Global insight through fully integrated ESB data,
Data As A Service and a range of Analytics tools
▶ An agile, scalable, resilient and secure platform that
can consume data from any source, consolidate,
enrich and expose global agricultural data from
everything soil to animals and all the way to satellites
▶ Precision Ag covering the full Ag value chain from
Mfr, through Agronomist, CO-OP, Farmer and
Distributor
▶ A range of Analytics solutions focused on Pesticides,
Herbicides, Fertilizers, Seeds, Cattle, Milk, etc. that
provide insight at market, region, farm, field and sub-
field levels
▶ Enabling the industry to increase yield and
profitability whilst reducing inputs and improving
environmental impact

14. A worked example – OSR aka Canola
▶ It is an edible crop
▶ Food oil
▶ High protein animal feed
▶ BioDiesel
▶ Lubricant
▶ Variable yield
▶ New diseases
▶ Pesticide limitations
▶ Tight profit margins

15. Patterns of OSR using Principal Component Analysis
▶ Why was the 2016 harvest in the UK so awful?
▶ What correlates to higher yields?
▶ How effective are pesticides?
▶ Are hybrids better?

16. Correlation doesn’t equal causation……

17. Some unusual yields…….
Maximum yield:
36,784,867 kg/ha

18. Growers aren’t very skilled at data entry
Planted Seed Variety
DK Excaliber
DK Excalibur
Excalibur + Coa@ng
Excalibur Stock
Excalibur and Catana
Rolled
OSR + 15:10:28
Planted Seed Variety
Excalibur
Excalibur
Excalibur
Excalibur
Other
Other
Other

19. Oh… in case I forgot to mention
THE DATA IS DIRTY…..

20. It can also be incomplete……..

21. How has yield varied over the last 10 years?
▶ Average yield is 3,766 kg/ha

22. The spread in yield
3,750 kg/ha
5,250 kg/ha
2,250 kg/ha
▶ Most growers are within 1,496 kg/ha of the average

23. …. But this isn’t constant!

24. Could it be related to variety choice?

25. Popularity of hybrid varieties by location

26. Variety trait differences by location

27. How do we visualise the data?
▶ Over 150 pairs of variables to investigate
▶ No idea what is linked before we start……

28. What’s this 2D picture of a 3D object?

29. How about now?

30. How to read the graphs
Two variables
that are high
at the same
@me

31. How to read the graphs
Two variables
that are high
at opposite
@mes

32. How to read the graphs
Two variables that
have nothing to do
with each other

33. The market for OSR varieties

34. What causes variation in yield – it’s a similar story
▶ Degree days
▶ Fertiliser treatments
▶ Fungicide treatments
▶ Insecticide treatments
▶ Month of first insecticide
application
▶ High wind events
▶ Temperatures
▶ Average rainfall
▶ Coldest week
▶ Total radiation
▶ Wettest weeks
▶ Driest weeks
▶ Soil moisture
▶ Precipitation
▶ Latitude & Longitude
▶ Soil content

35. Overall, what correlates with a high yield?
Warm winters
Dry springs
Warm
summers
FerIlisers not needed?
Larger and
Northern?
2016
2015

36. At last some answers…….
▶ Why was the 2016 harvest in the UK so awful?
▶ Wet spring and/or dry winter
▶ What correlates to higher yields?
▶ Warm spring, wet winters and proper pesticide application
▶ How effective are pesticides?
▶ Yes for fungicide, “perhaps” for insecticide
▶ Are hybrids better?
▶ Not really

37. Take home messages
▶ If you’re a farmer
▶ There are probably too many varieties of OSR in the world!
▶ OSR does better in wet springs and warm winters
▶ If you’re in to analytics
▶ Working with big data is a lot of fun
▶ Dimension reduction is great for picking out correlations in
complex data

38. Over 3,600 integrated agricultural customers

39. Thank you!!!!