In recent years, there has been a significant increase in the use of Smart Farming Technologies (SFTs), which are seen as key enablers in farm management for crop monitoring and reduction of chemical use. This presentation will cover a key component for the advancement of such systems – a data infrastructure which offers semantic and syntactic interoperability. Through the utilization of ontologies and smart data models in the agricultural domain, this kind of infrastructure can support actionable digital twins and advance farming capabilities.

1. Research Software Engineer @ VizLore Labs Foundation
PhD Candidate & TA @ University of Novi Sad, Faculty of Sciences
Lead Developer @ Cimet Software
Scalable and Interoperable Data
Flow Management for Digital
Twin Frameworks in Smart
Farming
Mihailo Ilić

2. • Interoperability – the ability of software systems to communicate
seamlessly with one another, share data, and work together to achieve a
common goal.
• Genius hardware and software solutions exist in almost every domain.
• Most systems have reached peak performance.
• The lack of interoperability between these systems is what is keeping us
from unlocking their full potential.
Introduction

3. • One such domain is agriculture.
• Enabling technologies of Smart
Agriculture include:
• Smart sensing, monitoring &
control;
• Data analysis & planning;
• Autonomous farm management.
Introduction
https://doi.org/10.1016/j.agsy.2017.01.023

4. • Data collection and monitoring allow for smart decision making.
• Interoperability between agriculture systems would reduce costs,
minimize operational overhead, and create optimal experiences when
integrating and using these systems, thus increasing adoption.
Introduction

5. • How is interoperability achieved?
• Ontology (or common vocabulary);
• The use of a common syntax and structure.
• Interoperability should originate as close to the data sources and control
points as possible.
• Limit the need for higher level interoperability middleware.
Introduction

6. • Horizon 2020 Project.
• smartdroplets.eu
• 9 partner organizations.
• Smart farming technologies as key enablers in
reduction of chemical use and negative
environmental impact.
• Validation to be done in test sites in Lithuania
(wheat farm) and Spain (apple orchard).
The Smart Droplets Project

7. • Conventional spraying is the most resource-intensive agricultural
operation, compromising natural, chemical, and human resources each
season.
• The EU Green Deal
• Reduce the use of pesticide by 50% and the use of fertilizers by 20% by 2030.
• The ideas behind Smart Droplets:
• Advancement of both hardware and software capabilities during chemical application;
• Data collection and use of actionable digital twins to make predictions and suggestions
on the best course of action;
• The use of advanced technologies (AI and robotics).
The Smart Droplets Project

8. • Technological building blocks:
• Data Infrastructure – enables
interoperability;
• AI Models & Digital Twins (DTs) –
intelligence systems capable of
simulating field conditions and
providing actionable decisions;
• Robotic Systems – autonomous
operation in the fields with spatial
awareness;
• Direct Injection Spraying – high
precision spraying.
The Smart Droplets Project

9. • General project architecture.
• Interaction between multiple
software systems:
• Field data sources;
• Intelligence components;
• Field systems – autonomous sprayer;
• 3rd party services – e.g. weather
services.
• Interoperability is a must.
The Smart Droplets Project

10. • Digital Twins allow us to run simulations on virtual counterparts of fields and
crops.
• WOFOST (WOrld FOod STudies) – a mechanistic model for simulating crops and running
quantitative analysis on growth and production;
• Maintained by WUR.
• DTs can help with:
• Precision Spraying – Minimizing environmental impact;
• Predictive Analysis – Predicting the spread of disease;
• Resource Optimization – Reduction of operational cost;
• Remote Monitoring – Allow monitoring from anywhere.
Actionable Digital Twins

11. • Digital Twin – virtual counterparts of
real-world entities.
• Data collection is conducted at the edge.
• Actionable decision-making is done in
the cloud.
Actionable Digital Twins
Data
management and
interoperability
Autonomous
Field systems
Legacy data
sets and
existing field
systems
Manually
provided by
farmers
3rd party
services
Digital Twins and AI - input data
● Crop characterizationand state.
● Parcel maps;soil characteristics
● Detected diseases, weeds.
● Operations(pesticideapplication,
fertilization).
● Field system status.
● Yield
● Pesticideand fertilizerapplication.
● Irrigation events.
● Weather
● Farm and crop onboarding.
● Sowing/flowering/harvest.
● Manual feedback on crop state and
autonomous field system operation.
● Corrective actions.
● Weather information - past and real
time.
● Pesticidedatabases.
● Satelliteimagery.
Data
warehouse

12. • Autonomous edge data collection is done with the Smart Droplets vision
system
• Mounted on retrofitted autonomous tractors;
• Equipped with RGB cameras, a set of stereo camera modules, and edge AI.
• Edge AI used to:
• Detect weeds, pests, and nitrogen deficiencies;
• Assess plant canopy density.
Autonomous Data Collection

13. • Pest detection
• Using RGB cameras;
• AI detects the presence of pests in the images;
• Continuous scanning of the canopy;
• Recorded with a geographical stamp;
• Post processing at the edge;
• Final upload to the Smart Droplets DT.
Autonomous Data Collection
Apple Scab
Altenaria
Sclerotinia

14. • Weed detection
• Also using RGB images;
• Dedicated AI model;
• Georeference stored as well.
Autonomous Data Collection
Tansy Mustard Weed

15. • Nitrogen deficiency
• Recognized on the plant leaves;
• Leaves become more yellow;
• Precision fertilization.
Autonomous Data Collection
Nitrogen deficiency in
apples
Nitrogen deficiency in wheat

16. • Canopy density estimation
• 3D profile of the canopy;
• Density estimation done through a set of
stereo cameras.
Autonomous Data Collection

17. • Actionable DTs:
• Data → Information → Knowledge →
Optimal Actions
• General workflow
• Field level observations are uploaded to the data
management platform.
• Sources: sensors, 3rd parties, farmers themselves.
• Decision making.
• DT simulations through WOFOST.
• Further actions are relayed back to the autonomous
sprayers.
Decision Making
Wheat Farm
Data
Management
Platform
Digital Twin &
AI Services

18. • High-level system interoperability is achieved through 2 concepts:
• Semantic interoperability – the use of a common vocabulary /
language / ontology;
• Syntactic interoperability – agreement on a common way of
representing data.
• Full semantic interoperability is hard to implement.
Interoperability

19. • Increasing communication efficiency between numerous systems.
• By using the same terminology, it becomes easier for systems to map and
integrate data from diverse sources.
• Ontologies
• Provide a shared understanding of concepts and relationships within a given domain;
• Standardized and formal representation of knowledge, using a common vocabulary.
Semantic Interoperability

20. • Choosing an ontology depends on the problem domain and information
which is being handled.
• Entities
• farms, fields, crops, soil, fertilizers, tractors, sensors …
• Events
• sowing, harvesting, irrigation, pesticide application …
Semantic Interoperability

21. • Smart Data Models – A collaborative effort to support the adoption of
common data models.
• FIWARE Foundation, TM Forum, OASC and IUDX;
• smartdatamodels.org
• Smart Agrifood – provides a vocabulary in the agricultural domain.
• One of numerous vocabularies in the Smart Data Models collection.
Semantic Interoperability

22. • The entities and relationships
identified in the project are covered by
Smart Agrifood, Smart Robotics, and
Smart Sensoring vocabularies.
Semantic Interoperability
Digital Twin Entity /
Event
Smart Agrifood Model
Farm AgriFarm
Field AgriParcel
Crop AgriCrop
Weed AgriPest
Pesticide Application
AgriParcelOperation
Irrigation Event
Fertilization
Sowing / Harvest

23. • How should the data be represented?
• The Smart Data Models can be described via NGSI-LD
• Next Generation Service Interface - Linked Data (information model & API);
• Used to support the exchange of structured data;
• An extension of standard JSON-LD (JSON Linked Data).
• Entities are described through:
• Properties
• Relationships
• These properties and relationships come from contexts
• E.g. The Agrifood context – A crop has its name and is planted on a certain
type of soil.
Syntactic Interoperability
Agri
Crop
Agri
Soil
hasAgriSoil
name
Agri
Product
Type
hasAgriFertilizer

24. • FIWARE Orion Context Broker
• Open-source software compliant with the NGSI standard;
• Serves and handles context information (the current state of
entities);
• Facilitates communication between all Smart Droplets
components.
Syntactic Interoperability
Context
Broker
Cloud
Intelligence
Edge
Components

25. Syntactic Interoperability
{
"@context": "https://json-ld.org/contexts/person.jsonld",
"@id": "http://dbpedia.org/resource/John_Lennon",
"name": "John Lennon",
"born": "1940-10-09",
"spouse": "http://dbpedia.org/resource/Cynthia_Lennon"
}
{
"@context": [
"https://json-ld.org/contexts/person.jsonld",
"https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld"
],
"id": "http://dbpedia.org/resource/John_Lennon",
"type": "Person",
"name": {"type": "Property", "value": "John Lennon"},
"born": {"type": "Property", "value": "1940-10-09"},
"spouse": {
"type": "Relationship", "object":
"http://dbpedia.org/resource/Cynthia_Lennon"
}
}
To JSON-LD
To NGSI-LD

26. Data
warehouse
Wheat Farm
Putting it all Together
Data
Management
Platform
Digital Twin &
AI Services
NGSI-LD
Edge Data
NGSI-LD
Actions

27. • Noticeable lack of interoperability between systems.
• Missing out on the full potential of software.
• Multiple levels of interoperability exist.
• Smart Droplets – Smart farming with the aim of reducing chemical use.
• Interoperability as an accelerator of smart farming.
Conclusion

28. Thank you!
Mihailo Ilić
LinkedIn
mihailo.ilic@vizlore.com
milic@dmi.uns.ac.rs