At BAADER, we design and engineer innovative and holistic solutions that ensure intelligent, safe, efficient and sustainable food processing in all phases, from the handling of live and raw protein materials to the finished food products. As a key player in the food value chain, we aim to take further significant steps toward greater efficiency, traceability, transparency, profitability, and sustainability through new digital solutions. During our digital transformation we are working on two ends: At one hand there are many brownfield factories unprepared for the digital journey and the other hand we have powerful greenfield technologies like Apache Kafka. Now, we have to bring two mindsets together – robust food processing machinery and highly scalable software technologies. In this talk, we will present how we successfully started to ingest various kinds of IoT data into our Kafka cluster - spotlighted from both ends.

1. 14. January 2019, Apache Kafka Meetup, Hamburg
Stefan Frehse, Software Architect, Digitalization
Ingesting IoT data in Food Processing

2. Machines for Food Processing
Example: fish machines
2

3. Machines for Food Processing
3

4. 11.01.20194
BAADER – Digital opportunities
A data driven food value chain
BAADER approach
Data-driven company by Apache Kafka
Summary
Agenda
BAADER Digitalization

5. OPPORTUNITIES OF FUTURE VALUE ADD
CURRENT VALUE CREATION
OPPORTUNITIES OF FUTURE VALUE
ADD
Machine information
runtime, speed, energy consumption
Product Data
Size, Weight, Quality
Business Process Data
Supplier, Order, Customer, Shipment
INFORMATION CHAIN
BAADER – Digital Opportunities
Expanding own value added through business process data capturing and using
BAADER Digitalization5
OPPORTUNITY OF FUTURE VALUE ADD OPPORTUNITY OF FUTURE VALUE ADD

6. 11.01.20196
Capturing data from farm to fork
A data-driven Food Value Chain
6
Farm inputs Primary sector Transport Processing Retail Consumers
Farmers/input suppliers
• Optimize yield and operational management
• Anticipate future developments
• Provide product and process guarantees
Offtakers
• Enable market and consumer-driven production
• Shorten and optimize production chains
Value Chain
• Predict market dynamics to anticipate supply/demand and negotiate better prices
• Meet demand by consumers for more transparency, e.g. provenance, production methods
Reduce food waste and increase production
BAADER Digitalization
Recycling

7. 11.01.20197 BAADER Digitalization7
ONE digital platform
as basis for all
future solutions in
the food processing
industry
Our vision
BAADER Approach
* Artificial Intelligence

8. 11.01.20198
Visibility, consensus, performance for a data-driven company
Collecting data along the food value chain
8
Shared, replicated, permissioned
Farm record Transport
record
Weather
record
Processing
record
Packaging
record
Retailers
record
Auditors
record
End user
record
BAADER Digitalization
Single source of truth for a
Data-driven company

9. 11.01.20199
Gathering data for food safety
Traceability
Temperature
Humidity
Air exchange
Transport time
Stunner settings
Time hanging
Transparency
Animal welfare indicators in poultry processing

10. Very brief overview
Vision: A data-driven company
10
Arbitrary
records
Apache Kafka
Transformation Aggregation
Central data log
Remote Service
Machine R&D
Predictive Maintenance
API
Dashboards / Reports

11. Collecting Farming and Transportation Records
Gathered from the farming process
● Farming: Chickens grow up and are catched after a
few weeks
● Critical parameters:
○ Temperature
○ Humidity
○ CO2
● GPS positions of the truck to forecast the ETA
○ Optimal preparation for reception
● Load information: number of birds, weight, sex
● => Animal welfare has a strong focus
11
+

12. Collecting Machine Data
Gathered from the slaughterhouse (factory)
● Installed OPC UA server (M2M protocol) provides
machine data
○ power consumption, voltage, line speed, air &
water temperature, bone size, fillet size,
quality, weight
● local IoT-gateway subscribes data from OPC UA
and forward the data to AWS IoT Core as they
occur
12

13. Processing record
Example: Collecting data from camera systems
11.01.201913
Camera before
chilling
Camera after
chilling
Upstreaming
steps+
Downstreaming
steps+
10:00 pm 1:00 pm Timeline
BAADER Digitalization
~190 birds/min
Evisceration Line CutUp Line
~190 birds/min

14. Product quality: Chicken
Collecting data from a camera system
11.01.201914 BAADER Digitalization
● Camera takes 2-4 pictures of
each chicken
● Images are analyzed in
real-time at the edge
● Quality and potential defects
are provided
● Sorting based on the quality
for downstreaming steps
(cutup, whole bird, …)

15. Data ingestion into Apache Kafka
Keeping the quality
11.01.2019BAADER Digitalization15
Apache Kafka
Cluster
MQTTAdaptor
AWSIoTCore(MQTT)
Elasticsearch / Kibana
MongoDB
AWS S3
KafkaConnector
Demo
Streaming SQL Processing
GPS
Machinedata
Weather
MQTT over TLS
Data source

16. KSQL and Chicken Quality
Creating a stream
CREATE STREAM quality (
thingId VARCHAR,
value STRUCT<quality STRUCT<code INT>>)
WITH (KAFKA_TOPIC='camera', VALUE_FORMAT='json');
● Creates a KSQL stream of the quality from the camera.
16

17. KSQL and Chicken Quality
Creating a windowed table for efficiency monitoring
CREATE TABLE empty_shackles_per_5_minutes AS
SELECT thingId, COUNT(*) FROM quality
WINDOW TUMBLING (SIZE 5 MINUTES)
WHERE value->quality->code = 4
GROUP by thingId;
● Create a table over a 5 minute window for empty shackles (code = 4)
● Application / Alerting: Notify factory manager about potential efficiency drop
17

18. Kafka and Transportation Data
Computing truck arrivals
● Topic that contains the GPS positions of a truck
18
{
"thingId": "4d5cc1d6-50ca-4bf9-bb6c-ddfaa9043d6a",
"thingType": "baader::dt::vehicle::daf::vehicle_geo_position",
"thingVersion": "0",
"value": {
"Latitude":"5X.6155128",
"Longitude":"-X.9469323",
“LastUpdateDataTimeInUtc":"2018-09-14T11:51:44.000Z"
}
"timestamp": 1536955501460
}

19. ETA Computation
Using KSQL for computing ETA
● KSQL: Added own stateless functions for SQL stream processing
● UDFs: User-defined functions in Java and just putting the compiled jar into the
confluent-platform environment
● Implemented a UDF for computing the ETA using Google Distance API
● Derived a new stream that calls the distance UDF
● The ETA is presented in a dashboard
19

20. Drive new businesses
Writing streaming applications
● Building a central data lake for machine data (power, voltage, water
consumptions) -> next machine generation
● Providing a transparent view of the entire food value chain
●
● Using easy-to-use streaming SQL (e.g., KSQL, LSQL) to create derived streams
○ Computing windowed reached SLA for the machines
○ Efficiency by computing number of empty shackles
○ Alerts when violating thresholds
20

21. Value of Apache Kafka
Context Digitalization
● Building streaming applications in the context of IoT
● Event-driven architectures
● Easy experimentation with low cost
● Single source of truth for a data-driven company
● … and the usual suspects
● Fault-tolerance
● Strict ordering
● High-Availability
● Scalability
21

22. Confluent Cloud Enterprise
● Fully managed Kafka Cluster for dev. and prod.
● Just configure: throughput and retention time
● Excellent support in terms of response and quality
● Secured environment using VPC peering on AWS
● Open-Source contributor to Apache Kafka and its entire ecosystem
22
Values

23. Today’s numbers
At the very early stage
● 40-50 msg/s
● Total: 140 data points
● 100 GB gathered from a single factory within a few months
● There is much more, but...
23

24. We are hiring:
https://stackoverflow.com/jobs/companies/baader

25. Backup
Kibana dashboard (15 min) of the camera system
25