Nowcasting Modelling of Volatile and Non-Volatile Food Prices Using Crowdsourcing Data (Case Study of Some Food Commodities Prices on Lombok Island in 2015)

1. c o m e . c o n n e c t . c r e a t e
CPS - 2564
Nowcasting Modelling of Volatile and
Non-Volatile Food Prices Using Crowdsourcing Data
(Case Study of Some Food Commodities Prices on
Lombok Island in 2015)
Wida Siddhikara Perwitasari
Tiffany Rizkika
BPS - Statistics Indonesia
22nd August 2019
Under the project of Big Data of STIS Polytecnic Statistic Indonesia
bigdata.stis.ac.id

2. 2
Outline
Copyright ISIWSC2019
Result
3
Conclusion
4
Crowdsourcing Data
1
WHY
Food Price Nowcasting
2
HOW

3. 3
Why?
Copyright ISIWSC2019
The advantage of using Big Data
Use of crowdsourcing data (PREMISE, Pulse Lab and SHK,BPS -
2015)
The importance of nowcasting food prices
Proof of concept in previous research (Pramana, et al., 2016)

4. 5
How?
Copyright ISIWSC2019
• Crowdsourcing Data
• Market Data
• Consumer Price Data
Data Source
chicken, beef, egg, onion, chili, low
quality rice and premium quality rice
Volatile Commodities
mackarel, long bean, instant dry noodles,
peanuts, and vegetable tomatoes
Non-volatile Commodities
March - July 2015

5. 6
How?
Copyright ISIWSC2019
Food
Commodity
Price
Volatile
Non-
Volatile
Present Based Model
Nowcastin
g
Historical Based Model
Filter (KDE)
Filter (IQR)
Nowcast (Modified)
Distributed Lag Model
Neural Network RPROP
(using daily data)
(using weekly data)
(using daily data)
(using weekly data)

6. 7
How?
Copyright ISIWSC2019
Non-volatile Food
Price
Time series-based
Statistical filtering-
based
Nowcast Model (Modified)
Followed by cubic smoothing
spline modelling
Filter : IQR dan KDE
Present data-based
approachment
Kim, J., Cha, M., & Lee, J. G. (2017).

7. 8
Statistics Filtering-based
How? Present data-based
approachment

8. 9
Time Series-based
(Nowcast)
How? Present data-based
approachment

9. 10
Time Series-based (Nowcast Model)
How? Present data-based
approachment
• 𝑃𝑡 =
𝛼𝑃𝑡−1+log 𝑄𝑡 +1 𝑃𝑡
𝑐𝑟𝑜𝑤𝑑
𝛼+log 𝑄𝑡 +1
• 𝑃𝑡
𝑐𝑟𝑜𝑤𝑑
=
𝑗=1
𝑄 𝑡 𝑤 𝑡
𝑗
𝑄𝑡
𝑗
𝑗=1
𝑄 𝑡 𝑤 𝑡
𝑗
• 𝑤𝑡
𝑗
= 1 −
𝑄 𝑡
𝑗
−𝑃 𝑡−1
𝑃 𝑡−1
𝛿
, 𝑗𝑖𝑘𝑎
𝑄𝑡
𝑗
−𝑃𝑡−1
𝑃𝑡−1
≤ 𝛿
0, 𝑦𝑎𝑛𝑔 𝑙𝑎𝑖𝑛𝑛𝑦𝑎
• 𝑃𝑡−1 =
𝑗=𝑡−𝑘
𝑡−1
𝑃 𝑗
𝑘
, 𝑘𝑒𝑡𝑖𝑘𝑎 𝑡𝑖𝑑𝑎𝑘 𝑎𝑑𝑎 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑠𝑖 𝑙𝑒𝑏𝑖ℎ 𝑛 ℎ𝑎𝑟𝑖

10. 11
Present data-based
approachment
Non-volatile FoodpriceResult
Data Prepocessing Result
Distribution of Mackerel Price Data Collection by
Contributors Based on Location, Day and Time
Point Location Data Collection Price of Instant Dry Noodle
During Enumeration
Criteria n
Outside of research time 123
Not in NTB province 599
In NTB province, not Lombok island 190
Amount of data removed
N data : 29806

11. 12
Result
Data Peprocessing Result
Statistic Filtering-based Result
MAPE Value for IQR-Smooth Spline Model and KDE-
Spline Spline Model in All Commodities
Commodities
MAPE
IQR Smooth
Spline (%)
MAPE
KDE Smooth
Spline (%)
Long beans 16,95 % 17,83 %
Mackerel 18,01 % 25,04 %
Instanr Dry
Noodles
25,62 % 0,81 %
Peanuts 118,42 % 62,33 %
Tomat Sayur 15,33 % 15,03 %

12. 13
Result
Data Preprocessing Result
Satistic Filtering-based Result
Nowcast Modelling Result
Do Six experiments with different parameters
Trial to- Outlier Filter Initial Price
1 Default form nowcast
model
1st Observation
2 Default form nowcast
model
HK BPS’s price
3 Default form nowcast
model
The average price of
the 1st day
observation
4 + KDE 1st Observation
5 + KDE HK BPS’s price
Previous
Research
Smallest
MAPEl
1st trial5th trial

13. 14
There are 3 models for each commodities in this research,
they are :
a.Smoothing Spline Model with IQR Filtering (IQR-Spline
Model)
b.Smoothing Spline Model with KDE Filtering (KDE-Spline
Model)
c. Nowcast Model (time series-based)
Conclusion
Non-volatile Foodprice

14. 15
How?
Copyright ISIWSC2019
Volatile Food
Price
Distributed Lag
Model
Historical data-based
approachment
Neural Network
RPROPResilient Backpropagation

15. 16
How?
Copyright ISIWSC2019
• Using Two Methods :
• Distributed Lag Model (DLM)
• Neural Network RPROP (NN RPROP)
• Two Types of Periods : Daily Data, Weekly Data
• Preprocessing : Data Cleaning, Data Transformation, Smoothing, Data
Transformation
𝑌𝑡 = 𝛼 + 𝛽0 𝑋𝑡 + 𝛽1 𝑋𝑡−1 +
… + 𝛽𝑠 𝑋𝑡−1 + 𝑢 𝑡
𝑡 = 1,2, … , 𝑇
Where :
𝑌𝑡 = 𝑡 𝑡ℎ
observation on the dependent variable 𝑋𝑡
𝑋𝑡−𝑠 = independent variable of 𝑡 𝑡ℎ
observation.
𝛼 = intercept
𝛽0, 𝛽1, … 𝛽𝑠 = coefficients at the present time and at lag time,
𝑢 𝑡 = stationary error.
𝜉𝑗
𝑙
= 𝑖=1
𝑁 𝑙−1
𝑤𝑗𝑖
𝑙
𝑥𝑖
𝑙−1
𝜎𝑗
𝑙
𝜉 =
1
1+𝑒
−𝜉 𝑗
𝑙
Where :
𝜉𝑗
𝑙
= net input from neurons to j at layer l;
𝑤𝑗𝑖
𝑙
= the weighting factor between neurons 𝑗 in layer 𝑙 and;
the neuron i in layer to (𝑙 − 1);
𝑥𝑖
𝑙−1
= the value of the i neuron in the layer(𝑙 − 1);
𝑁𝑙−1 = number of neurons in layer to(𝑙 − 1);
𝜎𝑗
𝑙
𝜉 = the sigmoid transfer function to calculate the final value from
neuron j at layer l.
Historical data-based
approachment

16. 17
Graphics
Copyright ISIWSC2019

17. 18Copyright ISIWSC2019
Step of
Study

18. 19
Result
Copyright ISIWSC2019
• Modelling with training data
• Get the Best Model with Testing Data
Commodity
DLM Lag 1
Neural Network
(2)
Neural Network
(3)
Neural Network
(3,2)
Daily Weekly Daily Weekly Daily Weekly Daily Weekly
Chicken 0,0833 0,0591 0,0827 0,0477 0,0831 0,0592 0,0834 0,0579
Beef 0,0019 0,0018 0,0020 0,0018 0,0020 0,0018 0,0020 0,0018
Egg 0,0198 0,0228 0,0171 0,0159 0,0171 0,0221 0,0174 0,0230
Chili 0,0770 0,0582 0,0578 0,1428 0,0757 0,1214 0,0651 0,1203
Onion 0,1082 0,0996 0,1024 0,0675 0,0970 0,0712 0,0937 0,0308
Low quality rice 0,0463 0,0380 0,0266 0,0172 0,0260 0,0081 0,0256 0,0222
Premium quality rice 0,0607 0,0582 0,0424 0,0233 0,0334 0,0165 0,0174 0,0147
Minimum MAPE for DLM model
(cell with yellow highlight) is
dominated by the model with
weekly period data.
MinimumMAPE value for NN RPROP
model (cell with blue highlight) also
dominated with by the model with
weekly period data, but in different
number of neuron.
Commodity Model Period
Chicken 𝑌𝑡 = −16146.441 + 1.03𝑋𝑡 + 0.55𝑋𝑡−1 Weekly
Beef 𝑌𝑡 = 99547.486 + 0.012𝑋𝑡 + 0.067𝑋𝑡−1 Weekly
Egg 1 Hidden layer with 2 neurons Weekly
Chili 𝑌𝑡 = 9253.229 + 0.186 + 0.643𝑋𝑡−1 Weekly
Onion 𝑌𝑡 = 26177.519 + 0.773𝑋𝑡 − 0.835𝑋𝑡−1 Weekly
Low quality rice 2 Hidden layer with 3 and 2 neurons Weekly
High quality rice 𝑌𝑡 = 7554.825 − 0.807𝑋𝑡 + 0.968𝑋𝑡−1 Weekly
Historical data-based
approachment
Volatile Foodprice

19. 20
Conclusion
Copyright ISIWSC2019
• The study have shown that the best model to nowcast
volatile food price (weekly) is Distributed Lag Model.
• For non-volatile commodities, the IQR-Spline model
perform best for long bean and mackerel commodities
whereas the KDE-Spline model best for the commodity of
instant dry noodles, peanuts, and vegetable tomato, long
beans, and mackerel.
Volatile Foodprice

20. c o m e . c o n n e c t . c r e a t e
THANK YOU