This paper proposes a ConvLSTM neural network architecture to forecast precipitation over the next 6 hours. Satellite data and terrain information are fed as inputs to capture atmospheric characteristics over time and space. The model is first trained generally then retrained specifically for a domain in Brazil to improve forecasts. Evaluation of forecasts for December 2022 showed the retrained model had higher precision, recall, and F1-scores, suggesting targeted training can enhance localized predictions. Future work will focus on increasing general training and applying the approach to other forecasting domains and data sources.

1. Data Fusion Approach for Precipitation
Nowcasting with ConvLSTM
Otavio M. Feitosa, Saulo R. Freitas, Haroldo F. C. Velho, Angel D. Chovert.

2. OUTLINE
● MOTIVATION
● METHODOLOGY:
○ DATA
○ PRECIPITATION
CATEGORIES
○ NN ARCHITECTURE
○ “RETRAINING”
● RESULTS
○ CASE ANALYSIS
○ OBJECTIVE ANALYSIS
● SUMMARY

3. MOTIVATION
● Meteorological Challenge: Tackling the 21st-century meteorological challenges in accurately predicting short-term
precipitation.
● Impact on Urban Planning: Significant implications for urban infrastructure and disaster management in densely
populated areas.
● Socio-Economic Considerations: Crucial for decision-making in sectors like agriculture, energy, transportation, and
tourism.
● Technological Advances: Leveraging the advancements in satellite technology (e.g., GOES-R series) and machine
learning.
● Filling the Gaps:
● Addressing the limitations in areas with inadequate radar coverage.
● Improving prediction accuracy and timeliness for better preparedness against extreme weather events.

4. DATA
● The table below shows the input data between the neural network, the training targets, which are
divided into different precipitation categories.
● The input data, such as from GOES-16, were chosen for their availability of temporal space, ideal for
nowcasting.
● For the use of terrain and relief, there are 19 categories used in addition to topography.
Variable Description Relation to Neural Network Input
Infrared Channels Atmospheric data from GOES satellites. Utilized to capture atmospheric characteristics for precipitation prediction.
GLM Data
Atmospheric electrical activity measured by GOES
satellites.
Serves as an indicator of significant meteorological events; a direct input to
the model.
MODIS Data Land use and topography information.
Influences atmospheric “behavior”; incorporated to improve forecast
accuracy.
IMERG Precipitation Estimates Precipitation data from the GPM constellation. Used as the target variable for network training, guiding model learning.

5. PRECIPITATION
CATEGORIES
Prediction Class Lower Limit (mm/6h) Upper Limit (mm/6h)
No Precipitation 0 1
Light Precipitation Accumulation (1) 1 6
Moderate Precipitation Accumulation (2) 6 20
Intense Precipitation Accumulation (3) 20 -
● Below in the table, the limits defined for each class of precipitation accumulated in
6 hours
● The limits were defined subjectively, with considerations of parameters used by
some centers in the central-western region of Brazil, where this work will initially be
applied

6. DATA
Model Number of Examples Reference Year
General Model (D01) 200,245 (balanced) 2019
Retrained Model (D02) 21,231,852 (balanced) 2021 (Summer D02)
The data for the model was collected in a
balanced way, for 2 domains, in the first
domain in a more general way as the entire
neural network will be trained with this
data, in the second set of data only the
vector resulting from the terrain training
and satellite data for a more specific area

7. NN ARCHITECTURE
ConvLSTM
Block
(4,63,63,19)
Conv Block
Terrain
(63,63,13)
Flatten 1D (512) Flatten 1D (512)
NN Dense
Output (4 categories)
● Neural network based model on
Convolutional neural networks (Conv,
(LECUN et al., 1989) and Convolutional
recurrent neural network (ConvLSTM,
SHI et al., 2015c)
● The model can enter the last 2 hours of
infrared and GLM data (with 30 minute
intervals), and as input data land use
and orographic data, which are static
over time in this model

8. NN ARCHITECTURE
● This type of architecture allows the
surrounding data to be considered for
each forecast point, in an attempt to
understand the advection of systems
and characteristics in the atmosphere,
as well as their evolution over time.
● This procedure is necessary for each
forecast point, so there is a certain type
of flexibility in the application, within the
forecast domain.

9. “RETRAINING”
Model Number of Examples Reference Year
General Model (D01) 200,245 (balanced) 2019
Retrained Model (D02) 21,231,852 (balanced) 2021 (Summer D02)
● The data for the model was collected in a
balanced way, for 2 domains.
● In the first domain in a more general way as
the entire neural network will be trained
with this data, in the second set of data only
the vector resulting from the terrain training
and satellite data for a more specific area.
● At this stage, the initial weights were from
the previous model in dense layers, and
with a reduced learning rate*

10. CASE ANALYSIS
In this case, focused more on something more
organized during the summer of (days 09 and
10 of december, 2022) near the capital of the
Brazilian state of Goias, Goiania, a case of
convection a little more organized and close to
the meso-ꞵ (20-200km) scale

11. CASE ANALYSIS

12. CASE ANALYSIS

13. OBJECTIVE ANALYSIS
● For the analysis, the month of
December 2022 was considered in
domain 2, between the general and
retrained model
● As target and correct output, data
estimation from IMERG-GPM late run
was considered, was considered
the closest point between
observation and prediction
Class No Precipitation Low Moderate High
Precision 0.83 0.50 0.47 0.26
Recall 0.54 0.76 0.50 0.42
F1-Score 0.65 0.60 0.48 0.32
Support 1,229,489 746,990 291,195 40,826
Metric Value
Average Precision 0.51
Weighted Recall 0.60
Weighted F1-Score 0.61
“General” Model

14. OBJECTIVE ANALYSIS
● For the analysis, the month of
December 2022 was considered in
domain 2, between the general and
retrained model
● As target and correct output, data
estimation from IMERG-GPM late run
was considered, was considered
the closest point between
observation and prediction
Class No Precipitation Low Moderate High
Precision 0.73 0.58 0.59 0.33
Recall 0.78 0.57 0.42 0.45
F1-Score 0.75 0.57 0.49 0.38
Support 1,229,489 746,990 291,195 40,826
Metric Value
Average Precision 0.56
Weighted Recall 0.66
Weighted F1-Score 0.66
“Retrained” Model

15. OBJECTIVE ANALYSIS
for the graphs above, only values above 1mm in 6h and precipitation probabilities of values above 1mm>h of the
models were considered

16. SUMMARY
● An architecture developed to try to capture space-time characteristics and
make a specific forecast for the next 6 hours
● The results suggest that training the neural dense block can help the model
respond better to a specific region.
● In future work, it is expected to increase the training of the general model,
so that it works as a vision transformer, and apply it to similar forecasting
tasks, with adjustments to other data sources (such as rain gauges).
Thank you for your time!
support agencies