This document describes a deep learning approach called c-ResUnet for counting cells in fluorescent microscopy images. It discusses fluorescent microscopy imaging techniques and applications in life sciences. It then introduces the Fluorescent Neuronal Cells dataset and challenges in counting cells, such as class imbalance, overcrowding, and noise. The c-ResUnet model is presented, which uses a convolutional neural network with residual blocks for semantic segmentation. Experiments show that c-ResUnet outperforms other architectures and achieves performance close to human experts on this dataset through the use of weight maps and oversampling artifacts during training. Both qualitative and quantitative evaluations demonstrate the effectiveness of c-ResUnet for automated cell counting.

1. c-ResUnet: un approccio
Deep Learning per contare
cellule in microscopie in
fluorescenza

2. About me
PhD in Data Science & Computation
Collaborazioni
22/06/2022 2
Luca
Clissa

3. Aree di ricerca
22/06/2022 3
Computer
Vision
Text
Processing Time Series
Quantum
Computing

4. Outline
• Microscopia in fluorescenza
• Come funziona
• Applicazioni
• Fluorescent Neuronal Cells dataset
• Cell ResUnet (c-ResUnet)
• Risultati
22/06/2022 4

5. Microscopia in Fluorescenza

6. Cos’è la fluorescenza?
• Tecnica di imaging
• Assorbimento/emissione luce
• Spesso utilizzata in life science
22/06/2022 6

7. Applicazioni
• Studio meccanismi responsabili del
torpore nei roditori (Hitrec [1], 2019)
• Importanti ricadute umane
• Neuroni marcati con fluoroforo giallo
• Varie tonalità, forme e grandezze
• Obiettivo: conteggio neuroni marcati
22/06/2022 7

8. Problema
• Manuale
• Time-consuming
• Error-prone
• Interpretazione soggettiva
casi limite
22/06/2022 8

9. Obiettivo
Automazione
Interpretabilità
22/06/2022 9

10. Strategia
• Counting by segmentation
• Semantic segmentation
• Convolutional neural network
22/06/2022 10

11. Fluorescent Neuronal Cells dataset
• 283 immagini ad alta risoluzione (1200x1600) (Hitrec et
al. 2019)
• Ottenere annotazioni è molto dispendioso
• Approccio semi-automatico
• 252 bozze basate su filtro luminosità, rifinite poi da esperti
• 31 immagini segmentate manualmente
22/06/2022 11

12. Canali RGB
red
7.32
16.81
0
0
2
5
9
12
252
quantile
mean
s.d.
min
10%
25%
50%
75%
90%
max
green
2.83
13.30
0
0
0
2
3
4
251
blue
0.20
1.43
0
0
0
0
0
0
87
22/06/2022 12

13. Encoding colore
22/06/2022 13
RGB HSV

14. Distribuzione conteggi
22/06/2022 14
# cells/image
27.05
21.75
0
4
7
21
48
59
68
quantile
mean
s.d.
min
10%
25%
50%
75%
90%
max

15. Area e diametro di Feret
Feret
Diameter (µm)
17.48
8.20
5.86
9.42
11.95
15.53
20.86
27.61
67.48
area (µm2)
119.30
97.96
15.94
35.23
55.51
89.86
148.02
237.09
796.39
quantile
mean
s.d.
min
10%
25%
50%
75%
90%
max
22/06/2022 15

16. Sfide
Sbilanciamento classi
Overcrowding
Rumore etichette
Artefatti
22/06/2022 16

17. Sbilanciamento classi
signal (%)
0.50
0.61
0
0
0.09
0.34
0.68
1.07
4.86
quantile
mean
s.d.
min
10%
25%
50%
75%
90%
max
signal ratio
367k
756k
19.57
92.39
145.35
291.10
1k
1.9M
1.9M
22/06/2022 17

18. 18

19. 19

20. 20

21. 21

22. 22

23. 23

24. 24

25. 25

26. Metodi
• c-ResUnet
• Ablation studies
• Weight maps (WM)
• Artifacts oversampling (AO)
22/06/2022 26

27. 27
Cell ResUnet
• Convoluzione 1x1 iniziale
• Conversione grayscale
→ Colourspace appreso
• Bottleneck
• Blocco residuo addizionale
• Filtri 5x5 (anzichè 3x3)
→ Maggiore field-of-view

28. Weight map
• Penalizzano errore in aree sovrafollate
• Effetti additivi
28
𝑤 = exp −
𝑑2
2𝜎2 ,
where
𝑑: distance from a cell
σ = 25px (mean cell radius)
22/06/2022

29. Weight maps
• Penalize errors in crowded areas
• Additive compound
29
22/06/2022

30. Artifacts oversampling
• Difficile da gestire
• Sottorappresentati
• Ricampionati con fattore 6
30
22/06/2022

31. Experiments
31
Ablation studies
• Full design (WM + AO)
• Weight maps only (no AO)
• Artifacts oversampling only (no WM)
Benchmark against
SOTA architectures
• c-ResUnet (1.7M params)
• ResUnet (887k params)
• Unet (14M params)
• small Unet (876k params)
Metrics of
performance
• Detection: F1 score, AUC
• Counting : R2, MAE
22/06/2022

32. Training pipeline
• 150 training; 63 validation; 70 test
• 512x512 crop
• Augmentation
• Rotazioni, rumore Gaussiano, variazione luminosità, deformazioni
elastiche
• Fattore re-sampling
• 4 per immagini segmentate in semi-automatically
• 10 per immagini segmentate manualmente
• 25 per artefatti
32
22/06/2022

33. Training settings
• Adam optimizer
• Learning rate (lr) starts at 0.06
• Decrease lr by 30% if no improvement in 4 epochs
• Weighted Binary Cross-Entropy loss
• 1.5 (cellule), 1 (sfondo)
• Criterio stopping: 20 epoche senza miglioramenti
• Implementazione
• Keras, TensorFlow
• 4 NVIDIA V100 GPUs (CNAF)
33
22/06/2022

34. Post-processing
• Output grezzo: heatmap
• Thresholding
• Introduce iper-parametro
• Ottimizzazione threshold
• Pulizia output
22/06/2022 34

35. Threshold optimization
35
22/06/2022

36. Post-processing
22/06/2022 36
(b) raw heatmap (c) thresholded (d) post-processed
(a) ground-truth

37. Qualitative evaluation: prediction
22/06/2022 37

38. Qualitative evaluation: artifacts
22/06/2022 38
c-ResUnet (no AO) c-ResUnet

39. Qualitative evaluation: weight maps
22/06/2022 39
c-ResUnet
c-ResUnet
(no
WM)

40. Quantitative evaluation
40
22/06/2022

41. Quantitative evaluation
41
22/06/2022

42. Quantitative evaluation
42
22/06/2022

43. Quantitative evaluation
43
22/06/2022

44. Quantitative evaluation
44
22/06/2022

45. Quantitative evaluation
45
22/06/2022

46. Quantitative evaluation
46
22/06/2022

47. Quantitative evaluation
47
22/06/2022

48. Conclusioni
c-ResUnet performa meglio dei competitor
Prestazioni simili ad operatore umano
“Effetto operatore” sistematico
Weight map utili
Pubblicazione dataset e modello allenato
Artefatti
22/06/2022 48

49. References
• Morelli et al. 2021, Automating cell counting in fluorescent
microscopy through deep learning with c-ResUnet:
https://rdcu.be/cB1Ds
• Clissa et al. 2021, Fluorescent Neuronal Cells dataset:
http://amsacta.unibo.it/6706/
• GitHub: https://github.com/robomorelli/cell_counting_yellow
22/06/2022 49

50. Backup

51. Metriche performance
• True positives (TP), False Positives (FP) e False Negatives (FN)
• Detection: F1 score, AUC
• Conteggio: R2, MAE
for center in predicted_centers:
compute distance (px) from all true_centers
if minimum_distance < 50:
increase TP count
remove corresponding true center
FN = ntrue – TP
FP = npred - TP
22/06/2022 51

52. Metriche detection: F1 score e AUC
• F1 score come metrica principale:
𝑎𝑐𝑐𝑢𝑟𝑎𝑐𝑦 =
𝑇𝑃
𝑇𝑃 + 𝐹𝑃 + 𝐹𝑁
,
𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 =
𝑇𝑃
𝑇𝑃 + 𝐹𝑃
,
𝑟𝑒𝑐𝑎𝑙𝑙 =
𝑇𝑃
𝑇𝑃 + 𝐹𝑁
,
𝐹1 =
2 ∙ 𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 ∙ 𝑟𝑒𝑐𝑎𝑙𝑙
𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 + 𝑟𝑒𝑐𝑎𝑙𝑙
,
• Area Under precision/recall Curve (AUC)
22/06/2022 52

53. Counting metrics: MAE and R2
Errore assoluto (AE) e percentuale (PE):
𝐴𝐸 = 𝑛𝑡𝑟𝑢𝑒 − 𝑛𝑝𝑟𝑒𝑑 ,
𝑃𝐸 =
𝑛𝑡𝑟𝑢𝑒 − 𝑛𝑝𝑟𝑒𝑑
𝑛𝑡𝑟𝑢𝑒
Coefficiente di determinazione, R2:
𝑅2 = 1 −
σ𝑖 𝑛𝑡𝑟𝑢𝑒
𝑖
− 𝑛𝑝𝑟𝑒𝑑
𝑖 2
σ𝑖 𝑛𝑡𝑟𝑢𝑒
𝑖
− ത
𝑛
2 ,
Dove 𝑛𝑡𝑟𝑢𝑒
𝑖
, 𝑛𝑝𝑟𝑒𝑑
𝑖
sono il numero di cellule vere e predette per l’immagine 𝑖,
ത
𝑛 è il numero medio di cellule per immagine
22/06/2022 53

54. ROC curve
54
22/06/2022