The document discusses the differences between overfitting and overparametrization in machine learning models. It explores how random forests may exhibit a phenomenon known as "double descent" where test error initially decreases then increases with more parameters before decreasing again. While double descent has been observed in other models, the document questions whether it is directly due to model complexity in random forests since very large trees may be unable to fully interpolate extremely large datasets.

1. Overfitting or overparametrization?
Overfitting or overparametrization?
Nathalie Vialaneix, INRAE/MIAT
Nathalie Vialaneix, INRAE/MIAT
SaAB meeting – March 18th, 2022
SaAB meeting – March 18th, 2022
1 / 9
1 / 9

2. Overfitting
2 / 9

3. Overparametrization / double descent
Michael Jordan / Journée IA SFDS 9/03/2021
NIPS & PNAS 2019:
3 / 9

4. Overparametrization / double descent
4 / 9

5. RF, interpolation and double descent
5 / 9

6. RF, interpolation and double descent
6 / 9

7. RF, interpolation and double descent
7 / 9

8. RF, interpolation and double descent
8 / 9

9. Conclusion
I still don't know if overparametrization/double descent really exist
but I'm sure that, for RF at least, double descent is not directly an effect of
model complexity
questions still to answer: for very large datasets, interpolating trees can
not be grown / what would be the proper RF heuristic in that case?
9 / 9