This document provides an overview of continual learning and state representation learning, including key papers in the fields. It summarizes various benchmark datasets used to evaluate continual learning models, such as MNIST, Fashion MNIST, and CORe50. Environments used to evaluate state representation and reinforcement learning are also cited, including ViZDoom and CRLMaze. Overall modeling approaches discussed include convolutional neural networks, policy distillation, and sim-to-real transfer for continual reinforcement learning.

3. ●
●
●

4. ●
●
●
●

5. ●
●
●
●

7. ●
●

8. ●
●
○
○

9. ●
○
https://continual-vista.github.io [Greco et al.'19]

10. ●
○
○
○
○
http://ai.berkeley.edu

11. ●
○
○
○
○
●
[McClelland'95, Parisi'18]

13. [Lesort, Lomonaco, Stoian, Maltoni, Filliat, Diaz-Rodriguez. 19]

14. [Lesort, Lomonaco, Stoian, Maltoni, Filliat, Diaz-Rodriguez. 19]

15. [Lesort, Lomonaco, Stoian, Maltoni, Filliat, Diaz-Rodriguez. 19]

16. [Lesort, Lomonaco, Stoian, Maltoni, Filliat, Diaz-Rodriguez. 19]

17. [Lesort, Lomonaco, Stoian, Maltoni, Filliat, Diaz-Rodriguez. 19]

18. [Lesort, Lomonaco, Stoian, Maltoni, Filliat, Diaz-Rodriguez. 19]

19. [Lesort, Lomonaco, Stoian, Maltoni, Filliat, Diaz-Rodriguez. 19]

20. [Lesort, Lomonaco, Stoian, Maltoni, Filliat, Diaz-Rodriguez. 19]

22. [Lesort, Lomonaco, Stoian, Maltoni, Filliat, Diaz-Rodriguez. 19]

25. ●
○
○

26. ●
○
○
●
○
○

27. ●
○
○
●
○
○
●
○
○

28. [Lesort, Lomonaco, Stoian, Maltoni, Filliat, Diaz-Rodriguez. 19]

29. [Lesort, Lomonaco, Stoian, Maltoni, Filliat, Diaz-Rodriguez. 19]

31. ●
○

32. ●
○
●

34. [Parisi & Ji'19]

35. [Parisi & Ji'19]
●
○

36. ●
○
[Parisi & Ji'19]

37. ●
○
●
[Parisi & Ji'19]

38. CLS Theory [McClelland'95, Parisi'18]

41. Zakharov, Shysheya

45. [Díaz-Rodríguez, Lomonaco et al. Don't forget, there is
more than forgetting: new metrics for Continual Learning.
CL Workshop, NeurIPS 2018].

46. CORe50 Dataset [Lomonaco & Maltoni’18]
Fashion MNIST [Xiao’17]
MNIST Dataset [LeCun & Cortes’98]
[Lesort, Lomonaco, Stoian, Maltoni, Filliat, Diaz-Rodriguez. 19]

47. CORe50 Dataset [Lomonaco & Maltoni’18]
[Lesort, Lomonaco, Stoian, Maltoni, Filliat, Diaz-Rodriguez. 19]
NICO (Non-I.I.D. Image dataset with
Contexts) [He'19]

48. CRLMaze [Lomonaco’19], ViZDoom

49. S-RL Toolbox: Environments, Datasets and Evaluation Metrics for State Representation Learning
[Raffin’18] https://arxiv.org/abs/1809.09369
CRLMaze [Lomonaco’19], ViZDoom
[Lesort, Lomonaco, Stoian, Maltoni, Filliat, Diaz-Rodriguez. 19]

52. [Díaz-Rodríguez, Lomonaco et al. Don't forget, there is
more than forgetting: new metrics for Continual Learning.
CL Workshop, NeurIPS 2018].
[Díaz-Rodríguez, Lomonaco et al. Don't forget, there is
more than forgetting: new metrics for Continual Learning.
CL Workshop, NeurIPS 2018].

53. [Lopez-Paz & Ranzato’17]
[Díaz-Rodríguez, Lomonaco et al. Don't forget, there is more than forgetting:
new metrics for Continual Learning. CL Workshop, NeurIPS 2018].
[Lesort, Lomonaco, Stoian, Maltoni, Filliat, Diaz-Rodriguez. 19]

54. [Lopez-Paz & Ranzato’17]
[Díaz-Rodríguez, Lomonaco et al. Don't forget, there is more than forgetting:
new metrics for Continual Learning. CL Workshop, NeurIPS 2018].
[Lesort, Lomonaco, Stoian, Maltoni, Filliat, Diaz-Rodriguez. 19]

55. [Díaz-Rodríguez, Lomonaco et al. Don't forget, there is
more than forgetting: new metrics for Continual Learning.
CL Workshop, NeurIPS 2018].

56. [Díaz-Rodríguez, Lomonaco et al. Don't forget, there is
more than forgetting: new metrics for Continual Learning.
CL Workshop, NeurIPS 2018].

57. [Díaz-Rodríguez, Lomonaco et al. Don't forget, there is
more than forgetting: new metrics for Continual Learning.
CL Workshop, NeurIPS 2018].

58. [Díaz-Rodríguez, Lomonaco et al. Don't forget, there is
more than forgetting: new metrics for Continual Learning.
CL Workshop, NeurIPS 2018].

59. [Lesort, Lomonaco, Stoian, Maltoni, Filliat, Diaz-Rodriguez. 19]

60. [Lesort, Lomonaco, Stoian, Maltoni, Filliat, Diaz-Rodriguez. 19]

62. [Lesort, Lomonaco, Stoian, Maltoni, Filliat, Diaz-Rodriguez. 19]

64. ●
○
○
○
●
○
[Lesort’18, State representation learning for control: An overview. Neural Networks]
DREAM EU H2020 Project: www.robotsthatdream.eu

65. Continual Reinforcement Learning deployed in Real-life using Policy Distillation and Sim2Real Transfer. [Traoré’19]
State representation learning for control: An overview. Neural Networks. [Lesort’18]

66. Continual Reinforcement Learning deployed in Real-life using Policy Distillation and Sim2Real Transfer. [Traoré’19]
State representation learning for control: An overview. Neural Networks. [Lesort’18]

67. [Lesort’18, State representation learning for control: An overview. Neural Networks]

68. [Lesort’18, State representation learning for control: An overview. Neural Networks]

70. ●
○
○
[Lesort’18, State representation learning for control: An overview. Neural
Networks] DREAM EU H2020 Project: www.robotsthatdream.eu

73. ●

74. ●
●

75. ●
●

77. ●
●
●
Continual Reinforcement Learning deployed in Real-life using Policy Distillation and Sim2Real Transfer. [Traoré’19]

80. 80

81. 81
S-RL Toolbox

82. 82
S-RL Toolbox

83. 83

84. 84

85. 85
S-RL Toolbox

88. [Díaz-Rodríguez, Lomonaco et al. Don't forget, there is
more than forgetting: new metrics for Continual Learning.
CL Workshop, NeurIPS 2018].

90. 90

91. 91
POSTER: Continual Reinforcement Learning deployed in Real-life using Policy Distillation and Sim2Real Transfer. [Traoré et. al.’19]
CODE: https://github.com/kalifou/robotics-rl-srl/tree/circular_movement_omnibot
S-RL Toolbox: RL & SRL for Robotics https://github.com/araffin/robotics-rl-srl

92. 92
APPENDIX

97. ●
●
○
○
[Greco et al.'19]

100. [Continual Reinforcement Learning deployed in Real-life using Policy Distillation and Sim2Real Transfer. Traoré et al. 2019]

103. [Pfülb and Gepperth’19]

104. MNIST Dataset [LeCun & Cortes’98]
Fashion MNIST [Xiao’17]

105. https://gym.openai.com/envs/#classic_control
Octopus arm [Engel’06, Munk’16]
Ball in cup

106. https://www.youtube.com/watch?v=qNsHMkIsqJc&feature=youtu.be

107. ●
●
●
Continual Reinforcement Learning deployed in Real-life using Policy Distillation and Sim2Real Transfer. [Traoré’19]

108. [Lesort’18, State representation learning for control: An overview. Neural Networks]
DREAM EU H2020 Project: www.robotsthatdream.eu

109. 109
Omni-robot sim2real replay