This paper proposes contractive auto-encoders as a method for feature extraction that explicitly enforces invariance to small perturbations of the input. Contractive auto-encoders train the model parameters to produce representations that are robust to noise by adding a regularization term to the standard auto-encoder objective function that penalizes the squared Frobenius norm of the Jacobian of the hidden unit activations with respect to the input. This forces nearby datapoints to be mapped to similar representations, improving generalization to new examples.

1. © SuperDataScienceDeep Learning A-Z

2. © SuperDataScienceDeep Learning A-Z
1
2
3
4

3. © SuperDataScienceDeep Learning A-Z
Contractive Auto-Encoders: Explicit
Invariance During Feature Extraction
By Salah Rifai et al. (2011)
Link:
http://machinelearning.wustl.edu/mlpapers/paper_files/ICML2011Rifai_455.pdf
Additional Reading: