This document proposes minimizing the curvature of trajectories in ODE-based generative models to improve sampling efficiency. It introduces a training objective that learns the dependence between data and noise to reduce trajectory curvature. Experimental results show this low-curvature approach requires fewer function evaluations and achieves better distillation performance than independent noise models. In conclusion, trajectory curvature is directly related to sampling efficiency in diffusion models, and the proposed method of reducing curvature through learned dependence cuts costs and enhances distillation quality.

1. Minimizing Trajectory Curvature of ODE-
based Generative Models
Sangyun Lee, Beomsu Kim, Jong Chul Ye

2. https://cvpr2022-tutorial-diffusion-models.github.io/

3. Probability Flow ODE
Truncation error

4. Probability Flow ODE
Accurate!
Straight generative path

7. intersection!

8. intersection!

10. Coupling Degree of intersection
Training objective (q)
Training objective (joint)
Upper bound

11. Coupling Degree of intersection
Training objective (q)
Training objective (joint)
Upper bound

12. Coupling Degree of intersection
Training objective (q)
Training objective (joint)
Upper bound

13. Coupling Degree of intersection
Training objective (q)
Training objective (joint)
Upper bound

14. Coupling Degree of intersection
Training objective (q)
Training objective (joint)
Upper bound

15. Coupling Degree of intersection
Training objective (q)
Training objective (joint)
Upper bound

16. NFEs
=
4
NFEs
=
128
Independent Ours

17. Low-curvature generative ODEs make less distillation error.

18. Conclusion
● Trajectory curvature is directly related to sampling efficiency of diffusion
models.
● The trajectory curvature can be reduced by learning dependence between
data and noise.
● Our method reduces sampling costs and improves distillation performance
of diffusion models.
Check out github.com/sangyun884/fast-ode for code and updates.