This document presents an adaptive model switching technique for variable fidelity optimization using population-based algorithms. The technique aims to provide reliable high-fidelity optimum designs with reasonable computational expense by leveraging multiple models of varying fidelity. It switches models by comparing the error distribution of the current model to the distribution of recent fitness function improvements over the population. The method was tested on airfoil and cantilever beam design problems, showing substantially better balance of optimum quality and efficiency than purely low- or high-fidelity optimizations.