This document discusses an approach for active learning to jointly discover rare classes and classify new instances with minimal supervision. It presents a unified active learning model that adapts query criteria online to discover new categories and learn classifiers. It also introduces a classifier combination algorithm that switches between generative and discriminative classifiers as learning progresses. Evaluation on standard datasets shows this approach outperforms existing methods.