This document describes a system for recognizing easily confused traditional Chinese medicine (TCM) herbs using convolutional neural networks (CNNs) on smartphones. The system was trained on a dataset of 2400 herb images from 24 categories collected using smartphone cameras. A hierarchical clustering CNN approach achieved higher accuracy than a standard CNN, with accuracy over 97% for most herb categories. Testing the system on different smartphone models found varying accuracy rates due to differences in camera quality. Data augmentation techniques like image rotation and brightness adjustment were also found to improve classification accuracy over using a single phone's dataset.