This document discusses an automatic methodology for indexing lecture videos to classify semantic fragments and enable context-based retrieval for e-learning materials. It presents a temporal state model for classifying lectures into states like introduction, definitions, theory, etc. based on extracted low-level audio, video and text features. An experiment applied the method to 26 videos and tested contextual searching and personalized retrieval based on a user's viewing purpose like first viewing or exam preparation.