This document describes a method called Inference by Learning (IbyL) that speeds up inference in Markov random fields (MRFs) while maintaining solution accuracy. IbyL builds a coarse-to-fine hierarchy of MRFs and learns pruning classifiers at each level to iteratively reduce the solution space. It exploits the piecewise smooth structure of MAP solutions. On stereo matching and optical flow problems, IbyL achieves significant speedups over direct optimization while agreeing highly with the best solutions and maintaining low energy.