The paper proposes the Efficient Deep-Learning Gradients Extraction Network (Edge-Net), which combines depthwise separable convolutions and deformable convolutional networks to enhance edge detection efficiency while maintaining state-of-the-art accuracy with only 500k parameters. By utilizing network pruning techniques and selecting appropriate components, Edge-Net addresses the inefficiencies of large CNNs commonly used for edge detection. Experimental results show that Edge-Net outperforms existing lightweight edge detectors on the BSDS500 and NYUDV2 datasets.