Existing filtering algorithms use all pixels within a window to filter out the impulse noise. They increase the size of neighboring pixels with the increase of noise density. In this paper, we propose an impulse noise removal algorithm for remote sensing images, that emphasis on few noise-free pixels. The detection map (DM) is constructed from the input noisy image, by assigning a binary value 1 for each corrupted pixel in the input image. By using the detection map, the proposed iterative algorithm searches the noise free pixels with in a small neighborhood. The noisy pixel is then replaced with the median value estimated from noise free pixels. In-order to better appraise the noise cancellation behavior of our filter from the point of view of human perception, we perform segmentation via spline regression on remote sensing image for both noisy image and filtered image. Experimental results show that the filtering performance of the proposed approach is very satisfactory providing better feature extraction in remote sensing images.