In this paper, a simple and flexible framework based on the duality of obstacle detection and ground homograph estimation is proposed for obstacle detecting during autonomous navigation. The virtue of which is that the obstacle can be detected and localized without requiring the parameters of the camera. The iterative frame is implemented in our method to avoid the influence of the obstacles when estimating the ground homography. We initially estimate a homograph for the ground using matching points to register the image pairs; then, the two images are differenced so that the homograph disparity image (HDI) can be generated; after that, we segment the HDI to obtain a coarse location of obstacles; and the matching points out of the obstacle regions will be considered as input for the latter loop. This process is iterated until the results converge. The feasibility of this scheme is analyzed, and the experimental results demonstrate it can gain robust results for obstacle detection.