Robust depth map refining using color image

Abstract

Depth maps are essential for various applications, providing spatial information about object arrangement in a scene. They play a crucial role in fields such as computer vision, robotics, augmented and virtual reality, autonomous systems, and medical imaging. However, generating accurate, high-quality depth maps is challenging due to issues like texture-copying artifacts, edge leakage, and depth edge distortion. This study introduces a novel method for refining depth maps by integrating information from color images, combining structural and statistical techniques for superior results. The proposed approach employs a structural method to calculate affinities within a regularization framework, utilizing minimum spanning trees (MST) and minimum spanning forests (MSF). Super-pixel segmentation is used to prevent MST construction across depth edges, addressing edge-leaking artifacts while preserving details. An edge inconsistency measurement model further reduces texture-copying artifacts. Additionally, an adaptive regularization window dynamically adjusts its bandwidth based on local depth variations, enabling effective handling of noise and maintaining sharp depth edges. Experimental evaluations across multiple datasets show the method's robustness and accuracy. It consistently achieves the lowest mean absolute deviation (MAD) compared to existing techniques across various upsampling factors, including 2×, 4×, 8×, and 16×. Visual assessments confirm its ability to produce depth maps free of texture-copying artifacts and blurred edges, yielding results closest to ground truth. Computational efficiency is ensured through a divide-and-conquer algorithm for spanning tree computations, reducing complexity while maintaining precision. This research underscores the importance of combining structural and statistical information in depth map refinement. By overcoming the limitations of existing methods, the proposed approach provides a practical solution for improving depth maps in applications requiring high precision and efficiency, such as robotics, virtual reality, and autonomous systems. Future work will focus on real-time applications and integration with advanced depth-sensing technologies.