Improved 3D measurement with a novel preprocessing method in DFP
© The Author(s) 2017
Received: 12 October 2017
Accepted: 8 November 2017
Published: 17 November 2017
Shadow and background are two common factors in digital fringe projection, which lead to ambiguity in three-dimensional measurement and thereby need to be seriously considered. Preprocessing is often needed to segment the object from invalid points. The existing segmentation approaches based on modulation normally perform well in pure dark background circumstances, which, however, lose accuracy in situations of white or complex background. In this paper, an accurate shadow and background removal technique is proposed, which segments the shadow by one threshold from modulation histogram and segments the background by the threshold in intensity histogram. Experiments are well designed and conducted to verify the effectiveness and reliability of the proposed method.
Digital fringe projection (DFP) techniques are widely employed in flexible, non-contact and high-speed 3D shape measurement . In a DFP system, a sequence of phase-shifted sinusoidal fringes is often projected on the object by the projector, and the fringes are distorted by the object surface and captured by a camera. Phase map can be retrieved from the deformed fringes, and the object height information is calculated from the phase map in a calibrated DFP system . However, shadow and the background are inevitable, since the projector and camera are arranged from different viewpoints. Invalid points such as shadow and background should be identified and removed from the object.
Researchers made great efforts to remedy the influence of invalid points including the shadow and background. Skydan et al.  utilized multiple projectors to probe the object from different viewpoints to achieve shadow free reconstruction. However, the increased cost of hardware keeps this method from commonly utilized. Zhang  proposed to employ the Gaussian filter on the fringes to remove random noise and identify the invalid points by the monotonicity of the unwrapped phase. However, the Gaussian filter introduces errors to the object details. Chen et al.  applied a threshold to the least-squares fitting errors in temporal phase unwrapping for invalid points detection. However, this method is vulnerable to noise .
Huang and Asundi  proposed a compact framework combining modulation, rms error and monotonicity for shadow and background removal and error detection. Intensity modulation is very effective in measuring how informative are the pixels, and can be used to detect background and shadow. However, manually adjusting the threshold is time-consuming. In practice, the threshold selection is subject to measurement conditions such as the environmental illumination and object surface characteristics. Lu et al.  proposed a technique to remove shadow points by mapping the 3D results into projector coordinates, and the modulation is not needed. However, this method can only detect shadow caused by the DFP system .
In this paper, we apply the multi-thresholding technique on modulation histogram and propose a preprocessing method to detect the valid points of the object by firstly segmenting the shadow using one threshold from the modulation histogram. Secondly, we project one more picture onto the object and reference plane and calculate the intensity difference of the captured images, and the histogram of the difference map is analyzed for the background detection. We call this one more picture the coding map.
The rest of this paper is organized as follows: We introduce the related principles and existing methods in Related work. In “Methods” section, we introduce the details of how to implement our proposed object segmentation technique. In the experiments and results part, we present and compare some segmentation results using our method and the expanded conventional method. The 3D shape reconstruction result is also presented in this section. In the end, we make a summary in “Conclusion”.
N-step phase shifting and modulation
Existing methods of threshold selection
The above two methods for automatic threshold selection are intended for image segment based on gray-level histogram. The literature  utilizes them in modulation histogram for object segmentation. However, in their work, the background is dark, so invalid points in shadow and background are with low modulation level, and the object is with higher modulation level; only one threshold is enough to segment the object. As shown in Fig. 1, Fig. 1a shows a captured fringe on the object with dark background, Fig. 1b shows the modulation map of the captured fringes, and Fig. 1c shows the histogram of the modulation map. The modulation histogram is within two classes, and it is easy to find the threshold t 1, to segment the valid points and invalid points.
In practical, the modulation histogram is not necessarily in two classes, such as when a white board is used as the background for system calibration, as shown in Fig. 1d. Figure 1e shows the modulation map of Fig. 1d, and Fig. 1f shows the histogram of the modulation map. As can be seen that when the background is a white board, the modulation level of the background will be high, and the modulation histogram in Fig. 1f is to be classified to three categories. The background is with middle to high modulation, the object is with medium modulation, and the shadow is with low modulation level. Two thresholds need to be calculated for shadow and the background segmentation separately. For this situation, the conventional method cannot be utilized directly.
Expanded thresholding method
Intensity-based background segmentation
Experiments and results
Shadow and background segmentation
Modulation and intensity thresholds calculated for two objects with different projector defocusing levels
Projector defocusing conditions
After we retrieved the phase map of the object, the height information can be calculated by system calibration . One commonly utilized method calibrates the camera and the projector separately to find the system parameters . This kind of method is easy to understand, because each system parameter has its geometric meaning, but is also time-consuming, and error prone . Because the projector is regarded as an inversed camera, its calibration accuracy depends on the camera calibration process. In this work, we apply the calibration framework presented in  to calculate the height information of the object.
In this paper, we proposed a novel preprocessing method for object segmentation in DFP 3D shape measurement. We firstly applied the multi-threshold Ng’s method on modulation histogram and then proposed our method for shadow and background detection based on modulation and intensity histogram. Experiments verified that our proposed method can improve the 3D shape measurement with white and complex background.
YX built the experiment system, implemented the algorithm, collected and analyzed the data, and wrote the manuscript. YL supervised the main idea and revised the manuscript. Both authors read and approved the final manuscript.
The authors declare that they have no competing interests.
This work was financially supported by the Research Grants Council of Hong Kong (Project No. CityU 11205015), the National Natural Science Foundation of China (Grant No. 61673329) and the Center for Robotics and Automation (CRA) at CityU. The funding body had no direct input on either data collection, experiments design or execution, or the writing of the manuscript.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- Gorthi SS, Rastogi P. Fringe projection techniques: whither we are? Opt Lasers Eng. 2010;48(2):133–40.View ArticleGoogle Scholar
- Guo Q, Xi J, Member S, Song L. Fringe pattern analysis with message passing based expectation maximization for fringe projection profilometry. IEEE Access. 2016;4:4310–20.View ArticleGoogle Scholar
- Skydan OA, Lalor MJ, Burton DR. Using coloured structured light in 3-D surface measurement. Opt Lasers Eng. 2005;43:801–14.View ArticleGoogle Scholar
- Zhang S. Phase unwrapping error reduction framework for a multiple-wavelength phase-shifting algorithm. Opt Eng. 2009;48(10):105601.View ArticleGoogle Scholar
- Chen F, Su X, Xiang L. Analysis and identification of phase error in phase measuring profilometry. Opt Exp. 2010;18(11):11300–7.View ArticleGoogle Scholar
- Huang L, Asundi AK. Phase invalidity identification framework with the temporal phase unwrapping method. Meas Sci Technol. 2011;22(3):35304.View ArticleGoogle Scholar
- Lu L, Xi J, Yu Y, Guo Q, Yin Y, Song L. Shadow removal method for phase-shifting profilometry. Appl Opt. 2015;54(19):6059.View ArticleGoogle Scholar
- Zhang W, Li W, Yan J, Yu L. Adaptive threshold selection for background removal in fringe projection profilometry. Opt Lasers Eng. 2017;90:209–16.View ArticleGoogle Scholar
- Otsu N. A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern. 1979;20(1):62–6.MathSciNetView ArticleGoogle Scholar
- Ng HF. Automatic thresholding for defect detection. Pattern Recognit Lett. 2006;27(14):1644–9.View ArticleGoogle Scholar
- Malacara D. Optical shop testing, vol. 59. New York: Wiley; 2007.View ArticleGoogle Scholar
- Su X, Chen W. Reliability-guided phase unwrapping algorithm: a review. Opt Lasers Eng. 2004;42(3):245–61.View ArticleGoogle Scholar
- Gdeisat M, Burton D, Lilley F, Arevalillo-Herráez M. Fast fringe pattern phase demodulation using FIR Hilbert transformers. Opt Commun. 2016;359:200–6.View ArticleGoogle Scholar
- Xiao Y, Li Y. High-quality binary fringe generation via joint optimization on intensity and phase. Opt Lasers Eng. 2017;90:19–26.View ArticleGoogle Scholar
- Vo M, Wang Z, Hoang T, Nguyen D. Flexible calibration technique for fringe-projection-based three-dimensional imaging. Opt Lett. 2010;35(15):3192–4.View ArticleGoogle Scholar
- Li Z, et al. Accurate calibration method for a structured light system. Opt Eng. 2008;47(5):053604. http://dx.doi.org/10.1117/1.2931517 View ArticleGoogle Scholar
- Zhang X, Zhu L. Projector calibration from the camera image point of view. Opt Eng. 2009;48(11):117208. http://dx.doi.org/10.1117/1.3265551 View ArticleGoogle Scholar
- Huang L, Chua P, Asundi A. Least-squares calibration method for fringe projection profilometry considering camera lens distortion. Appl Opt. 2010;49(9):1539–48.View ArticleGoogle Scholar
- Wang Z, Nguyen D, Barnes J. Some practical considerations in fringe projection profilometry. Opt Lasers Eng. 2010;48(2):218–25.View ArticleGoogle Scholar