The influence of nonuniform microstructure, uneven roughness and small thickness change of thermal barrier coating on terahertz waves is highly overlapping and reduce the accuracy of thickness measurement. To address this, a thickness measurement method based on a residual cross-domain network driven by an abnormal dominant loss function is proposed to minimize prediction outliers. First, an analytical model of terahertz signals is developed, demonstrating that two key parameters for thickness measurement—time of flight and refractive index—can be derived from time-domain and frequency-domain data. A Fast Fourier Transform layer is introduced to extract frequency-domain features, while a gated recurrent layer captures changes in the time-domain. Additionally, a division layer is designed based on transmission rules, enabling the construction of an interpretable cross-domain residual network. Given the overlapping effects of roughness and microstructural changes on terahertz signal peaks, an abnormal dominant loss function is established to assign greater weight to outliers. Finally, thermal barrier coating samples were prepared, and terahertz experiments were conducted. The results demonstrate that the proposed method achieves a maximum relative thickness measurement error of less than 2.5%.