Aiming at the limitations of the existing single image defogging algorithm in the case of uneven fog distribution and high time cost. In this paper, a fast image defogging algorithm based on encoder-decoder structure is proposed, which integrates multi-scale convolution and feature attention. Firstly, a lightweight encoder-decoder structure is used based on performance and memory storage trade-offs to ensure a low time cost. Secondly, considering that in the scene with uneven fog distribution, the atomized area may exceed the size of the convolution kernel, a multi-scale convolution structure is proposed. In the feature extraction stage, the parallel multi-scale convolution of 1×1, 3×3, 5×5, 7×7 is firstly used to extract features, so as to enlarge the sensitivity field and preserve more details of the input image. In addition, since different spatial domains of images can be affected by different levels of haze, the degradation degree of objects at different scene depths also varies greatly. Therefore, the basic block module of feature attention is introduced to treat the features on different channels and pixels differently. In order to verify the effectiveness of the algorithm, the proposed algorithm is compared with the current popular algorithms in three data sets. In the real NH-HAZE 2 data set with non-uniform haze distribution, the processing time of three evaluation indexes, namely, peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and single image resolution of 1600×1200, are 20.50 dB, 0.84 and 0.0038 seconds, respectively. Compared with DMPHN model , all the three indexes are at a higher level. At the same time, it also has a good performance in O-HAZE data set with uniform haze distribution. Experimental results show that the proposed algorithm can effectively solve the problem of unsatisfactory fog effect in the case of uneven fog distribution, and reduce the time cost. Meanwhile, the restored image has a better performance in color and brightness.