Aiming at the problem that the current pedestrian detection algorithm is not high in speed and accuracy due to the particularity and complexity of pedestrian targets, an improved pedestrian detection algorithm based on FCOS is proposed. First of all, in order to improve the accuracy of the algorithm, the efficient network VoVNet is used to replace ResNet for feature extraction. Meanwhile, the input-output residual connection is added on VoVNet to enhance the deep feature expression. Secondly, eSE attention mechanism is added to the last feature layer of the network to improve the ability of feature extraction. Finally, in the loss function, GIOU Loss is used as the regression branch loss function to solve the problem that IOU Loss can not reflect the coincidence degree of prediction box and real box. The experimental results show that compared with the existing algorithms, the improved FCOS algorithm improves the map by 9.5%, and the speed also meets the real-time requirements.