UPM Institutional Repository

Object detection of autonomous vehicle in complex traffic scenarios based on improved YOLOv8s


Citation

Ruixin, Zhao (2024) Object detection of autonomous vehicle in complex traffic scenarios based on improved YOLOv8s. Doctoral thesis, Universiti Putra Malaysia.

Abstract

With the rapid development of autonomous driving technology, road object detection has become a critical research field, as it is regarded as an essential component of the perception systems in autonomous driving. As deep learning based methods continue to improve in generalization capability and accuracy, they are gradually replacing traditional algorithms and have become the mainstream approach for object detection. However, the traffic scenario object detection algorithms proposed in the literature typically focus on a single field and still struggle to balance detection accuracy and real-time performance, thus limiting their practical applicability. Moreover, many methods fail to adapt to the diverse and complex conditions in real-world traffic, including occlusions, and the presence of small objects (in images, where object sizes are smaller than 32×32 pixels), all of which can significantly reduce detection accuracy. To address these problems, this study proposes two improved algorithms based on You Only Look Once version 8 small (YOLOv8s), namely Transformer Attention Mechanism-YOLOv8s (TAM-YOLOv8s), which significantly improves the detection of occluded objects, and Z-YOLOv8s, which significantly improves the detection of both small and occluded objects. Specifically, TAM-YOLOv8s introduces the Revisiting Perspective Vision Transformer with Cross-stage Partial Bottleneck and Two Convolutions (RepViTC2f) module into the backbone to strengthen global modeling capabilities, and integrates a Large Selective Kernel Network (LSKNet) in the detection head. Experimental results on the Berkeley Deep Drive 100K (BDD100K) dataset demonstrate that TAM-YOLOv8s achieves a mean Average Precision (mAP@0.5) of 70.7%, which is 2.8% higher than the original YOLOv8s, while maintaining a detection speed of 89.24 Frames Per Second (FPS). It also achieves consistent improvements on Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) (92.7%) and Malaysia-Universiti Putra Malaysia (Malaysia-UPM) (77.7%). Building on TAM-YOLOv8s, Z-YOLOv8s further incorporates a Space-to-Depth Convolution (SPD-Conv) module into the backbone and replaces the original Spatial Pyramid Pooling Fast (SPPF) module with Softpool- SPPF, thereby more effectively capturing the features of small objects in complex backgrounds. Consequently, Z-YOLOv8s achieves an mAP@0.5 of 75.2% on BDD100K (an improvement of 7.3% over YOLOv8s) at a detection speed of 78.41 FPS, and outperforms YOLOv8s by 3.8% and 4.8% on the KITTI and Malaysia-UPM datasets, respectively. Overall, TAM-YOLOv8s effectively addresses occlusion issues in complex traffic scenarios, while Z-YOLOv8s performs well in detecting small and occluded objects, achieving a balance between detection accuracy and processing speed. These findings underscore the significant potential of the proposed algorithms for applications in intelligent transportation systems and autonomous driving.


Download File

[img] Text
FK 2024 83 - Full Text.pdf
Available under License Creative Commons Attribution Non-commercial No Derivatives.

Download (12MB)
[img] Text
FK 2024 83.pdf
Restricted to Repository staff only
Available under License Creative Commons Attribution Non-commercial No Derivatives.

Download (13MB)

Additional Metadata

Item Type: Thesis (Doctoral)
Subject: Self-driving cars
Subject: Machine translating
Subject: Deep learning (Machine learning)
Call Number: FK 2024 83
Chairman Supervisor: Associate Professor Tang Sai Hong
Divisions: Faculty of Engineering
Keywords: Road environmental; Object detection; YOLOv8; Deep learning; Autonomous vehicle.
Sustainable Development Goals (SDGs): SDG 9: Industry, Innovation and Infrastructure, SDG 11: Sustainable Cities and Communities, SDG 12: Responsible Consumption and Production
Depositing User: MS. HADIZAH NORDIN
Date Deposited: 08 Jul 2026 03:41
Last Modified: 08 Jul 2026 03:41
URI: http://psasir.upm.edu.my/id/eprint/126942
Statistic Details: View Download Statistic

Actions (login required)

View Item View Item