A ranking-based approach for effort-aware software defect prediction.

Authors

  • Dr Muhammad Ramzan Software Engineering Department, University Of Sargodha
  • Areeba Zarnab
  • Muhammad Summair Raza
  • Mahwish Ilyas

Keywords:

Effort-aware defect prediction, Random-forest regression, Software quality assurance, Particle Swarm Optimization

Abstract

Predicting software defects is a critical aspect of software quality assurance, as early identification of potential faults enables better resource allocation and ensures high-quality software production. Effort-aware defect prediction enhances testing and maintenance by prioritizing software modules that maximize defect detection while minimizing inspection effort. Traditional methods typically predict defect probability or defect density and rank modules accordingly, often optimizing metrics such as the Proportion of Found Bugs at 20% LOC (PofB@20%) using linear regression models. In this study, we proposed a novel ranking approach that directly predicts the total number of defects per software module using a Random Forest Regressor. Each module is assigned a custom score that balances two objectives: selecting modules with more predicted defects and minimizing effort, measured as lines of code. Particle Swarm Optimization (PSO) is employed to optimize the scoring function with respect to effort-aware metrics PofB@20%, Initial False Alarms (IFA), and Popt, ensuring early defect detection and near-ideal module ranking. After the initial ranking, a defect-aware re-ranking strategy adjusts the top 20% of LOC modules by replacing them with better candidates from the remaining modules, provided that doing so improves defect coverage without exceeding the LOC budget. Experimental results demonstrate that the proposed approach outperforms baseline methods, achieving a higher PofB@20% (0.402), a lower IFA (5.1), and a better Popt (0.756). The findings indicate that ranking modules based on predicted defects and inspection effort effectively helps testers detect more faults with reduced effort, confirming the superiority of the PSO-optimized ranking methodology over traditional approaches.

References

X. Yu et al., “Finding the best learning to rank algorithms for effort-aware defect prediction,” Information and Software Technology, vol. 157, p. 107165, May 2023, doi: 10.1016/j.infsof.2023.107165.

M. I. Hossain, “Software Development Life Cycle (SDLC) Methodologies for Information Systems Project Management,” vol. 5, no. 5, 2023.

“Software Quality Assurance - Software Engineering - GeeksforGeeks.” Accessed: Jan. 28, 2025. [Online]. Available: https://www.geeksforgeeks.org/software-engineering-software-quality-assurance/

A. Bertolino, “Software Testing Research: Achievements, Challenges, Dreams,” in Future of Software Engineering (FOSE '07), Minneapolis, MN, USA: IEEE, May 2007, pp. 85–103. doi: 10.1109/FOSE.2007.25.

T. Zimmermann, N. Nagappan, H. Gall, E. Giger, and B. Murphy, “Cross-project defect prediction: a large scale experiment on data vs. domain vs. process,” in Proceedings of the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering, Amsterdam The Netherlands: ACM, Aug. 2009, pp. 91–100. doi: 10.1145/1595696.1595713.

T. Mende and R. Koschke, “Effort-Aware Defect Prediction Models,” in 2010 14th European Conference on Software Maintenance and Reengineering, Madrid: IEEE, Mar. 2010, pp. 107–116. doi: 10.1109/CSMR.2010.18.

Y. Kamei et al., “A large-scale empirical study of just-in-time quality assurance,” IEEE Trans. Software Eng., vol. 39, no. 6, pp. 757–773, Jun. 2013, doi: 10.1109/TSE.2012.70.

X. Yu et al., “Improving effort-aware defect prediction by directly learning to rank software modules,” Information and Software Technology, vol. 165, p. 107250, Jan. 2024, doi: 10.1016/j.infsof.2023.107250.

Q. Huang, X. Xia, and D. Lo, “Revisiting supervised and unsupervised models for effort-aware just-in-time defect prediction,” Empir Software Eng, vol. 24, no. 5, pp. 2823–2862, Oct. 2019, doi: 10.1007/s10664-018-9661-2.

C. Ni, X. Xia, D. Lo, X. Chen, and Q. Gu, “Revisiting Supervised and Unsupervised Methods for Effort-Aware Cross-Project Defect Prediction,” IEEE Trans. Software Eng., vol. 48, no. 3, pp. 786–802, Mar. 2022, doi: 10.1109/TSE.2020.3001739.

P. Yang et al., “Bug numbers matter: An empirical study of effort-aware defect prediction using class labels versus bug numbers,” Software: Practice and Experience, vol. 55, no. 1, pp. 49–78, 2025, doi: 10.1002/spe.. 3363.

S. S. Reddy and S. Pabboju, “Enhancement of Defect Prediction Using Regression Learning Method for Quality Software Development,” International Journal of Intelligent Engineering & Systems, vol. 17, no. 6, p. 424, Nov. 2024, doi: 10.22266/ijies2024.1231.33.

J. Çarka, M. Esposito, and D. Falessi, “On effort-aware metrics for defect prediction,” Empir Software Eng, vol. 27, no. 6, p. 152, Nov. 2022, doi: 10.1007/s10664-022-10186-7.

Y. Kamei et al., “A large-scale empirical study of just-in-time quality assurance,” IEEE Trans. Software Eng., vol. 39, no. 6, pp. 757–773, Jun. 2013, doi: 10.1109/TSE.2012.70.

X. Yu, L. Liu, L. Zhu, J. W. Keung, Z. Wang, and F. Li, “A multi-objective effort-aware defect prediction approach based on NSGA-II,” Applied Soft Computing, vol. 149, p. 110941, 2023.

X. Du et al., “CoreBug: Improving Effort-Aware Bug Prediction in Software Systems Using Generalized k-Core Decomposition in Class Dependency Networks,” Axioms, vol. 11, no. 5, p. 205, Apr. 2022, doi: 10.3390/axioms11050205.

R. Malhotra, R. Kapoor, P. Saxena, and P. Sharma, “SAGA: A Hybrid Technique to handle Imbalanced Data in Software Defect Prediction,” in 2021 IEEE 11th IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE), Apr. 2021, pp. 331–336. doi: 10.1109/ISCAIE51753.2021.9431842.

W. Li, W. Zhang, X. Jia, and Z. Huang, “Effort-aware semi-supervised just-in-time defect prediction,” Information and Software Technology, vol. 126, p. 106364, 2020.

X. Yang, H. Yu, G. Fan, and K. Yang, “DEJIT: A Differential Evolution Algorithm for Effort-Aware Just-in-Time Software Defect Prediction,” Int. J. Soft. Eng. Knowl. Eng., vol. 31, no. 03, pp. 289–310, Mar. 2021, doi: 10.1142/s0218194021500108.

X. Yu et al., “Finding the best learning to rank algorithms for effort-aware defect prediction,” Information and Software Technology, vol. 157, p. 107165, May 2023, doi: 10.1016/j.infsof.2023.107165.

F. Li, W. Lu, J. W. Keung, X. Yu, L. Gong, and J. Li, “The impact of feature selection techniques on effort?aware defect prediction: An empirical study,” IET Software, vol. 17, no. 2, pp. 168–193, Apr. 2023, doi: 10.1049/sfw2.12099.

F. Li, P. Yang, J. W. Keung, W. Hu, H. Luo, and X. Yu, “Revisiting ‘revisiting supervised methods for effort-aware cross-project defect prediction,’” IET Software, vol. 17, no. 4, pp. 472–495, Aug. 2023, doi: 10.1049/sfw2.12133.

Y. Guo, M. Shepperd, and N. Li, “Improving classifier-based effort-aware software defect prediction by reducing ranking errors,” May 13, 2024, arXiv: arXiv:2405.07604. doi: 10.48550/arXiv.2405.07604.

A. Boloori, A. Zamanifar, and A. Farhadi, “Enhancing software defect prediction models using metaheuristics with a learning to rank approach,” Discov Data, vol. 2, no. 1, p. 11, Nov. 2024, doi: 10.1007/s44248-024-00016-0.

Y. Zhou et al., “How Far We Have Progressed in the Journey? An Examination of Cross-Project Defect Prediction,” ACM Transactions on Software Engineering and Methodology, vol. 27, no. 1, pp. 1–51, Jan. 2018, doi: https://doi.org/10.1145/3183339.

S. Fatima, Z. Fatima, M.A. Hayat, M.H. Shahab, M.K. Meraj, R.M. Ibrahim, and S.M Muneeb, “Impact of Software Metrics on Software Quality using McCall Quality Model: In-Depth Analysis”, 2022.

Downloads

Published

2025-12-21 — Updated on 2025-12-26

Versions

How to Cite

Ramzan, D. M., Zarnab, A. ., Raza, M. S. ., & Ilyas, M. (2025). A ranking-based approach for effort-aware software defect prediction. University of Sindh Journal of Information and Communication Technology, 9(1), 50–64. Retrieved from https://sujo.usindh.edu.pk/index.php/USJICT/article/view/7787 (Original work published December 21, 2025)

Most read articles by the same author(s)

Obs.: This plugin requires at least one statistics/report plugin to be enabled. If your statistics plugins provide more than one metric then please also select a main metric on the admin's site settings page and/or on the journal manager's settings pages.