Document Type : Other
Authors
Department of engineering, Ferdowsi University of Mashhad, Mashhad, Iran.
10.22044/jadm.2026.17502.2890
Abstract
Spectrum-based fault localization (SBFL) is a widely used technique that utilizes coverage data and test outcomes to calculate a suspiciousness score for each program statement. The fundamental hypothesis of SBFL is that a statement covered by more failed test cases and fewer passed test cases is more likely to be faulty. However, the effectiveness of SBFL is hindered by coincidental correctness, which occurs when a fault is executed but no failure is detected. Additionally, traditional SBFL methods assign equal weight to all failed tests, despite some failed tests containing more valuable information. This study aims to enhance SBFL performance by employing a fuzzy expert system to address these challenges. Thirteen open-source subject programs were used to evaluate the efficiency of the proposed FSBFL method. Experimental results, assessed using four key metrics, demonstrate that FSBFL outperforms popular spectrum-based fault localization techniques.
Keywords
Main Subjects
[1] Y. Liu, M. Li, Y. Wu, and Z. Li, "A weighted fuzzy classification approach to identify and manipulate coincidental correct test cases for fault localization," Journal of Systems and Software, vol. 151, pp. 20-37, 2019.
[2] J. S. Collofello and L. Cousins, "Towards automatic software fault location through decision-to-decision path analysis," in Managing Requirements Knowledge, International Workshop on, 1987: IEEE Computer Society, pp. 539-539.
[3] W. E. Wong, R. Gao, Y. Li, R. Abreu, and F. Wotawa, "A survey on software fault localization," IEEE Transactions on Software Engineering, vol. 42, no. 8, pp. 707-740, 2016.
[4] R. Abreu, P. Zoeteweij, and A. J. Van Gemund, "An evaluation of similarity coefficients for software fault localization," in 2006 12th Pacific Rim International Symposium on Dependable Computing (PRDC'06), 2006: IEEE, pp. 39-46.
[5] J. A. Jones, M. J. Harrold, and J. T. Stasko, "Visualization for fault localization," in in Proceedings of ICSE 2001 Workshop on Software Visualization, 2001.
[6] R. Abreu, P. Zoeteweij, R. Golsteijn, and A. J. Van Gemund, "A practical evaluation of spectrum-based fault localization," Journal of Systems and Software, vol. 82, no. 11, pp. 1780-1792, 2009.
[7] W. E. Wong, V. Debroy, R. Gao, and Y. Li, "The DStar method for effective software fault localization," IEEE Transactions on Reliability, vol. 63, no. 1, pp. 290-308, 2013.
[8] L. Naish, H. J. Lee, and K. Ramamohanarao, "A model for spectra-based software diagnosis," ACM Transactions on software engineering and methodology (TOSEM), vol. 20, no. 3, pp. 1-32, 2011.
[9] Y. Li and C. Liu, "Using cluster analysis to identify coincidental correctness in fault localization," in 2012 Fourth International Conference on Computational and Information Sciences, 2012: IEEE, pp. 357-360.
[10] X. Xue, Y. Pang, and A. S. Namin, "Trimming test suites with coincidentally correct test cases for enhancing fault localizations," in 2014 IEEE 38th Annual Computer Software and Applications Conference, 2014: IEEE, pp. 239-244.
[11] F. Feyzi and S. Parsa, "Kernel-based detection of coincidentally correct test cases to improve fault localisation effectiveness," International Journal of Applied Pattern Recognition, vol. 5, no. 2, pp. 119-136, 2018.
[12] Y. Miao, Z. Chen, S. Li, Z. Zhao, and Y. Zhou, "Identifying Coincidental Correctness for Fault Localization by Clustering Test Cases," in SEKE, 2012, pp. 267-272.
[13] A. Sabbaghi, M. R. Keyvanpour, and S. Parsa, "FCCI: A fuzzy expert system for identifying coincidental correct test cases," Journal of Systems and Software, vol. 168, p. 110635, 2020.
[14] Z. Li, M. Li, Y. Liu, and J. Geng, "Identify coincidental correct test cases based on fuzzy classification," in 2016 International Conference on Software Analysis, Testing and Evolution (SATE), 2016: IEEE, pp. 72-77.
[15] L. Naish, H. J. Lee, and K. Ramamohanarao, "Spectral debugging with weights and incremental ranking," in 2009 16th Asia-Pacific Software Engineering Conference, 2009: IEEE, pp. 168-175.
[16] B. Baudry, F. Fleurey, and Y. Le Traon, "Improving test suites for efficient fault localization," in Proceedings of the 28th international conference on Software engineering, 2006, pp. 82-91.
[17] S. Artzi, J. Dolby, F. Tip, and M. Pistoia, "Directed test generation for effective fault localization," in Proceedings of the 19th international symposium on Software testing and analysis, 2010, pp. 49-60.
[18] S. Pearson et al., "Evaluating and improving fault localization," in 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE), 2017: IEEE, pp. 609-620.
[19] J. M. Voas, "PIE: A dynamic failure-based technique," IEEE Transactions on software Engineering, vol. 18, no. 8, p. 717, 1992.
[20] W. Masri and R. A. Assi, "Prevalence of coincidental correctness and mitigation of its impact on fault localization," ACM transactions on software engineering and methodology (TOSEM), vol. 23, no. 1, pp. 1-28, 2014.
[21] R. Abou Assi, C. Trad, M. Maalouf, and W. Masri, "Coincidental correctness in the Defects4J benchmark," Software Testing, Verification and Reliability, vol. 29, no. 3, p. e1696, 2019.
[22] W. Masri, R. Abou-Assi, M. El-Ghali, and N. Al-Fatairi, "An empirical study of the factors that reduce the effectiveness of coverage-based fault localization," in Proceedings of the 2nd International Workshop on Defects in Large Software Systems: held in conjunction with the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2009), 2009, pp. 1-5.
[23] Y. Miao, Z. Chen, S. Li, Z. Zhao, and Y. Zhou, "A clustering-based strategy to identify coincidental correctness in fault localization," International Journal of Software Engineering and Knowledge Engineering, vol. 23, no. 05, pp. 721-741, 2013.
[24] Y. Li and C. Liu, "Identifying coincidental correctness in fault localization via cluster analysis," Journal of Software Engineering, vol. 8, no. 4, pp. 328-344, 2014.
[25] X. Yang, M. Liu, M. Cao, L. Zhao, and L. Wang, "Regression identification of coincidental correctness via weighted clustering," in 2015 IEEE 39th Annual Computer Software and Applications Conference, 2015, vol. 2: IEEE, pp. 115-120.
[26] L. A. Zadeh, "Fuzzy sets," Information and control, vol. 8, no. 3, pp. 338-353, 1965.
[27] T. J. Ross, Fuzzy logic with engineering applications. John Wiley & Sons, 2005.
[28] L.-X. Wang, "A course in fuzzy systems," 1999.
[29] I. Iancu, "A Mamdani type fuzzy logic controller," Fuzzy logic-controls, concepts, theories and applications, vol. 15, no. 2, pp. 325-350, 2012.
[30] E. H. Mamdani and S. Assilian, "An experiment in linguistic synthesis with a fuzzy logic controller," International journal of man-machine studies, vol. 7, no. 1, pp. 1-13, 1975.
[31] T. Takagi and M. Sugeno, "Fuzzy identification of systems and its applications to modeling and control," IEEE transactions on systems, man, and cybernetics, no. 1, pp. 116-132, 1985.
[32] P. C. Shill, K. K. Pal, M. F. Amin, and K. Murase, "Genetic algorithm based fully automated and adaptive fuzzy logic controller," in 2011 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2011), 2011: IEEE, pp. 1572-1579.
[33] P. R. Srivastava, S. Kumar, A. P. Singh, and G. Raghurama, "Software testing effort: An assessment through fuzzy criteria approach," Journal of Uncertain Systems, vol. 5, no. 3, pp. 183-201, 2011.
[34] C.-T. Lin, W.-Y. Chen, and J. Intasara, "A framework for improving fault localization effectiveness based on fuzzy expert system," IEEE Access, vol. 9, pp. 82577-82596, 2021.
[35] V. Debroy and W. E. Wong, "A consensus‐based strategy to improve the quality of fault localization," Software: Practice and Experience, vol. 43, no. 8, pp. 989-1011, 2013.
[36] M. Hutchins, H. Foster, T. Goradia, and T. Ostrand, "Experiments on the effectiveness of dataflow-and control-flow-based test adequacy criteria," in Proceedings of 16th International conference on Software engineering, 1994: IEEE, pp. 191-200.
[37] R. Just, D. Jalali, and M. D. Ernst, "Defects4J: A database of existing faults to enable controlled testing studies for Java programs," in Proceedings of the 2014 international symposium on software testing and analysis, 2014, pp. 437-440.
[38] F. Feyzi and S. Parsa, "Inforence: effective fault localization based on information-theoretic analysis and statistical causal inference," Frontiers of Computer Science, vol. 13, no. 4, pp. 735-759, 2019.
[39] J. A. Jones and M. J. Harrold, "Empirical evaluation of the tarantula automatic fault-localization technique," in Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering, 2005, pp. 273-282.
[40] W. E. Wong, V. Debroy, and D. Xu, "Towards better fault localization: A crosstab-based statistical approach," IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 42, no. 3, pp. 378-396, 2011.
[41] X. Ju, S. Jiang, X. Chen, X. Wang, Y. Zhang, and H. Cao, "HSFal: Effective fault localization using hybrid spectrum of full slices and execution slices," Journal of Systems and Software, vol. 90, pp. 3-17, 2014.
[42] W. E. Wong, V. Debroy, and B. Choi, "A family of code coverage-based heuristics for effective fault localization," Journal of Systems and Software, vol. 83, no. 2, pp. 188-208, 2010.
[43] W. E. Wong, V. Debroy, R. Golden, X. Xu, and B. Thuraisingham, "Effective software fault localization using an RBF neural network," IEEE Transactions on Reliability, vol. 61, no. 1, pp. 149-169, 2011.
[44] W. E. Wong and Y. Qi, "BP neural network-based effective fault localization," International Journal of Software Engineering and Knowledge Engineering, vol. 19, no. 04, pp. 573-597, 2009.
[45] J. Sohn and S. Yoo, "Fluccs: Using code and change metrics to improve fault localization," in Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, 2017, pp. 273-283.
[46] F. Steimann, M. Frenkel, and R. Abreu, "Threats to the validity and value of empirical assessments of the accuracy of coverage-based fault locators," in Proceedings of the 2013 International Symposium on Software Testing and Analysis, 2013, pp. 314-324.
[47] P. S. Kochhar, X. Xia, D. Lo, and S. Li, "Practitioners' expectations on automated fault localization," in Proceedings of the 25th international symposium on software testing and analysis, 2016, pp. 165-176.
[48] F. Tip, "A survey of program slicing techniques." Journal of Programming Languages, vol. 3, pp. 121-189, 1994.
[49] E. Soremekun, L. Kirschner, M. Böhme, and A. Zeller, "Locating faults with program slicing: an empirical analysis," Empirical Software Engineering, vol. 26, no. 3, p. 51, 2021.
[50] J. Zhou, H. Zhang, and D. Lo, "Where should the bugs be fixed? more accurate information retrieval-based bug localization based on bug reports," in 2012 34th International conference on software engineering (ICSE), 2012: IEEE, pp. 14-24.
[51] A. H. Damia, M. Esnaashari, and M. R. Parvizimosaed, "Software Testing using an Adaptive Genetic Algorithm," Journal of AI and Data Mining, vol. 9, no. 4, pp. 465-474, 2021
)