Comparative Analysis of IPOG Strategy Variants for Enhanced Combinatorial T-way Testing Efficiency
DOI:
https://doi.org/10.56919/usci.2433.016Keywords:
One-parameter-at-a-time approachx, IPOG strategy, Combinatorial interaction t-way testing, Test case, Test suite, Harmony search algorithmAbstract
Study’s Excerpt/Novelty
- This study introduces a variant of the IPOG strategy, termed the enhancing IPOG strategy for uniform interaction testing (eIPOG), addressing the persistent challenge of combinatorial explosion in software testing.
- It uniquely integrates the harmony search algorithm to optimize t-way test suites, aiming to minimize test suite size while maximizing coverage.
- Through rigorous experimentation and statistical analysis on benchmarking configurations, the study demonstrates that eIPOG offers competitive results compared to existing IPOG-based strategies, highlighting its potential to significantly improve the efficiency and effectiveness of combinatorial interaction testing.
Full Abstract
Enhancing software quality and precision requires thorough testing of various configuration aspects. Combinatorial interaction testing has emerged as a potent method, utilizing strategies like Generalized Input Parameter Order (IPOG), which employs a one-parameter-at-a-time (OPAT) approach. However, the challenge of combinatorial explosion, where the number of inputs grows exponentially with test cases, remains significant. To address this, numerous strategies have been proposed, yet consistently generating optimal tests covering each t-way interaction efficiently remains elusive. This paper introduces a novel variant of the IPOG strategy, termed the enhancing IPOG strategy for uniform interaction testing (eIPOG). We investigate the application of the harmony search algorithm in introducing optimal combinatorial interaction t-way test suites through eIPOG. Evaluating the behavior of such strategies involves assessing their effectiveness in minimizing the test suite size while maximizing coverage. Through experiments on established benchmarking configurations and statistical analysis, we compare eIPOG with other IPOG-based strategies. Both approaches show promise, highlighting the efficacy of the OPAT approach, with eIPOG demonstrating particularly competitive results in nearly every case.
References
Alazzawi, A. K., Rais, H. M., Basri, S., Alsariera, Y. A., Imam, A. A., Abed, S. A., Balogun, A. O., & Kumar, G. (2022). Recent t-way Test Generation Strategies Based on Optimization Algorithms: An Orchestrated Survey. International Conference on Artificial Intelligence for Smart Community: AISC 2020, 17–18 December, Universiti Teknologi Petronas, Malaysia, 1055–1060.
Alsariera, Y. A., & Zamli, K. Z. (2015). A Bat-inspired strategy for t-way interaction testing. Advanced Science Letters, 21(7). https://doi.org/10.1166/asl.2015.6316
Alsewari, A., Mu’aza, A. A., Rassem, T. H., Tairan, N. M., Shah, H., & Zamli, K. Z. (2018). One-Parameter-at-a-Time Combinatorial Testing Strategy Based on Harmony Search Algorithm OPAT-HS. Advanced Science Letters, 24(10). https://doi.org/10.1166/asl.2018.12927
Alsewari, A. R. A., Poston, R., Zamli, K. Z., Balfaqih, M., & Aloufi, K. S. (2020). Combinatorial test list generation based on Harmony Search Algorithm. Journal of Ambient Intelligence and Humanized Computing. https://doi.org/10.1007/s12652-020-01696-7
Aminu Muazu, A., & Maiwada, U. D. (2020). PWiseHA: Application of Harmony Search Algorithm for Test Suites Generation using Pairwise Techniques. International Journal of Computer and Information Technology, (2279–0764), 9(4). https://doi.org/10.24203/ijcit.v9i4.23
Aminu Muazu, A., Sobri Hashim, A., & Sarlan, A. (2022). Application and Adjustment of “don’t care” Values in t-way Testing Techniques for Generating an Optimal Test Suite. Journal of Advances in Information Technology, 13(4), 347–357. https://doi.org/10.12720/jait.13.4.347-357
Aminu Muazu, A., Sobri Hashim, A., Sarlan, A., & Abdullahi, M. (2023). SCIPOG: Seeding and constraint support in IPOG strategy for combinatorial t-way testing to generate optimum test cases. Journal of King Saud University - Computer and Information Sciences, 35(1), 185–201. https://doi.org/10.1016/J.JKSUCI.2022.11.010
Chen, J., Chen, J., Cai, S., Chen, H., Zhang, C., & Huang, C. (2021). A Test Case Generation Method of Combinatorial Testing based on t-way Testing with Adaptive Random Testing. Proceedings - 2021 IEEE International Symposium on Software Reliability Engineering Workshops, ISSREW 2021. https://doi.org/10.1109/ISSREW53611.2021.00048
Fadhil, H. M., Abdullah, M. N., & Younis, M. I. (2022). Combinatorial Testing Approaches: A Systematic Review. IRAQI JOURNAL OF COMPUTERS, COMMUNICATIONS, CONTROL AND SYSTEMS ENGINEERING, 22(4), 60–79. https://doi.org/https://doi.org/10.33103/uot.ijccce.22.4.6
Fadhil, H. M., Abdullah, M. N., & Younis, M. I. (2023). Innovations in t-way test creation based on a hybrid hill climbing-greedy algorithm. IAES International Journal of Artificial Intelligence, 12(2), 794–805. https://doi.org/10.11591/ijai.v12.i2.pp794-805
Geem, Z. W., Kim, J. H., & Loganathan, G. V. (2001). A New Heuristic Optimization Algorithm: Harmony Search. Simulation, 76(2). https://doi.org/10.1177/003754970107600201
Hassan, A. A., Abdullah, S., Zamli, K. Z., & Razali, R. (2020). Combinatorial test suites generation strategy utilizing the whale optimization algorithm. IEEE Access, 8. https://doi.org/10.1109/ACCESS.2020.3032851
Khaleel, S. I., & Anan, R. (2023). A review paper: optimal test cases for regression testing using artificial intelligent techniques. International Journal of Electrical and Computer Engineering, 13(2). https://doi.org/10.11591/ijece.v13i2.pp1803-1816
Lei, Y., Kacker, R., Kuhn, D. R., Okun, V., & Lawrence, J. (2007). IPOG: A General Strategy for T-Way Software Testing. 14th Annual IEEE International Conference and Workshops on the Engineering of Computer-Based Systems (ECBS’07).
Lei, Y., Kacker, R., Kuhn, D. R., Okun, V., & Lawrence, J. (2008). IPOG-IPOG-D: Efficient test generation for multi-way combinatorial testing. Software Testing Verification and Reliability, 18(3), 125–148. https://doi.org/10.1002/stvr.381
Muazu, A. A., Hashim, A. S., Maiwada, U. D., Isma’ila, U. A., Yakubu, M. M., & Ibrahim, M. A. (2024). Pairwise test case generation with harmony search, one-parameter-at-at-time, seeding, and constraint mechanism integration. International Journal of Electrical and Computer Engineering, 14(3), 3137–3149. https://doi.org/10.11591/ijece.v14i3.pp3137-3149
Muazu, A. A., Hashim, A. S., Maiwada, U. D., & Muppidi, A. (2023). Enhanced Version of Seeding and Constraint support in IPOG strategy for Variable Strength Interaction T-way Testing. Malaysian Journal of Computer Science, 36(4), 381–403. https://doi.org/10.22452/mjcs.vol36no4.3
Muazu, A. A., Hashim, A. S., & Sarlan, A. (2022). Review of Nature Inspired Metaheuristic Algorithm Selection for Combinatorial t-Way Testing. IEEE Access, 10, 27404–27431. https://doi.org/10.1109/ACCESS.2022.3157400
Muazu, A. A., Hashim, A. S., Sarlan, A., & Maiwada, U. D. (2022). Proposed Method of Seeding and Constraint in One-Parameter-At-a- Time Approach for t-way Testing. 2022 International Conference on Digital Transformation and Intelligence (ICDI), 39–45. https://doi.org/10.1109/ICDI57181.2022.10007210
Nasser, A. B., Alsewari, A. A., Aminu Muazu, A., & Zamli, K. Z. (2016). Comparative Performance Analysis of Flower Pollination Algorithm and Harmony Search based strategies: A Case Study of Applying Interaction Testing in the Real World. 2nd International Conference on New Directions in Multidisciplinary Research & Practice (NDMRP), 3, 51–51. www.globalilluminators.org
Othman, R. R., & Zamli, K. Z. (2011). ITTDG: Integrated T-way test data generation strategy for interaction testing. Scientific Research and Essays, 6(17), 3638–3648. https://doi.org/10.5897/sre10.1196
Ramli, N., Othman, R. R., Abdul Khalib, Z. I., & Jusoh, M. (2017). A Review on Recent T-way Combinatorial Testing Strategy. MATEC Web of Conferences, 140. https://doi.org/10.1051/matecconf/201714001016
Soh, Z. H. C., Abdullah, S. A. C., & Zamli, K. Z. (2013). A Distributed T-Way Test Suite Generation Using “One-Parameter-at-a-Time” Approach. In Int. J. Advance Soft Compu. Appl (Vol. 5, Issue 3).
Wang Ziyuan, Nie Changhai, & Xu Baowen. (2007). Generating Combinatorial Test Suite for Interaction Relationship. Fourth International Workshop on Software Quality Assurance : In Conjunction with the 6th ESEC/FSE Joint Meeting , 3(4), 115.
Younis, M. (2020). GAMIPOG: A Deterministic Genetic Multi-Parameter-Order Strategy for the Generation of Variable Strength Covering Arrays. In Journal of Engineering Science and Technology (Vol. 15, Issue 5). https://www.researchgate.net/publication/344599430
Younis, M. I., & Zamli, K. Z. (2010). MC-MIPOG: A parallel t-way test generation strategy for multicore systems. ETRI Journal, 32(1), 73–83. https://doi.org/10.4218/etrij.10.0109.0266
Younis, M. I., Zamli, K. Z., & Ashidi Mat Isa, N. (2008). MIPOG-Modification of the IPOG Strategy for T-Way Software Testing. https://www.researchgate.net/publication/228858408
Younis, M. I., Zamli, K. Z., & Isa, N. A. M. (2008). A strategy for grid based T-Way test data generation. 1st International Conference on Distributed Frameworks and Application, DFmA 2008, 73–78. https://doi.org/10.1109/ICDFMA.2008.4784416
Yu, L., Lei, Y., Kacker, R. N., & Kuhn, D. R. (2013). ACTS: A combinatorial test generation tool. Proceedings - IEEE 6th International Conference on Software Testing, Verification and Validation, ICST 2013, 370–375. https://doi.org/10.1109/ICST.2013.52
Zamli, K. Z., Alkazemi, B. Y., & Kendall, G. (2016). A Tabu Search hyper-heuristic strategy for t-way test suite generation. Applied Soft Computing Journal, 44, 57–74. https://doi.org/10.1016/j.asoc.2016.03.021
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 UMYU Scientifica
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
&
