Optimal decisions on software release and post-release testing: A unified approach
DOI:
https://doi.org/10.2298/YJOR200218001KKeywords:
change-point, field-testing phase, imperfect debugging, multi-attribute utility theory, optimal software scheduling, software reliability assessment, testing effort, testing durationAbstract
In this research, a novel approach is developed where a testing team delivers the software product first and extends the testing process for additional time in the user environment. During the operational phase, users also participate in the fault detection process and notify the defects to the software. In this study, a reliability growth model is proposed using a unified approach based on the expenditure of efforts during the testing process. Besides, debugging process is considered imperfect as new faults may enter the software during each fault removal. The developed model further considers that the developer’s rate of defect identification changes with a software release. Thus, the software time-to-market acts as a change-point for the failure observation phenomenon. It is asserted that the accuracy of a software reliability estimation improves by implementing the concept of change-point. The main aim of the paper is to evaluate the optimal release time and testing termination time based on two attributes, particularly, reliability, and cost. A multi-attribute utility theory (MAUT) is applied to find a trade-off between the two conflicting attributes. Finally, a numerical example is presented by using the historical fault count data. The behavior of two decision variables is measured and compared with the existing release time strategy.References
Arora, A., Caulkins, J. P., Telang, R. (2006). "Research note—Sell first, fix later: Impact of patching on software quality", Management Science, 52(3), 465–471.
Cao, P., Yang, K., Liu, K. (2019). "Optimal selection and release problem in software testing process: a continuous time stochastic control approach", European Journal of Operational Research, 285(1), 211-222.
Goel, A. L., Okumoto, K. (1979). "Time-dependent error-detection rate model for software reliability and other performance measures", IEEE Transactions on Reliability, 28(3), 206-211.
Huang, C. Y. (2005). "Cost-reliability-optimal release policy for software reliability models incorporating improvements in testing efficiency", Journal of Systems and Software, 77(2), 139-155.
Huang, C. Y., Lyu, M. R. (2005). "Optimal release time for software systems considering cost, testing-effort, and test efficiency", IEEE Transactions on Reliability, 54(4), 583-591.
Jiang, Z., Sarkar, S., Jacob, V. S. (2012). "Postrelease testing and software release policy for enterprise-level systems", Information Systems Research, 23(3), 635-657.
Kapur, P. K., Garg, R. B. (1991). "Optimal release policies for software systems with testing effort", International Journal of Systems Science, 22(9), 1563-1571.
Kapur, P. K., Garg, R. B. (1992). "A software reliability growth model for an error-removal phenomenon", Software Engineering Journal, 7(4), 291-294.
Kapur, P. K., Goswami, D. N., Bardhan, A., Singh, O. (2008). "Flexible software reliability growth model with testing effort dependent learning process", Applied Mathematical Modelling, 32(7), 1298-1307.
Kapur, P. K., Khatri, S. K., Tickoo, A., Shatnawi, O. (2014). "Release time determination depending on number of test runs using multi attribute utility theory", International Journal of System Assurance Engineering and Management, 5(2), 186-194.
Kapur, P. K., Kumar, S., Garg, R. B. (1999). "Contributions to hardware and software reliability", World Scientific Publishing Company, Singapore.
Kapur, P. K., Panwar, S., Kumar, V., Singh, O. (2020). "Entropy-based two-dimensional software reliability growth modeling for open-source software incorporating change point", International Journal of Reliability, Quality and Safety Engineering, 27(05), 2040009.
Kapur, P. K., Panwar, S., Singh, O., Kumar, V. (2019a). "Joint optimization of software time to market and testing duration using multi-attribute utility theory", Annals of Operations Research. https://doi.org/10.1007/s10479-019-03483-w
Kapur, P. K., Panwar, S., Singh, O., Kumar, V. (2019b). "Joint release and testing stop time policy with testing effort and change point", In Risk Based Technologies, Springer, Singapore, 209-222.
Kapur, P. K., Pham, H., Gupta, A., Jha, P. C. (2011). "Software reliability assessment with OR applications", London, UK, Springer.
Keeney, R. L. (1971). "Utility independence and preferences for multi attributed consequences", Operations Research, 19(4), 875-893.
Kumar, V., Mathur, P., Sahni, R., Anand, M. (2016a). "Two-dimensional multi-release software reliability modeling for fault detection and fault correction processes", International Journal of Reliability, Quality and Safety Engineering, 23(03), 1640002.
Kumar, V., Sahni, R., Shrivastava, A. K. (2016b). "Two-dimensional multi-release software modelling with testing effort, time and two types of imperfect debugging", International Journal of Reliability and Safety, 10(4), 368-388.
Kumar, V., Singh, V. B., Dhamija, A., Srivastav, S. (2018). "Cost-reliability optimal release time of software with patching considered", International Journal of Reliability, Quality and Safety Engineering, 25(04), 1850018.
Li, Q., Pham, H. (2017). "NHPP software reliability model considering the uncertainty of operating environments with imperfect debugging and testing coverage", Applied Mathematical Modelling, 51, 68-85.
Minamino, Y., Sakaguchi, S., Inoue, S., Yamada, S. (2019). "Two-Dimensional NHPP Models Based on Several Testing-Time Functions and Their Applications", International Journal of Reliability, Quality and Safety Engineering, 26(04), 1950018.
Nagaraju, V., Fiondella, L., Wandji, T. (2019). "A heterogeneous single changepoint software reliability growth model framework", Software Testing, Verification and Reliability, 29(8), e1717.
Ohba, M., Yamada, S. (1984). "S-shaped software reliability growth models", In International Colloquium on Reliability and Maintainability, 4th, Tregastel, France, 430-436.
Okumoto, K. (2011). "Customer-perceived software reliability: Measurement, prediction, application", Keynote talk at the 22nd IEEE International Symposium on Software Reliability Engineering (ISSRE 2011), Tokyo, Japan.
Okumoto, K., Goel, A. L. (1980). "Optimum release time for software systems based on reliability and cost criteria", Journal of Systems and Software, 1(4), 315-318.
Panwar, S., Kapur, P. K., Singh, O. (2019). "Modeling technological substitution by incorporating dynamic adoption rate", International Journal of Innovation and Technology Management, 16(01), 1950010.
Peng, R., Li, Y. F., Zhang, J. G., Li, X. (2015). "A risk-reduction approach for optimal software release time determination with the delay incurred cost", International Journal of Systems Science, 46(9), 1628-1637.
Pham, H., Nordmann, L., Zhang, Z. (1999). "A general imperfect-software-debugging model with S-shaped fault-detection rate", IEEE Transactions on Reliability, 48(2), 169-175.
SAS, S. (2004). "STAT User guide, Version 9.1.2", SAS Institute Inc, Cary, NC, USA.
Singh, O., Panwar, S., Kapur, P. K. (2020). "Determining software time-to-market and testing stop time when release time is a change-point", International Journal of Mathematical, Engineering and Management Sciences, 5(2), 208-224.
Wang, J., Wu, Z., Shu, Y., Zhang, Z. (2015). "An imperfect software debugging model considering log-logistic distribution fault content function", Journal of Systems and Software, 100, 167-181.
Yamada, S. (2014). "Software reliability modeling: fundamentals and applications", Tokyo: Springer.
Yamada, S., Osaki, S. (1987). "Optimal software release policies with simultaneous cost and reliability requirements", European Journal of Operational Research, 31(1), 46-51.
Yamada, S., Ohba, M., Osaki, S. (1983). "S-shaped reliability growth modeling for software error detection", IEEE Transactions on Reliability, 32(5), 475-484.
Zhao, J., Liu, H. W., Cui, G., Yang, X. Z. (2006). "Software reliability growth model with change-point and environmental function", Journal of Systems and Software, 79(11), 1578-1587.
Zhao, M. (2003). "Statistical reliability change-point estimation models", In Handbook of Reliability Engineering, Springer, London, 157-163.
Downloads
Published
Issue
Section
License
Copyright (c) 2021 YUJOR
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
