International Journal of Mathematical, Engineering and Management Sciences

ISSN: 2455-7749

Determining Software Time-to-Market and Testing Stop Time when Release Time is a Change-Point

Ompal Singh
Department of Operational Research, University of Delhi, Delhi, India.

Saurabh Panwar
Department of Operational Research, University of Delhi, Delhi, India.

P. K. Kapur
Amity Center for Interdisciplinary Research, Amity University, Noida, Uttar Pradesh, India.


Received on April 04, 2019
Accepted on September 11, 2019


In software engineering literature, numerous software reliability growth models have been designed to evaluate and predict the reliability of the software products and to measure the optimal time-to-market of the software systems. Most existing studies on software release time assessment assumes that when software is released, its testing process is terminated. In practice, however, the testing team releases the software product first and continues the testing process for an added period in the operational phase. Therefore, in this study, a coherent reliability growth model is developed to predict the expected reliability of the software product. The debugging process is considered imperfect as new faults can be introduced into the software during each fault removal. The proposed model assumes that the fault observation rate of the testing team modifies after the software release. The release time of the software is therefore regarded as the change-point. It has been established that the veracity of the performance of the growth models escalates by incorporating the change-point theory. A unified approach is utilized to model the debugging process wherein both testers and users simultaneously identify the faults in the post-release testing phase. A joint optimization problem is formulated based on the two decision criteria: cost and reliability. In order to assimilate the manager’s preferences over these two criteria, a multi-criteria decision-making technique known as multi-attribute utility theory is employed. A numerical illustration is further presented by using actual data sets from the software project to determine the optimal software time-to-market and testing termination time.

Keywords- Field environment, Imperfect debugging, Multi-attribute utility theory (MAUT), Testing termination time, Software reliability.


Singh, O., Panwar, S., & Kapur, P. K. (2020). Determining Software Time-to-Market and Testing Stop Time when Release Time is a Change-Point. International Journal of Mathematical, Engineering and Management Sciences, 5(2), 208-224.

Conflict of Interest

The authors declare that there is no conflict of interest for this publication.


The research work presented in this paper is supported by grants to the first author from DST via DST PURSE phase II, India.


Arora, A., Caulkins, J.P., & Telang, R. (2006). Research note-sell first, fix later: impact of patching on software quality. Management Science, 52(3), 465-471.

Dalal, S.R., & Mallows, C.L. (1988). When should one stop testing software? Journal of the American Statistical Association, 83(403), 872-879.

Goel, A.L., & Okumoto, K. (1979). Time-dependent error-detection rate model for software reliability and other performance measures. IEEE Transactions on Reliability, 28(3), 206-211.

Jiang, Z., Sarkar, S., & Jacob, V.S. (2012). Postrelease testing and software release policy for enterprise-level systems. Information Systems Research, 23(3-part-1), 635-657.

Kapur, P.K., & Garg, R.B. (1992). A software reliability growth model for an error-removal phenomenon. Software Engineering Journal, 7(4), 291-294.

Kapur, P.K., Gupta, A., & Singh, O. (2005). On discrete software reliability growth model & categorization of faults. Opsearch, 42(4), 340-354.

Kapur, P.K., Gupta, A., Yadavalli, V.S.S., & Claasen, S.J. (2006). Flexible software reliability growth models. South African Journal of Industrial Engineering, 17(2), 109-125.

Kapur, P.K., Khatri, S.K., Tickoo, A., & Shatnawi, O. (2014). Release time determination depending on number of test runs using multi attribute utility theory. International Journal of System Assurance Engineering and Management, 5(2), 186-194.

Kapur, P.K., Kumar, S., & Garg, R.B. (1999). Contributions to hardware and software reliability. World Scientific, Singapore.

Kapur, P.K., Panwar, S., Singh, O., & Kumar, V. (2019). Joint release and testing stop time policy with testing-effort and change point. In Risk Based Technologies (pp. 209-222). Springer, Singapore.

Kapur, P.K., Pham, H., Anand, S., & Yadav, K. (2011). A unified approach for developing software reliability growth models in the presence of imperfect debugging and error generation. IEEE Transactions on Reliability, 60(1), 331-340.

Kapur, P.K., Pham, H., Gupta, A., & Jha, P.C. (2011). Software reliability assessment with OR applications. Springer, London.

Kapur, P.K., Singh, J.N., & Singh, O. (2015). Application of multi attribute utility theory in multiple releases of software. International Journal of System Assurance Engineering and Management, 6(1), 61-70.

Kapur, P.K., Singh, O., Garmabaki, A.S., & Singh, J. (2010). Multi up-gradation software reliability growth model with imperfect debugging. International Journal of System Assurance Engineering and Management, 1(4), 299-306.

Kapur, P.K., Singh, V.B., Anand, S., & Yadavalli, V.S.S. (2008). Software reliability growth model with change-point and effort control using a power function of the testing time. International Journal of Production Research, 46(3), 771-787.

Keeney, R.L. (1971). Utility independence and preferences for multi attributed consequences. Operations Research, 19(4), 875-893.

Li, X., Li, Y.F., Xie, M., & Ng, S.H. (2011). Reliability analysis and optimal version-updating for open source software. Information and Software Technology, 53(9), 929-936.

Majumdar, R., Shrivastava, A.K., Kapur, P.K., & Khatri, S.K. (2017). Release and testing stop time of a software using multi-attribute utility theory. Life Cycle Reliability and Safety Engineering, 6(1), 47-55.

Marquardt, D.W. (1963). An algorithm for least-squares estimation of nonlinear parameters. Journal of the Society for Industrial and Applied Mathematics, 11(2), 431-441.

McDaid, K., & Wilson, S.P. (2001). Deciding how long to test software. Journal of the Royal Statistical Society: Series D (The Statistician), 50(2), 117-134.

Ohba, M., & Yamada, S. (1984). S-shaped software reliability growth models. In International Colloquium on Reliability and Maintainability, 4th, Tregastel, France (pp. 430-436).

Okumoto, K., & Goel, A.L. (1980). Optimum release time for software systems based on reliability and cost criteria. Journal of Systems and Software, 1, 315-318.

Pham, H., Nordmann, L., & Zhang, Z. (1999). A general imperfect-software-debugging model with S-shaped fault-detection rate. IEEE Transactions on Reliability, 48(2), 169-175.

Roy, P., Mahapatra, G.S., & Dey, K.N. (2014). An NHPP software reliability growth model with imperfect debugging and error generation. International Journal of Reliability, Quality and Safety Engineering, 21(02), 1450008.

SAS, S. (2004). STAT User guide, Version 9.1.2. SAS Institute Inc, Cary, NC, USA.

Wang, J., Wu, Z., Shu, Y., & Zhang, Z. (2015). An imperfect software debugging model considering log-logistic distribution fault content function. Journal of Systems and Software, 100, 167-181.

Wood, A. (1996). Predicting software reliability. Computer, 29(11), 69-77.

Xie, M., & Yang, B. (2003). A study of the effect of imperfect debugging on software development cost. IEEE Transactions on Software Engineering, 29(5), 471-473.

Yamada, S., & Osaki, S. (1987). Optimal software release policies with simultaneous cost and reliability requirements. European Journal of Operational Research, 31(1), 46-51.

Yamada, S., Ohba, M., & Osaki, S. (1983). S-shaped reliability growth modeling for software error detection. IEEE Transactions on Reliability, 32(5), 475-484.

Zhu, M., & Pham, H. (2018a). A multi-release software reliability modeling for open source software incorporating dependent fault detection process. Annals of Operations Research, 269(1-2), 773-790.

Zhu, M., & Pham, H. (2018b). A two-phase software reliability modeling involving with software fault dependency and imperfect fault removal. Computer Languages, Systems & Structures, 53, 27-42.

Privacy Policy| Terms & Conditions