Navigation

Home

Musa's Book


Resources


SRE Orientation

Training Services

Consulting Services

Contact

Click Here for A Special Announcement!

User Experiences with Software Reliability Engineering



This is a list of published articles and papers by practitioners who have applied software reliability engineering in their projects and described their experiences with it. I have tried to exclude research papers in software reliability engineering, as they can be found with normal bibliographic sources. I have also excluded articles and papers that analyze software reliability engineering data after the fact and those where no practitioners were involved. Thus this list is intended for practitioners who want to understand what it is like to practice software reliability engineering in various settings. Please note that, unfortunately, practitioners are often too busy to write up their experiences and do not consider publication a priority; consequently, it is likely that software reliability engineering has been applied many more times than this list would indicate.

· Alam, M., W. Chen, W. Ehrlich, M. Engel, D. Kropfl, P. Verma. 1997. Assessing software reliability performance under highly critical but infrequent event occurrences. Proceedings 8th International Symposium on Software Reliability Engineering, Albuquerque, NM, November 1997, pp. 294-307.

· Arora, S., Misra, R.B., and Kumre, V.M. 2005. “Software reliability improvement through operational profile driven testing,” Proceedings Reliability and Maintainability Symposium, pp. 621 – 627.

· Beck, A. 1998. "ESSI process improvement experiment 23843 - USST usage specification and statistical testing." Proceedings 8th International Symposium on Software Reliability Engineering: Case Studies, Albuquerque, NM, November 1997, pp. 95-100.

· Bennett, J., Denoncourt, M., and Healy, J. D. 1992. "Software Reliability Prediction for Telecommunication Systems," Proc. 2nd Bellcore/Purdue Symposium on Issues in Software Reliability Estimation, Oct. 1992, pp. 85-102.

· Bentz, R. W. and C. D. Smith. 1996. Experience report for the software reliability program on a military system acquisition and development. Proceedings 7th International Symposium on Software Reliability Engineering - Industrial Track, White Plains NY, October 30-November 2, 1996, pp. 59-65.

· Bergen, L. A. 1989. "A Practical Application of Software Reliability to a Large Scale Switching System," IEEE International Workshop: Measurement of Quality During the Life Cycle, Val David, Quebec, Canada, April 25-27, 1989.

· Binder, R. 2003. “Experience with a profile-based automated testing environment. Proceedings 14th International Symposium on Software Reliability Engineering (Supplemental), Denver, CO, November 17 – 20, pp. 71-82.

· Binder, R. V. 2004. Automated testing with an operational profile. DOD Software Tech News (December): 7-10.

· Carman, D. W., Dolinsky, A. A., Lyu, M. R., and Yu, J. S. 1995. "Software Reliability Engineering Study of a Large-Scale Telecommunications Software System," Proc. 1995 International Symposium on Software Reliability Engineering, Toulouse, France, Oct. 1995, pp. 350-.

· Carnes, P. 1997. "Software reliability in weapon systems." Proceedings 8th International Symposium on Software Reliability Engineering: Case Studies, Albuquerque, NM, November 1997, pp. 95-100.

· Carnes, P. 1998. "Software reliability in weapon systems." Proceedings 9th International Symposium on Software Reliability Engineering: Industrial Practices, Paderborn, Germany, November 1998, pp.272-279.

· Carnes, P. 2001. Software reliability – theory vs practice in the operational environment. Proceedings 12th International Symposium on Software Reliability Engineering. Hong Kong, Nov. 27-30, 2001, pp. 72-82.

· Christenson, D. A. 1988. "Using Software Reliability Models to Predict Field Failure Rates in Electronic Switching Systems," Proc. 4th Annual National Joint Conference on Software Quality and Productivity, Washington, DC.

· Chruscielski, K. and J. Tian. 1997. An operational profile for the cartridge support software. Proceedings 8th International Symposium on Software Reliability Engineering, Albuquerque, NM, November 1997, pp. 203-212.

· Cramp, R., Vouk, M. A., and Jones, W. 1992. "On Operational Availability of a Large Software-Based Telecommunications System," Proc. 3rd International Symposium on Software Reliability Engineering, Research Triangle Park, NC, Oct. 7-10, 1992, pp. 358-366.

· Cusick, J. and M. Fine. 1997. Guiding reengineering with the operational profile. Proceedings 8th International Symposium on Software Reliability Engineering: Case Studies, Albuquerque, NM, November 1997, pp. 15-25.

· Derriennic, H. and G. Le Gall. 1995. Use of failure-intensity models in the software-validation phase for telecommunications. IEEE Transactions on Reliability 44(4), pp. 658-665.

· Dixit, P., M. A. Vouk, D. L. Bitzer, and C. Alix. 1996. Reliability and availability of a wide area network-based education system. Proceedings 7th International Symposium on Software Reliability Engineering, White Plains NY, October 30-November 2, 1996, pp. 213-218.

· Dixit, P., M. A. Vouk, and D. L. Bitzer. 1997. Reliability behavior of a large network based education system. Proceedings 8th International Symposium on Software Reliability Engineering: Case Studies, Albuquerque, NM, November 1997, pp. 43-56.

· Donnelly, M., Everett, B., Musa, J., and Wilson, G. 1996. "Best Current Practice of SRE," Lyu, M. R. (ed.), Handbook of Software Reliability Engineering, McGraw-Hill, 1996, pp. 219-254.

· Donovan, D., C. Dislis, R. Murphy, S.Unger, C. Kenneally, J. Young, and L. Sheehan. 2001. Incorporating software reliability engineering into the test process for an extensive GUI-based network management system. Proceedings 12th International Symposium on Software Reliability Engineering. Hong Kong, Nov. 27-30, 2001, pp. 44-53.

· Drake, H. D. and D. E. Wolting. 1987. Reliability theory applied to software testing. Hewlett-Packard Journal 38(4):35-39.

· Ehrlich, W. K., R. Chan, W. J. Donnelly, H. H. Park, M. B. Saltzman, and P Verma. 1996. Validating software architectures for high reliability. Proceedings 7th International Symposium on Software Reliability Engineering, White Plains NY, October 30-November 2, 1996, pp. 196-206.

· Ehrlich, W. K., Lee, K., and Molisani, R. H. 1990. "Applying Reliability Measurements: A Case Study," IEEE Software, March 1990.

· Ehrlich, W. K., Prasanna, B., Stampfel, J. P., and Wu, J. R. 1993. "Determining the Cost of a Stop-Test Decision," IEEE Software, March 1993, pp. 33-42.

· Ehrlich, W. K., Stampfel, J. P., and Wu, J. R. 1990. "Application of Software Reliability Modeling to Product Quality and Test Process," Proc. 12th International Conference on Software Engineering, Nice, France, March 1990.

· Elentukh, A. 1994. "System Reliability Policy at Motorola Codex," Proc. 5th International Symposium on Software Reliability Engineering, Monterey, CA, Nov. 6-9, 1994, pp. 289-293.

· Everett, W. W. and J. M. Gobat. 1996. DQS's experience with SRE. Proceedings 7th International Symposium on Software Reliability Engineering, White Plains NY, October 30-November 2, 1996, pp. 219-224.

· Everett, W.W. and Widmaier, J. 2004. Applications of SRE in the security domain. DOD Software Tech News (December): 11-14.

· Everett, W. W., H. Wilson, R. Johnson, J. Widmaier, and D. Cooper. 2004. “Measuring Reliability of High Integrity Systems,” Proc. 15th. International Symposium on Software Reliability Engineering - Supplement, Saint-Malo, France. November 2-5, 2004, pp 100-110.

· Fuoco, G., Irving, N., Juhlin B., Kropfl, D., and Musa, J. 1996. "The Operational Profile," Lyu, M. R. (ed.), Handbook of Software Reliability Engineering, McGraw-Hill, 1996, pp. 167-216 (includes three project applications).

· Hamilton, P. A. and Musa, J. D. 1978. "Measuring Reliability of Computation Center Software," Proc. 3rd International Conference on Software Engineering, pp. 29-36.

· Hartmann, H., J. Bokkerink, and V. Ronteltap. “How to Reduce Your Test Process With 30 Percent: The Application of Operational Profiles at Philips Medical Systems,” Proceedings 17th International Symposium on Software Reliability Engineering, Raleigh, NC, Nov. 7-10, 2006.

· Hill, S. W. and F. S. Kmetz. 1997. Application of software reliability engineered testing (SRET) to project accounting application (PAA). Proceedings 8th International Symposium on Software Reliability Engineering: Case Studies, Albuquerque, NM, November 1997, pp. 59-68.

· Hudepohl, J. P. Measurement of software service quality for large telecommunications systems. IEEE Journal on Selected Areas in Communications 8(2):210-218.

· Hudepohl, J. P., W. Snipes, T. Hollack, and W. Jones. A methodology to improve switching system software service quality and reliability. Proceedings IEEE Global Communications Conference, pp. 1671-1678.

· Iannino, A and Musa, J. D. 1991. "Software Reliability Engineering at AT&T," Apostolakis, G. (ed.) Probability Safety Assessment and Management - Vol. 1, Elsevier, New York.

· Jalote, P. and B. Murphy. 2004. “Reliability Growth in Software Products,” Proc. 15th. International Symposium on Software Reliability Engineering, Saint-Malo, France. November 2-5, 2004, pp 47-53.

· Jalote, P., B. Murphy, M. Garzia, and B. Errez. 2004. “Measuring Reliability of Products,” Proc. 15th. International Symposium on Software Reliability Engineering - Supplement, Saint-Malo, France. November 2-5, 2004, pp 269-276.

· Jenson, B. D. 1995. "A Software Reliability Engineering Success Story: AT&T's Definity( PBX," Proc. 1995 International Symposium on Software Reliability Engineering, Toulouse, France, Oct. 1995, pp. 338-343.

· Jones, W. D. 1991. "Reliability Models for Very Large Software Systems in Industry," Proc. 1991 International Symposium on Software Reliability Engineering, Austin, TX, May 17-18, 1991, pp. 35-42.

· Jones,W.D. 1998. "A Brief History of SRE in PCN," Proc. 9th Annual SRE Workshop 7/14-15/98, Ottawa, Ontario, Canada.

· Juhlin, B. D. 1992. "Implementing Operational Profiles to Measure System Reliability," Proc. 3rd International Symposium on Software Reliability Engineering, Research Triangle Park, NC, Oct. 7-10, 1992, pp. 286-295.

· Juhlin, B. D. 1992. "Applying Software Reliability Engineering to International PBX Testing," Proc. 9th International Conference on Testing Computer Software, Washington, DC, June 16-18, 1992, pp. 165-176.

· Juhlin, B. D. 1993. "Software Reliability Engineering in the System Test Process," Proc. 10th International Conference on Testing Computer Software, Washington, DC, June 14-17, 1993, pp. 97-115.

· Kaâniche, M. and K. Kanoun. 1996. Reliability of a commercial telecommunications system. Proceedings 7th International Symposium on Software Reliability Engineering, White Plains NY, October 30-November 2, 1996, pp. 207-212.

· Kanoun, K and Sabourin, T. 1987. "Software Dependability of a Telephone Switching System," Proc. 17th IEEE International Symposium on Fault-Tolerant Computing, Pittsburgh, June 1987, pp. 236-241.

· Kanoun, K., Bastos Martini, M., and Moreira de Souza, J. 1991. "A Method for Software Reliability Analysis and Prediction-Application to the TROPICO-R Switching System," IEEE Trans. Software Engineering, April 1991, pp. 334-344.

· Keller, T. and N. Schneidewind. 1997. Successful application of software reliability engineering for the NASA space shuttle. Proceedings 8th International Symposium on Software Reliability Engineering: Case Studies, Albuquerque, NM, November 1997, pp. 71-82.

· Kouchakdjian, A. 1999. Experience using statistical usage testing at the FAA. Proceedings Software Quality Week 1999.

· Kropfl, D. and Ehrlich, W. 1995. "Telecommunications Network Operating Systems: Experiences in Software Reliability Engineering," Proc. 1995 International Symposium on Software Reliability Engineering, Toulouse, France, Oct. 1995, pp. 344-349.

· Kruger, G. A. 1988. Project management using software reliability growth models. Hewlett-Packard Journal 39(6):30-35.

· Kruger, G. A. 1989. Validation and further application of software reliability growth models. Hewlett-Packard Journal 40(4):75-79.

· Lakey, Peter B. 1998. "How does any software organization proceed in incorporating SRE?" (Crusader self-propelled howitzer project) Proc. 9th Annual SRE Workshop 7/14-15/98, Ottawa, Ontario, Canada.

· Lee, I. and R. K. Iyer. 1995. Software dependability in the Tandem GUARDIAN system. IEEE Transactions on Software Engineering 21(5), pp. 455-467.

· Levendel, Y. 1989. "Defects and Reliability Analysis of Large Software Systems: Field Experience," Proc. 19th IEEE International Symposium on Fault-Tolerant Computing, Chicago, June 1989, pp. 238-244.

· Levendel, Y. 1990. "Reliability Analysis of Large Software Systems: Defect Data Modeling," IEEE Trans. Software Engineering, vol. SE-16, no. 2, February 1990, pp. 141-152.

· Levendel, Y. 1995. "The Cost Effectiveness of Telecommunication Service Dependability," Lyu, M. R. (ed.), Software Fault Tolerance, Wiley and Sons, pp. 279-314.

· Martini, M. R., Kanoun, K. and de Souza, J. M. 1990. "Software Reliability Evaluation of the TROPICO-R Switching System, IEEE Trans. Reliability, vol. 33, no. 3, pp. 369-379.

· Mattos, K., M. Zingarelli, and D. Coffey. 2004. “Achieving Reliable, Dependable, and Mature Software in Large-Scale Undersea Combat Systems,” Proc. 15th. International Symposium on Software Reliability Engineering - Supplement, Saint-Malo, France. November 2-5, 2004, pp 89-99.

· Mendiratta, Veena B. 1998. "Reliability Analysis of Clustered Architectures," Proc. 9th Annual SRE Workshop 7/14-15/98, Ottawa, Ontario, Canada.

· Murphy, B. and Garzia, M.R. 2004. Software reliability engineering for mass market products. DOD Software Tech News (December): 14-17.

· Musa, J. D., G. Fuoco, N. Irving, B. Juhlin, and D. Kropfl. 1996. The operational profile. In Handbook of Software Reliability Engineering, ed. M. R. Lyu, McGraw-Hill, 1996, pp. 167-216 (includes three project applications).

· Nikora, A. P. and Lyu, M. R. 1996. "Software Reliability Measurement Experiences," Lyu, M. R. (ed.), Handbook of Software Reliability Engineering, McGraw-Hill, pp. 255-301.

· Oshana, R. and F. P. Coyle. 1997. Improving a system regression test with usage models developed using field collected data. Proceedings Software Quality Week 1997.

· Oshana, R. 1999. An automated testing environment to support operational profiles of software intensive systems. Proceedings Software Quality Week 1999.

· Pemler, S. and Stahl, N. 1994. "An Automated Environment for Software Testing and Reliability Estimation," Proc. 5th International Symposium on Software Reliability Engineering, Monterey, CA, Nov. 6-9, 1994, pp. 312-317.

· Rapp, B. 1990. "Application of Software Reliability Models in Medical Imaging Systems," Proc. 1990 International Symposium on Software Reliability Engineering, Washington, DC, April 1990.

· Sandfoss, R. V. and S. A. Meyer. 1997. Input requirements needed to produce an operational profile for a new telecommunications system. Proceedings 8th International Symposium on Software Reliability Engineering: Case Studies, Albuquerque, NM, November 1997, pp. 29-39.

· Schneidewind, N.F. 2004. Application of SRE to ultrareliable systems – the Space Shuttle. DOD Software Tech News (December):19-23.

· Schneidewind, N. F. and Keller, T. W. 1992. "Application of Reliability Models to the Space Shuttle," IEEE Software, July 1992, pp. 28-33.

· Teresinski, J. A. 1996. Software reliability: getting started. Proceedings 7th International Symposium on Software Reliability Engineering - Industrial Track, White Plains NY, October 30-November 2, 1996, pp. 39-47.

· Tian, J. , P. Lu, and J. Palma. 1995. Test-execution based reliability measurement and modeling for large commercial software. IEEE Transactions on Software Engineering 21(5), pp. 405-414.

· Tierney, J. 1996. Putting aspects of software reliability engineering to use. Proceedings 7th International Symposium on Software Reliability Engineering - Industrial Track, White Plains NY, October 30-November 2, 1996, pp. 89-92.

· Weinberg, T. 1996. SoothSayer: a tool for measuring the reliability of Windows NT services. Proceedings 7th International Symposium on Software Reliability Engineering - Industrial Track, White Plains, NY, October 30-November 2, 1996, pp. 49-56.

· Wildenauer, P. and L. Eisner. 1989. Adaptive model for a software test to calculate a residual error forecast. Computing, vol. 42, pp. 141-158.

· Wills, M.S. 1999. Functional reliability. Proceedings 9th International Conference on Software Quality, Cambridge, Ma, October 4 - 6, 1999, pp. 43-54.

· Wills, M.S. 2001. Functional reliability in rapid application development. Proceedings 11th International Conference on Software Quality, Pittsburgh, PA, October 24, 2001.



If you have written or know of a published article not on this list, please send it to me in the citation format shown. Please send only references to articles written about the actual use by project personnel of software reliability engineering on real projects (data collection and analysis is not sufficient). To qualify as an application of software reliability engineering, the project must have developed and used operational profiles, set and applied failure intensity objectives, or measured failure intensity and used it in managing the project. The reason for imposing these requirements is to limit the list to articles that new users can learn from and apply.
Send E-mail to:
lauriewilliams@gmail.com


JOHN D. MUSA
Software Reliability Engineering and Testing Courses

s