Purpose and Difficulty of Software Testing T-76.5613 Software Testing and Quality Assurance 30.10.2015 Juha Itkonen Department of Computer Science
Is software quality a problem? 2
Famous examples of software quality 1985-1987 Therac-25 medical accelerator A radiation therapy device malfunctions and delivers lethal radiation doses at several medical facilities Bottom line: 5 deaths 1996 Arianne 5 rocket flight crash Conversion of rocket velocity caused an overflow, which resulted in a computer shutdown that and eventually led in destruction of the whole flight The same software was used in Arianne 4, but due to smaller rocket engines such high values that could have caused an overflow were not possible Bottom line: rocket $8 billion, payload $500 million Recent analysis by IEEE Spectrum: Lessons From a Decade of IT Failures http://spectrum.ieee.org/static/lessons-from-a-decade-of-it-failures 3
Is software testing needed, and why? 4
Someone will always test your software Customer or User Competitor A government inspector, an industrial analyst, an insurance inspector A newspaper or TV journalist Channel reseller Systems integrator and find Potential partner, subcontractor, employee the nastiest Software tester bugs 5
Someone? Who Tested My Software? Testing is not an action conducted only by specialized testers. Testing is performed by people in different roles like developers, managers, and consultants. Testing by individuals with customer contact and domain expertise is highly valued validation method. Mäntylä, M. V., Itkonen, J., Iivonen, J., "Who Tested My Software? Testing as an Organizationally Cross-Cutting Activity ", Software Quality Journal, vol. 20, issue 1, January 2012, pp. 145-172. 6
Testing is needed Because all software has faults Why? Impossible to prevent all errors in software development Often it is more effective to test than to write perfect code We need testing to reduce risk Find and fix the most important faults before the software is released to operation Failures that occur during live operation are usually more expensive to deal with than failures that occur during testing Higher the risks, more we need to test Level of confidence No risk no test 7
What is Software Testing? 8
Definition of Software Testing (Glenford Myers 1979) Testing is the execution of programs with the intent of finding defects. 9
Definition of Software Testing (Ilene Burnstein 2002) Testing is a the process of exercising a software component using a selected set of test cases, with the intent of revealing defects and evaluating quality. (Testing can be described as a process used for revealing defects in software, and for establishing that the software has attained a specified degree of quality with respect to selected attributes.) 10
Definition of Software Testing (Cem Kaner, 2004) Software testing is a technical investigation of a product, i.e., an empirical search for quality-related information of value to a project s stakeholders 11
Testing is an integral part of development Often, testing is seen as some separate, last phase of software development process That can be outsourced to separate testing team That only deeds to be done just before the release if there is any time Benefits of independent testing are questionable Testing can not be separated from rest of the software development Testing is much more than the final acceptance or gate keeper phase Testing has to be involved from the beginning Testers can, and should, contribute in each phase of the software development life-cycle Testing is not a phase 12
Lecture Assignment: The Challenges of Software Testing Form groups of 3-4 Study briefly the main messages of the given article, discuss in the group Whittaker, James A. 2000. What is Software Testing? And Why is it so Hard? IEEE Software, vol. 17, no. 1: 70-79. Summarize the key messages of the article and what are your own views on the topic 13
Major Challenges Testers Need to Overcome How to select what is and what is not tested Prioritization How to bring knowledge into testing Domain area knowledge Knowledge of the usage of the system How to manage testing work Assessing progress When you are done How to be a useful part of the development organization What is useful quality information How to effectively communicate and collaborate Timely early and often 14