Software testing: Difference between revisions

Revision as of 06:13, 3 April 2025

Revision as of 20:33, 1 May 2025

Line 9:

Software testing can provide objective, independent information about the [[Quality (business)|quality]] of software and the [[risk]] of its failure to a [[User (computing)|user]] or sponsor.<ref name="Kaner 1">{{Cite conference |last=Kaner |first=Cem |author-link=Cem Kaner |date=November 17, 2006 |title=Exploratory Testing |url=https://kaner.com/pdfs/ETatQAI.pdf |conference=Quality Assurance Institute Worldwide Annual Software Testing Conference |___location=Orlando, FL |access-date=November 22, 2014}}</ref>

Software testing can determine the [[Correctness (computer science)|correctness]] of software for specific [[Scenario (computing)|scenarios]] but cannot determine correctness for all scenarios.<ref name="pan">{{Cite web |last=Pan |first=Jiantao |date=Spring 1999 |title=Software Testing |url=httphttps://www.ece.cmu.edu/~koopman/des_s99/sw_testing/ |access-date=November 21, 2017 |publisher=Carnegie Mellon University |type=coursework}}</ref><ref name="Kaner2">{{Cite book |last1=Kaner |first1=Cem |title=Testing Computer Software |last2=Falk |first2=Jack |last3=Nguyen |first3=Hung Quoc |publisher=John Wiley and Sons |year=1999 |isbn=978-0-471-35846-6 |edition=2nd |___location=New York |author-link=Cem Kaner}}</ref> It cannot find all [[software bug|bugs]].

Based on the criteria for measuring correctness from an [[test oracle|oracle]], software testing employs principles and mechanisms that might recognize a problem. Examples of oracles include [[specification]]s, [[Design by Contract|contracts]],<ref>{{Cite conference |last1=Leitner |first1=Andreas |last2=Ciupa |first2=Ilinca |last3=Oriol |first3=Manuel |last4=Meyer |first4=Bertrand |author-link4=Bertrand Meyer |last5=Fiva |first5=Arno |date=September 2007 |title=Contract Driven Development = Test Driven Development – Writing Test Cases |url=httphttps://se.inf.ethz.ch/people/leitner/publications/cdd_leitner_esec_fse_2007.pdf |conference=ESEC/FSE'07: European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering 2007 |___location=Dubrovnik, Croatia |access-date=December 8, 2017}}</ref> comparable products, past versions of the same product, inferences about intended or expected purpose, user or customer expectations, relevant standards, and applicable laws.

Software testing is often dynamic in nature; running the software to verify actual output matches expected. It can also be static in nature; reviewing [[source code|code]] and its associated [[documentation]].

Line 115:

* [[Static testing]] methods

Code coverage tools can evaluate the completeness of a test suite that was created with any method, including black-box testing. This allows the software team to examine parts of a system that are rarely tested and ensures that the most important [[function points]] have been tested.<ref name="CornettCode96">{{Cite web |last=Cornett |first=Steve |date=c. 1996 |title=Code Coverage Analysis |url=httphttps://www.bullseye.com/coverage.html#intro |access-date=November 21, 2017 |publisher=Bullseye Testing Technology |at=Introduction}}</ref> Code coverage as a [[software metric]] can be reported as a percentage for:<ref name="LimayeSoftware09" /><ref name="CornettCode96" /><ref name="BlackPragmatic11">{{Cite book |last=Black, R. |url=https://books.google.com/books?id=n-bTHNW97kYC&pg=PA44 |title=Pragmatic Software Testing: Becoming an Effective and Efficient Test Professional |publisher=John Wiley & Sons |year=2011 |isbn=978-1-118-07938-6 |pages=44–6}}</ref>

:* ''Function coverage'', which reports on functions executed

Line 129:

Black-box testing (also known as functional testing) describes designing test cases without knowledge of the implementation, without reading the source code. The testers are only aware of what the software is supposed to do, not how it does it.<ref name="Patton">{{Cite book |last=Patton |first=Ron |url=https://archive.org/details/softwaretesting0000patt |title=Software Testing |publisher=Sams Publishing |year=2005 |isbn=978-0-672-32798-8 |edition=2nd |___location=Indianapolis}}</ref> Black-box testing methods include: [[equivalence partitioning]], [[boundary value analysis]], [[all-pairs testing]], [[state transition table]]s, [[decision table]] testing, [[fuzz testing]], [[model-based testing]], [[use case]] testing, [[exploratory testing]], and specification-based testing.<ref name="LimayeSoftware09" /><ref name="SalehSoftware09" /><ref name="BlackPragmatic11" />

Specification-based testing aims to test the functionality of software according to the applicable requirements.<ref>{{Cite thesis |last=Laycock |first=Gilbert T. |title=The Theory and Practice of Specification Based Software Testing |degree=dissertation |publisher=Department of Computer Science, [[University of Sheffield]] |url=httphttps://www.cs.le.ac.uk/people/glaycock/thesis.pdf |year=1993 |access-date=January 2, 2018}}</ref> This level of testing usually requires thorough [[test case]]s to be provided to the tester, who then can simply verify that for a given input, the output value (or behavior), either "is" or "is not" the same as the expected value specified in the test case. Test cases are built around specifications and requirements, i.e., what the application is supposed to do. It uses external descriptions of the software, including specifications, requirements, and designs, to derive test cases. These tests can be [[functional testing|functional]] or [[non-functional testing|non-functional]], though usually functional. Specification-based testing may be necessary to assure correct functionality, but it is insufficient to guard against complex or high-risk situations.<ref>{{Cite journal |last=Bach |first=James |author-link=James Bach |date=June 1999 |title=Risk and Requirements-Based Testing |url=httphttps://www.satisfice.com/articles/requirements_based_testing.pdf |journal=Computer |volume=32 |issue=6 |pages=113–114 |access-date=August 19, 2008}}</ref>

Black box testing can be used to any level of testing although usually not at the unit level.<ref name="AmmannIntro16" />

Line 139:

===== Visual testing =====

The aim of visual testing is to provide developers with the ability to examine what was happening at the point of software failure by presenting the data in such a way that the developer can easily find the information he or she requires, and the information is expressed clearly.<ref>{{Cite thesis |last=Lönnberg |first=Jan |title=Visual testing of software |date=October 7, 2003 |degree=MSc |publisher=Helsinki University of Technology |url=httphttps://www.cs.hut.fi/~jlonnber/VisualTesting.pdf |access-date=January 13, 2012}}</ref><ref>{{Cite magazine |last=Chima |first=Raspal |title=Visual testing |url=http://www.testmagazine.co.uk/2011/04/visual-testing |magazine=TEST Magazine |archive-url=https://web.archive.org/web/20120724162657/http://www.testmagazine.co.uk/2011/04/visual-testing/ |archive-date=July 24, 2012 |access-date=January 13, 2012}}</ref>

At the core of visual testing is the idea that showing someone a problem (or a test failure), rather than just describing it, greatly increases clarity and understanding. Visual testing, therefore, requires the recording of the entire test process – capturing everything that occurs on the test system in video format. Output videos are supplemented by real-time tester input via picture-in-a-picture webcam and audio commentary from microphones.

Line 209:

{{See also|Software release life cycle#Beta}}

Beta testing comes after alpha testing and can be considered a form of external [[user acceptance testing]]. Versions of the software, known as [[beta version]]s, are released to a limited audience outside of the programming team known as beta testers. The software is released to groups of people so that further testing can ensure the product has few faults or [[computer bug|bug]]s. Beta versions can be made available to the open public to increase the [[Feedback#In organizations|feedback]] field to a maximal number of future users and to deliver value earlier, for an extended or even indefinite period of time ([[perpetual beta]]).<ref>{{Cite web |last=O'Reilly |first=Tim |date=September 30, 2005 |title=What is Web 2.0 |url=httphttps://www.oreilly.com/pub/a/web2/archive/what-is-web-20.html?page=4 |access-date=January 11, 2018 |publisher=O'Reilly Media |at=Section 4. End of the Software Release Cycle}}</ref>

=== Functional vs non-functional testing ===

Line 382:

{{main|Verification and validation (software)|Software quality control}}

Software testing is used in association with [[Verification and validation (software)|verification and validation]]:<ref name="tran">{{Cite web |last=Tran |first=Eushiuan |year=1999 |title=Verification/Validation/Certification |url=httphttps://www.ece.cmu.edu/~koopman/des_s99/verification/index.html |access-date=August 13, 2008 |publisher=Carnegie Mellon University |type=coursework}}</ref>

* Verification: Have we built the software right? (i.e., does it implement the requirements).

Line 544:

== Further reading ==

* {{Cite magazine |last=Meyer |first=Bertrand |date=August 2008 |title=Seven Principles of Software Testing |url=httphttps://se.ethz.ch/~meyer/publications/testing/principles.pdf |magazine=Computer |volume=41 |pages=99–101 |doi=10.1109/MC.2008.306 |access-date=November 21, 2017 |number=8}}

== External links ==