Content deleted Content added
mNo edit summary |
|||
Line 9:
Software testing can provide objective, independent information about the [[Quality (business)|quality]] of software and the [[risk]] of its failure to a [[User (computing)|user]] or sponsor.<ref name="Kaner 1">{{Cite conference |last=Kaner |first=Cem |author-link=Cem Kaner |date=November 17, 2006 |title=Exploratory Testing |url=https://kaner.com/pdfs/ETatQAI.pdf |conference=Quality Assurance Institute Worldwide Annual Software Testing Conference |___location=Orlando, FL |access-date=November 22, 2014}}</ref>
Software testing can determine the [[Correctness (computer science)|correctness]] of software for specific [[Scenario (computing)|scenarios]]
Based on the criteria for measuring correctness from an [[test oracle|oracle]], software testing employs principles and mechanisms that might recognize a problem. Examples of oracles include
Software testing is often dynamic in nature; running the software to verify actual output matches expected. It can also be static in nature; reviewing [[source code|code]] and its associated [[documentation]].
Line 19:
Information learned from software testing may be used to improve the process by which software is developed.<ref name="kolawa">{{Cite book |last1=Kolawa |first1=Adam |url=http://www.wiley.com/WileyCDA/WileyTitle/productCd-0470042125.html |title=Automated Defect Prevention: Best Practices in Software Management |last2=Huizinga |first2=Dorota |publisher=Wiley-IEEE Computer Society Press |year=2007 |isbn=978-0-470-04212-0}}</ref>{{rp|41–43}}
Software testing should follow a "pyramid" approach wherein most of your tests should be [[unit tests]], followed by [[Integration testing|integration tests]] and finally [[End-to-end testing|end
== Economics ==
A study conducted by [[NIST]] in 2002 reported that software bugs cost the U.S. economy $59.5 billion annually. More than a third of this cost could be avoided
[[Outsourcing]] software testing because of costs is very common, with China, the Philippines, and India being preferred destinations.{{citation needed|date=March 2024}}
Line 96:
=== Black/white box ===
Software testing can often be divided into white-box and black-box. These two approaches are used to describe the point of view that the tester takes when designing test cases. A hybrid approach called grey-box that includes aspects of both boxes may also be applied to software testing methodology.<ref name="LimayeSoftware09">{{Cite book |last=Limaye, M.G. |url=https://books.google.com/books?id=zUm8My7SiakC&pg=PA108 |title=Software Testing |publisher=Tata McGraw-Hill Education |year=2009 |isbn=978-0-07-013990-9 |pages=108–11}}</ref><ref name="SalehSoftware09">{{Cite book |last=Saleh, K.A. |url=https://books.google.com/books?id=N69KPjBEWygC&pg=PA224 |title=Software Engineering |publisher=J. Ross Publishing |year=2009 |isbn=978-1-932159-94-3 |pages=224–41}}</ref>
==== White-box testing ====
Line 103:
[[File:White Box Testing Approach.png|alt=White Box Testing Diagram|thumb|White box testing diagram]]
White-box testing (also known as clear box testing, glass box testing, transparent box testing, and structural testing) verifies the internal structures or workings of a program, as opposed to the functionality exposed to the end-user. In white-box testing, an internal perspective of the system (the source code), as well as programming skills
While white-box testing can be applied at the [[unit testing|unit]], [[integration testing|integration]], and [[system testing|system]] levels of the software testing process, it is usually done at the unit level.<ref name="AmmannIntro16" /> It can test paths within a unit, paths between units during integration, and between subsystems during a system–level test. Though this method of test design can uncover many errors or problems, it might not detect unimplemented parts of the specification or missing requirements.
Line 129:
Black-box testing (also known as functional testing) describes designing test cases without knowledge of the implementation, without reading the source code. The testers are only aware of what the software is supposed to do, not how it does it.<ref name="Patton">{{Cite book |last=Patton |first=Ron |url=https://archive.org/details/softwaretesting0000patt |title=Software Testing |publisher=Sams Publishing |year=2005 |isbn=978-0-672-32798-8 |edition=2nd |___location=Indianapolis}}</ref> Black-box testing methods include: [[equivalence partitioning]], [[boundary value analysis]], [[all-pairs testing]], [[state transition table]]s, [[decision table]] testing, [[fuzz testing]], [[model-based testing]], [[use case]] testing, [[exploratory testing]], and specification-based testing.<ref name="LimayeSoftware09" /><ref name="SalehSoftware09" /><ref name="BlackPragmatic11" />
Specification-based testing aims to test the functionality of software according to the applicable requirements.<ref>{{Cite thesis |last=Laycock |first=Gilbert T. |title=The Theory and Practice of Specification Based Software Testing |degree=dissertation |publisher=Department of Computer Science, [[University of Sheffield]] |url=http://www.cs.le.ac.uk/people/glaycock/thesis.pdf |year=1993 |access-date=January 2, 2018}}</ref> This level of testing usually requires thorough [[test case]]s to be provided to the tester, who then can simply verify that for a given input, the output value (or behavior), either "is" or "is not" the same as the expected value specified in the test case. Test cases are built around specifications and requirements, i.e., what the application is supposed to do. It uses external descriptions of the software, including specifications, requirements, and designs, to derive test cases. These tests can be [[functional testing|functional]] or [[non-functional testing|non-functional]], though usually functional. Specification-based testing may be necessary to assure correct functionality, but it is insufficient to guard against complex or high-risk situations.<ref>{{Cite journal |last=Bach |first=James |author-link=James Bach |date=June 1999 |title=Risk and Requirements-Based Testing |url=http://www.satisfice.com/articles/requirements_based_testing.pdf |journal=Computer |volume=32 |issue=6 |pages=113–114 |access-date=August 19, 2008}}</ref>
Black box testing can be used to any level of testing although usually not at the unit level.<ref name="AmmannIntro16" />
Line 135:
'''Component interface testing'''
Component interface testing is a variation of [[black-box testing]], with the focus on the data values beyond just the related actions of a subsystem component.<ref name="MathurFound11-63">{{Cite book |last=Mathur, A.P. |url=https://books.google.com/books?id=hyaQobu44xUC&pg=PA18 |title=Foundations of Software Testing |publisher=Pearson Education India |year=2011 |isbn=978-81-317-5908-0 |page=63}}</ref> The practice of component interface testing can be used to check the handling of data passed between various units, or subsystem components, beyond full integration testing between those units.<ref name="Clapp">{{Cite book |last=Clapp |first=Judith A. |url=https://books.google.com/books?id=wAq0rnyiGMEC&pg=PA313 |title=Software Quality Control, Error Analysis, and Testing |year=1995 |isbn=978-0-8155-1363-6 |page=313 |publisher=William Andrew |access-date=January 5, 2018}}</ref><ref name="Mathur">{{Cite book |last=Mathur |first=Aditya P. |url=https://books.google.com/books?id=yU-rTcurys8C&pg=PR38 |title=Foundations of Software Testing |publisher=Pearson Education India |year=2007 |isbn=978-81-317-1660-1 |page=18}}</ref> The data being passed can be considered as "message packets" and the range or data types can be checked
===== Visual testing =====
Line 143:
At the core of visual testing is the idea that showing someone a problem (or a test failure), rather than just describing it, greatly increases clarity and understanding. Visual testing, therefore, requires the recording of the entire test process – capturing everything that occurs on the test system in video format. Output videos are supplemented by real-time tester input via picture-in-a-picture webcam and audio commentary from microphones.
Visual testing provides a number of advantages. The quality of communication is increased drastically because testers can show the problem (and the events leading up to it) to the developer as opposed to just describing it, and the need to replicate test failures will cease to exist in many cases. The developer will have all the evidence he or she requires of a test failure and can instead focus on the cause of the fault and how it should be fixed.
[[Ad hoc testing]] and [[exploratory testing]] are important methodologies for checking software integrity
{{further|Graphical user interface testing}}
Line 154:
Grey-box testing (American spelling: gray-box testing) involves using knowledge of internal data structures and algorithms for purposes of designing tests while executing those tests at the user, or black-box level. The tester will often have access to both "the source code and the executable binary."<ref name="RansomeCore13">{{Cite book |last1=Ransome, J. |url=https://books.google.com/books?id=MX5cAgAAQBAJ&pg=PA140 |title=Core Software Security: Security at the Source |last2=Misra, A. |publisher=CRC Press |year=2013 |isbn=978-1-4665-6095-6 |pages=140–3}}</ref> Grey-box testing may also include [[Reverse coding|reverse engineering]] (using dynamic code analysis) to determine, for instance, boundary values or error messages.<ref name="RansomeCore13" /> Manipulating input data and formatting output do not qualify as grey-box, as the input and output are clearly outside of the "black box" that we are calling the system under test. This distinction is particularly important when conducting [[integration testing]] between two modules of code written by two different developers, where only the interfaces are exposed for the test.
By knowing the underlying concepts of how the software works, the tester makes better-informed testing choices while testing the software from outside. Typically, a grey-box tester will be permitted to set up an isolated testing environment with activities, such as seeding a [[database]]. The tester can observe the state of the product being tested after performing certain actions such as executing [[SQL]] statements against the database and then executing queries to ensure that the expected changes have been reflected. Grey-box testing implements intelligent test scenarios
With the concept of grey-box testing, this "arbitrary distinction" between black- and white-box testing has faded somewhat.<ref name="AmmannIntro16">{{Cite book |last1=Ammann, P. |url=https://books.google.com/books?id=58LeDQAAQBAJ&pg=PA26 |title=Introduction to Software Testing |last2=Offutt, J. |publisher=Cambridge University Press |year=2016 |isbn=978-1-316-77312-3 |page=26}}</ref>
| |||