Software regression: Difference between revisions

Revision as of 05:22, 23 November 2021

Revision as of 00:19, 26 November 2021

Line 7:

Regressions are often caused by [[Hotfix|encompassed bug fixes]] included in [[software patch]]es. One approach to avoiding this kind of problem is [[regression testing]]. A properly designed [[test plan]] aims at preventing this possibility before releasing any software.<ref>{{cite book |last=Richardson |first=Jared |author2=Gwaltney, William Jr |title=Ship It! A Practical Guide to Successful Software Projects |url=https://archive.org/details/shipitpracticalg0000rich/page/32 |year=2006 |publisher=The Pragmatic Bookshelf |___location=Raleigh, NC |pages=[https://archive.org/details/shipitpracticalg0000rich/page/32 32, 193] |isbn=978-0-9745140-4-8 }}</ref> [[Automated testing]] and well-written [[test case]]s can reduce the likelihood of a regression.

==Prevention and Detectiondetection==

Techniques have been proposed that try to prevent regressions from being introduced into software at various stages of development, outlined below.

Line 15:

{{main|Regression testing}}

In order to avoid regressions from being seen by the [[end-user]] after release, developers regularly run [[regression tests]] after changes are introduced to the software. These tests can comprise ofinclude [[unit tests]] to catch local regressions, as well as [[integration tests]] to catch remote regressions.<ref>{{cite book |last1=Leung |first1=Hareton K.N. |last2=White |first2=Lee |title=Proceedings of the International Conference on Software Maintenance |date=November 1990 |publisher=IEEE |___location=San Diego, CA, USA |isbn=0-8186-2091-9 |url=https://ieeexplore.ieee.org/abstract/document/131377 |chapter=A study of integration testing and software regression at the integration level}}</ref> Regression testing techniques often leverage existing test cases to minimize the effort involved in creating them.<ref>{{cite journal |last1=Rothermel |first1=Gregg |last2=Harrold |first2=Mary Jean |last3=Dedhia |first3=Jeinay |title=Regression test selection for C++ software |journal=Software Testing, Verification and Reliability |date=2000 |volume=10 |issue=2 |pages=77–109 |doi=10.1002/1099-1689(200006)10:2<77::AID-STVR197>3.0.CO;2-E |url=https://onlinelibrary.wiley.com/doi/abs/10.1002/1099-1689(200006)10:2%3C77::AID-STVR197%3E3.0.CO;2-E |language=en |issn=1099-1689}}</ref> However, due to the volume of these existing tests, it is often necessary to select a representative subset, using techniques such as [[Regression_testing#Test_case_prioritization|test-case prioritization]].

For detecting performance regressions, [[software performance testing|software performance tests]] are run on a regular basis, to monitor the response time and resource usage metrics of the software after subsequent changes.<ref>{{cite journal |last1=Weyuker |first1=E.J. |last2=Vokolos |first2=F.I. |title=Experience with performance testing of software systems: issues, an approach, and case study |journal=IEEE Transactions on Software Engineering |date=December 2000 |volume=26 |issue=12 |pages=1147–1156 |doi=10.1109/32.888628 |url=https://ieeexplore.ieee.org/abstract/document/888628 |issn=1939-3520}}</ref> Unlike functional regression tests, the results of performance tests are subject to [[variance]] - that is, results can differ between tests due to variance in performance measurements; as a result, a decision must be made on whether a change in performance numbers constitutes a regression, based on experience and end-user demands. Approaches such as [[statistical significance test|statistical significance testing]] and [[change point detection]] are sometimes used to aid in this decision.<ref>{{cite book |last1=Daly |first1=David |last2=Brown |first2=William |last3=Ingo |first3=Henrik |last4=O'Leary |first4=Jim |last5=Bradford |first5=David |title=Proceedings of the International Conference on Performance Engineering |date=20 April 2020 |publisher=Association for Computing Machinery |isbn=978-1-4503-6991-6 |pages=67–75 |url=https://dl.acm.org/doi/abs/10.1145/3358960.3375791 |chapter=The Use of Change Point Detection to Identify Software Performance Regressions in a Continuous Integration System}}</ref>