Shift-left testing: Difference between revisions

Revision as of 16:29, 24 May 2023

Latest revision as of 12:06, 5 October 2024

(3 intermediate revisions by 3 users not shown)

Line 1:

{{Short description|Type of software testing}}

'''Shift-left''' testing refers to testing software early in the [[Software development|development]] process, while [https://www.dynatrace.com/news/blog/what-is-shift-left-and-what-is-shift-right/ shift-right] testing refers to testing towards the end of the development cycle. The term, "shift-left" is used to describe the idea of moving [[software testing]] earlier in the process, which [https://www.softwaretestinghelp.com/shift-left-testing-approach/ can help catch defects earlier and reduce the cost of fixing them later]. In contrast, "shift-right" refers to testing later in the process, which can help ensure that the software meets the intended requirements and functions correctly before it is released. SHift -left is also the first half of the maxim "test early and often",<ref>{{cite web|url=https://msdn.microsoft.com/en-us/library/vstudio/ee330950%28v=vs.110%29.aspx |title=Test Early and Often |author=Microsoft | date=2012 |access-date=27 March 2015}}</ref> coined by Larry Smith in 2001.<ref>{{Cite journal|last=Smith|first=Larry|date=September 2001|title=Shift-Left Testing|url=https://www.drdobbs.com/shift-left-testing/184404768|journal=Dr. Dobb's Journal|volume=26|number=9|pages=56, 62}}</ref><ref>{{Cite web|url=http://collaboration.cmc.ec.gc.ca/science/rpn/biblio/ddj/Website/articles/DDJ/2001/0109/0109e/0109e.htm|archive-url=https://web.archive.org/web/20140810171940/http://collaboration.cmc.ec.gc.ca/science/rpn/biblio/ddj/Website/articles/DDJ/2001/0109/0109e/0109e.htm|url-status=dead|archive-date=2014-08-10|title=Sep01: Shift-Left Testing|date=2014-08-10|access-date=2019-09-06}}</ref>

{{multiple issues|{{more footnotes|date=May 2015}}{{essay-like|date=May 2015}}

|}}

'''Shift-left testing'''<ref>{{cite testing refers to testing software early in the [[Software developmentweb|development]] process, while [url=https://wwwinsights.sei.dynatracecmu.comedu/newssei_blog/blog2015/what03/four-istypes-of-shift-left-and-what-is-shift-right/ shift-right] testing refers to testing towards the end of the development cycle.html The term, "shift-left" is used to describe the|title=Four ideaTypes of movingShift [[softwareLeft testing]]Testing earlier|author=Donald inFiresmith the| process,date=23 whichMarch [2015 |archive-url=https://wwwweb.softwaretestinghelparchive.comorg/web/20150905082941/https://insights.sei.cmu.edu/sei_blog/2015/03/four-types-of-shift-left-testing.html |access-approach/date=27 canMarch help2015|archive-date=2015-09-05 catch}}</ref> defectsis earlieran andapproach reduceto the[[software cost of fixing them latertesting]]. Inand contrast, "shift-right" refers to[[system testing later]] in the process, which cantesting helpis ensureperformed thatearlier in the [[software meetsdevelopment thelife intendedcycle|lifecycle]] requirements(i.e. andmoved functionsleft correctlyon beforethe itproject is releasedtimeline). SHift -leftIt is also the first half of the maxim "test early and often",.<ref>{{cite web |url=https://msdn.microsoft.com/en-us/library/vstudio/ee330950%28v=vs.110%29.aspx |title=Test Early and Often |author=Microsoft | date=2012 |access-date=27 March 2015 |archive-date=2 April 2015 |archive-url=https://web.archive.org/web/20150402174955/https://msdn.microsoft.com/en-us/library/vstudio/ee330950(v=vs.110).aspx |url-status=live }}</ref> It was coined by Larry Smith in 2001.<ref>{{Cite journal|last=Smith|first=Larry|date=September 2001|title=Shift-Left Testing|url=https://www.drdobbs.com/shift-left-testing/184404768|journal=Dr. Dobb's Journal|volume=26|number=9|pages=56, 62|access-date=2020-06-04|archive-date=2020-06-21|archive-url=https://web.archive.org/web/20200621040103/https://www.drdobbs.com/shift-left-testing/184404768|url-status=live}}</ref><ref>{{Cite web|url=http://collaboration.cmc.ec.gc.ca/science/rpn/biblio/ddj/Website/articles/DDJ/2001/0109/0109e/0109e.htm|archive-url=https://web.archive.org/web/20140810171940/http://collaboration.cmc.ec.gc.ca/science/rpn/biblio/ddj/Website/articles/DDJ/2001/0109/0109e/0109e.htm|url-status=dead|archive-date=2014-08-10|title=Sep01: Shift-Left Testing|date=2014-08-10|access-date=2019-09-06}}</ref>

== WhyHarm Shiftbecause Leftof Testinglate ?testing ==

Shift-left testing aims to prevent the following types of harm becausecaused ofby late testing:

Shift-left testing is used to catch defects earlier in the software development process before they become more costly and time-consuming to fix. This approach helps improve the overall quality of the software, reduces development costs, and shortens the time to market.

One of the main reasons for shift-left testing is to address defects as early as possible in the development process. According to a study by the IBM Systems Science Institute,<ref>{{cite book |last1=IBM Systems Science Institute |title=The impact of defects on productivity |date=2004 |publisher=IBM}}</ref> the cost of fixing defects increases dramatically as they move further along in the development lifecycle. In fact, the cost of fixing defects during production can be up to 100 times more expensive than if the defect had been caught during the requirements phase.

Another reason for shift left testing is to [https://www.lambdatest.com/learning-hub/shift-left-testing increase collaboration and communication] between different teams involved in software development. By catching defects early, development teams can work more closely with other stakeholders, such as testers and business analysts, to identify and resolve issues before they become more difficult to fix.

There are several tools available to support shift-left testing, including those that use [[artificial intelligence]] (AI) and [[machine learning]] (ML). AI and ML tools can be used to automate testing activities and optimize test coverage. By using these tools, software developers can identify defects earlier in the development cycle and improve the overall quality of their software products.<ref>{{cite book |last1=J. Gao, C. Tao, D. Jie, S. Lu |title=What is AI Software Testing? and Why |date=2019 |publisher=IEEE |url=https://ieeexplore.ieee.org/document/8705808/authors#authors}}</ref>

== Risks of testing later ==

Shift-left testing aims to prevent the following types of harm because of late testing:

* Insufficient resources allocated to testing.

* Undiscovered defects in requirements, architecture, and design, along with significant effort wasted while implementing them.

* Difficulty [[debugging]] (including identifying, localizing, fixing, and regression testing defects) as more software is produced and integrated.

* Reduced [[code coverage]] during testing{{Citation needed|date=October 2017}} as a result of [[Encapsulation (object-oriented programming)|encapsulation]] impeding [[white-box testing]].

* A “bow wave” of [[technical debt]] that can cause a project to fail.

== ExamplesTypes of shift-left testing ==

There are four basic ways to shift testing earlier in the life-cycle (that is, leftward on the classic [[V-model]]). These can be referred to as traditional shift-left testing,<ref>{{cite web |url=http://www.velocitypartners.net/blog/2014/01/28/agile-testing-the-agile-test-automation-pyramid/ |title=Agile Testing - The Agile Test Automation Pyramid |author=Velocity Partners |date=28 January 2014 |access-date=27 March 2015 |archive-date=2 April 2015 |archive-url=https://web.archive.org/web/20150402164204/http://www.velocitypartners.net/blog/2014/01/28/agile-testing-the-agile-test-automation-pyramid/ |url-status=live }}</ref> incremental shift-left testing, Agile/DevOps shift-left testing,<ref>{{cite web |url=http://www.slideshare.net/Urbancode/shift-left |title=Shift Left: Approaches and Practices |author=Paul Bahrs |date=6 November 2014 |access-date=27 March 2015 |archive-date=6 April 2015 |archive-url=https://web.archive.org/web/20150406145122/http://www.slideshare.net/Urbancode/shift-left |url-status=live }}</ref><ref>{{cite web |url=https://www.ibm.com/developerworks/community/blogs/invisiblethread/entry/enabling_devops_success_with_shift_left_continuous_testing?lang=en |title=Enabling DevOps Success with Shift Left Continuous Testing |author=Dibbe Edwards |website=[[IBM]] |date=18 September 2014 |access-date=27 March 2015 |archive-date=2 April 2015 |archive-url=https://web.archive.org/web/20150402110506/https://www.ibm.com/developerworks/community/blogs/invisiblethread/entry/enabling_devops_success_with_shift_left_continuous_testing?lang=en |url-status=live }}</ref> and model-based shift-left testing.<ref>{{cite web |url=http://blog.sei.cmu.edu/post.cfm/using-v-models-testing-315 |title=Using V Models for Testing |author=Donald Firesmith |date=11 November 2013 |access-date=27 March 2015 |archive-date=2 April 2015 |archive-url=https://web.archive.org/web/20150402110448/http://blog.sei.cmu.edu/post.cfm/using-v-models-testing-315 |url-status=live }}</ref>

Shift-left testing is an approach that involves moving testing activities to earlier stages of the software development lifecycle (SDLC) to catch defects early, improve software quality, and reduce costs. There are four basic ways to shift testing earlier in the life-cycle, leftward on the classic V-model, which include traditional shift-left testing,<ref>{{cite book |last1=Lisa Crispin & Janet Gregory |title=Agile Testing: A Practical Guide for Testers and Agile Teams |date=2014 |publisher=Velocity Partners}}</ref> incremental shift-left testing, Agile/DevOps shift-left testing, and model-based shift-left testing[8]. Traditional shift-left testing involves integrating testing activities into every phase of the process, while incremental shift-left testing gradually moves testing activities to earlier stages of the SDLC. Agile/DevOps shift-left testing involves continuous testing throughout the development process, using automated testing tools and practices like continuous integration and continuous testing (CI/CT). Model-based shift-left testing uses models to simulate software behavior and identify defects early in the development process.

=== Traditional shift-left testing ===

As illustrated in the following figure, traditional shift-left moves the emphasis of testing lower down (and therefore slightly to the left) on the right hand side of the classic V model. Instead of emphasizing acceptance and system level testing (e.g., [[GUI testing]] with record and playback tools<ref>{{cite web |url=https://msdn.microsoft.com/en-us/library/dd286714.aspx |title=Record and Playback Manual Tests |author=Microsoft |date=2013 |access-date=27 March 2015 |archive-date=2 April 2015 |archive-url=https://web.archive.org/web/20150402113241/https://msdn.microsoft.com/en-us/library/dd286714.aspx |url-status=live }}</ref>), traditional shift-left concentrates on [[unit testing]] and [[integration testing]] (e.g., using [[API testing]] and modern test tools). The transition to traditional shift-left testing has largely been completed.{{By whom|date=April 2016}}

Traditional shift left testing involves moving testing activities to earlier stages of the [[software development process]]. This approach aims to catch defects as early as possible, ideally during the requirements and design phases, to prevent them from becoming bigger issues later on. The goal of traditional shift left testing is to improve software quality and reduce costs by catching and [https://www.guru99.com/shift-left-testing.html resolving issues earlier in the development process].

In traditional shift-left testing, testing activities are typically carried out by [https://dzone.com/articles/a-beginners-guide-to-shift-left-testing dedicated testers] who work alongside developers to ensure that testing is integrated into every phase of the SDLC. Testers may be involved in activities such as reviewing requirements, writing test cases, performing code reviews, and executing automated tests.

One benefit of traditional shift-left testing is that it allows issues to be caught and resolved earlier in the SDLC, which can reduce the cost and time required to fix defects later on. Additionally, by involving testers earlier in the development process, [https://www.softwaretestinghelp.com/shift-left-testing/ communication and collaboration between testers and developers can be improved], leading to better software quality overall.

{{col-begin|width=100%}}

|-

|

<gallery>

Traditional-Shift-Left.jpg|Traditional shift-left testing
</gallery>

|

Traditional shift-left testing emphasizes unit and integration testing, as shown in the diagram, moving testing activities to earlier stages on the right-hand side of the V-model, rather than focusing on acceptance and system-level testing. This approach involves using modern testing tools and API testing instead of GUI testing with record and playback tools.

|}

=== Incremental shift-left testing ===

As illustrated in the following figure, many projects developing large and complex software-reliant systems decompose development into a small number of increments (Vs) having correspondingly shorter durations. The shift-left illustrated by the dashed red arrows occurs because parts of the single, large waterfall V model’s types of testing (shown in gray) are shifted left to become increments of the corresponding types of testing in the smaller incremental V models. When each increment is also a delivery to the customer and operations, then incremental shift-left testing shifts both developmental testing and operational testing to the left. Incremental shift-left testing is popular when developing large, complex systems, especially those incorporating significant amounts of hardware. Like traditional shift-left, the transition to incremental shift-left has also been largely completed.

Incremental shift-left testing is a software testing approach that gradually moves testing activities to earlier stages of the software development process in a phased manner. This approach enables organizations to implement shift-left testing at a comfortable pace, starting with small changes and gradually increasing the level of testing activity in earlier stages of the development process.

<gallery>

<gallery>Incremental-Shift-Left.jpg|Incremental shift-left testing</gallery>

In incremental shift-left testing, testing activities are moved incrementally to earlier stages of the development process, starting with the coding phase and moving backward to earlier stages such as requirements gathering and design. By doing this, issues can be identified and resolved earlier in the development process, leading to improved software quality and reduced costs.

</gallery>

For instance, organizations can start with unit testing and static code analysis during the coding phase, then gradually add more testing activities such as requirements testing and design review in earlier stages of the development process. [[Unit testing]] involves testing individual units or components of code to ensure they work correctly, while [[static code analysis]] helps to identify potential errors or bugs in the code before it is deployed.

{{col-begin|width=100%}}

|-

|

<gallery>Incremental-Shift-Left.jpg|Incremental shift-left testing</gallery>

|

Agile and DevOps projects use short sprints instead of a few large V models like in traditional shift-left testing. The types of testing in these sprints occur earlier in the development process compared to larger V models. Agile testing is limited to developmental testing, and the shift-left trend in Agile/DevOps testing is popular and ongoing.

|}

=== Agile/DevOps shift-left testing ===

As illustrated in the following figure, [[Agile software development|Agile]] and [[DevOps]] projects have numerous short duration Vs (sprints) in lieu of a single or small number of V as in the previous two examples of shift-left testing. These small Vs would also be modified if one or more early sprints are used to block out the basic requirements and architecture or if test-first and [[test-driven development]] (TDD) are being performed. The shift-left occurs because the types of testing on the right sides of the earliest of these tiny Vs are to the left of the corresponding types of testing on right side of the larger V(s) they replace. While the following figure appears remarkably the same for Agile and DevOps, Agile testing is typically restricted to developmental testing and does not include operational testing, which occurs once the system is placed into operation. The transition to Agile/DevOps shift-left testing is currently popular and ongoing.

[[Agile software development|Agile]]/[[DevOps]] shift-left testing is a way of testing software that is often used in Agile and DevOps software development methods. Instead of waiting until the end of the development process to perform testing, [https://www.softwaretestingnews.co.uk/agile-shift-left-testing/ testing activities are integrated] into the entire software development process.

<gallery>

<gallery>Agile_DevOps_Shift_Left_Testing.jpg|Agile/DevOps shift-left testing</gallery>

In Agile/DevOps shift-left testing, testing is performed continuously during the development process. Testers work closely with developers and other team members to catch and resolve defects as early as possible. This method emphasizes communication and collaboration between team members to make sure that testing is integrated into every part of the software development process.

</gallery>

One way of implementing Agile/DevOps shift-left testing is to use automated testing tools and practices like continuous integration and [https://www.ibm.com/topics/continuous-testing continuous testing]. These methods allow organizations to catch defects early and prevent them from causing bigger problems later on.

{{col-begin|width=100%}}

|-

|

<gallery>Agile_DevOps_Shift_Left_Testing.jpg|Agile/DevOps shift-left testing</gallery>

|

Agile and DevOps projects use many short sprints instead of a few larger ones, as shown in the figure. These sprints may include early stages for defining requirements or using [[test-driven development]] (TDD). The shift-left testing occurs because the tests done early in the sprints are further to the left of the corresponding tests in larger Vs. Agile testing is typically only for developmental testing, not operational testing. Many projects are transitioning to Agile/DevOps shift-left testing.

|}

=== Model-based shift-left testing ===

The previous forms all concentrated on testing earlier in the development cycle. However, they all test ''after'' software exists and seek to uncover only implementation defects.{{cn|date=January 2021}}

Model-based shift-left testing is a software testing approach that uses models to simulate how the software works. Testers use these models to detect defects early in the development process and make changes to the software design before the actual development begins. This approach can help ensure that the software meets the intended requirements and functions correctly before it is built.

Model-based testing moves testing to the left side of the Vs, by testing requirements, architecture, and design models. This shift begins testing almost immediately, instead of waiting a long time (traditional testing), medium time (incremental testing), or short time (Agile/DevOps) for software to become available to the right side of the Vs. This trend is just beginning.

To use model-based shift-left testing, testers create a graphical model that represents the software behavior and use automated tools to test the model. This allows testers to test different scenarios and validate the software design before the actual coding process starts. By identifying defects early in the development process, developers can avoid costly rework later on.

<gallery>

<gallery>Model-Shift-Left.jpg|Model-based shift-left testing</gallery>

According to an article in "[https://www.softwaretestinghelp.com/model-based-testing/ Software Testing Help]", model-based shift-left testing "can help to reduce the cost of software development by minimizing the need for rework and improving software quality through early detection and correction of defects."

</gallery>

{{col-begin|width=100%}}

|-

|

<gallery>Model-Shift-Left.jpg|Model-based shift-left testing</gallery>

|

Earlier testing approaches only aimed to detect implementation defects and were performed after the software was created. Model-based testing, on the other hand, tests the models used in software development such as requirements, architecture, and design models. This type of testing starts almost immediately, unlike traditional, incremental, and Agile/DevOps testing which all require waiting for software to be available. Model-based testing is a new trend that shifts testing to the left side of the Vs.

|}

== References ==