Operational acceptance testing: Difference between revisions

Revision as of 05:58, 25 November 2010

Latest revision as of 11:38, 16 August 2025

(48 intermediate revisions by 32 users not shown)

Line 1:

[[File:US Navy 070804-N-1745W-122 A Sailor assigned to Aircraft Intermediate Maintenance Department (AIMD) tests an aircraft jet engine for defects while performing Jet Engine Test Instrumentation, (JETI) Certification-Engine Runs.jpg|right|thumb|250px|Operational testing a jet engine]]

{{dead end|date=November 2010}}

'''Operational acceptance testing''' ('''OAT''') is used to conduct operational readiness (pre-release) of a product, service, or system as part of a [[quality management system]]. OAT is a common type of non-functional [[software testing]], used mainly in [[software development]] and [[software maintenance]] projects. This type of testing focuses on the operational readiness of the system to be supported, and/or to become part of the production environment. Hence, it is also known as '''operational readiness testing''' ('''ORT''') or '''operations readiness and assurance testing''' ('''OR&A'''). [[Functional testing]] within OAT is limited to those tests which are required to verify the ''non-functional'' aspects of the system.

{{wikify|date=November 2010}}

{{Inappropriate tone|date=October 2010}}

'''Operational Acceptance Testing (OAT)''' is a new type of testing in the market. This is now-a-days in very much demand in support and maintenance type projects. In this type of testing we only concentrate about the operational readiness of the system which would be supported or become the production environment later. Hence, it is also known as operational readiness testing. The functional testing of application should not be included or merged in OAT. This may include checking the backup facilities, maintenance and disaster recovery procedures. In OAT we play with those environmental things which our application uses to run smoothly. For a mixed/ hybrid architecture application, this may include: window services, config files, web services. XML files, COM+ components, web services, IIS, stored procedures in databases, etc. This type of testing should be conducted before the user acceptance testing.

The approach to be used in OAT is:

# Build the system,

# Deploy the application,

# Break the system.

# Validate the backup procedures setup for system

OAT elaborates upon and compartmentalises operational aspects of acceptance testing.<ref>{{Cite web |title=atos-operational-acceptance-testing-whitepaper.pdf |url=https://atos.net/wp-content/uploads/2017/01/atos-operational-acceptance-testing-whitepaper.pdf }}</ref>

Then check how our application is behaving and moreover how our system is behaving in these conditions? Are the backup procedures setup for emergency situation working properly?

According to the International Software Testing Qualifications Board (ISTQB), OAT may include checking the [[backup]]/restore facilities, IT [[IT disaster recovery|disaster recovery]] procedures, maintenance tasks and periodic check of security vulnerabilities.,<ref>ISTQB http://istqbexamcertification.com/what-is-acceptance-testing/</ref> and whitepapers on ISO 29119 and Operational Acceptance by Anthony Woods,<ref name="ISO 29119 OAT">{{cite document|author=Anthony Woods|title=Operational Acceptance - an application of the ISO 29119 Software Testing standard|date=2015|publisher=Capgemini and Sogeti|pages=1–12}}</ref> and ISO 25000 and Operational Acceptance Testing by Dirk Dach et al., OAT generally includes:<ref>White Paper: Operational Acceptance Testing, Business Continuity Assurance. December 2012 Dirk Dach, Dr Kai-Uwe Gawlik, Mark Mevert</ref>

For running the OAT test cases, the tester should have exclusive access to the system/ environment. This means that a single tester would be executing the test cases at a single point of time. Here breaking the system means; making the application dependent things (listed above) unaccessible to the application. Like, stopping the window services, deleting the SP in databases, taking the DB off-line and then access the system/application. So, this clearly indicates that this is a destructive type of testing.

* Component Testing

For OAT we should define the exact OR quality gates; Entry and Exit Gate. This should list down all the activities which would be part and covered in the different phases of testing. The main emphasis should be on the operational part of the system.

* Failover (Within the same data centre)

:* Component fail-over

:* Network fail-over

* Functional Stability

:* Accessibility

:* Conversion

:* Stability

:* Usability

* IT Service Management (Supportability)

* Monitoring and Alerts (to ensure proper alerts are configured in the system if something goes wrong)

* Portability

:* Compatibility

:* Interoperability

:* Installation and Backout

:* Localization

* Recovery (across data centres)

:* Application/system recovery

:* Data recovery

* Reliability

:* Backup and Restoration (Recovery)

:* Disaster Recovery

:* Maintainability

:* Performance, Stress and Volume,

:* Procedures (Operability) and Supporting Documentation (Supportability)

:* Security and Penetration

During OAT changes may be made to environmental parameters which the application uses to run smoothly. For example, with [[Microsoft Windows]] applications with a mixed or hybrid architecture, this may include: [[Windows service]]s, [[configuration file]]s, [[web services]], [[XML]] files, [[COM+]] components, web services, [[Internet Information Services|IIS]], stored procedures in databases, etc. Typically OAT should occur after each main phase of the development life cycle: design, build, and [[functional testing]]. In sequential projects it is often viewed as a ''final'' verification before a system is released; where in agile and iterative projects, a more frequent execution of OAT occurs providing stakeholders with assurance of continued stability of the system and its operating environment.

The main advantage of OAT is also to test the exception logging methods used by the application. Because, whenever there would be any system failure, it would write/ log an exception. So, automatically the exception logging part of application will be tested.

TheAn approach to be used in OAT ismay follow these steps:

Let us understand this with an example now:

#* BreakDesign the system.,

Say for there is a new production environment which need to be setup. The application is a 3 tier application and has a mixed architecture (''as the legacy application being built in VB 6/ ASP and the later enhancements in .Net technology''). And we need to perform the OAT for this environment. We will first identify all the environment related things which our application is using. For a simple example, let say our application is using window service to validate/ authenticate the user credentials. This window service takes the input from user and matches the credentials fetched from the database. What if this window services stops incidentally? This would surely be a problematic situation. But, as a proactive measure there is provision to deal with this situation. A mail with service details will be shot to responsible team for corrective action.

* Assess the design,

#* Build the system,

* Confirm if built to design,

* Evaluate the system addresses business functional requirements,

* Assess the system for compliance with non-functional requirements,

#* Deploy the applicationsystem,

* Assess operability and supportability of the system.

For running the OAT [[test case (software)|test case]]s, the tester normally has exclusive access to the system or environment. This means that a single tester would be executing the test cases at a single point of time. For OAT the exact '''Operational Readiness''' quality gates are defined: both entry and exit gates. The primary emphasis of OAT should be on the operational stability, portability and reliability of the system.

So, now our OAT test cases would become any of case give below:

==References==

# Stopping the window service. Stop the responsible window service and then try to access the application.

{{reflist}}

# Taking Database off-line. Take the DB off-line and then try to access the application.

In both the scenarios, when user access the application with valid credentials would result into failure/ crash. The mail will be automatically sent to the responsible team for taking the corrective action. So, this will help us to validate the backup procedures setup for any emergency.

[[Category:Software testing]]

{{Uncategorized|date=October 2010}}