Operational acceptance testing: Difference between revisions

Revision as of 02:30, 5 December 2010

Latest revision as of 11:38, 16 August 2025

(46 intermediate revisions by 30 users not shown)

Line 1:

[[File:US Navy 070804-N-1745W-122 A Sailor assigned to Aircraft Intermediate Maintenance Department (AIMD) tests an aircraft jet engine for defects while performing Jet Engine Test Instrumentation, (JETI) Certification-Engine Runs.jpg|right|thumb|250px|Operational testing a jet engine]]

{{Inappropriate tone|date=October 2010}}

'''Operational Acceptance Testing (OAT)''' is a new type of [[testing]] in the market. This is nowadays in very much demand in [[support]] and [[maintenance]] type projects. In this type of testing we only concentrate about the [[operational readiness]] of the system which would be supported or become the [[production environment]] later. Hence, it is also known as operational readiness testing. The [[functional testing]] of application should not be included or merged in OAT. This may include checking the backup facilities, maintenance and [[disaster recovery]] procedures. In OAT we play with those environmental things which our application uses to run smoothly. For a mixed/ hybrid architecture application, this may include: [[window services]], [[config file]]s, [[web services]], [[XML file]]s, [[COM+]] components, web services, [[IIS]], stored procedures in databases, etc. This type of testing should be conducted before the user acceptance testing.

The approach to be used in OAT is:

# Build the system,

# Deploy the application,

# Break the system.

# Validate the [[backup procedure]]s setup for system

'''Operational acceptance testing''' ('''OAT''') is used to conduct operational readiness (pre-release) of a product, service, or system as part of a [[quality management system]]. OAT is a common type of non-functional [[software testing]], used mainly in [[software development]] and [[software maintenance]] projects. This type of testing focuses on the operational readiness of the system to be supported, and/or to become part of the production environment. Hence, it is also known as '''operational readiness testing''' ('''ORT''') or '''operations readiness and assurance testing''' ('''OR&A'''). [[Functional testing]] within OAT is limited to those tests which are required to verify the ''non-functional'' aspects of the system.

Then check how our application is behaving and moreover how our system is behaving in these conditions? Are the backup procedures setup for [[emergency situation]] working properly?

OAT elaborates upon and compartmentalises operational aspects of acceptance testing.<ref>{{Cite web |title=atos-operational-acceptance-testing-whitepaper.pdf |url=https://atos.net/wp-content/uploads/2017/01/atos-operational-acceptance-testing-whitepaper.pdf }}</ref>

For running the OAT [[test case]]s, the tester should have exclusive access to the system/ environment. This means that a single tester would be executing the test cases at a single point of time. Here breaking the system means; making the application dependent things (listed above) inaccessible to the application. Like, stopping the window services, deleting the SP in databases, taking the DB off-line and then access the system/application. So, this clearly indicates that this is a destructive type of testing.

According to the International Software Testing Qualifications Board (ISTQB), OAT may include checking the [[backup]]/restore facilities, IT [[IT disaster recovery|disaster recovery]] procedures, maintenance tasks and periodic check of security vulnerabilities.,<ref>ISTQB http://istqbexamcertification.com/what-is-acceptance-testing/</ref> and whitepapers on ISO 29119 and Operational Acceptance by Anthony Woods,<ref name="ISO 29119 OAT">{{cite document|author=Anthony Woods|title=Operational Acceptance - an application of the ISO 29119 Software Testing standard|date=2015|publisher=Capgemini and Sogeti|pages=1–12}}</ref> and ISO 25000 and Operational Acceptance Testing by Dirk Dach et al., OAT generally includes:<ref>White Paper: Operational Acceptance Testing, Business Continuity Assurance. December 2012 Dirk Dach, Dr Kai-Uwe Gawlik, Mark Mevert</ref>

For OAT we should define the exact OR quality gates; Entry and Exit Gate. This should list down all the activities which would be part and covered in the different phases of testing. The main emphasis should be on the operational part of the system.

* Component Testing

The main advantage of OAT is also to test the [[exception logging]] methods used by the application. Because, whenever there would be any [[system failure]], it would write/ log an exception. So, automatically the exception logging part of application will be tested.

* Failover (Within the same data centre)

:* Component fail-over

:* Network fail-over

* Functional Stability

:* Accessibility

:* Conversion

:* Stability

:* Usability

* IT Service Management (Supportability)

* Monitoring and Alerts (to ensure proper alerts are configured in the system if something goes wrong)

* Portability

:* Compatibility

:* Interoperability

:* Installation and Backout

:* Localization

* Recovery (across data centres)

:* Application/system recovery

:* Data recovery

* Reliability

:* Backup and Restoration (Recovery)

:* Disaster Recovery

:* Maintainability

:* Performance, Stress and Volume,

:* Procedures (Operability) and Supporting Documentation (Supportability)

:* Security and Penetration

During OAT changes may be made to environmental parameters which the application uses to run smoothly. For example, with [[Microsoft Windows]] applications with a mixed or hybrid architecture, this may include: [[Windows service]]s, [[configuration file]]s, [[web services]], [[XML]] files, [[COM+]] components, web services, [[Internet Information Services|IIS]], stored procedures in databases, etc. Typically OAT should occur after each main phase of the development life cycle: design, build, and [[functional testing]]. In sequential projects it is often viewed as a ''final'' verification before a system is released; where in agile and iterative projects, a more frequent execution of OAT occurs providing stakeholders with assurance of continued stability of the system and its operating environment.

Let us understand this with an example now:

Say for there is a new production environment which need to be setup. The application is a 3 tier application and has a [[mixed architecture]] (''as the legacy application being built in VB 6/ ASP and the later enhancements in .Net technology''). And we need to perform the OAT for this environment. We will first identify all the environment related things which our application is using. For a simple example, let say our application is using window service to validate/ authenticate the [[user credentials]]. This window service takes the input from user and matches the credentials fetched from the database. What if this window services stops incidentally? This would surely be a problematic situation. But, as a proactive measure there is provision to deal with this situation. A mail with service details will be shot to responsible team for corrective action.

TheAn approach to be used in OAT ismay follow these steps:

So, now our OAT test cases would become any of case give below:

#* BreakDesign the system.,

* Assess the design,

#* Build the system,

* Confirm if built to design,

* Evaluate the system addresses business functional requirements,

* Assess the system for compliance with non-functional requirements,

#* Deploy the applicationsystem,

* Assess operability and supportability of the system.

For running the OAT [[test case (software)|test case]]s, the tester shouldnormally havehas exclusive access to the system/ or environment. This means that a single tester would be executing the test cases at a single point of time. HereFor breakingOAT the systemexact means;'''Operational makingReadiness''' thequality applicationgates dependentare thingsdefined: (listedboth above)entry inaccessibleand toexit the applicationgates. Like,The stoppingprimary theemphasis windowof services,OAT should be deletingon the SPoperational in databasesstability, taking the DB off-lineportability and thenreliability accessof the system/application. So, this clearly indicates that this is a destructive type of testing.

# Stopping the window service. Stop the responsible window service and then try to access the application.

# Taking Database off-line. Take the DB off-line and then try to access the application.

In both the scenarios, when user access the application with valid credentials would result into failure/ crash. The mail will be automatically sent to the responsible team for taking the [[corrective action]]. So, this will help us to validate the backup procedures setup for any emergency.

==References==

{{Uncategorized|date=October 2010}}

{{reflist}}

[[Category:Software testing]]