Wikipedia

Test and learn: Difference between revisions

Browse history interactively
← Previous edit
Content deleted Content added
VisualWikitext
m Reverted edits by Erikancox24 (talk) to last version by Monkbot
m Rollback edit(s) by Fakharaliseo321 (talk): Addition of unnecessary/inappropriate external links (RW 16.1)
 
(25 intermediate revisions by 18 users not shown)
Line 1:
'''Test and Learnlearn''' is a set of practices followed by retailers, banks and other consumer-focused companies to test ideas in a small number of locations or customers to predict impact. The process is often designed to answer three questions about any tested program before rollout:
 
# What impact will the program have on key performance indicators if executed across the network or customer base?
Line 7:
== Historical context ==
 
Test and Learnlearn has been systematically applied as far back as 1988 by [[Capital One]]. Capital One has been aggressive about testing since the firm was founded, testing everything from product design to marketing to customer selection to collection policies.<ref name="hbr">{{Cite news|url=https://hbr.org/2009/02/how-to-design-smart-business-experiments|title=How to Design Smart Business Experiments|last=Davenport|first=Thomas H.|date=February 2009|newspaper=[[Harvard Business Review]]|accessauthor-date=|postscript=<!--None-->|authorlinklink=Thomas H. Davenport}}</ref> In a single year, the company performs tens of thousands of tests, allowing it to offer thousands of different types of credit cards to customers, based on the knowledge gained from their tests. Richard Fairbank, [[CEO]] of Capital One, called test and learn "a marketing revolution that can be applied to many businesses".<ref>{{cite web
| last = Fishman
| first = Charles
Line 13:
| title = This is a Marketing Revolution
| publisher = [[Fast Company (magazine)|Fast Company]]
| accessdateaccess-date = September 22, 2009
| date = 1999-04-30
}}</ref>
Line 25:
* Retailers testing merchandising layouts across their door base
* The UK government, through research projects such as the College for Teaching and Learning's Close the Gap: Test and learn <ref>{{Cite web | url=https://www.gov.uk/government/publications/closing-the-gap-test-and-learn | title=Closing the gap: Test and learn}}</ref>
* [[Amazon (company)|Amazon]] frequently employs the Test and Learn methodology to determine the effectiveness of different homepage layouts, product recommendation algorithms, and pricing strategies.<ref>{{Cite web |date=2020-08-20 |title=Using A/B testing to measure the efficacy of recommendations generated by Amazon Personalize {{!}} AWS Machine Learning Blog |url=https://aws.amazon.com/blogs/machine-learning/using-a-b-testing-to-measure-the-efficacy-of-recommendations-generated-by-amazon-personalize/ |access-date=2023-07-26 |website=aws.amazon.com |language=en-US}}</ref><ref>{{Cite web |title=PFLB Platform - Exploring Test and Learn: A Scientific Examination of Predictive Business Strategies |url=https://platform.pflb.us/ |access-date=2023-07-26 |website=platform.pflb.us |language=en}}</ref>
* [[Netflix]] often employs the Test and Learn methodology for its algorithms, user interface, and even content creation.<ref>{{Cite journal |last=van Es |first=Karin |date=2022-09-26 |title=Netflix & Big Data: The Strategic Ambivalence of an Entertainment Company |url=http://journals.sagepub.com/doi/10.1177/15274764221125745 |journal=Television & New Media |language=en |pages=152747642211257 |doi=10.1177/15274764221125745 |issn=1527-4764|doi-access=free }}</ref>
 
== Test and learn in practice ==
 
* [[Woolworths Group (Australia)|Woolworths]] uses a test-and-learn approach to identify which stores should undergo renovations. <ref>{{cite web
| url = https://www.9news.com.au/technology/woolworths-refurbished-stores-decided-by-computer-software-coles-stores/6d512adf-baa4-4320-b817-457a2f8d5897
| title = High-tech secret behind the future of your local Woolworths
| publisher = 9News
| date = January 23, 2020
| access-date = July 6, 2020
}}</ref>
* [[Wawa Food Markets]], a Mid-Atlantic convenience store chain, uses Test and Learn across several aspects of its business:<ref name="storesmag">{{cite news
|last = Fleenor
Line 35 ⟶ 44:
|newspaper = STORES Magazine
|url = http://www.stores.org/InformationTechnology/2009/06/Edit03.asp
|accessdateaccess-date = 18 September 2009
|url-status = dead
|archiveurlarchive-url = https://web.archive.org/web/20091012122924/http://www.stores.org/InformationTechnology/2009/06/Edit03.asp
|archivedatearchive-date = 12 October 2009
}}</ref>
** Measuring the effect of introducing a new product
Line 51 ⟶ 60:
| publisher = Supermarket News
| date = December 22, 2008
| accessdateaccess-date = September 22, 2009
}}</ref>
** Determining the effect of a coffee aisle re-design on sales of premium coffees
** Measuring the impact on product sales of a change to in-store sales representatives' responsibilities
* [[Pier 1 Imports]], a recently bankrupt retail chain specializing in home decor and furnishings, uses testing to measure the effectiveness of [[search advertising]] campaigns.<ref>{{cite web
| last = Wong
| first = Elaine
Line 62 ⟶ 71:
| publisher = [[Brandweek]]
| date = June 24, 2009
| accessdateaccess-date = September 22, 2009
}}</ref>
* Test and Learnlearn is also methodically used online to test impact of various web page layouts on user behavior. Tools such as [[Google]] Website Analyzer provide every site with the ability to test various elements and their effect on user engagement.
*[[Pat McFadden]], the UK government's [[Chancellor of the Duchy of Lancaster]], announced on 9 December 2024 that the government would be using a "test and learn" approach to understanding and resolving [[public service]] design and delivery dilemmas.<ref>[[Cabinet Office]], [https://www.gov.uk/government/news/pat-mcfadden-vows-to-make-the-state-more-like-a-start-up-as-he-deploys-reform-teams-across-country Pat McFadden vows to make the state "more like a start up" as he deploys reform teams across country], published on 9 December 2024</ref>
 
==Test and learn software==
There are a variety of [[Computer software|software]] tools available today to support systematic testing within an organization. However, there is currently no one software package that covers all types of tests, and in some cases significant knowledge of statistics is still required for effective analysis.<ref name="hbr" />
 
Many companies, including [[Capital One]] and [[eBay]], have developed experiment management software within the company. Alternatives to internal solutions include broad [[statistical analysis]] software tools such as [[SAS Institute|SAS]] or test and learn focused software from [[Applied Predictive Technologies]], Trial Run, and MarketDial. There are also a variety of more narrowly tailored products, focused on particular problems or fields. In [[web analytics]], for example, firms such as [[Omniture]], [[WebTrends]], and [[Google]] have created specialized software.<ref name="hbr" />
 
== Criticism ==
Line 76 ⟶ 86:
 
== See also ==
 
* [[Clinical trials]]
* [[Scientific method]]
Line 86 ⟶ 95:
 
==References==
{{Reflist}}
 
{{reflist}}
 
{{DEFAULTSORT:Test And Learn}}
Retrieved from "https://en.wikipedia.org/wiki/Test_and_learn"