Benchmark (computing): Difference between revisions

Revision as of 06:16, 1 June 2025

Revision as of 06:19, 1 June 2025

Line 26:

Given the large number of benchmarks available, a vendor can usually find at least one benchmark that shows its system will outperform another system; the other systems can be shown to excel with a different benchmark.

Software vendors also use benchmarks in their marketing, such as the "benchmark wars" between rival [[relational database]] makers in the 1980s and 1990s. Companies commonly report only those benchmarks (or aspects of benchmarks) that show their products in the best light. They also have been known to mis-represent the significance of benchmarks, again to show their products in the best possible light. Software companies also use benchmarks in their marketing, such as the "benchmark wars" between rival [[relational database]] makers in the 1980s and 1990s.<ref name="rdbmsinformix20070612">{{Cite interview |interviewer=Luanne Johnson |title=RDBMS Workshop: Informix |url=https://archive.computerhistory.org/resources/access/text/2013/05/102702566-05-01-acc.pdf |access-date=2025-05-30 |publisher=Computer History Museum |date=2007-06-12}}</ref><ref name="rdbmsingressybase20070613">{{Cite interview |interviewer=Doug Jerger |title=RDBMS Workshop: Ingres and Sybase |url=https://archive.computerhistory.org/resources/access/text/2013/05/102702565-05-01-acc.pdf |access-date=2025-05-30 |publisher=Computer History Museum |date=2007-06-13}}</ref>

Ideally benchmarks should only substitute for real applications if the application is unavailable, or too difficult or costly to port to a specific processor or computer system. If performance is critical, the only benchmark that matters is the target environment's application suite.