Creative Computing Benchmark: Difference between revisions

Revision as of 13:39, 3 June 2019

Latest revision as of 04:27, 9 June 2025

(24 intermediate revisions by 11 users not shown)

Line 1:

{{Short description|Type of computer benchmark}}

The '''Creative Computing Benchmark''', also referred to ascalled '''David Ahl's benchmarkSimple Benchmark''', wasis a [[Benchmark (computing)|computer benchmark]] that was used to compare the performance of the [[BASIC]] programming language]] on various machines. It was first introduced in the November 1983 issue of ''[[Creative Computing (magazine)|Creative Computing]]'' magazine with the measures from a number of [[8-bit|8-bit computercomputers]]s that were popular at the time. Over a period of a few months, the list was greatly expanded to include practically every contemporary machine, topped by the [[Cray-1]] supercomputer, which ran it in 0.01 seconds.{{efn|Cray did not produce a BASIC for the Cray-1, so it is not clear in what language this version was written in.<ref>{{citeCite book |url=http://bitsavers.trailing-edge.com/pdf/cray/COS/T-0103C-CRAY_1_Computer_System-Operating_System_COS_Workbook-Training-September_1981.OCR.pdf |title=CRAY-1 Computer Operating System Handbook |date=September 1981 |publisher=Cray Computer}}</ref>}}

The Creative Computing Benchmark was one of three common benchmarks of the era. Its primary competition in the early 1980s in the United States was the [[Byte Sieve]], of September 1981, while the earlier [[Rugg/Feldman benchmarks]] of June 1977 were not as well known in the United States, but were widely used in the United Kingdom.

==History==

The benchmark first appeared in the November 1983 issue of ''[[Creative Computing]]'' under the title "Benchmark Comparison Test".{{sfn|Ahl|1983|p=259}} In the article, Ahl was careful to state that it tested only a few aspects of the BASIC language, mostly its looping performance. He stated:

== History ==

{{cquote|...the benchmark program presented here is not representative of the way computers are actually used; it measures only a few aspects of performance, and no one should buy a computer based solely on the results of these measures. Yet, the results provide some interesting comparative data.{{sfn|Ahl|1983|p=259}}}}

[[File:Creative_Computing_Benchmark_original_result_set.png|thumb|right|Original result set from November 1983 issue of Creative Computing]]

The benchmark first appeared in the November 1983 issue of ''[[Creative Computing]]'' under the title "Benchmark Comparison Test".{{sfn|Ahl|1983|p=259}} In the article, author [[David H. Ahl]] was careful to state that it tested only a few aspects of the BASIC language, mostly its looping performance. He stated:

The initial results were provided for common machines of the era, including the [[Apple II]], [[Commodore 64]] and the recently-released [[IBM PC]]. Most of these machines ran some variation of the stock [[Microsoft BASIC]] and thus provided similar times on the order of two minutes, while the [[16-bit]] PC was near the top of the list at only 24 seconds. the fastest machine in this initial suite was the [[Olivetti M20]] at 13 seconds, and the slowest was [[Atari BASIC]] on the [[Atari 800]] at 6 minutes 58 seconds.{{sfn|Ahl|1983|p=260}}

{{cquote|...&nbsp;the benchmark program presented here is not representative of the way computers are actually used; it measures only a few aspects of performance, and no one should buy a computer based solely on the results of these measures. Yet, the results provide some interesting comparative data.{{sfn|Ahl|1983|p=259}}}}

In the months following its publication, the magazine was inundated with results for other platforms. It became a regular feature for a time, placed prominently near the front of the magazine with an ever-growing list of results. By March the fastest machine on the list was the [[Cray-1]] at 0.01 seconds, and the slowest was the [[TI SR-50]] [[programmable calculator]] at 12.7 days.{{sfn|Ahl|1984|p=7}}

The initial results were provided for common machines of the era, including the [[Apple II]], [[Commodore 64]] and the recently- released [[IBM PCPersonal Computer]]. Most of these machines ran some variation of the stock [[Microsoft BASIC]] and thus provided similar times on the order of two minutes, while the [[16-bit]] PC was near the top of the list at only 24 seconds. the fastest machine in this initial suite was the [[Olivetti M20]] at 13 seconds, and the slowest was [[Atari BASIC]] on the [[Atari 8008-bit computers]] at 6 minutes 58 seconds.{{sfn|Ahl|1983|p=260}}

The benchmark had several problems that made it less useful for general purposes. For instance, the system did not test any string manipulation, who's performance varied widely across platforms. It also did not take advantage of any "speedups" available on different platforms, like the possible use of integer variables for loop indexes or turning off video access on machines with shared main memory.{{efn|Most 8-bit machines of the era had a single bank of RAM that was shared between the CPU and display driver, which led to bus contention issues that slowed performance as much as 30%. Turning off the display was a common way to improve compute-bound programs like this benchmark.{{sfn|Wilkinson|1985|p=140}}}} These limitations were widely debated at the time.{{sfn|Wilkinson|1985|p=139}}

In the months following its publication, the magazine was inundated with results for other platforms. It became a regular feature for a time, placed prominently near the front of the magazine with an ever-growing list of results. By March the fastest machine on the list was the [[Cray-1]] at 0.01 seconds, and the slowest was the [[TI SR-50]] [[programmable calculator]] at 12.7 days.{{sfn|Ahl|1984|p=7}}

Its last appearance is in the May 1984 issue, which included values for 183 machines. This issue included a note that the many criticisms of the system had been taken to heart and a new benchmark program was under design.<ref>{{cite magazine |magazine=Creative Computing |date=May 1984 |page=6 |first=David |last=Ahl |title=Creative Computing Benchmark |url=https://archive.org/details/creativecomputing-1984-05/page/n9}}</ref> However, such a program never appeared in the magazine, which increasingly focused on the business market that was coming to dominate [[personal computer]]s in that era.

The benchmark had several problems that made it less useful for general purposes. For instance, the system did not test any string manipulation, who'swhose performance varied widely across platforms. It also did not take advantage of any "speedups" available on different platforms, like the possible use of integer variables for loop indexes or turning off video access on machines with shared main memory.{{efn|Most 8-bit machines of the era had a single bank of RAM that was shared between the CPU and display driver, which led to [[bus contention]] issues that slowed performance as much as 30%. Turning off the display was a common way to improve compute-bound programs like this benchmark.{{sfn|Wilkinson|1985|p=140}}}} These limitations were widely debated at the time.{{sfn|Wilkinson|1985|p=139}} The November 1983 article stipulated using an "accurate stopwatch" to time the program execution on machines lacking a real-time clock: When applied to the faster machines, this would yield test results highly dependent on the reaction time of the individual operating the stopwatch.

The benchmark continued to be used as a general-purpose tool after this date,<ref> {{cite magazine |magazine=Creative Computing |date=May 1985 |first=Glenn |last=Hart |pages=46, 48, 51-52 |title=Microsoft BASIC 2.0 for the Mac |url=https://archive.org/stream/creativecomputing-1985-05/Creative_Computing_v11_n05_1985_May#page/n55}}</ref> but as the importance of BASIC dwindled it became less common.

Its last appearance is in the May 1984 issue, which included values for 183 machines. This issue included a note that the many criticisms of the system had been taken to heart and a new benchmark program was under design.{{sfn|Ahl|1984|p=6}} However, such a program never appeared in the magazine. In the September 1985 issue, David Ahl responded to a Letter to the Editor about the new benchmark program with "Several analysts spent many hours working out three new benchmark tests ... none gave different or better results".<ref>{{citeCite magazine |magazine=Creative Computing |date=MaySeptember 19841985 |page=6 |first=David |last=Ahl |title=Creative Computing BenchmarkInput/Output |url=https://archive.org/details/creativecomputing-19841985-0509/page/n9n7}}</ref> However, such a program never appeared in the magazine, which increasingly focused on the business market that was coming to dominate [[personal computer]]s in that era.

==Code==

The following is from later versions of the benchmark code, which reduced the number of compound statements on a line.{{sfn|Ahl|1984|p=7}}

The benchmark continued to be used as a general-purpose tool after this date,<ref> {{citeCite magazine |magazine=Creative Computing |date=May 1985 |first=Glenn |last=Hart |pages=46, 48, 51-52 |title=Microsoft BASIC 2.0 for the Mac |url=https://archive.org/stream/creativecomputing-1985-05/Creative_Computing_v11_n05_1985_May#page/n55}}</ref> but as the importance of BASIC dwindled it became less common.

10 ' Ahl's Simple Benchmark

20 FOR N=1 TO 100: A=N

==Code==

30 FOR I=1 TO 10

This is the original version from the November 1983 edition:{{sfn|Ahl|1983|p=259}}

40 A=SQR(A): R=R+RND(1)

{{sxhl|2=basic|1=<nowiki/>

50 NEXT I

1 ' Ahl's simple benchmark test

60 FOR I=1 TO 10

2 ' In Lines 30 and 40, some computers

70 A=A^2: R=R+RND(1)

3 ' may require RND(1) for correct results

80 NEXT I

10 PRINT "Accuracy Random"

90 S=S+A: NEXT N

20 FOR N=1 TO 100:A=N

100 PRINT ABS(1010-S/5)

30 FOR I=1 TO 10:A=SQR(A):R=R+RND(0):NEXT I

110 PRINT ABS(1000-R)

40 FOR I=1 TO 10:A=A^2:R=R+RND(0):NEXT I

50 S=S+A:NEXT N

60 PRINT ABS(1010-S/5);ABS(1000-R)

}}

The following is from later versions of the benchmark code, which reduced the number of compound statements on a line.:{{sfn|Ahl|1984|p=7}}{{efn|Likely to reduce the line length below 40 characters for all lines. In the original version, line 30 is 43 characters long.}}

{{sxhl|2=basic|1=<nowiki/>

10 ' Ahl's Simple Benchmark

20 FOR N=1 TO 100: A=N

30 FOR I=1 TO 10

40 A=SQR(A): R=R+RND(1)

50 NEXT I

60 FOR I=1 TO 10

70 A=A^2: R=R+RND(1)

80 NEXT I

90 S=S+A: NEXT N

100 PRINT ABS(1010-S/5)

110 PRINT ABS(1000-R)

}}

==Notes==

{{notelistNotelist}}

==References==

===Citations===

{{reflist|30em}}

===Bibliography===

* {{citeCite magazine |magazine=Creative Computing |date=November 1983 |pages=259-260259–260 |first=David |last=Ahl |title=Benchmark Comparison Test |url=https://archive.org/details/creativecomputing-1983-11/page/n269 |ref=harv}}

* {{citeCite magazine |magazine=Creative Computing |date=March 1984 |pages=7 |first=David |last=Ahl |title=Creative Computing Benchmark |url=https://archive.org/details/creativecomputing-1984-03/page/n7 |refpage=harv6}}

* {{citeCite magazine |magazine=Compute! |date=February 1985 |pages=139-142139–142 |title=INSIGHT: Atari |first=Bill |last=Wilkinson |url=https://archive.org/stream/1985-02-compute-magazine/Compute_Issue_057_1985_Feb#page/n139/ |ref=harv}}

[[Category:Benchmarks (computing)]]