|
The project consists of:
* A set of very simple algorithmic problems (thirteen in total)<ref>{{Cite book |last1=Couto |first1=Marco |last2=Pereira |first2=Rui |last3=Ribeiro |first3=Francisco |last4=Rua |first4=Rui |last5=Saraiva |first5=João |chapter=Towards a Green Ranking for Programming Languages |date=2017-09-21 |title=Proceedings of the 21st Brazilian Symposium on Programming Languages |chapter-url=https://dl.acm.org/doi/abs/10.1145/3125374.3125382 |series=SBLP '17 |___location=New York, NY, USA |publisher=Association for Computing Machinery |pages=1–8 |doi=10.1145/3125374.3125382 |isbn=978-1-4503-5389-2|hdl=1822/65360 |hdl-access=free }}</ref>
* A set of very simple algorithmic problems
* Various implementations to the above problems in various programming languages
* A set of unit tests to verify that the submitted implementations solve the problem statement
==Supported languages==
Due to resource constraints, only a small subset of common programming languages are supported, up to the discretion of the game's operator.<ref>{{cite web|url=https://benchmarksgame-team.pages.debian.net/benchmarksgame/|title=The Computer Language Benchmarks Game|website=benchmarksgame-team.pages.debian.net/benchmarksgame|accessdate=29 May 2018}}</ref>
{{Collapsible list
| title = List of supported languages|Ada|C|Chapel
|[[Ada (programming language)|Ada]]
| 5 = C#
|[[C (programming_language)|C]]
| 6 = C++
|[[Chapel (programming language)|Chapel]]
| 7 = Dart
| 5 = [[C Sharp (programming language)|C#]]
| 8 = Erlang
| 96 = F#[[C++]]
| 7 = [[Dart (programming language)|Dart]]
| 10 = Fortran
| 8 = [[Erlang (programming language)|Erlang]]
| 11 = Go
| 9 = [[F Sharp (programming language)|F#]]
| 12 = Haskell
| 1310 = Java[[Fortran]]
| 11 = [[Go (programming language)|Go]]
| 14 = JavaScript
| 1512 = Julia[[Haskell]]
| 13 = [[Java (programming language)|Java]]
| 16 = Lisp
| 1714 = Lua[[JavaScript]]
| 15 = [[Julia (programming language)|Julia]]
| 18 = OCaml
| 16 = [[Lisp (programming language)|Lisp]]
| 19 = Pascal
| 17 = [[Lua (programming language)|Lua]]
| 20 = Perl
| 2118 = PHP[[OCaml]]
| 19 = [[Pascal (programming language)|Pascal]]
| 22 = Python
| 2320 = Racket[[Perl]]
| 2421 = Ruby[[PHP]]
| 22 = [[Python (programming language)|Python]]
| 26 = Rust
| 23 = [[Racket (programming language)|Racket]]
| 24 = [[Ruby (programming language)|Ruby]]
| 29 = Swift
| 26 = [[Rust (programming language)|Rust]]
| 29 = [[Swift (programming language)|Swift]]
}}
* overall user [[Run_time_(program_lifecycle_phase)|runtime]]
* peak [[memory allocation]]
* [[gzip]]'pedgzipped size of the solution's source code
* sum of total CPU time over all [[Thread (computing)|threads]]
* individual CPU [[Load (computing)|utilization]]
==Impact==
{{Excessive citations|date=May 2018}}
The benchmark results have uncovered various compiler issues. Sometimes a given compiler failed to process unusual, but otherwise grammatically valid constructs. At other times, runtime performance was shown to be below expectations, which prompted compiler developers to revise their optimization capabilities.
Various research articles have been based on the benchmarks, its results and its methodology.<ref>
{{cite journal|author1=Kevin Williams|author2=Jason McCandless|author3=David Gregg|title=Dynamic Interpretation for Dynamic Scripting Languages|date=2009|url=https://www.scss.tcd.ie/publications/tech-reports/reports.09/TCD-CS-2009-37.pdf|accessdate=25 March 2017}}</ref><ref>
<ref>{{cite conference|author1=Tobias Wrigstad|author2=Francesco Zappa Nardelli|author3=Sylvain Lebresne Johan|author4=Ostlund Jan Vitek|title=Integrating Typed and Untyped Code in a Scripting Language|date=January 17–23, 2009|conference=POPL’10|url=https://www.di.ens.fr/~zappa/projects/liketypes/paper.pdf|accessdate=25 March 2017|___location=Madrid, Spain}}</ref><ref>
<ref>{{cite conference|last1=Lerche|first1=Carl|title=Write Fast Ruby: It's All About the Science|conference=Golden Gate Ruby Conference|date=April 17–18, 2009|url=http://2009.gogaruco.com/downloads/Wrap2009.pdf|accessdate=25 March 2017|___location=San Francisco, California}}</ref><ref>{{cite conference|author1=J. Shirako|author2=D. M. Peixotto|author3=V. Sarkar|author4=W. N. Scherer III|title=Phaser Accumulators: a New Reduction Construct for Dynamic Parallelism|date=2009|conference=IEEE International Symposium on Parallel & Distributed Processing|url=http://www.cs.rice.edu/~vs3/PDF/ipdps09-accumulators-final-submission.pdf|accessdate=25 March 2017}}</ref><ref>
<ref>{{cite conference|author1=J. Shirako|author2=D. M. Peixotto|author3=V. Sarkar|author4=W. N. Scherer III|title=Phaser Accumulators: a New Reduction Construct for Dynamic Parallelism|date=2009|conference=IEEE International Symposium on Parallel & Distributed Processing|url=http://www.cs.rice.edu/~vs3/PDF/ipdps09-accumulators-final-submission.pdf|accessdate=25 March 2017}}</ref>
author = Rajesh Karmani and Amin Shali and Gul Agha |
title = Actor frameworks for the JVM platform: A Comparative Analysis |
url = http://osl.cs.illinois.edu/docs/pppj09/paper.pdf |
accessdate = 26 March 2017
}}</ref><ref>
<ref>{{cite conference|author=Brunthaler Stefan|title=Inline Caching Meets Quickening|date=2010|volume=Object-Oriented Programming|conference=European Conference on Object-Oriented Programming (ECOOP)|pages=429–451|doi=10.1007/978-3-642-14107-2_21}}</ref><ref>
<ref>{{cite conference|author1=Prodromos Gerakios|author2=Nikolaos Papaspyrou|author3=Konstantinos Sagonas|title=Race-free and Memory-safe Multithreading: Design and Implementation in Cyclone|date=January 23, 2010|conference=Proceedings of the 5th ACM SIGPLAN workshop on Types in language design and implementation|pages=15–26|url=http://www.softlab.ntua.gr/research/techrep/CSD-SW-TR-8-09.pdf|accessdate=25 March 2017|___location=Madrid, Spain}}</ref><ref>
<ref>{{cite conference|author1=Slava Pestov|author2=Daniel Ehrenberg|author3=Joe Groff|title=Factor: A Dynamic Stack-based Programming Language|date=October 18, 2010|conference=DLS 2010|url=http://factorcode.org/littledan/dls.pdf|accessdate=25 March 2017|___location=Reno/Tahoe, Nevada, USA}}</ref><ref>
<ref>{{cite conference|author1=Andrei Homescu|author2=Alex Suhan|title=HappyJIT: A Tracing JIT Compiler for PHP|date=October 24, 2011|conference=DLS’11|url=https://www.ics.uci.edu/~ahomescu/happyjit_paper.pdf|accessdate=25 March 2017|___location=Portland, Oregon, USA}}</ref><ref>
<ref>{{cite conference|author1=Vincent St-Amour|author2=Sam Tobin-Hochstadt|author3=Matthias Felleisen|title=Optimization Coaching – Optimizers Learn to Communicate with Programmers|date=October 19–26, 2012|conference=OOPSLA’12|url=http://www.ccs.neu.edu/racket/pubs/oopsla12-stf.pdf|accessdate=25 March 2017|___location=Tucson, Arizona, USA}}</ref><ref>
<ref>{{cite conference|author1=Wing Hang Li|author2=David R. White|author3=Jeremy Singer|title=JVM-Hosted Languages: They Talk the Talk, but do they Walk the Walk?|date=September 11–13, 2013|conference=Proceedings of the 2013 International Conference on Principles and Practices of Programming on the Java Platform: Virtual Machines, Languages, and Tools|pages=101–112|url=http://www.dcs.gla.ac.uk/~wingli/jvm_language_study/jvmlanguages.pdf|accessdate=25 March 2017|___location=Stuttgart, Germany}}</ref><ref>
<ref>{{cite conference|author1=Aibek Sarimbekov|author2=Andrej Podzimek|author3=Lubomir Bulej|author4=Yudi Zheng|author5=Nathan Ricci|author6=Walter Binder|title=Characteristics of Dynamic JVM Languages|date=October 28, 2013|conference=VMIL ’13|url=http://d3s.mff.cuni.cz/publications/download/Sarimbekov-vmil13.pdf|accessdate=25 March 2017|___location=Indianapolis, Indiana, USA}}</ref><ref>
<ref>{{cite conference|author1=Bradford L. Chamberlain|author2=Ben Albrecht|author3=Lydia Duncan|author4=Ben Harshbarger|title=Entering the Fray: Chapel's Computer Language Benchmark Game Entry|date=2017|url=http://chapel.cray.com/CHIUW/2017/chamberlain-abstract.pdf|accessdate=25 March 2017}}</ref>{{Excessive citations inline|date=April 2023}}
==See also==
| 