The Great Computer Language Shootout

Back to the Language Shootout
Back to Doug's Homepage

[NEWS] [Editorial] [FAQ] [Methodology] [Performance Tips] [Download] [Activity Log] [Acknowledgements] [Scorecard] [Slash-Holed] [Rules for Benchmark Writers] [Conclusion] [Todo]

Hi, the shootout is an unfinished project. I've decided to discontinue updates to it for now while I work on some other things. Thanks for everyone's help.

[ 30 Language Implementations, 25 Benchmark Tests, 750 Total Possible Programs, 632 Written ]

Here's a list of which solutions have/have not been implemented.

The Benchmark Tests
Ackermann's Function
Array Access
Count Lines/Words/Chars
Echo Client/Server
Exception Mechanisms
Fibonacci Numbers
Hash (Associative Array) Access
Hashes, Part II
Heapsort
Hello World
List Operations
Matrix Multiplication
Method Calls
Nested Loops
Object Instantiation
Producer/Consumer Threads
Random Number Generator
Regular Expression Matching
Reverse a File
Sieve of Eratosthenes
Spell Checker
Statistical Moments
String Concatenation
Sum a Column of Integers
Word Frequency Count
(Not all languages are tested in every benchmark)
Other Language Comparisons
Creating Binary Extensions
(These are non-performance language comparisons)

A benchmark comparison of a number of programming languages.

Note: If you want a copy of the shootout, please get it from the Download Page, but please do not hammer my server requesting every page. The shootout tarball is updated nightly. Thanks.

Intro

When I started this project, my goal was to compare all the major scripting languages. Then I started adding in some compiled languages for comparison ... and it's still growing with no end in sight (so be sure to read the NEWS). I'm doing it so that I can learn about new languages, compare them in various (possibly meaningless) ways, and most importantly, have some fun.

Someday, maybe, the results I present might even be meaningful, but please take the current results with a grain of salt. You might get different results on a different OS, on different hardware, with newer releases of the languages, or even from run to run of the same test. You might even find that I have horrible bugs in my testing method.

This is very much a work in progress, as it evolves I may add, change or remove languages, tests, or solutions. Some solutions as currently presented are unoptimized, and may be optimized in the future (if I can do it myself or if someone contributes a better solution).

Disclaimer No. 1: I'm just a beginner in many of these languages, so if you can help me improve any of the solutions, please drop me an email. Thanks.

Disclaimer No. 2: These pages are provided for novelty purposes only. Any other use voids the manufacturer's warranty. Do not mix with alchohol. Some contents may consist of recycled materials.

Disclaimer No. 3: ditto.

Disclaimer No. 4: Please read the pages on Methodology, the FAQ, and my Conclusion before you flame.

By the way, the word Great in the title refers to quantity, not quality (I will let the reader judge that). I saw a need for a more comprehensive language comparison than what I could find out on the Net, and you are reading my solution. I wanted to see a comparison of more languages doing more tests, and with (hopefully) the participation of more people.

Aldo Calpini has put a huge amount of work into porting my shootout to Microsoft Windows. He even includes some new languages and some commercial compilers that run on Windows. Please click here to check it out. (Please note that there may be some differences in his port. It is really a separate, derivative work). Many thanks to Aldo!

The Languages

Language Imple-
mentation
(local summary) Version
(Official Homepage)

1. Awk gawk GNU Awk 3.0.6

2. Awk mawk mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan

3. Bash bash GNU bash, version 2.05.0(1)

4. C gcc 3.0

5. C g 3.0

6. Common Lisp cmucl CMU Common Lisp 18c

7. Eiffel se Release -0.76 (Saturday 11th november 2000)

8. Emacs Lisp xemacs XEmacs 21.5 (beta1) "anise" [Lucid] (i686-pc-linux)

9. Erlang erlang Erlang (BEAM) emulator version 5.0.2

10. Forth gforth gforth 0.5.0

11. Forth bigforth bigFORTH 386-Linux rev. 2.0.2

12. Haskell ghc The Glorious Glasgow Haskell Compilation System, version 5.00.1

13. Icon icon Icon Version 9.3.2. July 1, 1999

14. Java java Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.1-b24)

15. Javascript njs 0.2.5

16. Lisp rep rep version 0.15.2

17. Lua lua Lua 4.0

18. Mercury mercury Mercury 0.10.1

19. Ocaml ocaml The Objective Caml native-code compiler, version 3.04

20. Ocaml ocamlb The Objective Caml compiler, version 3.04 (Byte Code)

21. PHP php PHP 4.0.5

22. Perl perl This is perl, v5.6.1 built for i686-linux

23. Pike pike Pike v7.2 release 30 (configured: --with-double-precision).

24. Python python Python 2.1.0

25. Ruby ruby ruby 1.6.6 (2001-12-26) [i686-linux]

26. SML mlton MLton 20010706

27. SML smlnj Standard ML of New Jersey, Version 110.0.7, September 28, 2000 -

28. Scheme bigloo Bigloo (2.2b)

29. Scheme stalin 0.9

30. Scheme guile Guile 1.4

31. Tcl tcl Tcl 8.3.3

Languages that compile to native code are in Bold Italics.

About the Languages

The languages compared here are a mixture of compiled and interpreted, functional and imperative. Compiled languages have the natural advantage of running machine code that can be optimized by the compiler. The interpreted languages are often byte-compiled, and sometimes optimized. Many of the tests were designed to really test imperative features, and do not fairly test some languages (like Haskell). If you want to compare languages of the same type, consider:

The languages that are in bold italics compile to machine code. The others are either byte-compiled or just interpreted.
When using C and C , I think it is normal and expected to use external libraries. In this vein, we'll often use STL v3.3 with C .
However, with most other languages we will try to use only features in the language's core or its standard libraries.
There is also a very handy library for doing Perl Compatible Regular Expressions: PCRE (see the Regular Expression Matching test), which I will use with some languages.

Bigloo Scheme, CMU Common Lisp, SmallEiffel, GHC (Haskell), Mercury, Ocaml, and SML/NJ compile to native code. Some are actually compiled to intermediate C first.

Sun's Java uses a JIT compiler so the code is compiled to native code on the fly.

Guile, Lua, Perl, Python, Rep, Ruby and Tcl are all interpreted scripting / extension languages, and they can probably all be considered competitors in the same space. Perl, Python, Ruby and Tcl all come with extensive libraries in their standard distributions. Guile and Rep have less extensive libraries, and Lua is absolutely tiny, with no extra libraries.

Pike and Icon are not strictly extension languages but are similar in many ways to the previous group.

Erlang also compiles to bytecodes, but it isn't really thought of as a scripting language.

AWK is really just a scripting language, it's not typically embedded or extended with C or other compiled languages.

Gforth is an interpreted forth, and it can produce extremely fast code. But Forth is a pretty low-level language, and optimization usually left up to the programmer.

I include Bash just for fun, and often resort to the standard bash programming technique of calling the usual Unix shell commands.

Download

You can now download the entire shootout as a compressed tarball from the download page. The current distribution is about 1.5MB and it is approximately alpha quality (it is probably suitable only for the adventurous). The tarball is now updated nightly. I will try to keep the News up-to-date to explain the new stuff.