This code does a very straightforward calculation of the Mandelbrot set. There's nothing special about this test, other than the fact that it's calculation-heavy, so makes a nice semi-realistic speed test. Of course, as with all benchmarks, this should be taken for what it is: a single algorithm on a single test machine, tested with some care, but not perfectly. Fundamentally, it is a flawed benchmark, like every other benchmark.
Everything was run with the reference Python implementation (CPython) 2.7.3 on 64-bit Linux unless otherwise noted. In the code, I have ignored the built-in complex numbers type: I wanted to test straightforward numerical calculations, not the possibly-less-optimized complex type.
Here are the files used in the example:
main.py: main function to call any of these (change the
importline to select the file you want to work with)
dynamic.py: traditional Python implementation
static.pyx: statically-typed Cython implementation
setup.py: build script for Cython implementation.
The “static” implementation is based on Cython (
cython in Ubuntu). It is a very-Python-like language that adds optional static typing. This version,
static.pyx, is the same as
dynamic.py with static type declarations added to the arguments and variables in the
setup.py file, this module can be built with the command:
python setup.py build_ext --inplace
Since Cython can also compile (most) Python code, I also ran the original
dynamic.py version after compiling with Cython.
Having a somewhat arbitrary but functional benchmark sitting around, I was curious about other Python implementations, so I tested those as well.
Python 3 is the next version of Python. It contains some backwards-incompatible changes, so adoption has been slow. (In this case, only the
xrange function had to be changed to
PyPy is a reimplementation of the Python compiler/interpreter in Python itself. It includes a JIT compiler among other tricks.
And since I had done all of that, who could resist a inter-language shootout? I duplicated the calculations from the Python program as closely as I could (including continuing to ignore any complex number types).
(You can click on column headings in the table to sort.)
|Python||Python 2.7.3 with ||1.0|
|Haskell||GHC 7.4.1 with ||60.3|
|C||GCC 4.6.3 with ||109.9|
|C||Clang 3.0 with ||102.8|
|Java||GCJ 4.6.3 with ||109.5|
- Even though it was the original point of the example, I'm surprised how much of a difference the static typing in Cython made. Apparently this isn't unusual for numeric code (which this example is). Cython must be producing some nice C code to stack up to the native C implementation. (In fact, Cython has one more trick up its sleeve: it will produce an HTML file telling you exactly what C code it generated (and for the dynamic version). Double-click lines to expand.)
- This is very much a best case for the statically-typed options (or worst case for the Python reference implementation, depending on your point of view). Numeric code like this is where static binding makes the biggest difference. Most code's speedup will be more modest.
- There was a lot of variability in the speed of the “other” Python implementations.
- PyPy is an astonishing thing. Compare with in their own speed tests. PyPy seems to consistently give a several-times speedup over CPython.
- The JITs in modern web browsers are also amazing.
- Other languages are fun too.
Since I have all the code around, I seem to have the compulsion to keep trying it with other tools. None of these are serious “benchmarks”, but they're interesting anyway.
|Python||PyPy's rpython translator||105.4|
|C||PicoC 2.1 (compiled with ||0.13|
|Haskell||Hugs Sept 2006||0.04|