Mandelbrot Set Example

This code does a very straightforward calculation of the Mandelbrot set. There's nothing special about this test, other than the fact that it's calculation-heavy, so makes a nice semi-realistic speed test. Of course, as with all benchmarks, this should be taken for what it is: a single algorithm on a single test machine, tested with some care, but not perfectly. Fundamentally, it is a flawed benchmark, like every other benchmark.

Everything was run with the reference Python implementation (CPython) 2.7.3 on 64-bit Linux unless otherwise noted. In the code, I have ignored the built-in complex numbers type: I wanted to test straightforward numerical calculations, not the possibly-less-optimized complex type.

Here are the files used in the example:

• main.py: main function to call any of these (change the import line to select the file you want to work with)
• dynamic.py: traditional Python implementation
• static.pyx: statically-typed Cython implementation
• setup.py: build script for Cython implementation.

Cython Version

The “static” implementation is based on Cython (cython in Ubuntu). It is a very-Python-like language that adds optional static typing. This version, static.pyx, is the same as dynamic.py with static type declarations added to the arguments and variables in the escapes function.

With the setup.py file, this module can be built with the command:

python setup.py build_ext --inplace

Since Cython can also compile (most) Python code, I also ran the original dynamic.py version after compiling with Cython.

Other Runs

Having a somewhat arbitrary but functional benchmark sitting around, I was curious about other Python implementations, so I tested those as well.

Python 3 is the next version of Python. It contains some backwards-incompatible changes, so adoption has been slow. (In this case, only the xrange function had to be changed to range).

Jython compiles Python to the JVM (and allows integration with Java classes).

IronPython compiles Python to the .NET CLR. For IronPython, the comparison was made on a Windows 7 machine running Python for Windows.

PyPy is a reimplementation of the Python compiler/interpreter in Python itself. It includes a JIT compiler among other tricks.

Other Languages

And since I had done all of that, who could resist a inter-language shootout? I duplicated the calculations from the Python program as closely as I could (including continuing to ignore any complex number types).

I reimplemented the program in Haskell, C, Java, Javascript, and Ruby, with code and versions as below.

Results

(You can click on column headings in the table to sort.)

• Even though it was the original point of the example, I'm surprised how much of a difference the static typing in Cython made. Apparently this isn't unusual for numeric code (which this example is). Cython must be producing some nice C code to stack up to the native C implementation. (In fact, Cython has one more trick up its sleeve: it will produce an HTML file telling you exactly what C code it generated (and for the dynamic version). Double-click lines to expand.)
• This is very much a best case for the statically-typed options (or worst case for the Python reference implementation, depending on your point of view). Numeric code like this is where static binding makes the biggest difference. Most code's speedup will be more modest.
• There was a lot of variability in the speed of the “other” Python implementations.
• PyPy is an astonishing thing. Compare with in their own speed tests. PyPy seems to consistently give a several-times speedup over CPython.
• The JITs in modern web browsers are also amazing.
• Other languages are fun too.

Weird Stuff

Since I have all the code around, I seem to have the compulsion to keep trying it with other tools. None of these are serious “benchmarks”, but they're interesting anyway.