Benchmarks and Tools

Benchmarks

SPEC CPU 2000 (int and fp)

You can grab a copy of that benchmark at /cs/fac1/fedorova/Research/Benchmarks/SPECCPU2000.

Documentation describing how to build and run this benchmark suite can be found at the SPEC web site.

You will need to install the benchmarks in your home directory and build them for the platform where you plan to run them.

There benchmarks can be run via a startup script (included with the benchmark) or "by hand", i.e., starting the programs directly as opposed to via a script. If you run the benchmarks on a simulator, you'd want to run them by hand: this cuts simulation time and gives you a better control over your simulation.


TPC-C

SFU's systems group has their own implementation of the TPC-C benchmark on top of Berkeley DB. It is called TpccOverBkDB and currenlty runs only on 64-bit Solaris/SPARC platforms. It can be easily ported to other Unix platforms. Download the benchmark and documentation here.


Parallel benchmarks from UCSD

Brad Calder's research group at UCSD has shared with us their fine-grained parallel benchmarks. (Thanks to John Sampson for packaging the code). Talk to Sasha Fedorova if you'd like to use these benchmarks.


Olden benchmarks

If you are looking for other fine-grained parallel benchmarks, you could consider Olden benchmarks.


 
Tools

Performance monitoring on Solaris machines

You can monitor performance on Solaris using two methods:
  • cputrack - a performance monitoring tool that runs outside your process
  • via instrumentation of your benchmark program
Cputrack has a small performance overhead (I measured about 3% on Sparc T2000 when samples were taken every second).

The overhead can be avoided if you instrument your benchmark program to measure performance via cpc library. (Click here for an example code counting data cache misses. To compile this code, type: "cc -o trycpc trycpc.c -lcpc". This will work on Solaris10/Niagara only.

To further reduce the overhead, you can read and write performance counter values directly using assembly instructions such as "rd %pic" and "wr %pic".

Binding threads/processes to a CPU

You can bind your process or threads within the process onto a particular CPU (or thread context) using the following methods:
  • bind using an external pbind utility that takes process id as an argument
  • bind from within your program using processor_bind system call. Check out a code example here. To compile the program use Sun Studio cc compiler.
  • use an external program runbind (written by Sasha Fedorova) that launches your program after binding it to the specified CPU.

    You can grab a program for Solaris/Sparc from /cs/systems/Tools/Solaris-SPARC/Bind/Runbind/runbind

    To see how to run this program run runbind -h.

    In the same directory as the program there is an example program hello_world. You can try
    runbinding a single or multple instances of hello_world like this:

    ./runbind -p0 ./hello_world 1 2 3 -p1 ./hello_world 4 5 6

    You will see that each program prints its arguments and the CPU id to which it is bound.

    In an event you want to runbind a program whose arguments contain "-p", you need to use "runbind-one-command" program. You will be able to runbind one program at a time.
  • Note on Niagara:  to bind threads to a specific core, you need to use four consecutive CPU id's starting from a core ID. A core ID is any id evenly divisible by 4. For example, to bind four threads to core 0, you will use CPU ids 0, 1, 2, 3.