Assignment 3

In this assignment you will learn how to use performance asymmetric systems. As you recall, in a performance-asymmetric system some cores run faster than others.

For this assignment, we will have two systems set up for you to use. You should pick one system for the assignment.

  • octavia.cs.sfu.ca: on this machine, CPUs 0, 1, 2 and 3 will run at 2.0GHz, and CPUs 4, 5, 6, and 7 will run at 3.0GHz.
  • valencia.cs.sfu.ca: on this machine, CPUs 0-7 will run at 1.15GHz, and CPUs 8-15 will run at 2.3GHz.

Both systems are x86 machines running Solaris 11.

Your goal is to analyze how performance of different benchmark changes as the benchmark runs on a fast core vs. a slow core. Here are the steps you should follow:

  1. Choose the machine that you will use for experiments and make a reservation on it. When you prepare your final reports, you must run the benchmarks when nothing else is running on the machine, so for your final tests you will need to make an exclusive reservation.
  2. When you log on to the machine to do the assignment, CHECK THAT THE FREQUENCY IS SET AS SPECIFIED ABOVE. You can do so by running the command:
    kstat -m cpu_info -s current_clock_Hz
    If the frequencies are not set appropriately, notify the instructor immediately!
  3. Pick some benchmarks (from SPEC CPU2000 suite, or any others if you feel creative) that you will use in your experiments. You must pick at least two benchmarks. In the end, the benchmarks that you choose should show different behaviours as you run them on a fast vs. slow core. For example, one benchmark could experience a slowdown of a factor of two as it is moved from the fast core to the slow core on valencia, while another benchmark could experience a significantly smaller slowdown. In order to find two benchmarks with such different properties, you can pursue two approaches: (1) perform an exhaustive search over the whole benchmark suite -- i.e. run all of them on both cores and see how they respond, or (2) read this short paper, which will give you a clue how to pick the right benchmarks from the SPEC CPU2000 suite. Needless to say, the second approach will take considerably less time.
  4. Run each benchmark of your choice on the fast core and on the slow core. Run it at least three times on each core, compute the average and standard deviation of its completion time on each core type. To ensure that the benchmark runs on the right core type, use the runbind program that you used in Assignment 1. You should see that one benchmark slows down more than the other as you go from fast to slow core. (If you do not see that, refine your benchmark selection process, talk to your classmates if you need help).
  5. Write up a short but precise report of your results. Report all necessary details of your experimental setup -- you need to include details that would be necessary for me to replicate your experiments but not more. Report your data in the form (graphical or tabular) that best captures your findings. Do not report raw data. Explain why one of your benchmarks slows down more than the other when it runs on the slow core. The paper referenced above will be helpful in explaining the difference.
  6. If you want to surprise me and get an extra 5 points for the assignment, you can gather additional data that will explain the differences in the speedups between the benchmarks (hint: you'd have to use hardware performance counters similar to those you used in Assignment 1).

You must submit the final report describing your findings.