[Gems-users] MSI_MOSI_CMP_directory Cache Miss Numbers

[Joseph Greathouse <jlgreath@xxxxxxxxx>]

Hi there,

We are using GEMS 1.4 and Simics 3.0.27 to simulate a SPARC Solaris 9
Serengeti/Abisko machine.  We're attempting to keep track of L2 cache
misses.

We've run into a problem, though, in that our numbers stop making sense
after 4 processors.  Our testbench and problem is as follows:

We have a simple multithreaded test application.  It initializes a 64MB
array, then walks the array and updates each element to some value.  We
parallelized it with pthreads, and run it with 1, 2, 4, 8 and 16
threads.  We set g_NUM_PROCESSORS and g_PROCS_PER_CHIP to the number of
threads when we initialize Ruby, and we leave all other parameters
alone.  (The L2 cache size defaults to 16MB, banked differently
depending on the number of cores)

For 1, 2, and 4 threads, over the course of this program, we have about
the same number of L2 misses: about 3,400,000.  However, when we
increase it to 8 threads, the number decreases to around 1,700,000.  At
16 threads, the number of L2 misses is about 1,000,000.  We're not sure
why the number of L2 misses would change when the L2 is the same size in
all cases, and the program is moving the same amount of data.

Is there something we have configured incorrectly that would cause this
to happen?  The increased amount of L1D shouldn't matter, I would think,
since MSI_MOSI_CMP_directory is supposed to be inclusive.

Any help you could offer would be great.

Thanks!
Joe Greathouse