[Gems-users] Variability in STAMP's Labyrinth benchmark

[Ricardo Quislant del Barrio <quislant@xxxxxxxxx>]

Hi all,
I'm simulating STAMP's Labyrinth benchmark on GEMS 2.0 and Simics 2.2.19.
Below you can find some numbers I've gathered from Ruby and the parameters
I've been using:

   - Every simulation runs on top of a simulated Solaris 10 Sarek 16 
processor machine.
   - labyrinth input: -t15 -i inputs/random-x32-y32-z3-n16.txt

Following numbers were collected using "logtm_se_virtualization".
Random seed has been set to 213 for the next runs:

RD/WR filter                                       Ruby_cycles
---------------------------------------------------------------------------------
Perfect_ ------------------------------------------>  202691066
H3_2048_4_Regular ----------------------->  256785454
H3_2048_4_Parallel  ----------------------->  265880564

Random seed has been set to 56 for the next runs:

RD/WR filter                                       Ruby_cycles
---------------------------------------------------------------------------------
Perfect_ ------------------------------------------>  172010695
H3_2048_4_Regular ----------------------->  176020867
H3_2048_4_Parallel  ----------------------->  130056902
H3_1024_4_Parallel  ----------------------->    45808646

Now, when changing "logtm_se_virtualization" to 
"EagerCD_EagerVM_Hybrid_Pred":
Random seed has been set to 213 for the next runs:

RD/WR filter                                       Ruby_cycles
---------------------------------------------------------------------------------
Perfect_ ------------------------------------------>    17991611
H3_2048_4_Regular ----------------------->  Never ending
H3_2048_4_Parallel  ----------------------->   18236939


Random seed has been set to 56 for the next runs:

RD/WR filter                                       Ruby_cycles
---------------------------------------------------------------------------------
Perfect_ ------------------------------------------>    18146557
H3_2048_4_Regular ----------------------->  Never ending
H3_2048_4_Parallel  ----------------------->    18444055


Numbers above are not consistent.
H3_1024 should be slower than H3_2048 but it is around 60% faster instead.
Hybrid_Pred is 10x faster than logtm_se_virtualization.
Moreover, sometimes consistency checker problems arise and sometimes they
don't arise (the latter is more frequent).
Could anybody tell me what happens?



Thank you very much,
Ricardo