Re: [Gems-users] Variability in STAMP's Labyrinth benchmark

[Luke Yen <lyen@xxxxxxxxxxx>]

    I can say from my experience that for some STAMP workloads (e.g., 
Bayes, Delaunay) larger signatures may not improve performance. 
Furthermore, imperfect signatures can sometimes beat the performance of perfect 
signatures.
   The most intuitive explanantion (verified by looking at detailed stats) 
is that the conflict formation is different in each of those runs.  This 
can lead to different number of stalls and aborts, and these
can occur in different places in the code.  Another indicator is the 
difference in false positive rate for the signatures.
  Finally, I recommend multiple runs for the STAMP workloads to reduce 
variability.
   Luke

On Tue, 2 Dec 2008, Ricardo Quislant del Barrio wrote:

> Hi all,
> I'm simulating STAMP's Labyrinth benchmark on GEMS 2.0 and Simics 2.2.19.
> Below you can find some numbers I've gathered from Ruby and the parameters
> I've been using:
>
>    - Every simulation runs on top of a simulated Solaris 10 Sarek 16
> processor machine.
>    - labyrinth input: -t15 -i inputs/random-x32-y32-z3-n16.txt
>
> Following numbers were collected using "logtm_se_virtualization".
> Random seed has been set to 213 for the next runs:
>
> RD/WR filter                                       Ruby_cycles
> ---------------------------------------------------------------------------------
> Perfect_ ------------------------------------------>  202691066
> H3_2048_4_Regular ----------------------->  256785454
> H3_2048_4_Parallel  ----------------------->  265880564
>
> Random seed has been set to 56 for the next runs:
>
> RD/WR filter                                       Ruby_cycles
> ---------------------------------------------------------------------------------
> Perfect_ ------------------------------------------>  172010695
> H3_2048_4_Regular ----------------------->  176020867
> H3_2048_4_Parallel  ----------------------->  130056902
> H3_1024_4_Parallel  ----------------------->    45808646
>
> Now, when changing "logtm_se_virtualization" to
> "EagerCD_EagerVM_Hybrid_Pred":
>
> Random seed has been set to 213 for the next runs:
>
> RD/WR filter                                       Ruby_cycles
> ---------------------------------------------------------------------------------
> Perfect_ ------------------------------------------>    17991611
> H3_2048_4_Regular ----------------------->  Never ending
> H3_2048_4_Parallel  ----------------------->   18236939
>
>
> Random seed has been set to 56 for the next runs:
>
> RD/WR filter                                       Ruby_cycles
> ---------------------------------------------------------------------------------
> Perfect_ ------------------------------------------>    18146557
> H3_2048_4_Regular ----------------------->  Never ending
> H3_2048_4_Parallel  ----------------------->    18444055
>
>
> Numbers above are not consistent.
> H3_1024 should be slower than H3_2048 but it is around 60% faster instead.
> Hybrid_Pred is 10x faster than logtm_se_virtualization.
> Moreover, sometimes consistency checker problems arise and sometimes they
> don't arise (the latter is more frequent).
> Could anybody tell me what happens?
>
>
>
> Thank you very much,
> Ricardo
> _______________________________________________
> Gems-users mailing list
> Gems-users@xxxxxxxxxxx
> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
> Use Google to search the GEMS Users mailing list by adding "site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.