Re: [Gems-users] Ruby_cycles varying in a wide range

["Dan Gibson" <degibson@xxxxxxxx>]

Ricardo,

Some users have reported nondeterminism /within Simics/, which is rather problematic for this kind of work. We use Simics 2.2.19 and 2.0.23 (depending on who you talk to) internally, and we do not see any nondeterminism in Simics. You should try one of those versions if you're not already doing so. Thus, we run multiple runs with different random seeds for Ruby, and use those for our 95% confidence intervals.

Is Labyrinth the only workload for which this occurs? It seems that there are other STAMP workloads that suffer even more from wide variance, after a cursory glance at Notary. Furthermore, your configuration variables would be useful (not so much to me, but to others on the list). 

Regards,
Dan

On Thu, Nov 27, 2008 at 3:02 AM, Ricardo Quislant del Barrio <quislant@xxxxxxxxx> wrote:

Hi Dan,
where I wrote "You have published" I meant people from Mark Hill's group
as in the paper "Notary: Hardware Techniques to Enhance Signatures",
Micro '08. In that paper you can see results on Labyrinth benchmark with
none or little variance. So, I guess Labyrinth is a deterministic benchmark.
Then, why do I get this variance on ruby cycles (sometimes 3x)?
Should I set the "random_seed" to get Ruby deterministic and then run
multiple runs to
get rid of variance in Simics? Could setting "random_seed" to None lead
to such a variability?

Thanks a lot,
Ricardo.

   On Wed, Nov 26, 2008 at 9:07 AM, Ricardo Quislant del Barrio

   <quislant@xxxxxxxxx <mailto:quislant@xxxxxxxxx>> wrote:

       Yes, it is. You have published some results that haven't got that
       variability I'm experiencing.


   I have not published any results on Labyrinth. Those folks that have
   typically run multiple runs to get rid of variance due to
   timing-sensitive paths, and then report 95% confidence intervals.
   Timing variance is known to change entire execution paths inside of
   benchmarks that are sensitive to it. That is why I am wondering: Is
   Labyrinth deterministic? As in, does the result of the Labyrinth
   benchmark change under different coherence/transactional timing?



       Should I set "random_seed" in "config.py" to a value different
       than None?

       Thanks,
         Ricardo.
       _______________________________________________
       Gems-users mailing list

       Gems-users@xxxxxxxxxxx <mailto:Gems-users@xxxxxxxxxxx>

       https://lists.cs.wisc.edu/mailman/listinfo/gems-users
       Use Google to search the GEMS Users mailing list by adding
       "site:https://lists.cs.wisc.edu/archive/gems-users/" to your search.




   --

   http://www.cs.wisc.edu/~gibson <http://www.cs.wisc.edu/%7Egibson>
   [esc]:wq!

_______________________________________________
Gems-users mailing list
Gems-users@xxxxxxxxxxx
https://lists.cs.wisc.edu/mailman/listinfo/gems-users
Use Google to search the GEMS Users mailing list by adding "site:https://lists.cs.wisc.edu/archive/gems-users/" to your search.

-- 
http://www.cs.wisc.edu/~gibson [esc]:wq!