Re: [Gems-users] What happens if NUMBER_OF_VIRTUAL_NETWORKS is greater than needed?

[Ikhwan Lee <ikhwan@xxxxxxxxxxxxxxx>]

Dan,
I was not trying to compare the performance of VN=4 and VN=5. I was
rather concerned about the integrity of the simulation results since I
could not understand why they weren't exactly the same. But I do
appreciate your advice. Thanks,

Ikhwan

On Tue, Apr 6, 2010 at 8:17 AM, Dan Gibson <degibson@xxxxxxxx> wrote:
> Ikhwan,
> Let me put it another way. In multiprocessor simulations, no two runs are
> different in a /statistically significant way/. Only when two configurations
> (e.g., VN=4, VN=5) are run each MANY times, and averages taken, with nice,
> tight 95% confidence intervals, can one say with confidence that X is faster
> than Y.
>
> I.e., try VN=4 and VN=5 each with many different RANDOM_SEEDs.
>
> Regards,
> Dan
>
> See also:
> "Variability in Architectural Simulations of Multi-threaded Workloads", by
> Wisconsin's own Alaa Alameldeen and David A. Wood.
> LINK: http://www.cs.wisc.edu/~alaa/papers/hpca03_variability.pdf
>
>
> On Mon, Apr 5, 2010 at 5:07 PM, Ikhwan Lee <ikhwan@xxxxxxxxxxxxxxx> wrote:
>>
>> Thanks Dan. I was running MOESI_SMP_directory using a fixed
>> g_RANDOM_SEED for all simulations. I am still not sure why results are
>> different for having an extra VNET. I'll keep the list posted if I
>> figure out why.
>>
>> Ikhwan
>>
>> On Tue, Apr 6, 2010 at 7:57 AM, Dan Gibson <degibson@xxxxxxxx> wrote:
>> > It depends.
>> > 1. If you're not using a protocol ending in _m, then your protocol
>> > doesn't
>> > use the MemoryController module. If that is the case, random noise (aka
>> > rand() % 5) is simply added to MAIN_MEMORY_LATENCY_MINUS_2 to generate
>> > the
>> > 'actual' memory latency.
>> > 2. If your protocol ends in _m, then you are using the MemoryController
>> > model, which has some built-in features to add random delays, when
>> > activated. Check out MemoryController.[Ch] (specifically, the comments
>> > at
>> > the start) to see how that works.
>> >
>> > In either case, for a fixed RANDOM_SEED, you should get precisely the
>> > same
>> > results.
>> >
>> > Regards,
>> > Dan
>> >
>> > On Mon, Apr 5, 2010 at 4:43 PM, Ikhwan Lee <ikhwan@xxxxxxxxxxxxxxx>
>> > wrote:
>> >>
>> >> How is a simulation randomized? I've been getting exactly the same
>> >> results for each run with same simulation parameters.
>> >>
>> >> Thanks,
>> >> Ikhwan
>> >>
>> >> On Sat, Apr 3, 2010 at 12:24 PM, Dan Gibson <degibson@xxxxxxxx> wrote:
>> >> > Average many runs of each (e.g., 8 runs w/ VNs = 4, 8 runs w/ VNs =
>> >> > 5)
>> >> > and
>> >> > see if there is a statistically significant difference or not. Since
>> >> > the
>> >> > difference in runtime is pretty small, this could be due only to
>> >> > difference
>> >> > in RANDOM_SEED.
>> >> >
>> >> > That said, I don't actually know how Ruby might react to having
>> >> > 'extra'
>> >> > VNs.
>> >> >
>> >> > Regards,
>> >> > Dan
>> >> >
>> >> > On Fri, Apr 2, 2010 at 4:52 PM, Ikhwan Lee <ikhwan@xxxxxxxxxxxxxxx>
>> >> > wrote:
>> >> >>
>> >> >> The MOESI_SMP_directory protocol needs 4 virtual networks; however,
>> >> >> it
>> >> >> should also run fine with 5 virtual networks since the 5th virtual
>> >> >> network will never be used by the protocol. The question is, when I
>> >> >> compare the results of two simulations (one with 4 and one with 5
>> >> >> virtual networks), I get something like below.
>> >> >>
>> >> >> 153c153
>> >> >> < NUMBER_OF_VIRTUAL_NETWORKS: 4
>> >> >> ---
>> >> >> < NUMBER_OF_VIRTUAL_NETWORKS: 5
>> >> >> 1457c1458
>> >> >> < Ruby_current_time: 25741756
>> >> >> ---
>> >> >> > Ruby_current_time: 27894100
>> >> >> 1459c1460
>> >> >> < Ruby_cycles: 25741755
>> >> >> ---
>> >> >> > Ruby_cycles: 27894099
>> >> >> 1461,1468c1462,1469
>> >> >> < mbytes_resident: 480.223
>> >> >> < mbytes_total: 508.523
>> >> >> < resident_ratio: 0.944347
>> >> >> <
>> >> >> < Total_misses: 2736589
>> >> >> < total_misses: 2736589 [ 221437 184551 181803 181854 165005 165866
>> >> >> 172800 177348 162455 164810 168559 167311 146275 158797 158483
>> >> >> 159235
>> >> >> ]
>> >> >> < user_misses: 199059 [ 6640 12908 16388 12543 9998 14914 11967
>> >> >> 11834
>> >> >> 17170 11744 11685 12695 13389 11501 12351 11332 ]
>> >> >> < supervisor_misses: 2537530 [ 214797 171643 165415 169311 155007
>> >> >> 150952 160833 165514 145285 153066 156874 154616 132886 147296
>> >> >> 146132
>> >> >> 147903 ]
>> >> >> ---
>> >> >> > mbytes_resident: 177.781
>> >> >> > mbytes_total: 498.574
>> >> >> > resident_ratio: 0.356579
>> >> >> >
>> >> >> > Total_misses: 2976004
>> >> >> > total_misses: 2976004 [ 231381 201401 206063 197190 177795 182883
>> >> >> > 187239
>> >> >> > 196167 175355 180539 183532 179732 157077 173355 171734 174561 ]
>> >> >> > user_misses: 200169 [ 12482 12593 12510 11561 13957 9487 11581
>> >> >> > 12406
>> >> >> > 17732 13232 11811 11721 12437 12137 12108 12414 ]
>> >> >> > supervisor_misses: 2775835 [ 218899 188808 193553 185629 163838
>> >> >> > 173396
>> >> >> > 175658 183761 157623 167307 171721 168011 144640 161218 159626
>> >> >> > 162147
>> >> >> > ]
>> >> >> --- omitted the rest
>> >> >>
>> >> >> The differences are not too big, but still quite strange since I
>> >> >> expected them to be exactly the same. Any idea on what is happening?
>> >> >> I'm using Garnet fixed pipeline network.
>> >> >>
>> >> >> Thanks,
>> >> >> Ikhwan
>> >> >> _______________________________________________
>> >> >> Gems-users mailing list
>> >> >> Gems-users@xxxxxxxxxxx
>> >> >> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
>> >> >> Use Google to search the GEMS Users mailing list by adding
>> >> >> "site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > http://www.cs.wisc.edu/~gibson [esc]:wq!
>> >> >
>> >> > _______________________________________________
>> >> > Gems-users mailing list
>> >> > Gems-users@xxxxxxxxxxx
>> >> > https://lists.cs.wisc.edu/mailman/listinfo/gems-users
>> >> > Use Google to search the GEMS Users mailing list by adding
>> >> > "site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>> >> >
>> >> >
>> >> >
>> >> _______________________________________________
>> >> Gems-users mailing list
>> >> Gems-users@xxxxxxxxxxx
>> >> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
>> >> Use Google to search the GEMS Users mailing list by adding
>> >> "site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>> >>
>> >
>> >
>> >
>> > --
>> > http://www.cs.wisc.edu/~gibson [esc]:wq!
>> >
>> > _______________________________________________
>> > Gems-users mailing list
>> > Gems-users@xxxxxxxxxxx
>> > https://lists.cs.wisc.edu/mailman/listinfo/gems-users
>> > Use Google to search the GEMS Users mailing list by adding
>> > "site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>> >
>> >
>> >
>> _______________________________________________
>> Gems-users mailing list
>> Gems-users@xxxxxxxxxxx
>> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
>> Use Google to search the GEMS Users mailing list by adding
>> "site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>>
>
>
>
> --
> http://www.cs.wisc.edu/~gibson [esc]:wq!
>
> _______________________________________________
> Gems-users mailing list
> Gems-users@xxxxxxxxxxx
> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
> Use Google to search the GEMS Users mailing list by adding
> "site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>
>
>