Re: [Gems-users] Simics 2.2 vs Simics 3.0 statistics difference

["Berkin Ozisikyilmaz" <boz283@xxxxxxxxxxxxxxxxxxxx>]

Hi,

I am first launching the applications and continuing 200million instructions
which seem to be enough to get the applications running. Yes for this
testing I am running 100million, and aware that it might not be long enough
to be representative. I am using a quad-core machine and it takes 16-24
hours to run only this 100million instructions (its just a side note
regarding the performance). So according to the performance of the current
simulations 2billion instruction execution will take ~20x20hours=400 hours
about 17 days just for a configuration. Is this reasonable?

I have run the same script multiple times on Simics 2.2 and currently
running them under Simics 3.0. The results from the simics 2.2 are exactly
the same in between themselves. I have copied the disk image, checkpoint,
scripts from my Simics2.2 directory to 3.0. Both Simics and Gems are
compiled with the same compiler. 

Thanks
Berkin

-----Original Message-----
From: gems-users-bounces@xxxxxxxxxxx [mailto:gems-users-bounces@xxxxxxxxxxx]
On Behalf Of Philip Garcia
Sent: 2009-03-08 18:13
To: Gems Users
Subject: Re: [Gems-users] Simics 2.2 vs Simics 3.0 statistics difference

It appears that you are launching your jobs when you have ruby and  
opal loaded, and therefore measuring both the execution time of your  
jobs and the OS overhead of getting the jobs ready to execute.   If  
simics 2.2 and simics 3.0 are not starting at the exact same point in  
time, this alone will cause large variabilities in your execution  
time.  Personally, I recommend getting all programs you want to  
execute started and executing before you start measuring with simics/ 
opal.    In addition, it appears that you're only running gems for 100  
million cycles.  If workloads are not at the same point in time, this  
is likely not long enough of a sample to be representative of the  
programs you are attempting to measure.  Have you ensured that all  
your applications have reached the same point in each test?  Also, do  
they both use the same default simics configuration files?  There are  
many parameters that could cause such a large discrepancy in run times  
for such short tests.  In my experience runs between simics 2.2 and  
3.0 have experienced some variability (when issued from the same  
checkpoint), but the difference is negligible over a 2 billion cycle  
execution time.

Phil
On Mar 8, 2009, at 3:56 PM, Berkin Ozisikyilmaz wrote:

> I am running ./simics -stall myscript.simics
>
> My script is this
>
> read-configuration magicstart-micro
> con0.input "./example_O3 -i ./color100 -p 1 &\n"
> con0.input "./example_O3 -i ./color100 -f -p 3 &\n"
> con0.input "./para_hop 61440 ./particles_0_64.flp 64 16 -1 2 &\n"
> con0.input "./scalparc para_F26_A32-D125K/F26-A32-D125K.tab 125000  
> 32 2 10
> &\n"
> c 200000000
> cpu-switch-time 1
> instruction-fetch-mode instruction-fetch-trace
> istc-disable
> dstc-disable
> load-module ruby
> ruby0.setparam g_NUM_PROCESSORS 16
> ruby0.setparam g_MEMORY_SIZE_BYTES 536870912
> ruby0.setparam g_PROCS_PER_CHIP 16
> ruby0.setparam g_NUM_MEMORIES 1
> ruby0.setparam NUMBER_OF_VIRTUAL_NETWORKS 5
> ruby0.init
> load-module opal
> opal0.init
> opal0.sim-start "1-3-2-10-repeat2.opal"
> opal0.sim-step 100000000
> opal0.stats
> ruby0.dump-stats "1-3-2-10-repeat2.ruby"
> opal0.sim-stop
>
>
> -----Original Message-----
> From: gems-users-bounces@xxxxxxxxxxx
[mailto:gems-users-bounces@xxxxxxxxxxx 
> ]
> On Behalf Of Dan Gibson
> Sent: 2009-03-08 15:38
> To: Gems Users
> Subject: Re: [Gems-users] Simics 2.2 vs Simics 3.0 statistics  
> difference
>
> Berkin,
>
> How are you deciding how long to run your simulations? Are you using
> magic breakpoints before/after the benchmark? Or are you just running
> for a fixed number of instructions/steps/cycles?
>
> Can you please verify that you are setting both versions to emulate
> instruction fetch?
> (Simics command: instruction-fetch-mode instruction-fetch-trace)
>
> Can you also verify that both runs use cpu-switch-time 1?
> (Simics command: cpu-switch-time 1)
>
> Regards,
> Dan
>
> On Sun, Mar 8, 2009 at 2:05 PM, Berkin Ozisikyilmaz
> <boz283@xxxxxxxxxxxxxxxxxxxx> wrote:
>> Hi,
>>
>> Since my current licenses are expiring soon. I have been trying to  
>> run the
>> same scripts for the same experiments with the checkpoints created in
> simics
>> 2.2 in simics 3.0. The total statistics seems to be significantly
> different.
>> I am just posting a part of the logs to show the difference. Has  
>> anyone
>> observed this? Is there a solution? Should I contact Simics forum  
>> and what
>> should I tell?
>>
>> Thanks
>> Berkin
>> PS: If you need more details I can attach the whole logs too.
>>
>>
>> Simics 3.0:
>> Ruby_current_time: 56966471
>> Ruby_start_time: 1
>> Ruby_cycles: 56966470
>>
>> mbytes_resident: 415.297
>> mbytes_total: 443.906
>> resident_ratio: 0.935568
>>
>> Total_misses: 638457
>> total_misses: 638457 [ 76793 26643 472 14111 92155 440 2386 6512 9838
> 11455
>> 8045 793 279752 6949 91696 10417 ]
>> user_misses: 143334 [ 21434 11522 0 0 41821 0 1082 0 8810 10365  
>> 2177 0 0 0
>> 44952 1171 ]
>> supervisor_misses: 495123 [ 55359 15121 472 14111 50334 440 1304  
>> 6512 1028
>> 1090 5868 793 279752 6949 46744 9246 ]
>>
>> instruction_executed: 1942683921 [ 100000003 124129441 163319276  
>> 165435219
>> 53861776 163460755 161657288 175769500 38246531 38301267 159181278
> 168205246
>> 45314492 170155942 53902511 161743396 ]
>> simics_cycles_executed: 32 [ 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 ]
>> cycles_per_instruction: 0.469177 [ 0.569665 0.458928 0.348804  
>> 0.344343
>> 1.05764 0.348502 0.35239 0.324098 1.48945 1.48733 0.357872 0.338672
> 1.25714
>> 0.33479 1.05684 0.352203 ]
>> misses_per_thousand_instructions: 0.328647 [ 0.76793 0.214639  
>> 0.00289004
>> 0.0852962 1.71095 0.00269178 0.0147596 0.0370485 0.257226 0.299076
> 0.0505399
>> 0.00471448 6.17357 0.040839 1.70115 0.0644045 ]
>>
>> --------------------
>> Simics 2:
>> Ruby_current_time: 27449208
>> Ruby_start_time: 1
>> Ruby_cycles: 27449207
>>
>> mbytes_resident: 337.574
>> mbytes_total: 360.633
>> resident_ratio: 0.936082
>>
>> Total_misses: 296966
>> total_misses: 296966 [ 21740 589 369 7149 35485 266 349 8086 5925  
>> 134268
>> 39354 635 6461 33942 545 1803 ]
>> user_misses: 64611 [ 5863 0 0 474 16339 0 0 0 5232 0 20015 0 1669  
>> 15019 0
> 0
>> ]
>> supervisor_misses: 232355 [ 15877 589 369 6675 19146 266 349 8086 693
> 134268
>> 19339 635 4792 18923 545 1803 ]
>>
>> instruction_executed: 935171606 [ 17858141 78628658 78723317 78953073
>> 27873028 78695970 78809154 85782525 18431579 21706482 25875467  
>> 79044210
>> 77362284 27932361 78666708 80828649 ]
>> simics_cycles_executed: 32 [ 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 ]
>> cycles_per_instruction: 0.469633 [ 1.53707 0.349099 0.34868 0.347665
>> 0.984795 0.348801 0.3483 0.319986 1.48925 1.26456 1.06082 0.347264
> 0.354814
>> 0.982703 0.34893 0.339597 ]
>> misses_per_thousand_instructions: 0.317552 [ 1.21737 0.00749091  
>> 0.0046873
>> 0.0905475 1.27309 0.0033801 0.00442842 0.0942616 0.321459 6.18562  
>> 1.5209
>> 0.00803348 0.0835161 1.21515 0.00692796 0.0223064 ]
>>
>>
>> _______________________________________________
>> Gems-users mailing list
>> Gems-users@xxxxxxxxxxx
>> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
>> Use Google to search the GEMS Users mailing list by adding
> "site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>>
>>
>
>
>
> -- 
> http://www.cs.wisc.edu/~gibson [esc]:wq!
> _______________________________________________
> Gems-users mailing list
> Gems-users@xxxxxxxxxxx
> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
> Use Google to search the GEMS Users mailing list by adding
> "site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>
>
> _______________________________________________
> Gems-users mailing list
> Gems-users@xxxxxxxxxxx
> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
> Use Google to search the GEMS Users mailing list by adding
"site:https://lists.cs.wisc.edu/archive/gems-users/ 
> " to your search.
>

_______________________________________________
Gems-users mailing list
Gems-users@xxxxxxxxxxx
https://lists.cs.wisc.edu/mailman/listinfo/gems-users
Use Google to search the GEMS Users mailing list by adding
"site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.