Re: [Gems-users] Simics 2.2 vs Simics 3.0 statistics difference

["Berkin Ozisikyilmaz" <boz283@xxxxxxxxxxxxxxxxxxxx>]

I tried creating a checkpoint after starting the applications and continuing
for 200million instructions. Then I used this checkpoint to run the
applications with ruby and opal for another 100million. The result of the
simulation is the same as the original script under simics 2.2 with my old
checkpoint (I am still waiting for the simulation to continue with simics
3.0 since that machine is an older one).

I have also tried using only ruby with the newly generated tried it with
ruby only. The stats seem to be similar (less than 0.01%). As I said I am
waiting for the Ruby+Opal results with the new checkpoint. Even if this new
method gives similar statistics, since the combination of input parameters
that I feed the applications change and hence for each of those I would need
to create a checkpoint and it wouldn?t be a feasible way to do.

Thanks
Berkin

Simics 3.0
1-3-2-10-repeat200-100.ruby:instruction_executed: 1657018980 [ 100000001
127051186 127020115 123624522 95861765 127043386 126655381 69208028
104664132 58875416 70600025 125309505 84091654 62969275 127042452 127002137
]
1-3-2-10-repeat200-100.ruby:cycles_per_instruction: 0.306813 [ 0.317747
0.250093 0.250155 0.257026 0.331463 0.250109 0.250875 0.459118 0.303587
0.539693 0.450066 0.253569 0.377857 0.504606 0.250111 0.25019 ]
1-3-2-10-repeat200-100.ruby:misses_per_thousand_instructions: 0.299656 [
0.21901 0.000385671 0.00121241 0.0634906 0.683223 0.000377824 0.0117247
0.948691 0.138624 1.677 1.03222 0.0497728 0.634474 1.39682 0.000369955
0.00144879 ]
1-3-2-10-repeat200-100.ruby:instructions_per_transaction: 0 [ 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 ]
1-3-2-10-repeat200-100.ruby:  L1D_cache_misses_per_instruction: 0.000656192
1-3-2-10-repeat200-100.ruby:  L1D_cache_instructions_per_misses: 1523.95
1-3-2-10-repeat200-100.ruby:  L1I_cache_misses_per_instruction: 0.000103013
1-3-2-10-repeat200-100.ruby:  L1I_cache_instructions_per_misses: 9707.54
1-3-2-10-repeat200-100.ruby:  L2_cache_misses_per_instruction: 0.000299656
1-3-2-10-repeat200-100.ruby:  L2_cache_instructions_per_misses: 3337.16
1-3-2-10-repeat200-100.ruby:Duplicate instruction fetches: 170694
1-3-2-10-repeat200-100.ruby:Ruby_cycles: 31774655
Simics 2.2
1-3-2-10-repeat200-100-xps.ruby:instruction_executed: 1656821937 [ 100000001
127042624 127006834 123614070 95826221 127032265 126645571 69172174
104646470 58865879 70585352 125297324 84082583 62968509 127040242 126995818
]
1-3-2-10-repeat200-100-xps.ruby:cycles_per_instruction: 0.306831 [ 0.317727
0.250095 0.250165 0.257032 0.331566 0.250115 0.250879 0.459328 0.30362
0.539748 0.450132 0.253579 0.377875 0.504581 0.2501 0.250187 ]
1-3-2-10-repeat200-100-xps.ruby:misses_per_thousand_instructions: 0.300692 [
0.2185 0.000393569 0.00122041 0.0636254 0.683216 0.000377857 0.0117651
0.974756 0.138743 1.6771 1.03218 0.0497457 0.634638 1.39608 0.000385705
0.00146462 ]
1-3-2-10-repeat200-100-xps.ruby:instructions_per_transaction: 0 [ 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 ]
1-3-2-10-repeat200-100-xps.ruby:  L1D_cache_misses_per_instruction:
0.000655743
1-3-2-10-repeat200-100-xps.ruby:  L1D_cache_instructions_per_misses: 1524.99
1-3-2-10-repeat200-100-xps.ruby:  L1I_cache_misses_per_instruction:
0.000103029
1-3-2-10-repeat200-100-xps.ruby:  L1I_cache_instructions_per_misses: 9705.99
1-3-2-10-repeat200-100-xps.ruby:  L2_cache_misses_per_instruction:
0.000300692
1-3-2-10-repeat200-100-xps.ruby:  L2_cache_instructions_per_misses: 3325.66
1-3-2-10-repeat200-100-xps.ruby:Duplicate instruction fetches: 170701
1-3-2-10-repeat200-100-xps.ruby:Ruby_cycles: 31772728

-----Original Message-----
From: gems-users-bounces@xxxxxxxxxxx [mailto:gems-users-bounces@xxxxxxxxxxx]
On Behalf Of Dan Gibson
Sent: 2009-03-09 10:34
To: Gems Users
Subject: Re: [Gems-users] Simics 2.2 vs Simics 3.0 statistics difference

Try this with Ruby only.
Verify both stats files show similar instruction fetch statistics.

Regards,
Dan

On Sun, Mar 8, 2009 at 9:28 PM, Philip Garcia <pcgarcia@xxxxxxxx> wrote:
> the c 200000000 might be causing problems between the two versions of
> simics.  Try launching the simulation (in simics only) in 2.2, and
> after the 200M cycles (or more if you want, this should go fast), save
> a checkpoint.  Then load that same checkpoint with simics/gems for
> both the 2.2 and 3.0 case.  That way you know all processors are
> starting at the exact same point in each situation.
>
> Phil
> On Mar 8, 2009, at 6:24 PM, Berkin Ozisikyilmaz wrote:
>
>> Hi,
>>
>> I am first launching the applications and continuing 200million
>> instructions
>> which seem to be enough to get the applications running. Yes for this
>> testing I am running 100million, and aware that it might not be long
>> enough
>> to be representative. I am using a quad-core machine and it takes
>> 16-24
>> hours to run only this 100million instructions (its just a side note
>> regarding the performance). So according to the performance of the
>> current
>> simulations 2billion instruction execution will take ~20x20hours=400
>> hours
>> about 17 days just for a configuration. Is this reasonable?
>>
>> I have run the same script multiple times on Simics 2.2 and currently
>> running them under Simics 3.0. The results from the simics 2.2 are
>> exactly
>> the same in between themselves. I have copied the disk image,
>> checkpoint,
>> scripts from my Simics2.2 directory to 3.0. Both Simics and Gems are
>> compiled with the same compiler.
>>
>> Thanks
>> Berkin
>>
>> -----Original Message-----
>> From: gems-users-bounces@xxxxxxxxxxx
[mailto:gems-users-bounces@xxxxxxxxxxx
>> ]
>> On Behalf Of Philip Garcia
>> Sent: 2009-03-08 18:13
>> To: Gems Users
>> Subject: Re: [Gems-users] Simics 2.2 vs Simics 3.0 statistics
>> difference
>>
>> It appears that you are launching your jobs when you have ruby and
>> opal loaded, and therefore measuring both the execution time of your
>> jobs and the OS overhead of getting the jobs ready to execute.   If
>> simics 2.2 and simics 3.0 are not starting at the exact same point in
>> time, this alone will cause large variabilities in your execution
>> time.  Personally, I recommend getting all programs you want to
>> execute started and executing before you start measuring with simics/
>> opal.    In addition, it appears that you're only running gems for 100
>> million cycles.  If workloads are not at the same point in time, this
>> is likely not long enough of a sample to be representative of the
>> programs you are attempting to measure.  Have you ensured that all
>> your applications have reached the same point in each test?  Also, do
>> they both use the same default simics configuration files?  There are
>> many parameters that could cause such a large discrepancy in run times
>> for such short tests.  In my experience runs between simics 2.2 and
>> 3.0 have experienced some variability (when issued from the same
>> checkpoint), but the difference is negligible over a 2 billion cycle
>> execution time.
>>
>> Phil
>> On Mar 8, 2009, at 3:56 PM, Berkin Ozisikyilmaz wrote:
>>
>>> I am running ./simics -stall myscript.simics
>>>
>>> My script is this
>>>
>>> read-configuration magicstart-micro
>>> con0.input "./example_O3 -i ./color100 -p 1 &\n"
>>> con0.input "./example_O3 -i ./color100 -f -p 3 &\n"
>>> con0.input "./para_hop 61440 ./particles_0_64.flp 64 16 -1 2 &\n"
>>> con0.input "./scalparc para_F26_A32-D125K/F26-A32-D125K.tab 125000
>>> 32 2 10
>>> &\n"
>>> c 200000000
>>> cpu-switch-time 1
>>> instruction-fetch-mode instruction-fetch-trace
>>> istc-disable
>>> dstc-disable
>>> load-module ruby
>>> ruby0.setparam g_NUM_PROCESSORS 16
>>> ruby0.setparam g_MEMORY_SIZE_BYTES 536870912
>>> ruby0.setparam g_PROCS_PER_CHIP 16
>>> ruby0.setparam g_NUM_MEMORIES 1
>>> ruby0.setparam NUMBER_OF_VIRTUAL_NETWORKS 5
>>> ruby0.init
>>> load-module opal
>>> opal0.init
>>> opal0.sim-start "1-3-2-10-repeat2.opal"
>>> opal0.sim-step 100000000
>>> opal0.stats
>>> ruby0.dump-stats "1-3-2-10-repeat2.ruby"
>>> opal0.sim-stop
>>>
>>>
>>> -----Original Message-----
>>> From: gems-users-bounces@xxxxxxxxxxx
>> [mailto:gems-users-bounces@xxxxxxxxxxx
>>> ]
>>> On Behalf Of Dan Gibson
>>> Sent: 2009-03-08 15:38
>>> To: Gems Users
>>> Subject: Re: [Gems-users] Simics 2.2 vs Simics 3.0 statistics
>>> difference
>>>
>>> Berkin,
>>>
>>> How are you deciding how long to run your simulations? Are you using
>>> magic breakpoints before/after the benchmark? Or are you just running
>>> for a fixed number of instructions/steps/cycles?
>>>
>>> Can you please verify that you are setting both versions to emulate
>>> instruction fetch?
>>> (Simics command: instruction-fetch-mode instruction-fetch-trace)
>>>
>>> Can you also verify that both runs use cpu-switch-time 1?
>>> (Simics command: cpu-switch-time 1)
>>>
>>> Regards,
>>> Dan
>>>
>>> On Sun, Mar 8, 2009 at 2:05 PM, Berkin Ozisikyilmaz
>>> <boz283@xxxxxxxxxxxxxxxxxxxx> wrote:
>>>> Hi,
>>>>
>>>> Since my current licenses are expiring soon. I have been trying to
>>>> run the
>>>> same scripts for the same experiments with the checkpoints created
>>>> in
>>> simics
>>>> 2.2 in simics 3.0. The total statistics seems to be significantly
>>> different.
>>>> I am just posting a part of the logs to show the difference. Has
>>>> anyone
>>>> observed this? Is there a solution? Should I contact Simics forum
>>>> and what
>>>> should I tell?
>>>>
>>>> Thanks
>>>> Berkin
>>>> PS: If you need more details I can attach the whole logs too.
>>>>
>>>>
>>>> Simics 3.0:
>>>> Ruby_current_time: 56966471
>>>> Ruby_start_time: 1
>>>> Ruby_cycles: 56966470
>>>>
>>>> mbytes_resident: 415.297
>>>> mbytes_total: 443.906
>>>> resident_ratio: 0.935568
>>>>
>>>> Total_misses: 638457
>>>> total_misses: 638457 [ 76793 26643 472 14111 92155 440 2386 6512
>>>> 9838
>>> 11455
>>>> 8045 793 279752 6949 91696 10417 ]
>>>> user_misses: 143334 [ 21434 11522 0 0 41821 0 1082 0 8810 10365
>>>> 2177 0 0 0
>>>> 44952 1171 ]
>>>> supervisor_misses: 495123 [ 55359 15121 472 14111 50334 440 1304
>>>> 6512 1028
>>>> 1090 5868 793 279752 6949 46744 9246 ]
>>>>
>>>> instruction_executed: 1942683921 [ 100000003 124129441 163319276
>>>> 165435219
>>>> 53861776 163460755 161657288 175769500 38246531 38301267 159181278
>>> 168205246
>>>> 45314492 170155942 53902511 161743396 ]
>>>> simics_cycles_executed: 32 [ 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 ]
>>>> cycles_per_instruction: 0.469177 [ 0.569665 0.458928 0.348804
>>>> 0.344343
>>>> 1.05764 0.348502 0.35239 0.324098 1.48945 1.48733 0.357872 0.338672
>>> 1.25714
>>>> 0.33479 1.05684 0.352203 ]
>>>> misses_per_thousand_instructions: 0.328647 [ 0.76793 0.214639
>>>> 0.00289004
>>>> 0.0852962 1.71095 0.00269178 0.0147596 0.0370485 0.257226 0.299076
>>> 0.0505399
>>>> 0.00471448 6.17357 0.040839 1.70115 0.0644045 ]
>>>>
>>>> --------------------
>>>> Simics 2:
>>>> Ruby_current_time: 27449208
>>>> Ruby_start_time: 1
>>>> Ruby_cycles: 27449207
>>>>
>>>> mbytes_resident: 337.574
>>>> mbytes_total: 360.633
>>>> resident_ratio: 0.936082
>>>>
>>>> Total_misses: 296966
>>>> total_misses: 296966 [ 21740 589 369 7149 35485 266 349 8086 5925
>>>> 134268
>>>> 39354 635 6461 33942 545 1803 ]
>>>> user_misses: 64611 [ 5863 0 0 474 16339 0 0 0 5232 0 20015 0 1669
>>>> 15019 0
>>> 0
>>>> ]
>>>> supervisor_misses: 232355 [ 15877 589 369 6675 19146 266 349 8086
>>>> 693
>>> 134268
>>>> 19339 635 4792 18923 545 1803 ]
>>>>
>>>> instruction_executed: 935171606 [ 17858141 78628658 78723317
>>>> 78953073
>>>> 27873028 78695970 78809154 85782525 18431579 21706482 25875467
>>>> 79044210
>>>> 77362284 27932361 78666708 80828649 ]
>>>> simics_cycles_executed: 32 [ 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 ]
>>>> cycles_per_instruction: 0.469633 [ 1.53707 0.349099 0.34868 0.347665
>>>> 0.984795 0.348801 0.3483 0.319986 1.48925 1.26456 1.06082 0.347264
>>> 0.354814
>>>> 0.982703 0.34893 0.339597 ]
>>>> misses_per_thousand_instructions: 0.317552 [ 1.21737 0.00749091
>>>> 0.0046873
>>>> 0.0905475 1.27309 0.0033801 0.00442842 0.0942616 0.321459 6.18562
>>>> 1.5209
>>>> 0.00803348 0.0835161 1.21515 0.00692796 0.0223064 ]
>>>>
>>>>
>>>> _______________________________________________
>>>> Gems-users mailing list
>>>> Gems-users@xxxxxxxxxxx
>>>> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
>>>> Use Google to search the GEMS Users mailing list by adding
>>> "site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> http://www.cs.wisc.edu/~gibson [esc]:wq!
>>> _______________________________________________
>>> Gems-users mailing list
>>> Gems-users@xxxxxxxxxxx
>>> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
>>> Use Google to search the GEMS Users mailing list by adding
>>> "site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>>>
>>>
>>> _______________________________________________
>>> Gems-users mailing list
>>> Gems-users@xxxxxxxxxxx
>>> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
>>> Use Google to search the GEMS Users mailing list by adding
>> "site:https://lists.cs.wisc.edu/archive/gems-users/
>>> " to your search.
>>>
>>
>> _______________________________________________
>> Gems-users mailing list
>> Gems-users@xxxxxxxxxxx
>> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
>> Use Google to search the GEMS Users mailing list by adding
>> "site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>>
>>
>> _______________________________________________
>> Gems-users mailing list
>> Gems-users@xxxxxxxxxxx
>> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
>> Use Google to search the GEMS Users mailing list by adding
"site:https://lists.cs.wisc.edu/archive/gems-users/
>> " to your search.
>>
>
> _______________________________________________
> Gems-users mailing list
> Gems-users@xxxxxxxxxxx
> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
> Use Google to search the GEMS Users mailing list by adding
"site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>
>



-- 
http://www.cs.wisc.edu/~gibson [esc]:wq!
_______________________________________________
Gems-users mailing list
Gems-users@xxxxxxxxxxx
https://lists.cs.wisc.edu/mailman/listinfo/gems-users
Use Google to search the GEMS Users mailing list by adding
"site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.