Re: [Gems-users] ruby possibly hanging on 64p MSI_MOSI_CMP_directory

[Dan Gibson <degibson@xxxxxxxx>]

Changing the timing will definitely change dynamic execution paths. Even 
minor perturbations in apparent memory latencies can greatly affect 
executions, through different lock orderings, etc. Since Simics has the 
unrealistic model of zero-latency memory accesses, adding ruby will 
change things quite a bit.
As far as the question about Ruby's performance overall (which seems to 
come up rather frequently), I'll give some general remarks:
In general, Ruby's performance degrades with:
- Increases in processor count
- More complex networks, like DNUCA
- Longer memory latencies, and more memories
- Larger target address spaces
- More complicated protocols
- More contention in the target workloads
- Lots of other things.

As you might expect, Ruby's performance also depends on whether Opal is 
used, what version of Simics is used, what compiler was used, what the 
host's performance is, and lots more.
I'm not able to say with any certainty, for a given combination of host 
machine, target configuration, memory sizes, etc. what the expected 
slowdown of Ruby will be, except that it will almost always be significant.
Here are a things to do that sometimes improve simulator performance:
1) Reduce memory and cache latencies
2) Look into the simics command set-memory-limit to reduce Simics' footprint
3) Use simple networks and protocols
4) Be careful to only simulate the important regions of code -- do not 
include thread creation / destruction, process forking, etc.
5) Before loading ruby, do a dry-run of your algorithm to warm up the 
TLBs and the OS file cache, if necessary.
Regards,
Dan

Brad Moore wrote:
> Would Ruby affect the execution of a given benchmark?  As in, could 
> different processes be scheduled at different times, etc?  I'm trying 
> to rule out deadlock in my benchmark (when I run the checkpoint from 
> Simics, it terminates - but maybe loading Ruby results in a different 
> execution trace?).
> My current test code takes less than 30 secs in simics, under -stall.  
> (It's more or less 64 processes trying to acquire a lock once).
> Here's a snippet of my mid-run stats,
>
>
> Profiler Stats
> --------------
> Elapsed_time_in_seconds: 12534
> Elapsed_time_in_minutes: 208.9
> Elapsed_time_in_hours: 3.48167
> Elapsed_time_in_days: 0.145069
>
> Virtual_time_in_seconds: 12471.5
> Virtual_time_in_minutes: 207.858
> Virtual_time_in_hours:   3.4643
> Virtual_time_in_days:    3.4643
>
> Ruby_current_time: 4579041
> Ruby_start_time: 1
> Ruby_cycles: 4579040
>
> mbytes_resident: 742.039
> mbytes_total: 814.199
> resident_ratio: 0.911373
>
> Total_misses: 5030
> total_misses: 5030 [ 2149 126 117 105 108 77 76 79 76 74 72 78 86 87 
> 74 80 81 78
>  72 80 95 89 72 78 76 75 71 77 67 56 34 8 7 21 21 6 5 53 6 7 5 5 8 7 6 
> 8 8 7 7 7
>  6 6 7 20 20 7 107 22 6 7 5 7 26 17 ]
> user_misses: 578 [ 7 6 6 6 6 5 8 7 8 7 6 6 20 20 7 8 7 8 7 7 20 22 7 7 
> 6 5 5 6 6
>  7 7 6 7 21 21 6 5 6 6 7 5 5 8 7 6 8 8 7 7 7 6 6 7 20 20 7 10 21 6 7 5 
> 7 26 17 ]
> supervisor_misses: 4452 [ 2142 120 111 99 102 72 68 72 68 67 66 72 66 
> 67 67 72 7
> 4 70 65 73 75 67 65 71 70 70 66 71 61 49 27 2 0 0 0 0 0 47 0 0 0 0 0 0 
> 0 0 0 0 0
>  0 0 0 0 0 0 0 97 1 0 0 0 0 0 0 ]
>
> instruction_executed: 548307672 [ 175506 7875662 7774131 7951346 
> 8155794 8028192
>  8342766 8337001 8520547 8521181 8360406 8523937 8502288 8507459 
> 8518105 8505419
>  8477870 8505652 8373339 8360242 8324068 8502216 8543667 8376771 
> 8396250 8497359
>  8394274 8551848 8626100 8754440 8854887 8968650 8994113 8951270 
> 8999835 8999393
>  9027096 8516963 9005266 9013142 9039001 9007965 9031391 9018369 
> 9038944 9031420
>  9010380 9005542 9006547 9019249 9016592 9010371 9004193 8998379 
> 8988859 9007826
>  8541924 8986005 9023253 9009101 9021840 9007900 8921952 8946218 ]
>
> Thanks,
> -Brad
>
> On 3/1/07, *Mike Marty* <mikem@xxxxxxxxxxx <mailto:mikem@xxxxxxxxxxx>> 
> wrote:
>     Also when it took 5 minutes with Simics-only, was this Simics "fast
>     mode" or with "stall mode".  We always run Simics with the -stall
>     option.
>
>     Brad Moore wrote:
>     > Hey all,
>     >
>     > I'm currently running a benchmark that has been executing for over
>     > 24hrs.  I haven't successfully executed it with simics + ruby, but I
>     > have executed it with simics and it took under 5 minutes.  I'm
>     trying
>     > to figure out if it's just going to take this long, or if I may
>     have
>     > done something that would cause ruby to hang silently.
>     >
>     > 64 process Sarek machine running Solaris 9.
>     >
>     > Before compiling MSI_MOSI_CMP_directory, I edited
>     rubyconfig.defaults:
>     > g_NUM_PROCESSORS: 64
>     > g_NUM_L2_BANKS: 8
>     > g_NUM_MEMORIES: 1
>     > g_PROCS_PER_CHIP: 64
>     >
>     > And I created a file,
>     > NUCA_Procs-64_ProcsPerChip-64_L2Banks-8_Memories-1.txt
>     > After that, I went into ./ruby and called built
>     MSI_MOSI_CMP_directory.
>     >
>     >
>     > Now, if it is just taking too long...  Could I reduce the memory
>     > requirements of the simulation by reducing the memory size parameter
>     > to ruby?  Does that parameter have to be equivalent to the
>     > corresponding simics parameter?
>     >
>     > Thanks!
>     > -Brad
>     >
>     ------------------------------------------------------------------------
>     >
>     > _______________________________________________
>     > Gems-users mailing list
>     > Gems-users@xxxxxxxxxxx <mailto:Gems-users@xxxxxxxxxxx>
>     > https://lists.cs.wisc.edu/mailman/listinfo/gems-users
>     <https://lists.cs.wisc.edu/mailman/listinfo/gems-users>
>     > Use Google to search the GEMS Users mailing list by adding
>     "site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>     >
>     >
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Gems-users mailing list
> Gems-users@xxxxxxxxxxx
> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
> Use Google to search the GEMS Users mailing list by adding "site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>
>   
--
http://www.cs.wisc.edu/~gibson [esc]:wq!