[Gems-users] question about migrations in DNUCA and mapping of L2 banks to bankclusters

["Dimitris Kaseridis" <kaseridi@xxxxxxxxxxxxxx>]

Hello,

I am trying to understand and hopefully use the CMP_NUCA protocol and
more specifically I am using the DNUCA one. According the
documentation wiki here:

http://www.cs.wisc.edu/gems/doc/gems-wiki/moin.cgi/CMP-NUCA

it says
"The CMP-NUCA protocol is based on the protocol described in
[http://www.cs.wisc.edu/multifacet/papers/micro04_cmp_wire_delay.pdf].";

Based on the assumption that the whole implementation should follow
the description of the paper I am really confused with the migration
of blocks.

Let's assume that we have 8 cores and 256 L2 banks as the paper does
and I am using the gems-1.2 to have the implementation of CMP-NUCA

The paper separates the banks in 16 banksets/bankclusters -> 8 local,
4 inter and 4 center as the figure 1 of the paper shows.
According to section 5.2 from the same paper, the order of migrations
from one local bankset, that belongs to one core, e.g cpu1, to another
local bankset that belongs to another core e.g cpu2 should follow the
order:

local bank cpu1 --> inter bankcluster of cpu1  --> center bankcluster
of cpu1--> center bankcluster of cpu2 --> inter bankcluster of cpu2
--> local bank cpu2

SO far, I am just referring to the paper.

Now the actual code seems little bit different or I am missing sth
real important.

If someone sees the code in $GEMS/ruby/nuca/DNUCAPredictor.C for the
DNUCAPredictor::getNUCARequest function it is almost
straight forward to see that the order of migrations is the following:

local bank cpu1 --> cpu1 center bankcluster --> cpu2 center
bankcluster --> local bank of cpu2


Someone could say that the cpu1 center bankcluster might do a
different mapping actually have the inter and center inside it .. but
again if you see the code for the functions:

 MachineID myLocalBank =
map_L1CacheMachId_to_L2CacheLocalBank(infoMsg.getAddress(), m_machId);
  MachineID myCenterBank =
map_L1CacheMachId_to_L2CacheCenterBank(infoMsg.getAddress(),
m_machId);

in $GEMS/ruby/slicc_interface/RubySlicc_ComponentMapping.h file

(the functions actually select the banks that maps to an address)

you will see that the local banks are the first (num_L2_banks/2) and
the other half of them are the center ones. To make it even more
difficult for me to understand it ;-) it seems like by inspecting the
code of the map_L1CacheMachId_to_L2CacheCenterBank, each local L1 maps
to one only center bank

Check this code line for the mapping functions abovep:

L2bank = addr.bitSelect(RubyConfig::dataBlockBits(),

RubyConfig::dataBlockBits()+RubyConfig::DNUCABankSetBits()-1) // lower
bits for the bank set
        + RubyConfig::numberOfDNUCABankSets()*L1CacheMachId.num
        + RubyConfig::numberOfL2CachePerChip()/2;  // center banks are
the higher half of

the first part addr.bitSelect chooses which one bank set in a
bankcluster maps the address to due to the static mapping of addresses
to banks and the
"RubyConfig::numberOfDNUCABankSets()*L1CacheMachId.num" multiplies the
bank id with the actual name of the L1 bank.

Because of that each L1 bank and therefore each core can only access
one center bank!

I really need the concept of having both inter and center bankclusters
for my implementation.

On a 8 cores with 256 banks and 16 bankclusters altogether, I should
have 8 local bankclusters, 4 inter and 4 center which means that some
processors should share the same inter (like 2 cores per inter) and
all of them should have access to the center one or sth like that.

SO my final question is if I am missing sth and the implementation is
exactly the same with the paper. I don't want to start changing the
code before making sure that what I have understand so far is correct.

I apologize for the long email but I wanted to provide you as much
info as possible to avoid any misinterpretation of my question.

Regards,
Dimitris
-- 
Dimitris