[Gems-users] Regarding memory consistency issues

[Enrique Vallejo Gutiérrez <enrique@xxxxxxxxxxxxx>]

Title: Regarding memory consistency issues

Dear list,

        I have some doubts about consistency issues in Gems. In $GEMS/opal/Readme.memory_consistency it is clear that Opal does not enforce SC. In that file it is said that, given that Simics only allows SC memory orderings and opal checks against Simics on instruction retirements, other memory orderings would be detected as consistency violations in the opal stats file.

        The main reasons to argue that Simics follows a SC memory model are that:

- there is only a single global memory image (for example, no cached data)

- Simulation is performed alternating processor flows, this is, there is no parallelism on the requests (one at a time), and each request accesses the same global memory image.

This is the typical programmer view of SC: "a single global memory and a switch that connects an arbitrary processor to memory at any time step" (from the Consistency tutorial by S. Adve and K. Gharachorloo)

        However, I understand that Ruby by itself can model systems with different memory models, forcing Simics to follow that model. Ruby does not follow these limitations:

- There is parallelism in the processor requests, as processors get stalled until their data is ready. Requests from all of them can be performed at the same time on the network.

- Usually, there is a single global memory, as caches do not hold actual values. However, in some cases this might not be true:

        1.- The TSO parameter determines if a StoreBuffer is used for stores sent to the main memory. When a load hits the local store buffer, data is not received from the single global memory, but from the private StoreBufferEntry's Subblock (which holds the actual data)

        2.- If the DATA_BLOCK parameter is true (default: false) or the coherence protocol has m_XactMemory set to true (i.e., LogTM), then the current cache data is actually held in the m_DataBlk DataBlock (as I guess from the code)

Given that Ruby is installed with the snoop_device interface, it can modify the values that each processor reads, and then it can force a different memory model.

        So, my main questions are:

- Can Ruby really model a different memory consistency model, or would Simics "break down" in case of any SC violation? I have checked that LogTM protocols hold cache data, but I don't know how to make a quick test about the memory consistency model presented (Something like TSOTool or similar). The ideas in the previous paragraph are what I understand from the code, but I am not 100% sure about it.

- What about Ruby + Opal? Opal's Readme file states that only SC memory orderings are allowed as registers are checked against Simics, but Simics load results can be modified by Ruby, which might allow for other (TSO) consistency models. I am not sure if the snoop_device interface is installed when both Opal and Ruby are loaded.

- In conclusion, can Ruby be used to study different consistency models performance?

        I would appreciate if anyone could correct any wrong ideas in this mail, and could give some hints on his own experience.

        Thanks in advance,

Enrique Vallejo

University of Cantabria, Spain