Re: [DynInst_API:] Creating ParseAPI graphs to CFGsforexternalrepresentations; adding line map information from externalsources


Date: Mon, 25 Jun 2018 21:42:44 -0700
From: Keren Zhou <kz21@xxxxxxxx>
Subject: Re: [DynInst_API:] Creating ParseAPI graphs to CFGsforexternalrepresentations; adding line map information from externalsources

Hi Xiaozhu,

 

Thanks for your advice!

 

Even with a public set_cache_valid method, our example will not be working. As we leave parse_data_ in Parser as NULL, Function::setEntryBlock method which invokes obj()->parser->move_func will throw an error because we donât have a Parser instance. Note Function::setEntryBlock is not a virtual method so that inheritance couldnât solve the problem. For the reason, I think my previous solution that instantiate a âdummyâ parser might be better.

 

I also suffered from a similar problem in Block where getInsns instantiates a InstructionCoder instance that we do not need. Here I change Block::getInsns to virtual so that I can replace it with my own procedure, getting away from InstructionCoder structure.

 

Regards,

Keren

 

Sent from Mail for Windows 10

 

From: Xiaozhu Meng
Sent: Friday, June 22, 2018 8:10 AM
To: Keren Zhou
Cc: dyninst-api; John Mellor-Crummey; Mark W. Krentel
Subject: Re: [DynInst_API:] Creating ParseAPI graphs to CFGsforexternalrepresentations;adding line map information from externalsources

 

 

On Wed, Jun 20, 2018 at 11:44 AM Keren Zhou <kz21@xxxxxxxx> wrote:

Hi Xiaozhu,

 

I have built a first version to extract loops from the dot files.

 

I changed two places in the dyninst repo.

 

  1. Here I just make Function::add_block public.

https://github.com/Jokeren/dyninst/commit/38155e39bde2910780c0f2c8e74c5f9f5ef3f176 

 

  1. This may be a tricky one. We cannot leave _parse_data as NULL, since a bunch of methods in dyninst rely on _parse_data, such as Parser::add_hint(Function * f). Thus, I just assign an StandardParseData object to _parse_data. Maybe we could come up with better ideas.

https://github.com/Jokeren/dyninst/commit/74f0a082fa1bf0e4bcd5c05e0ac446d3c34f9300

 

Do you have advices on my modifications? Looking forward to your reply.

 

For the _parse_data issue, I guess it is probably because function queries triggered finalize(). In principle, you should not see any calls to Parser. Based on the code, it seems like I was wrong to claim that you don't need Function::_cache_valid. If _cache_valid is false, it will trigger the parser and adding a dummy _parse_data may temporarily fix the issue, but could have other side effects. So, I would recommend that you add a public interface to set the _cache_valid.

 

 

Regards,

Keren

 

Sent from Mail for Windows 10

 

From: Keren Zhou
Sent: Thursday, June 14, 2018 10:42 PM
To: Xiaozhu Meng
Cc: dyninst-api; John Mellor-Crummey; Mark W. Krentel
Subject: RE: [DynInst_API:] Creating ParseAPI graphs to CFGsforexternalrepresentations; adding line map information from externalsources

 

Thanks for the response!

 

I will follow your instructions and let you know if I have further questions.

 

Sent from Mail for Windows 10

 

From: Xiaozhu Meng
Sent: Thursday, June 14, 2018 3:40 AM
To: Keren Zhou
Cc: dyninst-api; John Mellor-Crummey; Mark W. Krentel
Subject: Re: [DynInst_API:] Creating ParseAPI graphs to CFGs forexternalrepresentations; adding line map information from external sources

 

Hi Keren,

 

I am sorry for the late response. See my inlined comments.

 

      //ret_func->_cache_valid = true;

 

In your case, I don't think you need to maintain _cache_valid, because _cache_valid is used by the Dyninst parser. You should be able to ignore it.

  

        //ret_func->add_block(ret_block);

 

Right now, there is indeed no easy way to add a block to a function, because Function::add_block is declared as a private member function. Personally, I am fine with changing Function::add_block to a public function. Making it public is consistent with an existing interface Edge::install(), which adds an edge to the source and target block.  

 

 

            //ret_func->_call_edge_list.insert(ret_edge);

 

There is no good interface for inserting call edges to a function. But I don't think this information is needed for loop analysis because the loop analysis in ParseAPI just iterate every edge of a basic block and ignore the call edges. If you want to maintain a list of call edges in a function, you can either create a new public interface in parseAPI/h/CFG.h to access _call_edge_list, or you can inherit the ParseAPI::Function class and maintain your own version of call edge list. 

 

Considering that you may want to maintain other information about a function, I would recommend inherit the ParseAPI::Function. 

 

 

          } else {  // TODO(Keren): Add more edge types

            ret_edge = new Edge(ret_block, ret_target_block, DIRECT);

          }

 

          ret_edge->install();

          edges_.add(*ret_edge);

        }

      }

 

      return ret_func;

    }

  }

 

  return NULL;

  // iterate blocks

  // add blocks

  // iterate targets

  // add edges

}

 

Regards,

Keren

 

Sent from Mail for Windows 10

 

From: John Mellor-Crummey
Sent: Thursday, May 31, 2018 9:58 PM
To: Xiaozhu Meng
Cc: John Mellor-Crummey; Keren Zhou
Subject: Re: [DynInst_API:] Creating ParseAPI graphs to CFGs for externalrepresentations; adding line map information from external sources

 

Xiaozhu,

 

Thanks! Weâll give the CFG construction a try.

 

I believe that Bill thought that I could push in line maps, but I havenât tried it yet.

--
John Mellor-Crummey            Professor
Dept of Computer Science      Rice University
email: johnmc@xxxxxxxx                     phone: 713-348-5179

 

On May 31, 2018, at 8:55 AM, Xiaozhu Meng <mxz297@xxxxxxxxx> wrote:

 

Hi John,

 

I can answer the part about ParseAPI. You can definitely use the CFGFactory to construct your ParseAPI CFG, which should contains a bunch of ParseAPI::Function, ParseAPI::Block, and ParseAPI::Edge objects. Then, you can invoke loop analysis through Function objects. You won't have the SymtabCodeSource or CodeObject, but they are not needed for loop analysis.

 

Thanks,

 

--Xiaozhu 

 

On Tue, May 29, 2018 at 2:53 PM, John Mellor-Crummey <johnmc@xxxxxxxx> wrote:

We are interested in building ParseAPI CFGs for a GPU binary given a representation of the CFG for a GPU binary in as a graph in dot format. We are currently parsing the GPU CFG for each function out of dot, understand its blocks, edges, and calls. From scanning through the documentation in ParseAPI, it is not clear whether it would be feasible to simply use the CFG factory interface to supply graph elements to Dyninst to construct a CFG. If we did so, would it then be possible to use the Loop Analyzer to analyze this CFG? (At present, we have a copy of the LoopAnalyzer code that analyzes our lightweight internal representation of the dot graph, but it would be better for HPCToolkitâs hpcstruct to just work with one representation  - Dyninst CFGs for binaries.)

 

Also, can I push line map information into dyninst from the outside? Line maps for optimized GPU binaries canât be read from libdw. If I write my own reader, can I push information associating address ranges with source file and line? There is no information about compilation units, which is what makes NVIDIAâs cubin line maps unreadable with libdw. If I were to push information in from the outside about source lines and files, would I have to fake a compilation unit, or could I just put it in the default module?

 

We would appreciate any advice. If is easier to have a conversation than sending a long email, let me know.

--
John Mellor-Crummey Professor
Dept of Computer Science Rice University
email: johnmc@xxxxxxxx phone: 713-348-5179

 


_______________________________________________
Dyninst-api mailing list
Dyninst-api@xxxxxxxxxxx
https://lists.cs.wisc.edu/mailman/listinfo/dyninst-api

 

 

 

 

 

 

[← Prev in Thread] Current Thread [Next in Thread→]