Re: [DynInst_API:] Creating ParseAPI graphs to CFGs for externalrepresentations; adding line map information from external sources

[Keren Zhou <kz21@xxxxxxxx>]

Hi all,

 

We are trying to create CFG by parsing a dot file generated from NVIDIAâs binary utilities. We followed Xiaozhuâs method to inherit a CFGFactory class and implement mkfunction, mkblock, and mkedge methods. However, there are some obstacles that prevent us from adding blocks into their corresponding functions.

 

The main steps can be described as following:

 

ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ CodeSource * code_src = CudaCodeSource(dot_file);

ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ CodeObject * code_obj = CodeObject(code_src);

ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ code_obj->parse();

ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ const CodeObject::funclist & funcList = code_obj->funcs();

 

CodeObject calls parse to build necessary records from an internal Parser, which is created for binaries. We want to go around the parser because it gets information by interpreting binary encodings through InstructionDecoder that we do not need. We have already got all necessary information in the dot file, including function name, block relations, and interpreted instructions. Therefore, we tried to create Functions, Edges, Blocks independently without parser, but it turned out that some interfaces in Function Class are forbidden for outer usage. For instance,

 

1. After creating all blocks, thereâs no public method for adding blocks into the corresponding function.
2. Thereâs no public method for adding call edges into the corresponding function.

 

And here I attached our initial try by creating all functions in mkfunc interface. We failed because the commented lines are not open for access. Is there any way to pass our own parser into CodeObject? If not, could you add necessary interfaces for us? Or do you have other recommendations?

 

Function *CudaCFGFactory::mkfunc(Address addr, FuncSource src,

ÂÂstd::string name, CodeObject * obj, CodeRegion * region,

ÂÂDyninst::InstructionSource * isrc) {

 // find function by name

 for (auto *function : _functions) {

ÂÂÂ if (function->name == name) {

ÂÂÂÂÂ Function *ret_func = new Function(addr, name, obj, region, isrc);

ÂÂÂÂÂ //ret_func->_cache_valid = true;

ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ Â

ÂÂÂÂÂÂfuncs_.add(*ret_func);

ÂÂÂÂÂ bool first_entry = true;

ÂÂÂÂÂ for (auto *block : function->blocks) {

ÂÂÂÂÂÂÂ Block *ret_block = NULL;

ÂÂÂÂÂÂÂ if (_block_filter.find(block->id) == _block_filter.end()) {

ÂÂÂÂÂÂÂÂÂ ret_block = new Block(obj, region, block->insts[0]->offset);

ÂÂÂ ÂÂÂÂÂÂblocks_.add(*ret_block);

ÂÂÂÂÂÂÂ } else {

ÂÂÂÂÂÂÂÂÂ ret_block = _block_filter[block->id];

ÂÂÂÂÂÂÂ }

 

ÂÂÂÂÂÂÂ if (first_entry) {

ÂÂÂÂÂÂÂÂÂ ret_func->setEntryBlock(ret_block);

ÂÂÂÂÂÂÂÂÂ first_entry = false;

ÂÂÂÂÂÂÂ }

 

ÂÂÂÂÂÂÂ //ret_func->add_block(ret_block);

 

ÂÂÂÂÂÂÂ for (auto *target : block->targets) {

ÂÂÂÂÂÂÂÂÂ Block *ret_target_block = NULL;

ÂÂÂÂÂÂÂÂÂ if (_block_filter.find(target->block->id) == _block_filter.end()) {

ÂÂÂÂÂÂÂÂÂÂÂ ret_target_block = new Block(obj, region, target->block->insts[0]->offset);

ÂÂÂÂÂÂÂÂÂÂÂ blocks_.add(*ret_target_block);

ÂÂÂÂÂÂÂÂÂ } else {

ÂÂÂÂÂÂÂÂÂÂÂ ret_target_block = _block_filter[target->block->id];

ÂÂÂÂÂÂÂÂÂ }

 

ÂÂÂÂÂÂÂÂÂ Edge *ret_edge = NULL;

ÂÂÂÂÂÂÂÂÂ if (target->type == CudaParse::CALL) {

ÂÂÂÂÂÂÂÂÂÂÂ ret_edge = new Edge(ret_block, ret_target_block, CALL);

ÂÂÂÂÂÂÂÂÂÂÂ //ret_func->_call_edge_list.insert(ret_edge);

ÂÂÂÂÂÂÂÂÂ } else {Â // TODO(Keren): Add more edge types

ÂÂÂÂÂÂÂÂÂÂÂ ret_edge = new Edge(ret_block, ret_target_block, DIRECT);

ÂÂÂÂÂÂÂÂÂ }

 

ÂÂÂÂÂÂÂÂÂ ret_edge->install();

ÂÂÂÂÂÂÂÂÂ edges_.add(*ret_edge);

ÂÂÂÂÂÂÂ }

ÂÂÂÂÂ }

 

ÂÂÂÂÂ return ret_func;

ÂÂÂ }

 }

 

 return NULL;

 // iterate blocks

 // add blocks

 // iterate targets

 // add edges

}

 

Regards,

Keren

 

Sent from Mail for Windows 10

 

From: John Mellor-Crummey
Sent: Thursday, May 31, 2018 9:58 PM
To: Xiaozhu Meng
Cc: John Mellor-Crummey; Keren Zhou
Subject: Re: [DynInst_API:] Creating ParseAPI graphs to CFGs for externalrepresentations; adding line map information from external sources

 

Xiaozhu,

 

Thanks! Weâll give the CFG construction a try.

 

I believe that Bill thought that I could push in line maps, but I havenât tried it yet.

--
John Mellor-CrummeyÂÂÂÂÂÂÂÂÂÂÂ Professor
Dept of Computer ScienceÂÂÂÂÂ Rice University
email: johnmc@xxxxxxxxÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ phone: 713-348-5179

On May 31, 2018, at 8:55 AM, Xiaozhu Meng <mxz297@xxxxxxxxx> wrote:

 

Hi John,

 

I can answer the part about ParseAPI. You can definitely use the CFGFactory to construct your ParseAPI CFG, which should contains a bunch of ParseAPI::Function, ParseAPI::Block, and ParseAPI::Edge objects. Then, you can invoke loop analysis through Function objects. You won't have the SymtabCodeSource or CodeObject, but they are not needed for loop analysis.

 

Thanks,

 

--Xiaozhu 

 

On Tue, May 29, 2018 at 2:53 PM, John Mellor-Crummey <johnmc@xxxxxxxx> wrote:

We are interested in building ParseAPI CFGs for a GPU binary given a representation of the CFG for a GPU binary in as a graph in dot format. We are currently parsing the GPU CFG for each function out of dot, understand its blocks, edges, and calls. From scanning through the documentation in ParseAPI, it is not clear whether it would be feasible to simply use the CFG factory interface to supply graph elements to Dyninst to construct a CFG. If we did so, would it then be possible to use the Loop Analyzer to analyze this CFG? (At present, we have a copy of the LoopAnalyzer code that analyzes our lightweight internal representation of the dot graph, but it would be better for HPCToolkitâs hpcstruct to just work with one representation  - Dyninst CFGs for binaries.)

 

Also, can I push line map information into dyninst from the outside? Line maps for optimized GPU binaries canât be read from libdw. If I write my own reader, can I push information associating address ranges with source file and line? There is no information about compilation units, which is what makes NVIDIAâs cubin line maps unreadable with libdw. If I were to push information in from the outside about source lines and files, would I have to fake a compilation unit, or could I just put it in the default module?

 

We would appreciate any advice. If is easier to have a conversation than sending a long email, let me know.

--
John Mellor-Crummey Professor
Dept of Computer Science Rice University
email: johnmc@xxxxxxxx phone: 713-348-5179

 

_______________________________________________
Dyninst-api mailing list
Dyninst-api@xxxxxxxxxxx
https://lists.cs.wisc.edu/mailman/listinfo/dyninst-api