Re: [DynInst_API:] BPatch_image->findFunction() behavior with weak symbols

[Tim Haines <thaines.astro@xxxxxxxxx>]

>I can tell you already that there are differences between both demanglers (GNU vs. libiberty)

I have seen this before. Having another source for details on this is something I would be very interested in reading.

>In our system, I have noticed that the binary gets linked both to the libmpi.so (c implementation of the MPI library), but also against the libmpif.so (fortran implementation).

Are you using the mpicc compiler wrapper when you build/link? If so, you might try manually linking to just libmpi- assuming, of course, that it doesn't automatically transitively link to libmpif.

>So, my question is, is it possible to filter out the symbols in a particular library?

I will have to defer to someone else's expertise on this one. @Xiaozhu Meng, what are your thoughts on this?

Thanks.

- Tim

On Tue, Jul 23, 2019 at 11:18 AM GermÃn Llort <gllort@xxxxxx> wrote:

Hi!

I have been doing some more tests, and while I haven't compiled a full report yet, I can tell you already that there are differences between both demanglers (GNU vs. libiberty), and also between both parsing methods (findFunction vs getProcedures). However, in order to be able to make a report that makes sense, I'd need to be able to filter some symbols that come from a shared library in particular.

I am looking for the name of MPI functions in a binary that is linked against the MPI library. In our system, I have noticed that the binary gets linked both to the libmpi.so (c implementation of the MPI library), but also against the libmpif.so (fortran implementation). What I am trying to do is to search for the symbols that are used in the binary, in order to detect dynamically whether the application is C or Fortran. But since the app is linked against both MPI libraries, it doesn't matter if I look for the symbol "MPI_Init" (c implementation) or "mpi_init__" (one of the possible aliases for the fortran implementation), because if the symbol is found in any of the libraries, all the different possible names will be found in one or the other.

What I'd need precisely is to look for those symbols except in the MPI library. This is to say, I need to exclude the objects libmpi.so / libmpif.so from the search. I've seen that there are methods to list all modules and objects from a BPatch_image, but when I use those, I get only 1 module/object which is named as the binary.

So, my question is, is it possible to filter out the symbols in a particular library? Could you please point me in the right direction?

Thanks!!
-- G.

On 18/7/19 19:30, GermÃn Llort wrote:

Ok, thank you both for the suggestions. I will test all these different things and get back to you with the results in a few days :)

Thanks!
-- G.

On 18/7/19 19:20, Tim Haines wrote:

> Should I try the GNU demangler instead?

If you have the time, I would try rebuilding Dyninst with it just to double-check. Admittedly, this is an area where Dyninst could use some improvements. Please note that changes to the build system require you to delete any current installation you have in CMAKE_INSTALL_PREFIX.

Thanks.

Â- Tim

On Thu, Jul 18, 2019 at 3:11 AM German Llort Sanchez <gllort@xxxxxx> wrote:

Hi Tim!

I haven't tried Ben's suggestions yet, but I confirm that I'm using the
latest release 10.1, and also using the libiberty demangler. Should I
try the GNU demangler instead?

Thanks!
-- G.

El 2019-07-18 00:09, Tim Haines escribiÃ:
> Hi, German.
>
> Are you using Dyninst-10.1? If not, I would strongly recommend
> upgrading as there was an enhancement to the build system where the
> USE_GNU_DEMANGLER flag was fixed to work correctly. Previously, there
> was an ambiguity about which demangler was compiled into Dyninst:
> either GNU demangler or libiberty.
>
> Thanks.
>
>Â - Tim
>
> On Wed, Jul 17, 2019 at 3:45 PM German Llort Sanchez
> <german.llort@xxxxxx> wrote:
>
>> Hi,
>>
>> Just adding a little extra information, but I'm not sure if
>> clarifies
>> anything or opens a second issue. Checking the API of
>> BPatch_function I
>> noticed the method getNames (in plural), with this comment saying
>> that
>> it returns all names of the symbol including those weak. So I
>> figured
>> out that maybe the match returned by findFunction corresponds to
>> the
>> first name in the list, but maybe in the whole list of names all
>> the
>> other decorations appear, so I could check if the one I am
>> expecting for
>> is present in the list.
>>
>> I tried that, dumping the names like this:
>>
>> BPatch_Vector<const char *> names;
>> found_funcs[0]->getNames(names);
>>
>> cerr << "getRoutine: Searching for " << routine << " found " <<
>> found_funcs.size() << " with " << names.size() << " names: ";
>> for (int i=0; i<names.size(); i++)
>> {
>> cerr << names[i] << " ";
>> }
>>
>> And the output looks like this:
>>
>> getRoutine: Searching for MPI_Init found 1 with 4 names: `a `a `a
>> `a
>> getRoutine: Searching for mpi_init found 1 with 16 names: `a `a `a
>> `a `a
>> `a `a `a `a `a `a `a `a `a `a `a
>> getRoutine: Searching for mpi_init_ found 1 with 16 names: `a `a `a
>> `a
>> `a `a `a `a `a `a `a `a `a `a `a `a
>> getRoutine: Searching for mpi_init__ found 1 with 16 names: `a `a
>> `a `a
>> `a `a `a `a `a `a `a `a `a `a `a `a
>> getRoutine: Searching for MPI_INIT found 1 with 16 names: `a `a `a
>> `a `a
>> `a `a `a `a `a `a `a `a `a `aStarting program execution
>>
>> For each symbol that I look for, there's only 1 BPatch_function
>> match,
>> and for each object, there's several names found, but the names
>> look
>> like that (`a).
>>
>> Also note that the getNames method that I'm using from
>> BPatch_function
>> class is this one:
>>
>> bool getNames(BPatch_Vector<const char *> &names);
>>
>> There's another one with strings:
>>
>> bool getNames(std::vector<std::string> &names);
>>
>> But if I try to use the second one, I'm getting this linking error:
>>
>> launcher.cpp:(.text+0x2aa): undefined reference to
>> `BPatch_function::getNames(std::vector<std::string,
>> std::allocator<std::string> >&)'
>>
>> I have the same problem with several other methods that use the
>> string
>> interface.
>>
>> Thanks for your help!
>> -- G.
>>
>> El 2019-07-15 17:22, GermÃn Llort escribiÃ:
>>> Hi,
>>>
>>> We're using BPatch_image->findFunction() to look for MPI symbols
>> in a
>>> binary. When the binary is Fortran code, different compilers may
>> add
>>> different decorations to the MPI functions. For example, for
>>> "MPI_Init", different compilers may end up naming this function
>> like:
>>>
>>> - mpi_init (all lowercase)
>>> - mpi_init_ (1 underscore at the end)
>>> - mpi_init__ (2 underscores at the end)
>>> - MPI_INIT (all uppercase)
>>>
>>> And sometimes, all these variants are present.
>>>
>>> When we use findFunction to look for these different names, ,
>> there's
>>> always a match, but in all cases it matches with "mpi_init__" (2
>>> underscores). For example:
>>>
>>> found_funcs.clear();
>>> if (appImage->findFunction ("mpi_init", found_funcs) != NULL)
>>> {
>>> fprintf(stderr, "match %s\n",
>> found_funcs[0]->getName().c_str());
>>> }
>>>
>>> This results in finding "mpi_init__".
>>>
>>> We use this to discover dynamically the name mangling scheme that
>> was
>>> applied by the compiler, and since all these searches end up
>> matching
>>> with the symbol with 2 underscores, it's misleading for us.
>>>
>>> The particularity with these symbols is that they're defined as
>> weak.
>>> Is this the expected behavior of the search when the symbols are
>> weak?
>>> Or maybe we're doing something wrong?
>>>
>>> Thanks!
>>> -- G.
>>
>> http://bsc.es/disclaimer [1]
>> _______________________________________________
>> Dyninst-api mailing list
>> Dyninst-api@xxxxxxxxxxx
>> https://lists.cs.wisc.edu/mailman/listinfo/dyninst-api [2]
>
>
> Links:
> ------
> [1] http://bsc.es/disclaimer
> [2] https://lists.cs.wisc.edu/mailman/listinfo/dyninst-api


http://bsc.es/disclaimer

WARNING / LEGAL TEXT: This message is intended only for the use of the individual or entity to which it is addressed and may contain information which is privileged, confidential, proprietary, or exempt from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to the intended recipient, you are strictly prohibited from disclosing, distributing, copying, or in any way using this message. If you have received this communication in error, please notify the sender and destroy and delete any copies you may have received.

http://www.bsc.es/disclaimer

_______________________________________________
Dyninst-api mailing list
Dyninst-api@xxxxxxxxxxx
https://lists.cs.wisc.edu/mailman/listinfo/dyninst-api

WARNING / LEGAL TEXT: This message is intended only for the use of the individual or entity to which it is addressed and may contain information which is privileged, confidential, proprietary, or exempt from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to the intended recipient, you are strictly prohibited from disclosing, distributing, copying, or in any way using this message. If you have received this communication in error, please notify the sender and destroy and delete any copies you may have received.

http://www.bsc.es/disclaimer