Re: [DynInst_API:] Hung process

[Bill Williams <bill@xxxxxxxxxxx>]

On 02/18/2015 01:00 PM, Gerard wrote:
> I'm not sure at which point it hangs, but I have set up an example that
> I think mimics what happens with my application. In this example the
> mutatee hangs after creating 32294 threads. I'm using the latest master
> version.
>
> Thanks,
>
> Gerard
I'll take a look and see what I can turn up.

How reproducible/deterministic is this hang for you?

> 2015-02-16 18:51 GMT+01:00 Bill Williams <bill@xxxxxxxxxxx
> <mailto:bill@xxxxxxxxxxx>>:
>
>     On 02/16/2015 10:35 AM, Gerard wrote:
>
>         Hi,
>
>         I'm having another problem. I increased the MAX_THREADS variable to
>         10240 because I need to instrument a mutatee that creates around
>         10000
>         threads. After increasing the variable I could instrument the
>         mutatee
>         without problems, but I added more snippets and the process hung
>         again,
>         this time after creating 3999 threads. Now, it doesn't matter if I
>         increase even more the constant, it still hangs.
>
>         Is there any other reason why the mutatee could hang or is there any
>         other limit somewhere?
>
>     I'm not aware of other hardcoded limits, and if you already had
>     tramp guards disabled, it seems likely that you're running into some
>     form of bad behavior at scale that's independent of MAX_THREADS.
>
>     Is it possible for you to put together a simple reproducer
>     (mutator/mutatee) and send that to me? Do you know whether the
>     process is hanging at a particular point (thread
>     creation/destruction, process exit)?
>
>         Thanks,
>
>         Gerard
>
>         2015-02-04 11:22 GMT+01:00 Gerard <nouboh@xxxxxxxxx
>         <mailto:nouboh@xxxxxxxxx>
>         <mailto:nouboh@xxxxxxxxx <mailto:nouboh@xxxxxxxxx>>>:
>
>              Thanks! That was exactly the problem. I have increased the
>         constant
>              MAX_THREADS and now I don't have this problem anymore. Is
>         there any
>              reason why the constant is set to 32 instead of a higher value?
>
>              I also have tried to find how to dynamically change
>              the DYNINST_max_num_threads value but I haven't found where
>         is it
>              implemented (version 8.2.1). And about disabling tramp
>         guards, I
>              already had them disabled so it seems that this workaround
>         does not
>              solve the problem.
>
>              Gerard
>
>              2015-02-03 20:13 GMT+01:00 Barton Miller <bart@xxxxxxxxxxx
>         <mailto:bart@xxxxxxxxxxx>
>              <mailto:bart@xxxxxxxxxxx <mailto:bart@xxxxxxxxxxx>>>:
>
>                  Disabling tramp guards certainly works if you really
>         know that
>                  you're not recursing.  That can be subtle and error
>         prone, which
>                  is why tramp guards were invented.  Proceed cautiously
>         here.
>
>                  --bart
>
>
>                  On 2/3/2015 12:02 PM, Matthew LeGendre wrote:
>
>                      Another possible fix may be to disable tramp
>         guards.  Tramp
>                      guards are used to prevent recursive
>         instrumentation.  For
>                      example, if you instrument malloc() with
>         instrumentation
>                      that calls malloc(), then tramp guards will prevent
>         you from
>                      going into infinite recursion.
>
>                      If you already know that your instrumentation can't
>                      infinitely recurse,
>                      then disabling tramp guards will give a big
>         performance win
>                      and may work around this hang.  To disable tramp
>         guards, put
>                      a call to BPatch::setTrampRecursive(____true)
>         before you
>                      insert instrumentation.
>
>                      -Matt
>                      ___________________________________________________
>                      Dyninst-api mailing list
>         Dyninst-api@xxxxxxxxxxx <mailto:Dyninst-api@xxxxxxxxxxx>
>         <mailto:Dyninst-api@xxxxxxxxxxxxx <mailto:Dyninst-api@xxxxxxxxxxx>>
>         https://lists.cs.wisc.edu/____mailman/listinfo/dyninst-api
>         <https://lists.cs.wisc.edu/__mailman/listinfo/dyninst-api>
>
>         <https://lists.cs.wisc.edu/__mailman/listinfo/dyninst-api
>         <https://lists.cs.wisc.edu/mailman/listinfo/dyninst-api>>
>
>
>                  ___________________________________________________
>                  Dyninst-api mailing list
>         Dyninst-api@xxxxxxxxxxx <mailto:Dyninst-api@xxxxxxxxxxx>
>         <mailto:Dyninst-api@xxxxxxxxxxxxx <mailto:Dyninst-api@xxxxxxxxxxx>>
>         https://lists.cs.wisc.edu/____mailman/listinfo/dyninst-api
>         <https://lists.cs.wisc.edu/__mailman/listinfo/dyninst-api>
>
>         <https://lists.cs.wisc.edu/__mailman/listinfo/dyninst-api
>         <https://lists.cs.wisc.edu/mailman/listinfo/dyninst-api>>
>
>
>
>
>
>         _________________________________________________
>         Dyninst-api mailing list
>         Dyninst-api@xxxxxxxxxxx <mailto:Dyninst-api@xxxxxxxxxxx>
>         https://lists.cs.wisc.edu/__mailman/listinfo/dyninst-api
>         <https://lists.cs.wisc.edu/mailman/listinfo/dyninst-api>
>
>
>
>     --
>     --bw
>
>     Bill Williams
>     Paradyn Project
>     bill@xxxxxxxxxxx <mailto:bill@xxxxxxxxxxx>

--
--bw

Bill Williams
Paradyn Project
bill@xxxxxxxxxxx