Re: [DynInst_API:] Hung mutator on exit

[Emily Gember-Jacobson <jacobson@xxxxxxxxxxx>]

Hi Marc,

I haven't been able to reproduce this (8.2 on RHEL 6). What OS are you on?

Nothing stands out to me as problematic in your mutator.Â

Matt orÂJosh, do either of you have ideas here?

Best,

Emily

On Thursday, April 2, 2015, Marc BrÃnink <marc@xxxxxxxxx> wrote:

Hi Emily,

actually I have a hanging mutator every now and then with this very simple mutator/mutatee pair during destruction of the mutator. Using 8.2.

If you enable the thread in the mutatee, you can get a hanging mutator at a different location.

All hangs happen on exit.

Marc




On 31/03/2015 21:59, Emily Gember-Jacobson wrote:

Hi Marc,

Bill is out of the office this week, so I'll try to look into this for
you. Would it be possible for you to share the code to reproduce this hang?

Thanks,
Emily

On Tue, Mar 31, 2015 at 5:55 AM, Marc BrÃnink <marc@xxxxxxxxx
<mailto:marc@xxxxxxxxx>> wrote:

  I am struggling with a hung mutator on exit of the mutatee.
  It looks like it is related to event ordering.

  The process hangs if the post-UserThreadDestroy event is queued
  before the pre-Exit.

  --------------
  [pcEventMuxer.C/83]: PCEventMuxer waiting for events, non-blocking
  pcEventMuxer.C[407]: Begin callbackMux, process pointer =
  0x33ae2dc0, event post-UserThreadDestroy
  pcEventMuxer.C[439]: Added event post-UserThreadDestroy to mailbox,
  size now 1
  pcEventMuxer.C[411]: after muxing event, mailbox size is 1
  pcEventMuxer.C[407]: Begin callbackMux, process pointer =
  0x33ae2dc0, event pre-LWPDestroy
  pcEventMuxer.C[439]: Added event pre-LWPDestroy to mailbox, size now 2
  pcEventMuxer.C[411]: after muxing event, mailbox size is 2
  [pcEventMuxer.C:86] after PC event handling, 2 events in mailbox
  pcEventMuxer.C[464]: Returning event post-UserThreadDestroy from mailbox
  pcEventHandler.C[65]: attempting to handle event
  post-UserThreadDestroy on thread 4976/4977
  pcEventHandler.C[109]: stopping process for event handling
  dynProcess.C[1048]: Stopping process 4976
  pcEventMuxer.C[231]: Begin callbackMux, process pointer =
  0x33ae2dc0, event pre-Exit
  [pcEventMuxer.C:232] Exit callback
  pcEventMuxer.C[439]: Added event pre-Exit to mailbox, size now 2
  pcEventMuxer.C[236]: after muxing event, mailbox size is 2
  ... hang ...
  --------------


  In contrast if the pre-Exit is queued before, everything work fine
  because the stopping of the process is skipped.


  --------------
  [pcEventMuxer.C/83]: PCEventMuxer waiting for events, non-blocking
  [46912496468160]pcEventMuxer.__C[231]: Begin callbackMux, process
  pointer = 0x20cde620, event pre-Exit
  [pcEventMuxer.C:232] Exit callback
  pcEventMuxer.C[439]: Added event pre-Exit to mailbox, size now 1
  pcEventMuxer.C[236]: after muxing event, mailbox size is 1
  pcEventMuxer.C[407]: Begin callbackMux, process pointer =
  0x20cde620, event post-UserThreadDestroy
  pcEventMuxer.C[439]: Added event post-UserThreadDestroy to mailbox,
  size now 2
  pcEventMuxer.C[411]: after muxing event, mailbox size is 2
  pcEventMuxer.C[407]: Begin callbackMux, process pointer =
  0x20cde620, event pre-LWPDestroy
  pcEventMuxer.C[439]: Added event pre-LWPDestroy to mailbox, size now 3
  pcEventMuxer.C[411]: after muxing event, mailbox size is 3
  pcEventMuxer.C[407]: Begin callbackMux, process pointer =
  0x20cde620, event post-LWPDestroy
  pcEventMuxer.C[439]: Added event post-LWPDestroy to mailbox, size now 4
  pcEventMuxer.C[411]: after muxing event, mailbox size is 4
  [pcEventMuxer.C:86] after PC event handling, 4 events in mailbox
  pcEventMuxer.C[464]: Returning event pre-Exit from mailbox
  pcEventHandler.C[65]: attempting to handle event pre-Exit on thread
  4952/4952
  --------------


  A truncated backtrace:
  --------------
  #0 pthread_cond_wait@@GLIBC_2.3.2 () at
  ../nptl/sysdeps/unix/sysv/__linux/x86_64/pthread_cond___wait.S:185
  #1 0x00002aaaaee25c38 in wait<Mutex<false> > (m=...,
  this=0x2aaab40009b8)
    Âat /usr/include/boost/thread/__pthread/condition_variable.__hpp:184
  #2 wait (this=0x2aaab40009b8) at
  /home/marc/Downloads/__DyninstAPI-8.2/common/src/__dthread.h:121
  #3 MailboxMT::dequeue (this=0x2aaab40008c0, block=true)
    Âat
  /home/marc/Downloads/__DyninstAPI-8.2/proccontrol/__src/mailbox.C:156
  #4 0x00002aaaaee4aa01 in int_process::__waitAndHandleEvents
  (block=block@entry=false)
    Âat
  /home/marc/Downloads/__DyninstAPI-8.2/proccontrol/__src/process.C:1045
  #5 0x00002aaaaee733af in
  Dyninst::ProcControlAPI::__ProcessSet::stopProcs (this=0xca39e0)
    Âat
  /home/marc/Downloads/__DyninstAPI-8.2/proccontrol/__src/procset.C:1495
  #6 0x00002aaaaee39953 in
  Dyninst::ProcControlAPI::__Process::stopProc (this=<optimized out>)
    Âat
  /home/marc/Downloads/__DyninstAPI-8.2/proccontrol/__src/process.C:6406
  --------------


  Questions:
  1. Is this a known issue in Dyninst 8.2?
  2. Any idea what's wrong? It might very well be a problem in my mutator.


  Marc
  _________________________________________________
  Dyninst-api mailing list
  Dyninst-api@xxxxxxxxxxx <mailto:Dyninst-api@xxxxxxxxxxx>
  https://lists.cs.wisc.edu/__mailman/listinfo/dyninst-api
  <https://lists.cs.wisc.edu/mailman/listinfo/dyninst-api>