Re: [DynInst_API:] Hung mutator on exit

[Emily Gember-Jacobson <jacobson@xxxxxxxxxxx>]

Hi Marc,

Bill is out of the office this week, so I'll try to look into this for you. Would it be possible for you to share the code to reproduce this hang?

Thanks,

Emily

On Tue, Mar 31, 2015 at 5:55 AM, Marc BrÃnink <marc@xxxxxxxxx> wrote:

I am struggling with a hung mutator on exit of the mutatee.
It looks like it is related to event ordering.

The process hangs if the post-UserThreadDestroy event is queued before the pre-Exit.

--------------
[pcEventMuxer.C/83]: PCEventMuxer waiting for events, non-blocking
pcEventMuxer.C[407]: Begin callbackMux, process pointer = 0x33ae2dc0, event post-UserThreadDestroy
pcEventMuxer.C[439]: Added event post-UserThreadDestroy to mailbox, size now 1
pcEventMuxer.C[411]: after muxing event, mailbox size is 1
pcEventMuxer.C[407]: Begin callbackMux, process pointer = 0x33ae2dc0, event pre-LWPDestroy
pcEventMuxer.C[439]: Added event pre-LWPDestroy to mailbox, size now 2
pcEventMuxer.C[411]: after muxing event, mailbox size is 2
[pcEventMuxer.C:86] after PC event handling, 2 events in mailbox
pcEventMuxer.C[464]: Returning event post-UserThreadDestroy from mailbox
pcEventHandler.C[65]: attempting to handle event post-UserThreadDestroy on thread 4976/4977
pcEventHandler.C[109]: stopping process for event handling
dynProcess.C[1048]: Stopping process 4976
pcEventMuxer.C[231]: Begin callbackMux, process pointer = 0x33ae2dc0, event pre-Exit
[pcEventMuxer.C:232] Exit callback
pcEventMuxer.C[439]: Added event pre-Exit to mailbox, size now 2
pcEventMuxer.C[236]: after muxing event, mailbox size is 2
... hang ...
--------------


In contrast if the pre-Exit is queued before, everything work fine because the stopping of the process is skipped.


--------------
[pcEventMuxer.C/83]: PCEventMuxer waiting for events, non-blocking
[46912496468160]pcEventMuxer.C[231]: Begin callbackMux, process pointer = 0x20cde620, event pre-Exit
[pcEventMuxer.C:232] Exit callback
pcEventMuxer.C[439]: Added event pre-Exit to mailbox, size now 1
pcEventMuxer.C[236]: after muxing event, mailbox size is 1
pcEventMuxer.C[407]: Begin callbackMux, process pointer = 0x20cde620, event post-UserThreadDestroy
pcEventMuxer.C[439]: Added event post-UserThreadDestroy to mailbox, size now 2
pcEventMuxer.C[411]: after muxing event, mailbox size is 2
pcEventMuxer.C[407]: Begin callbackMux, process pointer = 0x20cde620, event pre-LWPDestroy
pcEventMuxer.C[439]: Added event pre-LWPDestroy to mailbox, size now 3
pcEventMuxer.C[411]: after muxing event, mailbox size is 3
pcEventMuxer.C[407]: Begin callbackMux, process pointer = 0x20cde620, event post-LWPDestroy
pcEventMuxer.C[439]: Added event post-LWPDestroy to mailbox, size now 4
pcEventMuxer.C[411]: after muxing event, mailbox size is 4
[pcEventMuxer.C:86] after PC event handling, 4 events in mailbox
pcEventMuxer.C[464]: Returning event pre-Exit from mailbox
pcEventHandler.C[65]: attempting to handle event pre-Exit on thread 4952/4952
--------------


A truncated backtrace:
--------------
#0Â pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1Â 0x00002aaaaee25c38 in wait<Mutex<false> > (m=..., this=0x2aaab40009b8)
  at /usr/include/boost/thread/pthread/condition_variable.hpp:184
#2Â wait (this=0x2aaab40009b8) at /home/marc/Downloads/DyninstAPI-8.2/common/src/dthread.h:121
#3Â MailboxMT::dequeue (this=0x2aaab40008c0, block=true)
  at /home/marc/Downloads/DyninstAPI-8.2/proccontrol/src/mailbox.C:156
#4Â 0x00002aaaaee4aa01 in int_process::waitAndHandleEvents (block=block@entry=false)
  at /home/marc/Downloads/DyninstAPI-8.2/proccontrol/src/process.C:1045
#5Â 0x00002aaaaee733af in Dyninst::ProcControlAPI::ProcessSet::stopProcs (this=0xca39e0)
  at /home/marc/Downloads/DyninstAPI-8.2/proccontrol/src/procset.C:1495
#6Â 0x00002aaaaee39953 in Dyninst::ProcControlAPI::Process::stopProc (this=<optimized out>)
  at /home/marc/Downloads/DyninstAPI-8.2/proccontrol/src/process.C:6406
--------------


Questions:
1. Is this a known issue in Dyninst 8.2?
2. Any idea what's wrong? It might very well be a problem in my mutator.


Marc
_______________________________________________
Dyninst-api mailing list
Dyninst-api@xxxxxxxxxxx
https://lists.cs.wisc.edu/mailman/listinfo/dyninst-api