[DynInst_API:] [PATCH] Fix DYNINST_index_lock state and ppc64 writeFunctionPtr

[Josh Stone <jistone@xxxxxxxxxx>]

There are two fixes in this patch to resolve hangs that we've seen on
ppc64 tests, most notably in test_thread_5.

The first is that DYNINST_index_lock may be left in a locked state from
DYNINSTthreadIndexSLOW when DYNINST_thread_hash_size is 0.  This simply
needs an unlock in that error path.

The second resolves *why* DYNINST_thread_hash_size is 0, even after it
was correctly initialized to 40.  This turned out to be corruption when
the mutator writeFunctionPtr sets DYNINST_pthread_self.  Those symbols
in libdyninstAPI_RT.so happen to be arranged like so:

    0000000000031180 B DYNINST_pthread_self
    0000000000031188 B DYNINST_sysEntry
    0000000000031190 B DYNINST_thread_hash_size

So writeFunctionPtr was sending three longs: the function descriptor
correctly in DYNINST_pthread_self; the toc in DYNINST_sysEntry, a dead
variable; and the guilty 0x0 in DYNINST_thread_hash_size.  The only
thing a function pointer actually needs is the function descriptor.

For comparison, on EL5 and EL6 our build has the symbols like so:

    000000000002c400 B DYNINST_pthread_self
    000000000002c408 B DYNINSTlinkSave
    000000000002c410 B DYNINSTtocSave
    000000000002c418 B DYNINST_sysEntry
    000000000002c420 B DYNINST_thread_hash_tids
    000000000002c428 B DYNINST_thread_hash_size

So that still clobbered data, but DYNINSTlinkSave and DYNINSTtocSave are
both unused variables -- no harm done.

Signed-off-by: Josh Stone <jistone@xxxxxxxxxx>
---
 dyninstAPI/src/inst-power.C  | 13 ++-----------
 dyninstAPI_RT/src/RTthread.c |  1 +
 2 files changed, 3 insertions(+), 11 deletions(-)

diff --git a/dyninstAPI/src/inst-power.C b/dyninstAPI/src/inst-power.C
index 6bda333..157dc3f 100644
--- a/dyninstAPI/src/inst-power.C
+++ b/dyninstAPI/src/inst-power.C
@@ -2589,20 +2589,11 @@ bool writeFunctionPtr(AddressSpace *p, Address addr, func_instance *f)
 #else
     // 64-bit ELF PowerPC Linux uses r2 (same as AIX) for TOC base register
     if (p->getAddressWidth() == sizeof(uint64_t)) {
-        Address buffer[3];
         Address val_to_write = f->addr();
         // Use function descriptor address, if available.
         if (f->getPtrAddress()) val_to_write = f->getPtrAddress();
-        assert(p->proc());
-        Address toc = p->proc()->getTOCoffsetInfo(f);
-        buffer[0] = val_to_write;
-        buffer[1] = toc;
-        buffer[2] = 0x0;
-
-        if (!p->writeDataSpace((void *) addr, sizeof(buffer), buffer))
-            fprintf(stderr, "%s[%d]:  writeDataSpace failed\n",
-                            FILE__, __LINE__);
-        return true;
+        return p->writeDataSpace((void *) addr,
+                                 sizeof(val_to_write), &val_to_write);
     }
     else {
         // Originally copied from inst-x86.C
diff --git a/dyninstAPI_RT/src/RTthread.c b/dyninstAPI_RT/src/RTthread.c
index 9897563..2a80f58 100644
--- a/dyninstAPI_RT/src/RTthread.c
+++ b/dyninstAPI_RT/src/RTthread.c
@@ -119,6 +119,7 @@ unsigned DYNINSTthreadIndexSLOW(dyntid_t tid) {
      **/
     if (!DYNINST_thread_hash_size) {
         //Uninitialized tramp guard.
+        tc_lock_unlock(&DYNINST_index_lock);
         return DYNINST_max_num_threads;
     }
 
-- 
1.8.3.1