[DynInst_API:] [dyninst/dyninst] 091029: Don't use software breakpoints when creating traps...


Date: Wed, 26 Feb 2020 09:47:53 -0800
From: Xiaozhu Meng <noreply@xxxxxxxxxx>
Subject: [DynInst_API:] [dyninst/dyninst] 091029: Don't use software breakpoints when creating traps...
  Branch: refs/heads/capstone_integration
  Home:   https://github.com/dyninst/dyninst
  Commit: 091029ad62801355787390393f213bed9def0978
      https://github.com/dyninst/dyninst/commit/091029ad62801355787390393f213bed9def0978
  Author: Xiaozhu Meng <xmeng@xxxxxxxxxxx>
  Date:   2019-09-02 (Mon, 02 Sep 2019)

  Changed paths:
    M dyninstAPI/src/dynProcess.C

  Log Message:
  -----------
  Don't use software breakpoints when creating traps for springboards (#637)

We should only emit trap instructions and record trap mapping for trap springboards, without involving software breakpoints.


  Commit: a7d412137271bf0bd545151675c81d6f1b9be0c1
      https://github.com/dyninst/dyninst/commit/a7d412137271bf0bd545151675c81d6f1b9be0c1
  Author: Jonathon Anderson <17242663+blue42u@xxxxxxxxxxxxxxxxxxxxxxxx>
  Date:   2019-09-04 (Wed, 04 Sep 2019)

  Changed paths:
    M instructionAPI/src/InstructionDecoder-aarch64.C
    M instructionAPI/src/InstructionDecoder-aarch64.h
    M instructionAPI/src/aarch64_opcode_tables.C

  Log Message:
  -----------
  Use static AArch64 decoder tables. (#633)

Fixes dyninst/dyninst#630

Other improvements include faster compile time (for affected files), ~1.2s less lazy init time, ~7.8MiB smaller binary, and ~10MiB less heap usage.

* Fixes for the more pedantic issues

* Splice out the operandFactory tables.

* Splice out the nodeBranches tables.


  Commit: b69554bd2c70260298d609af95bb2f3a43a9893a
      https://github.com/dyninst/dyninst/commit/b69554bd2c70260298d609af95bb2f3a43a9893a
  Author: Tim Haines <thaines.astro@xxxxxxxxx>
  Date:   2019-09-07 (Sat, 07 Sep 2019)

  Changed paths:
    M cmake/Boost.cmake
    M cmake/ElfUtils.cmake
    M cmake/LibIberty.cmake
    M cmake/ThreadingBuildingBlocks.cmake

  Log Message:
  -----------
  Allow CMake files for dependencies to be included multiple times (#639)

Previously, multiple inclusions would result in the library being looked
up or downloaded/built multiple times. This ensures these actions happen
only once.


  Commit: af01eab4d78af03b493f52c8157a70144f106469
      https://github.com/dyninst/dyninst/commit/af01eab4d78af03b493f52c8157a70144f106469
  Author: Tim Haines <thaines.astro@xxxxxxxxx>
  Date:   2019-10-04 (Fri, 04 Oct 2019)

  Changed paths:
    M CMakeLists.txt
    M cmake/Boost.cmake
    M cmake/ElfUtils.cmake
    M cmake/LibIberty.cmake
    M cmake/ThreadingBuildingBlocks.cmake

  Log Message:
  -----------
  Use CMAKE_xx_COMPILER when building external dependencies (#636)


  Commit: 798fd8731e82c4a28432307ceef2f4697dcd2b2e
      https://github.com/dyninst/dyninst/commit/798fd8731e82c4a28432307ceef2f4697dcd2b2e
  Author: Tim Haines <thaines.astro@xxxxxxxxx>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M symtabAPI/doc/API/Types/Type.tex
    M symtabAPI/doc/symtabAPI.pdf

  Log Message:
  -----------
  Change dataTypeDefine -> DataTypedef in DataClass enum documentation (#650)

Fixes #623
* Change dataTypeDefine -> DataTypedef in DataClass enum
* Update PDF


  Commit: 8d9a99e99358f2c4ae46c7c932384439dfbe48d1
      https://github.com/dyninst/dyninst/commit/8d9a99e99358f2c4ae46c7c932384439dfbe48d1
  Author: Tim Haines <thaines.astro@xxxxxxxxx>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M symtabAPI/doc/API/Types/Type.tex
    M symtabAPI/doc/symtabAPI.pdf

  Log Message:
  -----------
  Fix typo in declaration of 'create' in Section 8.3 (#659)


  Commit: 67948100c6e7bdb8d23b4ca101eaf2d4a46855e2
      https://github.com/dyninst/dyninst/commit/67948100c6e7bdb8d23b4ca101eaf2d4a46855e2
  Author: Tim Haines <thaines.astro@xxxxxxxxx>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M cmake/ThreadingBuildingBlocks.cmake

  Log Message:
  -----------
  Update handling of TBB CMake file for clang (#654)

* Update version handling in TBB CMake file
* Fix passing 'compiler' to TBB build system for clang


  Commit: 210729ef1b95ba4e7b43a33ed613f5b4de655134
      https://github.com/dyninst/dyninst/commit/210729ef1b95ba4e7b43a33ed613f5b4de655134
  Author: Tim Haines <thaines.astro@xxxxxxxxx>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M symtabAPI/doc/API/Types/Type.tex
    M symtabAPI/doc/symtabAPI.pdf

  Log Message:
  -----------
  Fix spelling of getTypedefType in Section 8.1 of SymtabAPI docs (#660)


  Commit: 08743d9446c4ce2a494702087caedf10f82394b1
      https://github.com/dyninst/dyninst/commit/08743d9446c4ce2a494702087caedf10f82394b1
  Author: Tim Haines <thaines.astro@xxxxxxxxx>
  Date:   2019-10-10 (Thu, 10 Oct 2019)

  Changed paths:
    M symtabAPI/doc/API/Types/Type.tex
    M symtabAPI/doc/symtabAPI.pdf

  Log Message:
  -----------
  Rename getType() -> getDataClass() in Section 8.1 of SymtabAPI docs (#661)


  Commit: e5850fecd70d4a20b93f3fd442508dc7f16adc5d
      https://github.com/dyninst/dyninst/commit/e5850fecd70d4a20b93f3fd442508dc7f16adc5d
  Author: Tim Haines <thaines.astro@xxxxxxxxx>
  Date:   2019-10-11 (Fri, 11 Oct 2019)

  Changed paths:
    M parseAPI/h/CFG.h

  Log Message:
  -----------
  Delete duplicate friend declaration of Parser in Block (#649)

Fixes #615


  Commit: 22110074036737595aec6c2a73c0d1d578a3ff94
      https://github.com/dyninst/dyninst/commit/22110074036737595aec6c2a73c0d1d578a3ff94
  Author: Tim Haines <thaines.astro@xxxxxxxxx>
  Date:   2019-10-11 (Fri, 11 Oct 2019)

  Changed paths:
    M CMakeLists.txt

  Log Message:
  -----------
  Deprecate Blue Gene/Q support (#662)


  Commit: b7c9eee605e6067bd4744f38c96527134c01a9a7
      https://github.com/dyninst/dyninst/commit/b7c9eee605e6067bd4744f38c96527134c01a9a7
  Author: Tim Haines <thaines.astro@xxxxxxxxx>
  Date:   2019-10-16 (Wed, 16 Oct 2019)

  Changed paths:
    M CMakeLists.txt

  Log Message:
  -----------
  Reorder includes to fix hidden build dependencies (#665)

cmake/shared.cmake includes cmake/cap_def_arch.cmake which conditionally
sets cap_thread_db only if ThreadDB has been included (but does not
include it, itself).


  Commit: 2094db4aefcf4705fcec07ebe28a740ccec16582
      https://github.com/dyninst/dyninst/commit/2094db4aefcf4705fcec07ebe28a740ccec16582
  Author: Sasha NÃcolas <sasha@xxxxxxxxxxx>
  Date:   2019-10-30 (Wed, 30 Oct 2019)

  Changed paths:
    A .github/ISSUE_TEMPLATE/bug_report.md

  Log Message:
  -----------
  Update issue templates


  Commit: d8a5fb05bf9a255482686078306dd7980249053d
      https://github.com/dyninst/dyninst/commit/d8a5fb05bf9a255482686078306dd7980249053d
  Author: Sasha NÃcolas <sasha@xxxxxxxxxxx>
  Date:   2019-10-30 (Wed, 30 Oct 2019)

  Changed paths:
    A .github/ISSUE_TEMPLATE/feature_request.md
    A .github/ISSUE_TEMPLATE/question.md

  Log Message:
  -----------
  Update issue templates


  Commit: 03a33fb1d56403df4c57500700e34bd83e8d7ad9
      https://github.com/dyninst/dyninst/commit/03a33fb1d56403df4c57500700e34bd83e8d7ad9
  Author: Tim Haines <thaines.astro@xxxxxxxxx>
  Date:   2019-11-13 (Wed, 13 Nov 2019)

  Changed paths:
    M CMakeLists.txt
    M cmake/Boost.cmake
    M cmake/ElfUtils.cmake
    M cmake/LibIberty.cmake
    M cmake/ThreadingBuildingBlocks.cmake

  Log Message:
  -----------
  Allow sterile builds (#641)

A sterile build requires that all dependencies are already installed on the system and thus cannot be downloaded/built from source by the Dyninst build system.


  Commit: 1c738e787e619f2b4d838a2c373247e79163bb2c
      https://github.com/dyninst/dyninst/commit/1c738e787e619f2b4d838a2c373247e79163bb2c
  Author: Tim Haines <thaines.astro@xxxxxxxxx>
  Date:   2019-11-14 (Thu, 14 Nov 2019)

  Changed paths:
    M cmake/Modules/FindLibElf.cmake

  Log Message:
  -----------
  Add $INSTALL/lib/elfutils subdirectory to build paths (#680)


  Commit: 32abe5da78f1bd76e7c681b14200677c85c43b95
      https://github.com/dyninst/dyninst/commit/32abe5da78f1bd76e7c681b14200677c85c43b95
  Author: Tim Haines <thaines.astro@xxxxxxxxx>
  Date:   2019-11-15 (Fri, 15 Nov 2019)

  Changed paths:
    M examples/CMakeLists.txt
    M parseThat/CMakeLists.txt

  Log Message:
  -----------
  Add explicit ElfUtils dependency for ParseThat and examples (#678)

When linking the binaries for ParseThat and the examples, the transitive linkage between libsymtabAPI and libelf was not correctly resolved by the linker. Instead, it used the system libelf which may or may not be correct. This was discovered when using a non-system compiler with the --sysroot switch.


  Commit: 47336e6ac185bd2134ff9484c55ef62b7d76f9b7
      https://github.com/dyninst/dyninst/commit/47336e6ac185bd2134ff9484c55ef62b7d76f9b7
  Author: Xiaozhu Meng <mxz297@xxxxxxxxx>
  Date:   2019-11-15 (Fri, 15 Nov 2019)

  Changed paths:
    M dyninstAPI/h/BPatch_process.h
    M dyninstAPI/src/BPatch_process.C
    M dyninstAPI/src/dynProcess.C
    M dyninstAPI/src/dynProcess.h

  Log Message:
  -----------
  Implement the check of ThreadDB at BPatch level (#667)


  Commit: cf195af6655d688a611e499c43a88efc8dd2e0fb
      https://github.com/dyninst/dyninst/commit/cf195af6655d688a611e499c43a88efc8dd2e0fb
  Author: Xiaozhu Meng <mxz297@xxxxxxxxx>
  Date:   2019-11-16 (Sat, 16 Nov 2019)

  Changed paths:
    M dataflowAPI/src/liveness.C
    M dyninstAPI/src/dynProcess.C
    M dyninstAPI/src/dynProcess.h
    M dyninstAPI/src/emit-aarch64.C
    M dyninstAPI/src/pcEventHandler.C
    M dyninstAPI/src/pcEventHandler.h
    M dyninstAPI_RT/h/dyninstAPI_RT.h
    M dyninstAPI_RT/src/RTcommon.c
    M dyninstAPI_RT/src/RTcommon.h
    M dyninstAPI_RT/src/RTlinux.c
    M dyninstAPI_RT/src/RTthread-aarch64.c
    M proccontrol/src/linux.C
    M stackwalk/src/linuxbsd-swk.C

  Log Message:
  -----------
  Fix instrumentation regressions for libc-2.29 on ARM (#653)

* Fixes loading pre-instrumentation register values on aarch64
* Fix the handling of trap instruction on ARM for register liveness analysis
* Implement a byte matching heuristic for determining the length of _start
* Implement a spin lock use __atomic_test_and_set on ARM
*   1. Change stop signal from SIGBUS to (SIGRTMIN + 4) in dyninstAPI_RT.
*   2. In dyninstAPI, only do dyninstRT signal check if the signal is indeed (SIGRTMIN + 4)
* Refacotr BPatch and DyninstRT to use breakpoint as the way of communication


  Commit: 7718cda22a0119ab6281d2a771da3cc1ab70c789
      https://github.com/dyninst/dyninst/commit/7718cda22a0119ab6281d2a771da3cc1ab70c789
  Author: Tim Haines <thaines.astro@xxxxxxxxx>
  Date:   2019-11-16 (Sat, 16 Nov 2019)

  Changed paths:
    M .gitignore

  Log Message:
  -----------
  Ignore additional Eclipse file and spurious .gitignore (#681)

The instructionAPI/doc/.gitignore seems to be auto-generated when
building the documentation. It tries to ignore instructionAPI/doc/API
which is already in the git history, so it's best to just ignore it to
prevent its accidental inclusion.


  Commit: 18364c8a6160925f582ffa5bc37b24cc54854d45
      https://github.com/dyninst/dyninst/commit/18364c8a6160925f582ffa5bc37b24cc54854d45
  Author: Sasha NÃcolas <sasha@xxxxxxxxxxx>
  Date:   2019-11-16 (Sat, 16 Nov 2019)

  Changed paths:
    M symtabAPI/src/dwarfWalker.C

  Log Message:
  -----------
  Parse dwarf variables with abstract origin attribute (#642)


  Commit: 9fb741f03657036c96bd1c02ccba60489014349d
      https://github.com/dyninst/dyninst/commit/9fb741f03657036c96bd1c02ccba60489014349d
  Author: Sasha NÃcolas <sasha@xxxxxxxxxxx>
  Date:   2019-11-17 (Sun, 17 Nov 2019)

  Changed paths:
    M common/src/debug_common.C
    M common/src/debug_common.h
    M symtabAPI/h/RangeLookup.h
    M symtabAPI/src/Object-elf.C

  Log Message:
  -----------
  Allow same address range to map to different source lines in DWARF (#643)


  Commit: 0dffe9f69384b49858006c21667387053d3abcf2
      https://github.com/dyninst/dyninst/commit/0dffe9f69384b49858006c21667387053d3abcf2
  Author: Xiaozhu Meng <mxz297@xxxxxxxxx>
  Date:   2019-11-17 (Sun, 17 Nov 2019)

  Changed paths:
    M dyninstAPI/src/emit-x86.C

  Log Message:
  -----------
  Prevent corruption to rax during stack alignment on x86-64 (#670)


  Commit: d233ae7596cd74201d634c2f0f7d7a0e3d628e79
      https://github.com/dyninst/dyninst/commit/d233ae7596cd74201d634c2f0f7d7a0e3d628e79
  Author: Xiaozhu Meng <xmeng@xxxxxxxxxxx>
  Date:   2020-01-13 (Mon, 13 Jan 2020)

  Changed paths:
    M CMakeLists.txt
    M cmake/ElfUtils.cmake
    A cmake/Modules/FindValgrind.cmake
    M common/CMakeLists.txt
    M common/h/IBSTree-fast.h
    M common/h/IBSTree.h
    A common/h/concurrent.h
    R common/h/mcs-lock.h
    R common/h/pfq-rwlock.h
    R common/h/race-detector-annotations.h
    M common/src/arch-x86.C
    A common/src/concurrent.C
    M common/src/linuxKludges.C
    R common/src/mcs-lock.C
    M common/src/pfq-rwlock.C
    R common/src/race-detector-annotations.C
    M common/src/singleton_object_pool.h
    A common/src/vgannotations.h
    M dataflowAPI/h/slicing.h
    M dataflowAPI/src/debug_dataflow.C
    M dwarf/CMakeLists.txt
    M dwarf/h/dwarfFrameParser.h
    M dwarf/src/dwarfFrameParser.C
    M dyninstAPI/h/BPatch_type.h
    M dyninstAPI/src/BPatch.C
    M dyninstAPI/src/BPatch_addressSpace.C
    M dyninstAPI/src/BPatch_function.C
    M dyninstAPI/src/BPatch_module.C
    M dyninstAPI/src/BPatch_snippet.C
    M dyninstAPI/src/BPatch_type.C
    M dyninstAPI/src/Parsing.C
    M dyninstAPI/src/Parsing.h
    M dyninstAPI/src/Relocation/Springboard.C
    M dyninstAPI/src/function.C
    M dyninstAPI/src/parse-cfg.h
    M dyninstAPI/src/unix.C
    M dyninstAPI_RT/CMakeLists.txt
    M examples/CMakeLists.txt
    M instructionAPI/src/InstructionDecoderImpl.C
    M instructionAPI/src/Operation.C
    M parseAPI/CMakeLists.txt
    M parseAPI/h/CFG.h
    M parseAPI/h/CFGFactory.h
    M parseAPI/h/CodeSource.h
    M parseAPI/h/LockFreeQueue.h
    M parseAPI/src/Block.C
    M parseAPI/src/BoundFactCalculator.C
    M parseAPI/src/CFGFactory.C
    M parseAPI/src/CFGModifier.C
    M parseAPI/src/CodeObject.C
    M parseAPI/src/Function.C
    M parseAPI/src/IA_IAPI.C
    M parseAPI/src/IA_IAPI.h
    M parseAPI/src/IA_aarch64.h
    M parseAPI/src/IA_power.h
    M parseAPI/src/IndirectASTVisitor.C
    M parseAPI/src/IndirectASTVisitor.h
    M parseAPI/src/IndirectAnalyzer.C
    M parseAPI/src/IndirectAnalyzer.h
    M parseAPI/src/JumpTableFormatPred.C
    M parseAPI/src/JumpTableFormatPred.h
    M parseAPI/src/LoopAnalyzer.C
    M parseAPI/src/ParseData.C
    M parseAPI/src/ParseData.h
    M parseAPI/src/Parser-speculative.C
    M parseAPI/src/Parser.C
    M parseAPI/src/Parser.h
    M parseAPI/src/ParserDetails.C
    M parseAPI/src/ProbabilisticParser.C
    M parseAPI/src/ProbabilisticParser.h
    M parseAPI/src/SymtabCodeSource.C
    M parseAPI/src/ThunkData.C
    M parseAPI/src/debug_parse.C
    M parseThat/CMakeLists.txt
    M stackwalk/src/x86-swk.C
    M symtabAPI/CMakeLists.txt
    M symtabAPI/h/Aggregate.h
    M symtabAPI/h/Collections.h
    M symtabAPI/h/Function.h
    M symtabAPI/h/Module.h
    M symtabAPI/h/StringTable.h
    M symtabAPI/h/Symbol.h
    M symtabAPI/h/Symtab.h
    M symtabAPI/h/Type.h
    M symtabAPI/h/Variable.h
    M symtabAPI/src/Aggregate.C
    M symtabAPI/src/Collections.C
    M symtabAPI/src/Function.C
    M symtabAPI/src/Module.C
    M symtabAPI/src/Object-elf.C
    M symtabAPI/src/Object-elf.h
    M symtabAPI/src/Object.C
    M symtabAPI/src/Object.h
    M symtabAPI/src/Symtab-edit.C
    M symtabAPI/src/Symtab-lookup.C
    M symtabAPI/src/Symtab.C
    M symtabAPI/src/Type-mem.h
    M symtabAPI/src/Type.C
    M symtabAPI/src/Variable.C
    M symtabAPI/src/dwarfWalker.C
    M symtabAPI/src/dwarfWalker.h
    M symtabAPI/src/parseStab.C

  Log Message:
  -----------
  Parallel DWARF parsing and improved parallel code parsing (#651)

* changes for parallelizing symtabapi

* Tweaks to use OpenMP (still one last reducer left)

* Mark a false race in Type.C.
    std::call_once has a barrier at the end, so anything that happens inside is visible to anything that happens after.

* Replace the Cilk reducer with an OpenMP reduction

* Remove some dead code, and move the reducer into the source file.

* Mark the last race so far, move VG macros to a separate file.

* Annotate the two core locks, negating a lot of the race reports
    Implementation is not great though, I moved mcs_init to be a real function call.
    There might be a better way to do this, will investigate at some point.

* Very awkwardly annotate a parallel hashmap by key. Consider implementing better magic.

* Wrap the callback in `omp critical`, to make it parallel-safe no matter what happens.

* Swap out the vector for a TBB concurrent_queue.

* Remove some unnessesary annotations, using a proper OpenMP handles them.

* Revert the core lock annotations, they don't actually really work.

* Macro-replace the locks with boost equivalents. Revert this commit later once the testing is complete, or refactor it fully out.

* Replace the 1-entry cache with a vector that expands with the number of threads.

* Reuse the DwarfWalker between loop iterations, to save some work

* Use the standard allocator, so that Valgrind can track it properly

* Silence some warnings

* Replace the write with a CAS, letting Valgrind mostly ignore it.

* Strengthen the condition, its good enough for now.

* Re-annotate the hash map similar to an RW lock (which it technically is)

* Nearly fully fix the function-static annotations, using a very small constructor.

* Add libc++ annotations to vgannotations, and shift includes around to make them work

* Actually do the annotations right. I learned things today.

* Tell Helgrind to ignore a few more things

* Disable checking on everything

* A few tweaks to annotation

* Add h-b arcs to the pfq rwlock

* Apparently vectors didn't work like I thought they did. Whoops.

* Elfutils is now more thread-safe, and fix the annotations to keep DRD quiet.

* Make vgannotations.h local to Dyninst, and do something C++ for the lazy inits.

* Make the custom locks first-class C++ types, and compatible with C++17's syntax.

* Replace the lock implementations with more reasonable alternatives.

* Move the annotation to handle the case where std::pair does the write

* Unify all the TBB types under a Dyninst-tagged namespace, to permit refactor

Also fix a few whitespace errors near affected lines.

* Move concurrent_hash_map annotations into the unified class.

* Shift the thread-local stuff into a template class

* Wiggle the parallel loops and fine-grain the locking for a performance boost.

* Use Boost's call_once to try and be a bit more portable

* Put some parallelism into DwarfFrameParser. Fixes blue42u/dyninst#18 and blue42u/dyninst#19

* Use atomics for the reference counting. Fixes blue42u/dyninst#4.

* Add an extra lock to the StringTable, and use it to mediate access to the internals. Fixes blue42u/dyninst#20.

* Add a lock for inlines management, fixes blue42u/dyninst#22.

* Add a lock to the function frame vectors, fixes blue42u/dyninst#21.

* Replace a map+lock with a proper parallel hashmap for performance.

* Replace a mutex'd multi_index_container with a series of concurrent_hash_maps.

* Remove the comments that were left over from Cilkscreen race detection.

...I meant to do this a long time ago...

* Fix a number of minor issues, and one possibly important typo.

* Use RAII-style classes when handling the locks.

* Add a few defines for when Valgrind annotations are turned off.

* Parallelize some stuff in Object, makes the loading of files a little faster.

* Swap out some hash_maps for their concurrent forms, more easy parallelism.

* Actually do the parsing properly, and add a lock where it was needed.

* Adding more parallelism.

* Get rid of unnecessary serial code in finalization and add parallelism for hints initialization

* Removed a lock, and attempts to fix the resulting issues that arose afterwards.

* Tweaks to remove the phase-based approach

* 1. Fix missing parsed edges caused by early resuming frames
2. Always choosing the alphabetically smallest name for a function if there are multiple

* Delete swap_busy

* 1. Handle ud2 instruction, which will raise an undefined opcode exception. Therefore, control flow should not fall through
2. When deleting a bogus function, the reference counts of the blocks in the function should be decremented

* 1. Rewrite createAndRecordFrame to allow concurrent frame creations.
2. Simplify the use of frame status: BAD_LOOKUP means frame does not exists
   and a caller should only create a new frame when the result is BAD_LOOKUP

* 1. Resume functions as soon as a function finds a ret instruction
2. Rewrite parts of the tail call correction in finalization

* Estimate parsing task size by function symbol size
and launch large tasks first

* Delete omp critical in parisng of a fram

* Get rid of unnecessary assert

* Small changes to fix some maybe-races

* Munge the annotations for c_hash_map a little, and hotfix a possible race.

* Silence a number of warnings. Identations a mess but its quieter.

* Rewrite the c_hash_map to expose the accessors as a rwlock.

* Replace lock for delayed frames with concurrent hash map

* Parallelize SymtabCodeSource::init_hints

* Parallelize CodeObject::process_hints

* Fix an infinite loop in the parsing finalization stages.

* 1. Fix parallelization for CodeObject::process_hint(): a local variable declared is moved from outside loop to inside loop
2. Fix debug print crash in Parser.C
3. Some code cleanup

* ParseAPI now initialize hints in parallel. So, if the user of ParseAPI overloads the function for creating
ParseAPI::Function, that code should be thread-safe.

Change the DynCFGFactor in dyninstAPI to use mutex to be thread-safe.

* Lasily preparing range data for functions and blocks.

* Add block ranges and clean some dead code

* Add a CMake flag to enable Valgrind annotations

* Use dyn_c_vector, and disable the parallelism for ELF stuff for now.

* Rough additions of locks in various places to make things work.

* Properly initialize the Module in all constructors.

* Fix compilation

* Get rid unnecessary boost::lock_guard and use entry lookup in loop tree construction

* Use static AArch64 decoder tables. Fixes dyninst/dyninst#630.

Other improvements include faster compile time (for affected files),
~1s less load time, ~7.8MiB smaller binary, and ~200KiB more memory usage.

Performance effects not yet tested.

* Should not delete unused ParseAPI::Function during parsing because CFGFactory will do delete all created functions in its destructor.

* Use exchange instead of store to keep Valgrind happy.

* Several fixes for analyzing .a files

1. Rewrite the OverlappingParseData to correct handle overlapping code regions.
2. In .o files, code starts at address 0, so address 0 can be a valid address.
   So, change indirect call target from address 0 to max address to represent indiret calls
3. Use CodeSource to check address validity, which would allow cross CodeRegions valid address;
   but use CodeRegion for raw code bytes, because using CodeSource may return code bytes
   from other regions that are overlapping.

* Fixes for gap parsing, which uses a different interface to call Parser

* Fix range data related to gap parsing

* Fix loop tree callee construction on Power

* Fix function removing in parsing finalization

* Fix crahses of symtabAPI tests on power

* Fix deadlock in constructing analysis graph for jump table analysis.

* Type refcount refactor, part 1: s/Type*/boost::shared_ptr<Type>/g

* Purge Type::refCount and all related code bits. The shared_ptr's handle it now.

Also add proper SFINAE on that one template, since now typeScalar and Type
have the same size (thank GCC's automatic bitfield construction).

* Add the backwards compatbility layer

* Fixup for a minor issue that should have popped up sooner

* CFGFactory class does not need to inherit boost_guard

* Fix compilation on ARM, and fix part of the backwards-compat layer.

* Adjust cmake file to new elfutils

* libdyninstAPI_RT.so should not link against libgomp, which would
cause crash at program startup time due to calling into uninitialized
rewritten libc.

* Cleanup OpenMP handling in build system

* Fix compilation when USE_OpenMP is set to OFF

* Fix a debug print crash

* Fix infinite recursion caused by missing stack unwind debug info

* Disable installing trampolines in instrumentation

* Stack walk should always have an increasing SP on x86

* Several fixes for parallel code parsing

1. Add a jump table finalization step. The assumption here is that different jump tables
   do not share entries. So, if one jump table runs into another one, we know that the
   entries that are overlapping with the next table are out-of-bound.
2. Remove edges and blocks for created by out-of-bound jump table entries
3. Handle problems of failing to resolve jump tables caused by out-of-bound entries from
   other jump table entries.

Co-authored-by: Jonathon Anderson <17242663+blue42u@xxxxxxxxxxxxxxxxxxxxxxxx>
Co-authored-by: Tim Haines <thaines.astro@xxxxxxxxx>


  Commit: a0afdbb7993cb046edf404d556e32b4ead2374b7
      https://github.com/dyninst/dyninst/commit/a0afdbb7993cb046edf404d556e32b4ead2374b7
  Author: Xiaozhu Meng <mxz297@xxxxxxxxx>
  Date:   2020-01-23 (Thu, 23 Jan 2020)

  Changed paths:
    M dataflowAPI/src/AbslocInterface.C
    M parseAPI/src/SymbolicExpression.C

  Log Message:
  -----------
  Fixes for AbsRegion and jump table index slicing involving global variable (#695)

1. When converting AbsRegion, PC on x86/64 should use post-instruction address

2. During jump table index slicing, we should only read read-only memory locations.
   Note that we still allow reading writable memory locations during jump table reads
   as jump tables may be put in writable sections by compilers


  Commit: ee541552195c4b437b86f3ee3ee5fd9bb86fc704
      https://github.com/dyninst/dyninst/commit/ee541552195c4b437b86f3ee3ee5fd9bb86fc704
  Author: Xiaozhu Meng <mxz297@xxxxxxxxx>
  Date:   2020-01-29 (Wed, 29 Jan 2020)

  Changed paths:
    M dataflowAPI/doc/Examples.tex
    M dataflowAPI/doc/dataflowAPI.pdf

  Log Message:
  -----------
  Update Examples in dataflowAPI (#700)

* Update Examples.tex file in dataflowAPI
* Update pdf

Co-authored-by: Tim Haines <thaines.astro@xxxxxxxxx>


  Commit: f04cbfaf8fce079cccec64b6946fef9f631f2167
      https://github.com/dyninst/dyninst/commit/f04cbfaf8fce079cccec64b6946fef9f631f2167
  Author: Xiaozhu Meng <mxz297@xxxxxxxxx>
  Date:   2020-02-26 (Wed, 26 Feb 2020)

  Changed paths:
    A .github/ISSUE_TEMPLATE/bug_report.md
    A .github/ISSUE_TEMPLATE/feature_request.md
    A .github/ISSUE_TEMPLATE/question.md
    M .gitignore
    M CMakeLists.txt
    M cmake/Boost.cmake
    M cmake/ElfUtils.cmake
    M cmake/LibIberty.cmake
    M cmake/Modules/FindLibElf.cmake
    A cmake/Modules/FindValgrind.cmake
    M cmake/ThreadingBuildingBlocks.cmake
    M common/CMakeLists.txt
    M common/h/IBSTree-fast.h
    M common/h/IBSTree.h
    A common/h/concurrent.h
    R common/h/mcs-lock.h
    R common/h/pfq-rwlock.h
    R common/h/race-detector-annotations.h
    M common/src/arch-x86.C
    A common/src/concurrent.C
    M common/src/debug_common.C
    M common/src/debug_common.h
    M common/src/linuxKludges.C
    R common/src/mcs-lock.C
    M common/src/pfq-rwlock.C
    R common/src/race-detector-annotations.C
    M common/src/singleton_object_pool.h
    A common/src/vgannotations.h
    M dataflowAPI/doc/Examples.tex
    M dataflowAPI/doc/dataflowAPI.pdf
    M dataflowAPI/h/slicing.h
    M dataflowAPI/src/AbslocInterface.C
    M dataflowAPI/src/debug_dataflow.C
    M dataflowAPI/src/liveness.C
    M dwarf/CMakeLists.txt
    M dwarf/h/dwarfFrameParser.h
    M dwarf/src/dwarfFrameParser.C
    M dyninstAPI/h/BPatch_process.h
    M dyninstAPI/h/BPatch_type.h
    M dyninstAPI/src/BPatch.C
    M dyninstAPI/src/BPatch_addressSpace.C
    M dyninstAPI/src/BPatch_function.C
    M dyninstAPI/src/BPatch_module.C
    M dyninstAPI/src/BPatch_process.C
    M dyninstAPI/src/BPatch_snippet.C
    M dyninstAPI/src/BPatch_type.C
    M dyninstAPI/src/Parsing.C
    M dyninstAPI/src/Parsing.h
    M dyninstAPI/src/Relocation/Springboard.C
    M dyninstAPI/src/dynProcess.C
    M dyninstAPI/src/dynProcess.h
    M dyninstAPI/src/emit-aarch64.C
    M dyninstAPI/src/emit-x86.C
    M dyninstAPI/src/function.C
    M dyninstAPI/src/parse-cfg.h
    M dyninstAPI/src/pcEventHandler.C
    M dyninstAPI/src/pcEventHandler.h
    M dyninstAPI/src/unix.C
    M dyninstAPI_RT/CMakeLists.txt
    M dyninstAPI_RT/h/dyninstAPI_RT.h
    M dyninstAPI_RT/src/RTcommon.c
    M dyninstAPI_RT/src/RTcommon.h
    M dyninstAPI_RT/src/RTlinux.c
    M dyninstAPI_RT/src/RTthread-aarch64.c
    M examples/CMakeLists.txt
    M instructionAPI/src/InstructionDecoder-aarch64.C
    M instructionAPI/src/InstructionDecoder-aarch64.h
    M instructionAPI/src/InstructionDecoderImpl.C
    M instructionAPI/src/Operation.C
    M instructionAPI/src/aarch64_opcode_tables.C
    M parseAPI/CMakeLists.txt
    M parseAPI/h/CFG.h
    M parseAPI/h/CFGFactory.h
    M parseAPI/h/CodeSource.h
    M parseAPI/h/LockFreeQueue.h
    M parseAPI/src/Block.C
    M parseAPI/src/BoundFactCalculator.C
    M parseAPI/src/CFGFactory.C
    M parseAPI/src/CFGModifier.C
    M parseAPI/src/CodeObject.C
    M parseAPI/src/Function.C
    M parseAPI/src/IA_IAPI.C
    M parseAPI/src/IA_IAPI.h
    M parseAPI/src/IA_aarch64.h
    M parseAPI/src/IA_power.h
    M parseAPI/src/IndirectASTVisitor.C
    M parseAPI/src/IndirectASTVisitor.h
    M parseAPI/src/IndirectAnalyzer.C
    M parseAPI/src/IndirectAnalyzer.h
    M parseAPI/src/JumpTableFormatPred.C
    M parseAPI/src/JumpTableFormatPred.h
    M parseAPI/src/LoopAnalyzer.C
    M parseAPI/src/ParseData.C
    M parseAPI/src/ParseData.h
    M parseAPI/src/Parser-speculative.C
    M parseAPI/src/Parser.C
    M parseAPI/src/Parser.h
    M parseAPI/src/ParserDetails.C
    M parseAPI/src/ProbabilisticParser.C
    M parseAPI/src/ProbabilisticParser.h
    M parseAPI/src/SymbolicExpression.C
    M parseAPI/src/SymtabCodeSource.C
    M parseAPI/src/ThunkData.C
    M parseAPI/src/debug_parse.C
    M parseThat/CMakeLists.txt
    M proccontrol/src/linux.C
    M stackwalk/src/linuxbsd-swk.C
    M stackwalk/src/x86-swk.C
    M symtabAPI/CMakeLists.txt
    M symtabAPI/doc/API/Types/Type.tex
    M symtabAPI/doc/symtabAPI.pdf
    M symtabAPI/h/Aggregate.h
    M symtabAPI/h/Collections.h
    M symtabAPI/h/Function.h
    M symtabAPI/h/Module.h
    M symtabAPI/h/RangeLookup.h
    M symtabAPI/h/StringTable.h
    M symtabAPI/h/Symbol.h
    M symtabAPI/h/Symtab.h
    M symtabAPI/h/Type.h
    M symtabAPI/h/Variable.h
    M symtabAPI/src/Aggregate.C
    M symtabAPI/src/Collections.C
    M symtabAPI/src/Function.C
    M symtabAPI/src/Module.C
    M symtabAPI/src/Object-elf.C
    M symtabAPI/src/Object-elf.h
    M symtabAPI/src/Object.C
    M symtabAPI/src/Object.h
    M symtabAPI/src/Symtab-edit.C
    M symtabAPI/src/Symtab-lookup.C
    M symtabAPI/src/Symtab.C
    M symtabAPI/src/Type-mem.h
    M symtabAPI/src/Type.C
    M symtabAPI/src/Variable.C
    M symtabAPI/src/dwarfWalker.C
    M symtabAPI/src/dwarfWalker.h
    M symtabAPI/src/parseStab.C

  Log Message:
  -----------
  Merge branch 'master' into capstone_integration


  Commit: bd71e33191515a94b106891d7e81c813944f4e4a
      https://github.com/dyninst/dyninst/commit/bd71e33191515a94b106891d7e81c813944f4e4a
  Author: Xiaozhu Meng <mxz297@xxxxxxxxx>
  Date:   2020-02-26 (Wed, 26 Feb 2020)

  Changed paths:
    M dataflowAPI/src/stackanalysis.C

  Log Message:
  -----------
  Update stack analysis with capstone intregration


  Commit: d4d0e19a8865d331093d438577c5d8a72374cc1e
      https://github.com/dyninst/dyninst/commit/d4d0e19a8865d331093d438577c5d8a72374cc1e
  Author: Xiaozhu Meng <mxz297@xxxxxxxxx>
  Date:   2020-02-26 (Wed, 26 Feb 2020)

  Changed paths:
    M instructionAPI/src/Instruction.C

  Log Message:
  -----------
  Do not re-decode operands


  Commit: 34c57a60f2659265307a9c37b6f91f58c71c3e41
      https://github.com/dyninst/dyninst/commit/34c57a60f2659265307a9c37b6f91f58c71c3e41
  Author: Xiaozhu Meng <mxz297@xxxxxxxxx>
  Date:   2020-02-26 (Wed, 26 Feb 2020)

  Changed paths:
    M dataflowAPI/src/stackanalysis.C

  Log Message:
  -----------
  More updates to stack analysis


  Commit: 697d9e3d70f09f6f072893c7c9a84fb3c088a090
      https://github.com/dyninst/dyninst/commit/697d9e3d70f09f6f072893c7c9a84fb3c088a090
  Author: Xiaozhu Meng <mxz297@xxxxxxxxx>
  Date:   2020-02-26 (Wed, 26 Feb 2020)

  Changed paths:
    M instructionAPI/src/InstructionDecoder-Capstone-x86.C

  Log Message:
  -----------
  When Capstone fails to report read/written properties of an operand,
mark the operand as both read and written. So that the user of instructionAPI
can perform correct, consertative (while less precise)  analysis.


  Commit: 4febbe439f3335430fc36e6fbda653276505faba
      https://github.com/dyninst/dyninst/commit/4febbe439f3335430fc36e6fbda653276505faba
  Author: Xiaozhu Meng <mxz297@xxxxxxxxx>
  Date:   2020-02-26 (Wed, 26 Feb 2020)

  Changed paths:
    M instructionAPI/src/Operation.C

  Log Message:
  -----------
  movsd seems to represent both "move string to string" and "move from scalar to floating point".

"move string to string" is old instructions that uses and writes esi and edi, while "move from scalar" is AVX series,
which does not implicitly write to rsi or rdi.


  Commit: 65ce98e6f2294d6471e1c3aeada44e1c76b2ae55
      https://github.com/dyninst/dyninst/commit/65ce98e6f2294d6471e1c3aeada44e1c76b2ae55
  Author: Xiaozhu Meng <mxz297@xxxxxxxxx>
  Date:   2020-02-26 (Wed, 26 Feb 2020)

  Changed paths:
    M dataflowAPI/src/convertOpcodes.C

  Log Message:
  -----------
  Add missing instruction semantics for e_shl


Compare: https://github.com/dyninst/dyninst/compare/1d8816ded03e...65ce98e6f229

[← Prev in Thread] Current Thread [Next in Thread→]
  • [DynInst_API:] [dyninst/dyninst] 091029: Don't use software breakpoints when creating traps..., Xiaozhu Meng <=