Re: [DynInst_API:] instructionAPI fails on ARM 64


Date: Mon, 12 Sep 2016 19:33:43 -0500
From: Sunny Shah <ssunny@xxxxxxxxxxx>
Subject: Re: [DynInst_API:] instructionAPI fails on ARM 64
Hi Mark,

This message couldn't have come at a better time.

The "Instruction processing threw exception for instruction" messages are thrown when Dyninst cannot find the semantics for said instruction. The master branch does not contain semantics for load/store instructions; in fact, I just added them last Friday and are available on the arm64/feature/semantics branch.

The abort you're seeing with the hpcstruct-bin is because we added new binary functions in Dyninst over the last year, but ROSE (which we use for semantics) doesn't support them. I made and committed the necessary changes a few hours ago. Again, they are on the arm64/feature/semantics branch.

Using that branch should make these errors go away. However, I should add that ARM semantics are a work-in-progress: I'm continuing to work on adding support for more instructions and fixing whatever bugs I find. Do let me know about other asserts and errors you see!

Thanks,
Sunny


On 9/12/2016 5:59 PM, Mark W. Krentel wrote:
I think instructionAPI (or maybe parseAPI) is broken in master branch
on ARM (aarch64).  I'm seeing two problems.  The small one is a spew
of error messages, "Instruction processing threw exception for
instruction: ldr X17, [X16 + 3c0]".  The bigger one is raising an
abort.

I'm running this on leela inside Wisconsin CS.  Dyninst master branch
as of today (Sept 12).

commit 69783ea9172db27d3fe14cf3f9d36c3fd84fb66f
Merge: da2da48 3402dc2
Author: Bill Williams <wwilliam47@xxxxxxxxx>
Date:   Fri Sep 2 10:59:36 2016 -0500

    Merge pull request #164 from dyninst/release9.2/fixes/jenkins-fix
    CMake fixes for Cotire and GCC 4.4 compatibility

I have a unit test (parse.cpp) that calls Symtab::openFile() and
parseTypesNow() and then makes a CodeObject and calls parse(). Then,
the program iterates through every ParseAPI::Function, every basic
block and every instruction and prints the line map and inline
sequence for every instruction.

(1) The small problem is: running parse on a small, simple program,
(eg, itself), I get a spew of exceptions:

Instruction processing threw exception for instruction: ldr X17, [X16 + 878] Instruction processing threw exception for instruction: ldr X17, [X16 + 7e0] Instruction processing threw exception for instruction: ldr X17, [X16 + 840] Instruction processing threw exception for instruction: ldr X17, [X16 + 808] Instruction processing threw exception for instruction: ldr X17, [X16 + 800]
...

These seem to be coming from the PLT section in the binary.
0x800 = 2048, and 0x808 = 2056, etc.

0000000000401480 <_ZNSsC1EPKcRKSaIcE@plt>:
401480: b0000090 adrp x16, 412000 <__FRAME_END__+0xfbb0>
  401484:       f9440211        ldr     x17, [x16,#2048]
  401488:       91200210        add     x16, x16, #0x800
  40148c:       d61f0220        br      x17

0000000000401490 <__cxa_atexit@plt>:
401490: b0000090 adrp x16, 412000 <__FRAME_END__+0xfbb0>
  401494:       f9440611        ldr     x17, [x16,#2056]
  401498:       91202210        add     x16, x16, #0x808
  40149c:       d61f0220        br      x17

In this case, my unit test runs to completion, seems to produce valid
output and exits with code 0 (success).

However, the warnings suggest that something internal to instructionAPI
is unknown or unimplemented.

(2) Running parse on a larger, more complicated binary (eg, hpcstruct-bin)
triggers a sig abort.

$ ./parse hpcstruct-bin
Instruction processing threw exception for instruction: ldr X17, [X16 + d20] Instruction processing threw exception for instruction: ldr X17, [X16 + 7e0] Instruction processing threw exception for instruction: ldr X17, [X16 + bb0] Instruction processing threw exception for instruction: ldr X17, [X16 + 640]
parse[19973] 0.00010s  [FATAL]: not implemented yet:
parse[19973] 0.00032s [FATAL]: /home/krentel/newarch/externals/symtabAPI/dyninst/dataflowAPI/rose/semantics/BaseSemantics2.C:695 parse[19973] 0.00103s [FATAL]: virtual rose::BinaryAnalysis::InstructionSemantics2::BaseSemantics::SValuePtr rose::BinaryAnalysis::InstructionSemantics2::BaseSemantics::Dispatcher::read(SgAsmExpression*, size_t, size_t)
parse[19973] 0.00235s  [FATAL]:   SgAsmBinaryLsl
Aborted (core dumped)

(gdb) bt
#0  0x000003ff970c2d38 in raise () from /lib64/libc.so.6
#1  0x000003ff970c4aa8 in abort () from /lib64/libc.so.6
#2 0x000003ff97ad77d4 in Sawyer::Assert::fail (mesg=mesg@entry=0x3ff97b759f0 "not implemented yet", expr=expr@entry=0x0, note="SgAsmBinaryLsl", filename=filename@entry=0x3ff97b7cec8 "/home/krentel/newarch/externals/symtabAPI/dyninst/dataflowAPI/rose/semantics/BaseSemantics2.C",
    linenum=linenum@entry=695,
funcname=funcname@entry=0x3ff97b7ccd8 <rose::BinaryAnalysis::InstructionSemantics2::BaseSemantics::Dispatcher::read(SgAsmExpression*, unsigned long, unsigned long)::__PRETTY_FUNCTION__> "virtual rose::BinaryAnalysis::InstructionSemantics2::BaseSemantics::SValuePtr rose::BinaryAnalysis::InstructionSemantics2::BaseSemantics::Dispatcher::read(SgAsmExpression*, size_t, size_t)") at /home/krentel/newarch/externals/symtabAPI/dyninst/dataflowAPI/rose/util/Assert.C:39 #3 0x000003ff97b00d04 in rose::BinaryAnalysis::InstructionSemantics2::BaseSemantics::Dispatcher::read (this=0x1b8c8330, e=0x155ffd20, value_nbits=32, addr_nbits=<optimized out>) at /home/krentel/newarch/externals/symtabAPI/dyninst/dataflowAPI/rose/semantics/BaseSemantics2.C:695 #4 0x000003ff97b0c924 in rose::BinaryAnalysis::InstructionSemantics2::ARM64::IP_add_addsub_ext_execute::p (this=<optimized out>, d=0x1b8c8330,
    ops=0x1b8ac8d0, insn=<optimized out>, args=..., raw=2334173216)
at /home/krentel/newarch/externals/symtabAPI/dyninst/dataflowAPI/rose/semantics/DispatcherARM64.C:216 #5 0x000003ff97b03484 in rose::BinaryAnalysis::InstructionSemantics2::ARM64::InsnProcessor::process (this=0x15ff8e50, dispatcher_=..., insn_=<optimized out>) at /home/krentel/newarch/externals/symtabAPI/dyninst/dataflowAPI/rose/semantics/DispatcherARM64.C:47 #6 0x000003ff97afeecc in rose::BinaryAnalysis::InstructionSemantics2::BaseSemantics::Dispatcher::processInstruction (this=0x1b8c8330, insn=0x15a64890) at /home/krentel/newarch/externals/symtabAPI/dyninst/dataflowAPI/rose/semantics/BaseSemantics2.C:460 #7 0x000003ff97a2503c in Dyninst::DataflowAPI::SymbolicExpansion::expandAarch64 (rose_insn=rose_insn@entry=0x15a64890, ops=..., insn_dump="add X0, X1, W0 << 2") at /home/krentel/newarch/externals/symtabAPI/dyninst/dataflowAPI/src/SymbolicExpansion.C:90 #8 0x000003ff97abed38 in Dyninst::DataflowAPI::SymEval::expandInsn (insn=..., addr=<optimized out>, res=std::map with 1 elements = {...}) at /home/krentel/newarch/externals/symtabAPI/dyninst/dataflowAPI/src/SymEval.C:505 #9 0x000003ff97abfcc4 in Dyninst::DataflowAPI::SymEval::expand (res=std::map with 1 elements = {...}, failedInsns=std::set with 0 elements, applyVisitors=applyVisitors@entry=false) at /home/krentel/newarch/externals/symtabAPI/dyninst/dataflowAPI/src/SymEval.C:94 #10 0x000003ff97abff74 in Dyninst::DataflowAPI::SymEval::expand (assignment=..., applyVisitors=applyVisitors@entry=false) at /home/krentel/newarch/externals/symtabAPI/dyninst/dataflowAPI/src/SymEval.C:75 #11 0x000003ff9798ac30 in JumpTablePred::ExpandAssignment (this=this@entry=0x3ffc34a97d0, assign=...) at /home/krentel/newarch/externals/symtabAPI/dyninst/parseAPI/src/JumpTablePred.C:401 #12 0x000003ff9798e574 in JumpTablePred::addNodeCallback (this=0x3ffc34a97d0, ap=..., visitedEdges=std::set with 0 elements) at /home/krentel/newarch/externals/symtabAPI/dyninst/parseAPI/src/JumpTablePred.C:192 #13 0x000003ff979f1190 in Dyninst::Slicer::updateAndLink (this=this@entry=0x3ffc34a99c0, g=..., dir=dir@entry=Dyninst::Slicer::backward, cand=..., cache=..., p=...) at /home/krentel/newarch/externals/symtabAPI/dyninst/dataflowAPI/src/slicing.C:398 #14 0x000003ff979f1fe0 in Dyninst::Slicer::sliceInternalAux (this=this@entry=0x3ffc34a99c0, g=..., dir=dir@entry=Dyninst::Slicer::backward, p=..., cand=..., skip=skip@entry=false, visited=std::map with 1 elements = {...}, singleCache=std::map with 2 elements = {...}, cache=std::map with 0 elements) at /home/krentel/newarch/externals/symtabAPI/dyninst/dataflowAPI/src/slicing.C:231 #15 0x000003ff979f266c in Dyninst::Slicer::sliceInternalAux (this=this@entry=0x3ffc34a99c0, g=..., dir=dir@entry=Dyninst::Slicer::backward, p=..., cand=..., skip=skip@entry=true, visited=std::map with 1 elements = {...}, singleCache=std::map with 2 elements = {...}, cache=std::map with 0 elements) at /home/krentel/newarch/externals/symtabAPI/dyninst/dataflowAPI/src/slicing.C:307 #16 0x000003ff979f2de0 in Dyninst::Slicer::sliceInternal (this=this@entry=0x3ffc34a99c0, dir=dir@entry=Dyninst::Slicer::backward, p=...) at /home/krentel/newarch/externals/symtabAPI/dyninst/dataflowAPI/src/slicing.C:195 #17 0x000003ff979f321c in Dyninst::Slicer::backwardSlice (this=this@entry=0x3ffc34a99c0, predicates=...) at /home/krentel/newarch/externals/symtabAPI/dyninst/dataflowAPI/src/slicing.C:1394 #18 0x000003ff979aa8d4 in IndirectControlFlowAnalyzer::NewJumpTableAnalysis (this=this@entry=0x3ffc34a9ca0, outEdges=std::vector of length 0, capacity 0) at /home/krentel/newarch/externals/symtabAPI/dyninst/parseAPI/src/IndirectAnalyzer.C:45 #19 0x000003ff97984034 in Dyninst::InsnAdapter::IA_aarch64Details::parseJumpTable (this=<optimized out>, currFunc=<optimized out>, currBlk=0x1b8c7400, outEdges=std::vector of length 0, capacity 0) at /home/krentel/newarch/externals/symtabAPI/dyninst/parseAPI/src/IA_aarch64Details.C:172 #20 0x000003ff9796a6e0 in Dyninst::InsnAdapter::IA_IAPI::parseJumpTable (this=this@entry=0x1b8c7480, currFunc=currFunc@entry=0x1b4f1d50, currBlk=0x1b790800, currBlk@entry=0x1b8c7400, outEdges=std::vector of length -274768239832, capacity -274768239832 = {...}) at /home/krentel/newarch/externals/symtabAPI/dyninst/parseAPI/src/IA_IAPI.C:958 #21 0x000003ff9796ffe4 in Dyninst::InsnAdapter::IA_IAPI::getNewEdges (this=0x1b8c7480, outEdges=std::vector of length 0, capacity 0, context=0x1b4f1d50, Python Exception <type 'exceptions.ValueError'> Cannot find type const std::set<unsigned long, std::less<unsigned long>, std::allocator<unsigned long> >::_Rep_type: currBlk=0x1b8c7400, num_insns=56, plt_entries=<optimized out>, knownTargets=std::set with 11 elements) at /home/krentel/newarch/externals/symtabAPI/dyninst/parseAPI/src/IA_IAPI.C:681 #22 0x000003ff9793d3fc in Dyninst::ParseAPI::Parser::ProcessCFInsn (this=this@entry=0x1a1f3f60, frame=..., cur=cur@entry=0x1b8c7400, ah=...) at /home/krentel/newarch/externals/symtabAPI/dyninst/parseAPI/src/ParserDetails.C:425 #23 0x000003ff97943cf0 in Dyninst::ParseAPI::Parser::parse_frame (this=this@entry=0x1a1f3f60, frame=..., recursive=recursive@entry=true) at /home/krentel/newarch/externals/symtabAPI/dyninst/parseAPI/src/Parser.C:1122 #24 0x000003ff979464dc in Dyninst::ParseAPI::Parser::parse_frames (this=this@entry=0x1a1f3f60, work=std::vector of length 3491, capacity 4096 = {...}, recursive=recursive@entry=true) at /home/krentel/newarch/externals/symtabAPI/dyninst/parseAPI/src/Parser.C:409 #25 0x000003ff97947c28 in Dyninst::ParseAPI::Parser::parse_vanilla (this=this@entry=0x1a1f3f60) at /home/krentel/newarch/externals/symtabAPI/dyninst/parseAPI/src/Parser.C:282 #26 0x000003ff97947df0 in Dyninst::ParseAPI::Parser::parse (this=0x1a1f3f60) at /home/krentel/newarch/externals/symtabAPI/dyninst/parseAPI/src/Parser.C:160 #27 0x0000000000401e48 in main (argc=<optimized out>, argv=<optimized out>) at parse.cpp:149

----------

My working directory on leela is: /home/krentel/newarch/.  It should
be open to someone with an account on leela.  I've copied these files
into newarch/Files to save for you.

-r--r--r--. 1 krentel krentel 155320320 Sep 12 18:21 core.19973
-rwxr-xr-x. 1 krentel krentel  19330363 Sep 12 18:20 hpcstruct-bin
-rwxr-xr-x. 1 krentel krentel    472639 Sep 12 18:10 parse
-rw-r--r--. 1 krentel krentel      3530 Sep 12 18:10 parse.cpp

Running parse on itself produces the small problem, running it on
hpcstruct-bin produces the sig abort.

Is there a different branch that I should try for ARM?  I thought most
of the arm64 branch was merged into trunk a couple months ago.

--Mark

_______________________________________________
Dyninst-api mailing list
Dyninst-api@xxxxxxxxxxx
https://lists.cs.wisc.edu/mailman/listinfo/dyninst-api

[← Prev in Thread] Current Thread [Next in Thread→]