Some mips64 binaries have ELFCLASS32

[nimrodpar]

irsb = pyvex.IRSB('\xdf\xbf\x00\x18\xdf\xbc\x00\x10\xdf\xb1\x00\x08\xdf\xb0\x00\x00', 0x5193c, archinfo.ArchMIPS32('Iend_BE'), opt_level=0)

gives:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/workspace/angr-dev/pyvex/pyvex/block.py", line 64, in __init__
    lift(self, data, max_bytes, max_inst, bytes_offset, opt_level, traceflags)
  File "/home/user/workspace/angr-dev/pyvex/pyvex/lift/__init__.py", line 100, in lift
    raise PyVEXError('\n\n'.join(errors))
pyvex.errors.PyVEXError: 
vex: priv/guest_mips_toIR.c:1219 (putIReg): Assertion `typeOfIRExpr(irsb->tyenv, e) == ty' failed.

The bytes should be:

0x5193c ld      $ra, 0x20+var_8($sp)
0x51940 ld      $gp, 0x20+var_10($sp)
0x51944 ld      $s1, 0x20+var_18($sp)
0x51948 ld      $s0, 0x20+var_20($sp)

(ODA confirms)

[rhelmot]

As far as I can tell, those instructions are only valid in MIPS64. This is what VEX seems to expect, and this is what the MIPS instruction reference seems to tell me.

[nimrodpar]

Indeed when i try it with MIPS64 it works, but the file it originated from is a MIPS32, and is read as such by cle.backends.elf.ELF.

I've added a file where this happens.
Running:

cle.backends.elf.ELF('ucc-1.0__Ou__dlinkDSR500-1.09.B61__libcurl.so.4.1.0', loader=cle.Loader('ucc-1.0__Ou__dlinkDSR500-1.09.B61__libcurl.so.4.1.0')).arch

Returns <Arch MIPS32 (BE)>.
If i try to IRSB the block at 0x3C5D8 (for instance):

bytes = [0x8f, 0xc2, 0x0, 0x2c, 0xac, 0x40, 0x0, 0x0, 0x8f, 0xc2, 0x0, 0x14, 0x8f, 0xc3, 0x0, 0xc, 0x8f, 0xc6, 0x0, 0x24, 0x27, 0xc8, 0x0, 0x10, 0x0, 0x40, 0x20, 0x2d, 0x0, 0x60, 0x28, 0x2d, 0x8f, 0xc7, 0x0, 0x28, 0x8f, 0x99, 0x83, 0x24]
irsb = pyvex.IRSB(''.join(map(chr,bytes)), 0x3c5d8, archinfo.ArchMIPS32(endness='Iend_BE'), opt_level=0)

I get:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/mnt/d/GitHub/angr-dev/pyvex/pyvex/block.py", line 64, in __init__
    lift(self, data, max_bytes, max_inst, bytes_offset, opt_level, traceflags)
  File "/mnt/d/GitHub/angr-dev/pyvex/pyvex/lift/__init__.py", line 100, in lift
    raise PyVEXError('\n\n'.join(errors))
PyVEXError:
vex: priv/guest_mips_toIR.c:1219 (putIReg): Assertion `typeOfIRExpr(irsb->tyenv, e) == ty' failed.

ucc-1.0__Ou__dlinkDSR500-1.09.B61__libcurl.so.4.1.0.zip

[rhelmot]

This is...... horrifying.

This binary is not in fact 32 bit but is 100% a mips64 binary. It's easy to tell this because all the addresses are 8-aligned instead of just 4-aligned, and it definitely expects the registers to be 8 bytes wide. Furthermore, readelf -h reveals that the e_flags section of the header includes a bit for mips64r2. HOWEVER, the binary is distributed as an elf with ELFCLASS32. I.... genuinely don't understand why anyone who's ever designed anything for MIPS thought anything they were doing was a good idea.

The best solution I can come up with is modifying angr/project.py to add a line, at the very end of Project.__init__, to say self.arch = archinfo.ArchMIPS64('IEnd_BE') just for this project. There's a very strong assumption baked into pretty much all of angr that the bitness of the binary and the bitness of the cpu it runs on are the same.

[nimrodpar]

Right. I don't want to name names, but the filename gives some hints as to the origin of the file o_0.

From what i gather, this is actually an "issue" in cle.backends.elf.ELF and maybe the fix should be there? I hacked this by retrying the creation of the IRSB with MIPS64 whenever i get the aforementioned error, and things seem to be legit.

[rhelmot]

Probably. I took a look at what it would take to fix and it's very scary - the "architecture" of the binary is used in several places to load elfclass-sized pieces or data, often implicitly, via clemory.read_addr_at. The fix would probably be adding a separate field for the elf wordsize and changing all those references to use that, and then adding a clause near where we look into the elf flags to determine the ARM and PPC abi weirdnesses and add a special case for this kind of binary, such that we set the arch correctly. We might also have to patch the "compatibility" checks in a couple of places to make sure that it's not seeing the 64 bit arch and assuming this binary can't link with 32 bit binaries.

[rhelmot]

(I have entirely too much stuff on my plate to do this patch myself, so this one is on your or any other contributor who wants to give it a shot)

[nimrodpar]

Another probably related error (comes from the same kind of weird files):

vex: priv/guest_mips_toIR.c:1020 (jmp_lit64): Assertion `dres->whatNext == Dis_Continue' failed.`

when trying to translate:

0x513ac li      $v0, 1 4
0x513b0 dsll32  $v0, 29 4
0x513b4 sltu    $v0, $a1, $v0 4
0x513b8 bnez    $v0, loc_513CC 4
0x513bc cins32  $v1, $a0, 0x14, 0xA 4

Bytes:

[0x24, 0x2, 0x0, 0x1, 0x0, 0x2, 0x17, 0x7c, 0x0, 0xa2, 0x10, 0x2b, 0x14, 0x40, 0x0, 0x4, 0x70, 0x83, 0x55, 0x33]

Arch: <Arch MIPS64 (BE)>

Originated from:
ucc-1.0__Ou__dlinkDWC2000-4.5.0.2__libcurl.so.4.1.zip

[rhelmot]

ok that one was weird. there were actually two bugs, one in pyvex, one in vex, intersecting very weirdly. The bug in pyvex was that we were claiming that our emulated mips cpu was baseline, which was causing MIPS to mark your cins32 instruction as undecodable. The second bug was that vex will assert out if it hits an undecodable instruction found in the branch delay slot. I've fixed both of these, you'll need to pull the newest pyvex and vex.

[github-actions]

This issue has been marked as stale because it has no recent activity. Please comment or add the pinned tag to prevent this issue from being closed.