Detect more endings

[STKFLT]

I found some cases where Ropper would output fewer instructions for a given binary than ROPGadget. I found that the regex for end instructions on both x86 and ARM are overly specific:

x86: missing the retf/retn distinction

ARM: only included one type of pop when imo any pop into pc (aside from conditionals) should qualify as an end instruction. also the JOP code for ARM was missing some forms of bl and blx

Most of the regex here is pulled from ROPGadget and I attempted to validate as best I could against the instruction set documentation (https://iitd-plos.github.io/col718/ref/arm-instructionset.pdf)

There is also a regex compilation caching feature I added that relates to another pull request I am going to send soon.

[STKFLT]

after thinking more about this and experimenting some, it may make sense to modify this somewhat. on ARM, the pop (i.e. ldm) base register can be any register and as such, an instruction like ldmda r4!, {r0, r8, sb, sl, fp, sp, pc} is better categorized as a JOP gadget

[STKFLT]

Other case in question:
add sp, sp, #0xc; ldm sp!, {pc}

[STKFLT]

The refined split between ROP and JOP for ldm* instructions is:
its ROP if:

• its unconditional
• up/down bit set to 1, meaning it must go up the stack
• PSR & force user bit set to 0 or 1. I don't know enough about arm to understand the implications of this bit but it doesn't seem to change that we are changing pc based on a stack value and moving sp
• write back bit set to 1. If we aren't also moving sp then this doesn't behave like a ret style instruction
• base register is sp
• pc is in registers to load

its JOP if:

• its unconditional
  -up/down bit set to whatever
• PSR & force user bit set to whatever
• write back bit set to whatever
• base register is NOT sp
• pc is in registers to load

[sashs]

Hi.
Many thanks for you pull requests. I will check it.
One question:
In the file rop.py: Why did you add the imagebase in the for loop if you subtract it again afterwards?

[STKFLT]

I add it when disassembling the gadget so that relative branch targets are correct (I can show an example if you'd like), but then subtract it for the gadget address because there's logic elsewhere that adds the imageBase back on for printing e.g.

Ropper/ropper/gadget.py

Line 209 in 9a94802

toReturn = '%s (%s): ' % (cstr(toHex(self._lines[0][0] + self.imageBase, self.__arch.addressLength), analyseColor),cstr(toHex(address + self.imageBase, self.__arch.addressLength), Color.GREEN))

I'm open to trying to combine all of that so it's consistent, I just wanted to keep the changes narrowed down until you weighed in.

Are there other areas where the distinction between address and address+imageBase are important beyond the string functions in gadget.py?

[STKFLT]

Also I've totally just mixed together these pull requests at this point, sorry :/