Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PPC Support #204

Merged
merged 141 commits into from
Feb 3, 2022
Merged

PPC Support #204

merged 141 commits into from
Feb 3, 2022

Conversation

zbanks
Copy link
Collaborator

@zbanks zbanks commented Jan 30, 2022

This is the big PR that merges in the work of the ppc2cpp branch to add support for PPC & C++. Progress was primarily tracked in #187.

This branch has one commit on top of ppc2cpp which changes the default target arch & makes some small changes to the website. (The default target on ppc2cpp is ppc-mwcc-c++, but on the mips_to_c mainline we want to use mips-ido-c as the default.)

Known caveats (including comments marked with 🚀)

  • Not every instruction is (fully) implemented (see Common PPC instructions #187 & end-to-end tests)
  • PpcArch.function_abi() is relatively basic, and does not support passing args on the stack.
  • Correct NaN handling in float comparisons comment
  • Unify make_storex with handle_loadx comment
  • Refactor output_regs_for_instr to avoid deviating from process_instr comment

Diffs from OOT/MM (MIPS-only). The only diffs are related to error messages, and some constant folding (0x10000 + 0x4908 --> 0x14908).

matt-kempster and others added 30 commits December 5, 2021 16:09
@zbanks
Copy link
Collaborator Author

zbanks commented Jan 31, 2022

I'm still going through the comments (super helpful) - there are a few "heavier" changes that I might want to separate out into their own commits, so only things that I've 👍 are things that I have resolved. (Some things I commented on but did not change yet)

src/translate.py Outdated Show resolved Hide resolved
src/types.py Outdated Show resolved Hide resolved
src/demangle_codewarrior.py Outdated Show resolved Hide resolved
src/demangle_codewarrior.py Outdated Show resolved Hide resolved
src/arch_ppc.py Outdated Show resolved Hide resolved
src/translate.py Outdated Show resolved Hide resolved
src/translate.py Outdated Show resolved Hide resolved
src/translate.py Outdated Show resolved Hide resolved
src/translate.py Show resolved Hide resolved
Register("cr0_eq"),
Register("cr0_so"),
] + reg
return reg
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, pre-existing: the

            elif (len(mn_parts) >= 2 and mn_parts[1] == "d") or mnemonic == "ldc1":                   
                set_reg(target.other_f64_reg(), SecondF64Half())                                      

case isn't covered here. All these mismatches scare me, given that they may result in subtle errors with phis. I wonder if we can assert somehow that output_regs_for_instr returns the exact set of output registers... or make that somehow obviously true by construction, but that seems hard given current architecture.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed the case for these mnemonics, but I agree it'd be nice to revisit this and figure out how to remove the duplication between output_regs_for_instr and process_instr.

src/translate.py Outdated Show resolved Hide resolved
src/translate.py Outdated
branch_condition = arch.instrs_decctr_branches[mnemonic](args)

ctr = Register("ctr")
set_reg(ctr, BinaryOp.int(args.regs[ctr], "-", Literal(1)))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the decrement happens before the comparison. Maybe we should add an asm pattern that splits this into two fictive instructions (decrement + conditional branch) to reduce the number of instruction categories?

Copy link
Collaborator Author

@zbanks zbanks Feb 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did this refactor, but it did end up making the output a bit worse...

a8ac096#diff-425a544ce90858581aeed48b74e4a718eacbaf51919d7af56c1be05eaf802686

And despite the ordering bug here, most of the C output was "correct" before (I think it could be from a bug from the way that phi assignments happen before branch_conditions are evaluated?)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see... The worse output is unfortunate, albeit in line with how regular for (i = 0; i < N; i++) output also tends to look (do { ... temp = phi + 1; phi = temp; } while (temp < lim);). It would be nice to improve that if we can figure out a good way. If it's a big worsening we could consider keeping the incorrect version, e.g. by changing the asm pattern to move $temp, $ctr, addi $ctr, $ctr, -1, <branch> $temp, though it'd be a bit uncomfortable. I wonder if there's cases where that gives buggy output in practice, e.g. non-loop occurrences of $ctr.

And yeah, phi assignments are treated as happening all at once at the end of the block, past branch_condition's, which isn't possible to represent in C. That's definitely something that would be nice to fix, it's tripped people up in the past, but it's not obvious how. Would need temps for phis or some more involved refactoring of the whole phi/temp system. Or some wallpapering hacks for the most common cases.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah - I like that if we decide we want to go back to the incorrect version, it's possible to do so by modifying BranchCtrPatter (we don't need to bring back instrs_branch_decctr).

src/translate.py Outdated Show resolved Hide resolved
- Remove `PPC:` from comments in `arch_ppc.py`
- Rename `read` -> `read_exact` helper for clarity
- Store `demangled_str` instead of the `CxxSymbol` in `GlobalSymbol` to
keep MWCC specifics under a tighter abstraction.
- Fix bug in refacotring `handle_addi`
- In `output_regs_for_instr`:
    - Replace `mnemonic == "jal"` with `instrs_fn_call` to handle PPC
    - Add cases for `instrs_store_update`/`instrs_load_update`
    - (NB: Need to revisit & re-audit)
- Assert that registers returned by `function_return` exactly match
those declared in `all_return_regs`
- Remove `.weaken_void_ptr()` in parsing demangled types to be more
consistent with CTypes parsing.
- Assert that `ELLIPSIS` does not show up outside of function args when
parsing demangled types
- Add "b" to instrs_ignore in PPC
    - Remove "b" handling from instrs_jumps
- Clarify rlwinm instruction, factor out an assert
src/translate.py Outdated Show resolved Hide resolved
src/translate.py Show resolved Hide resolved
Copy link
Collaborator

@simonlindholm simonlindholm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Through translate.py!

src/translate.py Outdated Show resolved Hide resolved
src/translate.py Outdated Show resolved Hide resolved
src/translate.py Outdated Show resolved Hide resolved
src/translate.py Outdated Show resolved Hide resolved
src/translate.py Outdated Show resolved Hide resolved
src/translate.py Show resolved Hide resolved
if not isinstance(update, AddressMode):
raise DecompFailure(
f"Unhandled store-and-update arg in {instr}: {update!r}"
)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error occurs for all storex's, right? Would be nice to unify code for that with loadx. (We can fix this later.)

src/translate.py Show resolved Hide resolved
Copy link
Collaborator

@simonlindholm simonlindholm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done reviewing!

src/translate.py Show resolved Hide resolved
src/arch_ppc.py Outdated Show resolved Hide resolved
# varargs functions. Decompiling multiple functions at once
# would help.
# TODO: don't do this in the middle of the argument list,
# except for f12 if a0 is passed and such.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment will eventually need to be updated, and I wonder if we want to share code with the mips implementation. Can wait for later though, I know this part is very much wip.

src/arch_ppc.py Outdated Show resolved Hide resolved
"cmpwi": lambda a, op: BinaryOp.sintptr_cmp(a.reg(0), op, a.imm(1)),
"cmplw": lambda a, op: BinaryOp.uintptr_cmp(a.reg(0), op, a.reg(1)),
"cmplwi": lambda a, op: BinaryOp.uintptr_cmp(a.reg(0), op, a.imm(1)),
# TODO: There is a difference in how these two instructions handle NaN
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(mips has a similar difference and handles this by swapping op -> {'>': '<', '<': '>', '!=': '==', '==': '!='}[op] and calling .negate() on the result in the u case. I don't know if the ppc behavior is exactly comparable)

src/arch_ppc.py Outdated Show resolved Hide resolved
src/arch_ppc.py Outdated Show resolved Hide resolved
src/arch_ppc.py Outdated Show resolved Hide resolved
src/arch_ppc.py Outdated Show resolved Hide resolved
src/arch_ppc.py Outdated Show resolved Hide resolved
zbanks added 11 commits February 1, 2022 16:00
- Move get_branch_target into ArchAsm (shared between MIPS/PPC)
- Remove LENGTH_TWO/LENGTH_THREE normalization from PPC
- Tidy up "PPC:" comments
- Add isync
- Remove break/trapuv.fictive
- Logically order instructions in instrs_destination_first map
- Change `normalize_instruction` to a `@classmethod`
- Remove `$f2` from `all_return_regs` in PPC
- Normalize `cmp{,l}w{,i}` instructions to have 3 args (implicit cr0)
- Assert instrs_ppc_compare have cr0 as first arg
- Comment make_storex/handle_loadx
- Emit `MIPS2C_OVERFLOW(...)` when reading `cr0_so` instead of an error
- Require rA != rD in load-and-update instructions
- Clean up comments around PPC macros
- Remove `& 0xFFFF` from `literal@l` values
- Simplify `sym-_SDA_BASE_(r13)` parsing
- Split bdnz/bdz instructions into a pair (decrement; then branch)
    - Removes the whole `instrs_decctr_branches` map
    - This changes the output
- Add docstrings to all `Cxx*` classes
- Document `CxxSymbol` parsing
- Add quick unit tests
- Fix `OPS` to match with underscores
- Fix parsing unqualified symbol references in templates
Everywhere in `translate.py` where we manually parse an instruction
mnemonic (i.e. that isn't abstracted in `Arch`), use an arch-prefixed
version instead. (ex: `"mips:jr"` instead of `"jr"`)

This makes it easier to follow the code, and avoids situations where
mnemonic names may overlap between architectures.

Also add missing case for `ldc1` to `output_regs_for_instr`
This map should now contains all instructions that have special behaior
for `$r0`, including unimplemented instructions.

This also changes the set into a dict to support instructions where
`$r0` is the first argument.
@zbanks zbanks requested a review from simonlindholm February 2, 2022 22:07
README.md Show resolved Hide resolved
@zbanks
Copy link
Collaborator Author

zbanks commented Feb 2, 2022

Okay, I think I've responded to all the comments and marked them with 👍!

As mentioned in Discord, for comments that we're punting to later, I marked them with 🚀 and added a link to them in the PR description.

src/translate.py Outdated Show resolved Hide resolved
src/translate.py Show resolved Hide resolved
src/translate.py Show resolved Hide resolved
src/translate.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@simonlindholm simonlindholm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's go!

src/translate.py Outdated Show resolved Hide resolved
@zbanks zbanks merged commit 46c55a0 into matt-kempster:master Feb 3, 2022
@zbanks zbanks deleted the ppc2cpp-merge branch February 3, 2022 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants