Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow new file format #849

Merged
merged 5 commits into from
Jun 26, 2022
Merged

Allow new file format #849

merged 5 commits into from
Jun 26, 2022

Conversation

hugsy
Copy link
Owner

@hugsy hugsy commented Jun 24, 2022 •
edited
Loading

Description/Motivation/Screenshots

Create a new root class for file formats (FileFormat) that subclasses will automatically use to register as a file format candidate. Each subclass must have a is_valid(pathlib.Path) -> bool function to be valid. When checking for a file format, the first valid class will be taken.

Currently, only Elf stays builtin to GEF but see PR hugsy/gef-extras#68 for PE and MachO support.

How Has This Been Tested ?

Architecture Yes/No Comments
x86-32 ✅
x86-64 ✅
ARM ❌
AARCH64 ❌
MIPS ❌
POWERPC ❌
SPARC ❌
RISC-V ❌
make test ✅

Checklist

  • My PR was done against the dev branch, not main.
  • My code follows the code style of this project.
  • My change includes a change to the documentation, if required.
  • If my change adds new code, adequate tests have been added.
  • I have read and agree to the CONTRIBUTING document.

hugsy added 2 commits June 23, 2022 10:29
New abstraction class for fileformat, but defaults to GEF. This allow…
0100c7b 
…s PE/MachO fileformat to be supported via gef-extras
🐛 fixes
5dd793d 
- moved prompt init after gef initial loading
- added a `__str__` for `FileFormat`
- `FileFormat` also requires a section member
@hugsy hugsy mentioned this pull request Jun 24, 2022
Draft
5 tasks
@hugsy hugsy added this to the Release: 2022.06 milestone Jun 25, 2022
@hugsy hugsy marked this pull request as ready for review June 25, 2022 16:26
@hugsy hugsy merged commit 430d9d3 into dev Jun 26, 2022
@hugsy hugsy deleted the allow_new_file_format branch June 26, 2022 16:53
hugsy pushed a commit that referenced this pull request Jun 26, 2022
Squashed commit for 2022.06
dd55a10 
commit 430d9d3
Author: hugsy <[email protected]>
Date:   Sun Jun 26 09:53:07 2022 -0700

    Allow new file format (#849)

    * New abstraction class for fileformat, but defaults to GEF. This allows PE/MachO fileformat to be supported via gef-extras

    * 🐛 fixes
    - moved prompt init after gef initial loading
    - added a `__str__` for `FileFormat`
    - `FileFormat` also requires a section member

    * [CI] Fixed missing import

commit d594eb3
Author: hugsy <[email protected]>
Date:   Sun Jun 26 09:43:05 2022 -0700

    Fix 850 (#851)

    * Make sure GefSetting `on_write` hooks are called even on creation

    * Removed `SettingHookType`, not useful

    * in `GefSetting` replaced  `dict` with `super`

    * Added loading counter

commit a2a4bfa
Author: hugsy <[email protected]>
Date:   Sun Jun 26 09:42:25 2022 -0700

    Minor update to unittest of `print-format` after #843 (#852)

commit 9834252
Author: hugsy <[email protected]>
Date:   Sat Jun 25 14:08:13 2022 -0700

    Better `gef-remote` - part 2: Remote Qemu (#846)

    * - rewrite of `gef-remote` to properly manager remote session
    - removed unused functions (& tests)

    * fixes gef's got function fails in remote debug sessions #806

    * fully restore `qemu-user` + `test`

    * added more `__str__` for clarity

    * better __str__

    * better __str__

    * better __str__

    * last changes to `__str__`

    * add `qemu_user` support

    * stupid typo

    * stupid typo

    * qemu-system works too

    * [gef-remote] Updated the docs

    * Apply suggestions from code review

    * Update docs/commands/gef-remote.md

commit 546f4b1
Author: hugsy <[email protected]>
Date:   Sat Jun 25 09:11:28 2022 -0700

    Better `gef-remote` : rewrite and add `remote` full support (#830)

    * - rewrite of `gef-remote` to properly manager remote session

    * removed unused functions (& tests)

    * fixes gef's got function fails in remote debug sessions #806

    * fully restore `qemu-user` + `test`

    * added more `__str__` for clarity

    * better `__str__`  for more classes, making them more meaningful

commit 343cd23
Author: Dreg <[email protected]>
Date:   Fri Jun 24 15:40:12 2022 +0200

    Fix typo in patch byte help (#848)

    `WORD` -> `BYTE`

commit 1e8f55f
Author: Dreg <[email protected]>
Date:   Fri Jun 24 06:26:20 2022 +0200

    `print-format` and `patch byte` commands can work together (#843)

commit f890579
Author: hugsy <[email protected]>
Date:   Mon Jun 20 12:51:28 2022 -0700

    `master` -> `main` (#845)

commit 0fcd20a
Author: Dreg <[email protected]>
Date:   Mon Jun 20 00:37:15 2022 +0200

    keep x86 and x86_64 FLAGS Register when calls to mprotect. Added pushfd, popfd, pushfq, popfq instructions (#844)

commit bf959a3
Author: hugsy <[email protected]>
Date:   Sun Jun 19 14:18:15 2022 -0700

    Type hinting fixes - part 2 (#839)

    * make `@deprecated` more informative on what must be changed

    * - `gef.binary` is being set only from the `newobjfile` handler
    - removed useless `get_elf_headers`
    - more pythonic gef.doc

    * simplified command/function registration

    * fixed `functions` command

    * [ci] roper was never executing correctly

    * Don't delete the `gef` object on `exit_handler` so we can still use it to get info even when the current debugging has exited

    * more type hinting fixes, now down to 75

    * - make all tests run in a tempdir (in tmp)
    - fixed gef install to use the tempdir config setting, not gef-extras dir

    * Formatting issues

    * add the possibility to specify read/write callbacks to `GefSetting`

    * [gef] updated unicode code strings to glyphs

    * allow directories inside `extra_plugin_dir` to act as python package

    * command examples can be provided as a list of strings

    * Allowing context disassembler to be monkey-patched

commit dbf2372
Author: hugsy <[email protected]>
Date:   Sat Jun 18 11:40:58 2022 -0700

    Remove `ida-interact` (#842) from GEF, entirely moved to GEF-Extras

    * removed all last parts of `ida-interact`, everything is moved to `gef-extras`

    * [docs] added a page to track command/function deprecation

commit 1c295fc
Author: hugsy <[email protected]>
Date:   Fri Jun 17 09:40:45 2022 -0700

    restoring temporarily rtfd for redirection to the new docs

commit a78915d
Author: hugsy <[email protected]>
Date:   Fri Jun 17 07:42:14 2022 -0700

    remove rtfd cfg file

commit f375efc
Merge: d463f7d 8ee281d
Author: hugsy <[email protected]>
Date:   Thu Jun 16 14:19:09 2022 -0700

    Merge branch 'dev' of github.com:hugsy/gef into dev

commit 8ee281d
Author: hugsy <[email protected]>
Date:   Thu Jun 16 09:30:52 2022 -0700

    Update README.md

commit d463f7d
Author: hugsy <[email protected]>
Date:   Tue Jun 14 11:02:06 2022 -0700

    [docs] hardcoded emojis instead of md

commit 37ddf86
Author: hugsy <[email protected]>
Date:   Tue Jun 14 10:29:20 2022 -0700

    Updated `gef-extras.sh` installation script

    Force script to pip-install the requirements of `gef-extras`

commit c671bed
Author: hugsy <[email protected]>
Date:   Tue Jun 14 10:10:09 2022 -0700

    Update install.md

commit ca67ced
Author: hugsy <[email protected]>
Date:   Mon Jun 13 20:03:33 2022 -0700

    :bug: Last doc fixes, GEF API is now generated from Actions

commit 5b468d0
Author: hugsy <[email protected]>
Date:   Mon Jun 13 19:16:39 2022 -0700

    :bug: forcing docs to run the same version than gdb

commit d836054
Author: hugsy <[email protected]>
Date:   Mon Jun 13 19:07:36 2022 -0700

    Squashed commit of the following:

        [docs] make actions regenerate the api file

        [docs] regenerated gef api file

        [docs] cleanup

commit a7d8fa5
Author: hugsy <[email protected]>
Date:   Mon Jun 13 14:50:35 2022 -0700

    Updated documentation appearance

commit dca3cb1
Author: hugsy <[email protected]>
Date:   Mon Jun 13 13:46:35 2022 -0700

    Update generate-docs.yml

commit d7d64a2
Author: hugsy <[email protected]>
Date:   Mon Jun 13 13:43:21 2022 -0700

    Update generate-docs.yml

commit a89f91c
Author: hugsy <[email protected]>
Date:   Mon Jun 13 13:37:37 2022 -0700

    Move 3rd party to extras (#841)

    * moved keystone, capstone, unicorn and ropper to gef-extras

    * moved test files to extras

    * moved docs

    * [docs] removed references to 3rd party packages

    * [actions] mlc -> lychee

    * updated PR & contrib templates and fixed actions command for lychee

    * moved requirements.txt to tests/ so people don't think there's a need to install any package

    * no need for readthedocs anymore

    * replaced references to rtfd to gh-pages

commit 7f45550
Author: hugsy <[email protected]>
Date:   Sun May 22 08:14:12 2022 -0700

    Type Hinting Fixes - Part 1 (#827)

    * better linting, by creating a proper .editorconfig & pylintrc
    * major drop of linting errors, from 786 errors to 145
    * using Py3.6 `__init_subclass__` allows to remove the use of `abc` and `@register_architecture`
    * using `__init_subclass__` to create base class for commands
    * using `__init_subclass__` to create base class for functions

commit e50af77
Author: Khaotic <[email protected]>
Date:   Sat Apr 9 10:36:53 2022 -0500

    Improve identifying Stack Canaries (#833)

commit d540abd
Author: hugsy <[email protected]>
Date:   Mon Mar 7 10:52:17 2022 -0800

    pushing urgent fix for #831
    (even tho `get_arch` will be deprecated after #827)

commit 93f3010
Merge: 7817bea 6e3cd5c
Author: hugsy <[email protected]>
Date:   Fri Mar 4 09:35:13 2022 -0800

    Merge branch 'dev' of github.com:hugsy/gef into dev

commit 7817bea
Author: hugsy <[email protected]>
Date:   Fri Mar 4 09:35:06 2022 -0800

    [docs] http -> https
    because apparently it's a big deal for debugging stuff

commit 6e3cd5c
Author: hugsy <[email protected]>
Date:   Sat Feb 26 12:29:58 2022 -0800

    Update gef.md

commit 0954a16
Author: hugsy <[email protected]>
Date:   Sat Feb 26 12:25:53 2022 -0800

    New subcommand `gef install` (#825)

    * Adding new command `gef install`

    * added doc

    * added tests

    * test: changing `remote` with `skel` because `remote` uses an external dependency (`rpyc`)

    * PR review additions

commit 18c40b6
Author: Boris-Chengbiao Zhou <[email protected]>
Date:   Tue Feb 22 05:18:33 2022 +0100

     Add mechanism to let architectures specify whether they support a gdb arch (#822)

    * Fix gdb arch parsing for auto-detected archs

    * Add mechanism to let architectures specify whether they support a gdb arch

    * Add documentation about adding architectures and supports_gdb_arch()

    * Address review comments

commit 2b7f315
Author: Ebubekir Türker <[email protected]>
Date:   Mon Feb 14 23:30:43 2022 +0300

    remove unnecessary warnings (#824)

    * remove unnecessary warnings

commit 18c7ba4
Author: hugsy <[email protected]>
Date:   Sun Feb 13 18:38:44 2022 -0800

    Better tests (#813)

    * first shot at refactoring ci testing

    * - Added skeleton for most missing command test modules
    - Fixed a logic error in `NopCommand`

    * [ci] added tests for `nop`

    * [ci] extra test for nop for memory check

    * added benchmarking capability, can be triggered directly from `pytest`

    * - fixed `pcustom` command test for 32b
    - fixed calculation of tcache in gef
    - added tests for `gef.heap`
    - improved tests for `heap` command
    - fixes #641

    * [ci] cmd/heap - adjusted tcachebins indexes for 32b

    * damnit

    * - fixed linting
    - fixed test `highlight` for 32b
    - fixed test `pattern_search` for x86

    * last fixes for tonight

    * fixed `pattern` and `heap` tests for good

    * - add 3rd party module check for `capstone`, `keystone`, `unicorn` and `ropper`

    * added `test_func_update_gef`

    * `make test` doesn't execute benchmark

    * - fixed errors in the `pie` subcommands
    - added tests for `pie`

    * `theme` added more tests

    * - improved tests for `pattern` and `edit-flags`

    * [ci] created cases for all arches for bin tests in `tests/heap.py`

    * fixed `heap` tests for good

    * - added ci test for `glibcarena`
    - fixed `theme` missing comma (original PR #808 by @mrshu )
    - added missing values for `theme`

    * - added tests for deprecated API
    - added test for smart eval
    - more function tests

    * started `gef` test module

    * - added tests for `syscall-args` and `is-syscall`

    * - fixed `syscall-args` to also get catchpoints + tests

    * - test `show_last_exception`

    * make sure `syscall-args` test collects the ABI files from `gef-extras`

    * linting

    * only enable `syscall-args` test for x86

    * `syscall-args` fixed typo in i686 test

    * Fix RISCV arch detection (#790)

    * Add RISCV alias so arch can be determined by ELF
    * Add ptrsize property to RISCV arch
    * Allow riscv tests to run

    * Update tests/api/gef_arch.py

    Co-authored-by: Grazfather <[email protected]>

    * fix: make shebang lines portable (#814)

    * fix: make shebang lines portable

    * fix: SC2006, SC2086, SC2016, SC2059

    * make `heap` tests work universally

    * disabling capstone/keystone/unicorn for some arches for now

    * - fixed tests for ppc64
    - added static `ptrsize` for ppc & ppc64 in gef

    * - `BIN_LS` -> `_target("default")`
    - removed auto demangle for now

    * - disable pytest `--pdb` from makefile
    - added doc for testing

    * Apply suggestions from code review

    Co-authored-by: Grazfather <[email protected]>

    * fixing ci

    * [tests] use camel case for format string helper test class

    * [tests] added docstring to `GefFuncDeprecatedApi`

    * [tests] `edit-flags` are only for known arches for now

    * PR review changes

    * PR review last batch

    Co-authored-by: Grazfather <[email protected]>
    Co-authored-by: theguy147 <[email protected]>

commit 37bb542
Author: hugsy <[email protected]>
Date:   Sun Feb 13 10:54:28 2022 -0800

    Add class factory support for `pcustom` (#819)

    * [pcustom] add class factory support
    instead of creating a static class, class factories allow to generate a `ctypes.Structure` class with information from the runtime, which can drastically simplify (and unify) classes declaration (from on libc version, architecture, ptrsize, etc.)

    * [pcustom] added doc for class factory

    * linting

    * [pcustom] completed the documentation for class factory

    * Better filter of external attribute in the `Structure.__init__` to catch both classes and class factory methods

    * Apply suggestions from code review

    Co-authored-by: Grazfather <[email protected]>

commit 82b2570
Author: hugsy <[email protected]>
Date:   Sun Feb 13 09:52:18 2022 -0800

    Remove `ida-interact` (#817)

    * remove `ida-interact` from gef, replaced by a better version in `gef-extras`

    * Fix double loading of external plugins (#816)

    `register_external_command` was receiving an instance of a class for each new external script. This lead to a double initialization when calling `gef.gdb.load(cls)`. Fixed by registering directly a class (just like `register_command`)

    * [lint] removed `xmlrpclib` unused import

    * restored doctstring of `IdaInteractCommand`

    * restoring python path insertion for python plugins for extra pacakges

commit d86e7a0
Author: hugsy <[email protected]>
Date:   Wed Feb 9 13:04:48 2022 -0800

    Minor follow-up of #821

commit 8c0f625
Author: hugsy <[email protected]>
Date:   Tue Feb 8 17:59:08 2022 -0800

    - create `gef.ui.libc_args_table` (#821)

    - make the old  `libc_args_definitions` dict point to `gef.ui.libc_args_table`

commit 4365d9c
Author: hugsy <[email protected]>
Date:   Fri Feb 4 09:55:08 2022 -0800

    Fix double loading of external plugins (#816)

    `register_external_command` was receiving an instance of a class for each new external script. This lead to a double initialization when calling `gef.gdb.load(cls)`. Fixed by registering directly a class (just like `register_command`)

commit d1fa00f
Author: theguy147 <[email protected]>
Date:   Tue Feb 1 19:28:30 2022 +0100

    fix: make shebang lines portable (#814)

    * fix: make shebang lines portable

    * fix: SC2006, SC2086, SC2016, SC2059

commit c078733
Author: Grazfather <[email protected]>
Date:   Tue Feb 1 10:57:08 2022 -0500

    Fix RISCV arch detection (#790)

    * Add RISCV alias so arch can be determined by ELF
    * Add ptrsize property to RISCV arch
    * Allow riscv tests to run

commit 48d39bb
Author: code-byter <[email protected]>
Date:   Mon Jan 31 05:36:52 2022 +0100

    Update gef config parameters of gef-extras installation script (#807)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
api-change code-quality documentation enhancement new feature
Projects
None yet
Milestone
Release: 2022.06
Development

Successfully merging this pull request may close these issues.
1 participant
@hugsy