Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update packets.hexpat to support PPAC 4 (zstd) #3

Open
rockisch opened this issue Sep 10, 2024 · 1 comment
Open

Update packets.hexpat to support PPAC 4 (zstd) #3

rockisch opened this issue Sep 10, 2024 · 1 comment

Comments

@rockisch
Copy link

rockisch commented Sep 10, 2024

Thought it would be interesting to create a tracking issue for this, since I was looking into it.

One of the reasons I think parsing PPAC v4 wasn't possible is because the current implementation uses the zstd streaming API to save the files, and ImHex's zstd_decompress didn't support that.

This PR fixes that:
WerWolv/ImHex#1898

With that out of the way, it is now possible to parse the zstd data... but the way it is displayed is kinda weird. Here's the updated parts of the script:

import hex.dec;

...

struct file {
    Header header;
    if (header.Ver >= 3) {
        u8 is_ngs;
    }
    if (header.Ver >= 4) {
        u8 is_zstd;
    }
    if (header.Ver == 1) {
        Packet_v1 packet[while(!std::mem::eof())];
    } else if (header.Ver == 2 || header.Ver == 3 || (header.Ver == 4 && !is_zstd)) {
        Packet_v2 packet[while(!std::mem::eof())];
    } else if (header.Ver == 4 && is_zstd) {
        u8 packet_zstd[while(!std::mem::eof())] [[hidden]];
        std::mem::Section packet_section = std::mem::create_section("data_section") [[export]];
        hex::dec::zstd_decompress(packet_zstd, packet_section);
        Packet_v2 packet[while(!std::mem::eof())] @ 0x00 in packet_section;
    }
};

This works under the PR branch, but it seems like the way ImHex pattern scripts treats @ 0x00 in <section> is a bit weird:
WerWolv/ImHex#1899

In order to access the data you need to go to the sections tab which looks like this:
image

As of right now, you can't resize this window, which makes it pretty much useless. I'm trying to see if there are any better options on the above ticket.

@rockisch rockisch changed the title Update ppac_tools to support PPAC 4 (zstd) Update packets.hexpat to support PPAC 4 (zstd) Sep 10, 2024
@AntonnMal
Copy link
Member

Little background on the packets.hexpat: when I was writing the pattern file, I wasn't planning on adding compression, but when the time came, I was already using other tools to extract data (ppak_reader), so I've kinda gave up on the pattern. I didn't know that there were decompression functions. Also I believe that viewing large packet files in ImHex is not a nice experience.

Anyway, when compression was added to the format, I've written a small program (reppac) to decompress the files, so they could be viewed without trouble in other editors.

Looking through the linked issue I've noticed a mention about virtual files. In the pattern docs there is one function mentioned that is linked to virtual files: hex::core::add_virtual_file

struct file {
    Header header;
    if (header.Ver >= 3) {
        u8 is_ngs;
    }
    if (header.Ver >= 4) {
        u8 is_zstd;
    }
    if (header.Ver == 1) {
        Packet_v1 packet[while(!std::mem::eof())];
    } else if (header.Ver == 2 || header.Ver == 3 || (header.Ver == 4 && !is_zstd)) {
        Packet_v2 packet[while(!std::mem::eof())];
    } else if (header.Ver == 4 && is_zstd) {
        u8 packet_zstd[while(!std::mem::eof())] [[hidden]];
        std::mem::Section packet_section = std::mem::create_section("data_section");
        hex::dec::zstd_decompress(packet_zstd, packet_section);
        Packet_v2 packet[while(!std::mem::eof())] @ 0x00 in packet_section;
        hex::core::add_virtual_file("a", packet);
    }
};

In the virtual files section a new entry appears:
image

And it can be opened as a reguar file. However it requires slightly modified hexpat to parse. Maybe there is some way to copy data in the pattern between files or views.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants