Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate the reading and interpreting of a replay file #142

Closed
wants to merge 5 commits into from

Conversation

Garanas
Copy link
Member

@Garanas Garanas commented May 6, 2024

Factors out the reading of the file on disk. This is now a separate class. I intentionally did not touch the ReplayDataParser class because that would only complicate the pull request.

The next pull request will separate the header from the body of a replay. The header is not to be confused with the additional JSON context that FAF introduces, that's a different concept. We'll extend the replay container in the process.

Related pull requests:

@Garanas
Copy link
Member Author

Garanas commented May 6, 2024

I did a hiccup, all the changes are in a8889f8 . I also tried to use LFS and that turned it into an even bigger mess when trying to remove it again.

@Garanas Garanas marked this pull request as draft May 6, 2024 19:28
@Garanas
Copy link
Member Author

Garanas commented May 6, 2024

I'll fix the pull request tomorrow.

@Sheikah45 Sheikah45 force-pushed the rework/separate-file-reading branch from 0608f8b to 01f0f6e Compare May 6, 2024 21:01
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if there is a strong case for having the FromMemory public api methods. I don't really see a time when the replay will already be in memory rather than read directly from a file. Making these not public also removes the need for the ReplayBinaryFormat Wrapper classes as then the search for the separation between the metadata and command bytes is just an implementation detail.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taken this into account - in future pull requests the wrapper will go again but at this stage it helps me understand the code as I rework it 😄

@Garanas Garanas marked this pull request as ready for review May 7, 2024 06:35
@Garanas
Copy link
Member Author

Garanas commented May 7, 2024

The concept is not complete yet, but in the current state it is ready for a merge in my opinion. It doesn't impact existing functionality and it's covered by a few tests.

The end goal is to use the newly created Replay class to manage the reading, tokenizing and parsing of the replay. This specifically excludes the semantics of the replay. The introduction of meaning should be entirely separate from the reading and understanding the replay on a binary level.

Eventually the ReplayContainer class will consist of:

  • (1) Context: Introduced by FAF, the JSON on the 1st line in the fafreplay format
  • (2) Header: In the SCFA replay, information about the scenario
  • (3) Body: in the SCFA replay, information about what happened in the scenario (commands, callbacks, etc)

The Replay class will be responsible for populating the ReplayContainer. With this pull request we separate out (1) from ((2), (3)). With #140 we separated (3) from ((1), (2)). With the next pull request I'll separate out the tokenizing and parsing of the header (2). This will also improve the container where we can then specify the header separate from the body and properly type it instead of an array of bytes. The output of loadSCFAReplayFromMemory will manage this in future pull requests.

Then, from that point onwards we can apply semantics to the parsed replay. This is now still combined in one class: the replay data parser. We can keep that class backwards compatible with the intended direction.

import java.util.Arrays;
import java.util.Objects;

public class Replay {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think calling this ReplayLoader makes it clearer what this class is

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Processed in #143

Comment on lines +3 to +9
public sealed interface ReplayBinaryFormat {

record WithContext (byte[] bytes) implements ReplayBinaryFormat {}

record BinarySCFA(byte[] bytes) implements ReplayBinaryFormat {}

}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think these classes are necessary anymore. As it is know they are only used as a wrapper around the bytes and only a member of the ReplayContainer. But they are only created and used inside the ReplayLoader without being exposed publicly. So I think the ReplayLoader can just handle the bytes appropriately without these types.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Processed in #143

@Garanas
Copy link
Member Author

Garanas commented May 11, 2024

Closed in favor of #143

@Garanas Garanas closed this May 11, 2024
Sheikah45 added a commit that referenced this pull request May 18, 2024
* Separate out the reading and interpreting of the file

* Undo lfs of fafreplay

* Fix replays that were damaged by LFS

* Make the memory-based functions private

* Clean up of the code

* Improve documentation of unknown values

* Document the header of a replay

* First working end-to-end example of the new replay parsing

* Restructure the project, process feedback by Sheikah in #142

* Extend tests and fix a few bugs

* Discover the byte that is set when queueing orders

* Introduce the first semantics

* Add interpretation of chat messages

* Introduce additional semantics

* Use atomics

* Rework the enums related to game options

* Remove excessive whitespace

* Add documentation that it still requires to be implemented

* Fix typo in name

* Rename 'Utils' to 'LoadUtils'

* Use a better describing exception

* Undo formatting changes

* Remove the tokenizer of the header

* Extend pattern matching of modern Java

* DO THAT SCREAMING SNAKE CASE THING

* Process feedback

---------

Co-authored-by: Sheikah45 <[email protected]>
@Garanas Garanas deleted the rework/separate-file-reading branch June 3, 2024 11:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants