Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible improvement for the manifest file #12

Open
apirogov opened this issue Aug 17, 2022 · 0 comments
Open

Possible improvement for the manifest file #12

apirogov opened this issue Aug 17, 2022 · 0 comments
Labels
enhancement New feature or request internals Technical and subtle issues users are (usually) not aware of note Something to keep in mind

Comments

@apirogov
Copy link
Collaborator

The manifest exists for the usecase of patching a container that is locally not available.

But there is another possible use for the manifest: avoid opening all containers to access a single data entity.

Each data entity exists only in one file - by definition the most recent patch defining it. Therefore, when requesting data from a path in an immutable container, we can avoid opening all the files, constructing the overlay, etc and simply serve the data from the correct patch.

To do that, the manifest should store patch index + patch containers UUIDs (or something like that) which helps figuring out which file to open for serving a given path.

Some careful thought is required so that this works smoothly and correctly when combined with stub containers and merged containers.
In these cases the skeleton is flattened (so you cannot use a stub to infer correct files to use), in stubs we must preserve the patch indices of the original patch sequence! With merged containers its not really clear what should happen - the same patch on a merge or the same patch on the original container sequence are observationally equivalent.

Maybe, in fact, this info should be stored ad-hoc for a specific context, and not added to the manifest. In any case, this needs further thought.

@apirogov apirogov added enhancement New feature or request note Something to keep in mind labels Aug 17, 2022
@apirogov apirogov added the internals Technical and subtle issues users are (usually) not aware of label Aug 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request internals Technical and subtle issues users are (usually) not aware of note Something to keep in mind
Projects
None yet
Development

No branches or pull requests

1 participant