Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow usage of non-default copyManager in JacksonDeSerializer #75

Open
martin-bpw opened this issue Nov 7, 2024 · 5 comments
Open

Allow usage of non-default copyManager in JacksonDeSerializer #75

martin-bpw opened this issue Nov 7, 2024 · 5 comments

Comments

@martin-bpw
Copy link
Contributor

martin-bpw commented Nov 7, 2024

This issue/suggestion is closely related to spdx/Spdx-Java-Library#257 . Basically if we use MultiFormatStore, we cannot specify custom copyManager, which lead to case when default copyManager was used. Instead, it would be better to be able to use garbage-collectible copyManager instance instead.

We came across this issue as it led to OOM error on our system:

image
@goneall
Copy link
Member

goneall commented Nov 7, 2024

The MultiFormatStore should be picking up the copyManager from the SPDX model objects being "copied", not from any parameter or property stored in the MultiFormatStore. If you're creating the objects directly, you can just pass in a different copyManager. From the other issue, it looks like you're using the LicenseInfoFactory which doesn't give you an option of passing in null.

I think I have a work around - I'll post it as a comment in the library issue.

@goneall
Copy link
Member

goneall commented Nov 7, 2024

BTW - That's a lot of IDs in the copy manager - is it possible there is another issue? Perhaps in the copy manager or SPDX java library?

@goneall
Copy link
Member

goneall commented Nov 7, 2024

@martin-bpw - Seeing the pull request - I now understand where the use of the copy manager is used.

I'll put a little more thought and review into the PR.

@martin-bpw
Copy link
Contributor Author

martin-bpw commented Nov 7, 2024

"BTW - That's a lot of IDs in the copy manager - is it possible there is another issue? Perhaps in the copy manager or SPDX java library?"

Thanks for quick response. I analyzed today and from my pov it is combination of lot of DATA and long running time.
image
We dont use DefaultModelStore, but the "trick" is that its lifecycle starts at the start of application, and it has static member copyManager, which is also singleton and also lives "forever". And on various places in code, only this forever-living singleton is referrenced, and thus is just gradually accumulating the data, until it gets full. Unless it is interrupted by restart (and we were running I think 1-2 months without restart).

You correctly pointed that out in the DefaultModelStore javadoc (No GC), but its more subtle that copyManager inherits this limitation too.

@martin-bpw
Copy link
Contributor Author

martin-bpw commented Nov 7, 2024

Just for completeness - this is our (single) entry point or boundary between our code and spdx lib code:

    MultiFormatStore multiFormatStore = new MultiFormatStore(new InMemSpdxStore(), format);
    try (InputStream input = new ByteArrayInputStream(spdxDocument.getBytes())) {
        multiFormatStore.deSerialize(input, false);
    }

And that alone already led to this OOM issue.

goneall added a commit that referenced this issue Nov 7, 2024
Allow usage of non-default copyManager in JacksonDeSerializer
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants