Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: follow indirect repository URLs #629

Merged
merged 7 commits into from
Feb 14, 2024

Conversation

benmss
Copy link
Member

@benmss benmss commented Feb 6, 2024

This PR allows the Repo Finder to return GitHub, etc. URLs that are initially presented as a redirect URL. URLs that can be considered valid redirects are included within a list in the defaults configuration file. URLs found from redirects are validated as per any other URL. A flag is used to prevent infinite redirect loop shenanigans.

Closes #626

@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Feb 6, 2024
Signed-off-by: Ben Selwyn-Smith <[email protected]>
@benmss benmss force-pushed the 626-follow-indirect-apache-urls branch from d5d50e7 to 3df10da Compare February 6, 2024 23:12
@benmss benmss marked this pull request as ready for review February 7, 2024 00:06
@nathanwn
Copy link
Member

nathanwn commented Feb 7, 2024

I'm unsure if redirection should be handled in the parse_remote_url function in the git_url module.

It seems like this redirection handling should only happen once at the beginning of the analysis of a target. It may be during/right after repo finding instead (which I guess may be part of your intention, confirmed by the fact that you added some redirection setting fields into the Repo Finder section in the .ini config).

Otherwise, I don't see why we should modify the parse_remote_url function. As you have mentioned previously, this function is used in different places to split a URL into different parts. Its name does not suggest that it should also handle redirection, and if that is the case then this function is doing too many things at once.

@behnazh-w
Copy link
Member

behnazh-w commented Feb 7, 2024

@benmss I think this PR can be considered as a new feature. We should update the documentation for the repo finder too.

Signed-off-by: Ben Selwyn-Smith <[email protected]>
@benmss benmss changed the title chore: follow indirect repository URLs feat: follow indirect repository URLs Feb 8, 2024
Comment on lines 295 to 307
("", ""),
("scm:git:[email protected]:FasterXML/jackson-databind.git", "[email protected]:FasterXML/jackson-databind.git"),
("https://github.com/commons-io/commons-io", "https://github.com/commons-io/commons-io"),
("askjdlkajsdlkajsdlkjasldjlk:scm", ""),
],
)
def test_clean_url(url: str, expected: str) -> None:
"""Test the functionality of the clean_url function."""
result = git_url.clean_url(url)
if result is None:
assert expected == ""
else:
assert expected == result.geturl()
Copy link
Member

@nathanwn nathanwn Feb 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Hmm, why not have the expected results as None instead of empty strings so that we can avoid this branching in the test?

Copy link
Member

@nathanwn nathanwn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi. I just had one minor comment for the test. Otherwise, it looks good.

Thanks for the PR.

@behnazh-w behnazh-w self-requested a review February 12, 2024 05:24
@benmss benmss added the repository_finder The issues related to the repository finder label Feb 13, 2024
@benmss benmss merged commit 31f1c87 into staging Feb 14, 2024
9 checks passed
art1f1c3R pushed a commit that referenced this pull request Nov 29, 2024
This PR adds support for SCM metadata URLs found by the Repo Finder that are indirect links to actual repositories.

Signed-off-by: Ben Selwyn-Smith <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OCA Verified All contributors have signed the Oracle Contributor Agreement. repository_finder The issues related to the repository finder
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants