Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scrape all the specfiles from Fedora for source URLs #2390

Open
1 of 8 tasks
mfocko opened this issue Apr 4, 2024 · 3 comments
Open
1 of 8 tasks

Scrape all the specfiles from Fedora for source URLs #2390

mfocko opened this issue Apr 4, 2024 · 3 comments
Assignees
Labels
area/fedora Related to Fedora ecosystem complexity/single-task Regular task, should be done within days. gain/high This brings a lot of value to (not strictly a lot of) users. impact/low This issue impacts only a few users. kind/internal Doesn't affect users directly, may be e.g. infrastructure, DB related.

Comments

@mfocko
Copy link
Member

mfocko commented Apr 4, 2024

To be able to adjust the firewall rules »beforehand« on the MP+, we need to know the domains/forges from which users are using sources for the sync-release actions.

Scraping the specfiles and configs of current users didn't provide enough benefit, since it can be still a pain point during an onboarding of new users

  • scrape all the specfiles
  • pinpoint forges/URLs that we do not support as of now
    • (optionally) ask the packagers if they'd be interested in onboarding
    • also check for activity within the packages
  • edit comment with the “waiting list” in Firewall rules that need to be enabled deployment#575
  • (optionally) open a ticket to adjust the firewall rules
    • (“optional”) ticket per URL :)
  • (optionally) verify that the sources can be obtained
@mfocko mfocko added area/fedora Related to Fedora ecosystem complexity/single-task Regular task, should be done within days. impact/low This issue impacts only a few users. gain/high This brings a lot of value to (not strictly a lot of) users. kind/internal Doesn't affect users directly, may be e.g. infrastructure, DB related. labels Apr 4, 2024
@xsuchy xsuchy self-assigned this Apr 4, 2024
@xsuchy
Copy link
Contributor

xsuchy commented Apr 4, 2024

I am running:

for i in *spec; do
spectool $i |grep Source
done >/tmp/output.txt

@xsuchy
Copy link
Contributor

xsuchy commented Apr 4, 2024

Here is the list of all Sources from Fedora:
output.txt.gz

@xsuchy xsuchy removed their assignment Apr 4, 2024
@nforro
Copy link
Member

nforro commented May 20, 2024

  • scrape all the specfiles

Maybe we should go also through Packit configs, because (at least in theory) upstream_project_url can be blocked while Source0 URL is fine.

@mfocko mfocko self-assigned this Jul 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/fedora Related to Fedora ecosystem complexity/single-task Regular task, should be done within days. gain/high This brings a lot of value to (not strictly a lot of) users. impact/low This issue impacts only a few users. kind/internal Doesn't affect users directly, may be e.g. infrastructure, DB related.
Projects
Status: in-progress
Development

No branches or pull requests

3 participants