Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packit SRPM build takes too much time #289

Closed
psss opened this issue Dec 5, 2019 · 12 comments
Closed

Packit SRPM build takes too much time #289

psss opened this issue Dec 5, 2019 · 12 comments
Labels
area/user-experience Usability issue deployment Related to our deployment. stale Is the issue still valid? triaged This issue was already processed by the team.

Comments

@psss
Copy link
Contributor

psss commented Dec 5, 2019

Several times I've observed that building the SRPM takes quite some time. I tried to do some basic time check on the teemtee/tmt#55 pull request. Here are the results:

15:20 ... commit pushed
15:21 ... travis started, packit SRPM build has just started...
15:23 ... travis tests completed, packit still building SRPM...
15:33 ... SRPM was built successfully (finally)

So it takes almost quarter of an hour to build the SRPM which is about 1 second on my laptop. Is there anything blocking the build? I think this delay should definitely be improved.

@TomasTomecek
Copy link
Member

it seems like it's time to scale # of workers of packit-service up

@TomasTomecek TomasTomecek added deployment Related to our deployment. area/user-experience Usability issue labels Dec 6, 2019
@TomasTomecek TomasTomecek added this to the Finish by DevConf.cz 2020 milestone Dec 6, 2019
@TomasTomecek TomasTomecek added the triaged This issue was already processed by the team. label Dec 6, 2019
@TomasTomecek
Copy link
Member

We have now 3 workers so this should be resolved.

@psss
Copy link
Contributor Author

psss commented Jan 6, 2020

Tested today on psss/edd#10. Looks better but still the difference is quite significant:

  • local make srpm takes 1 second
  • packit/srpm-build takes 5 minutes

Is there anything else making this simple initial step so slow?

@psss psss reopened this Jan 6, 2020
@TomasTomecek
Copy link
Member

This is why it's slow in the service (and sadly we can't change any of those, short-term):

  • all the commands are running in a sandbox - the whole repo is being copied to the sandbox, this is usually a network operation which is brutally slow in the cluster
  • it make take a while to get your job scheduled in packit-service
  • in the new cluster, we no longer need to copy stuff - we'll be able to use network volumes and share the content like that

Long-term, we are planning on moving packit-service to a cluster with much better performance.

@psss
Copy link
Contributor Author

psss commented Jan 7, 2020

I see, thanks for clarification. When do you plan to migrate to the new cluster?

@TomasTomecek
Copy link
Member

as soon as possible, which will likely take weeks/months :/ since the new place is not even close to being ready

@TomasTomecek TomasTomecek removed this from the Finish by DevConf.cz 2020 milestone Jan 16, 2020
@stale

This comment has been minimized.

@stale stale bot added the stale Is the issue still valid? label Mar 16, 2020
@psss
Copy link
Contributor Author

psss commented Mar 16, 2020

Lately the response time seems to be better. Already moved to the new cluster?

@stale stale bot removed the stale Is the issue still valid? label Mar 16, 2020
@lachmanfrantisek
Copy link
Member

Already moved to the new cluster?

No, unfortunately not yet. Maybe some small tweaks helped that.

@TomasTomecek
Copy link
Member

Sadly, we're not migrating any time soon :/ we were promised a new place, but it didn't turn out in the end.

Maybe we could revisit this and try to do some optimizations on p-s' side.

@lachmanfrantisek
Copy link
Member

lachmanfrantisek commented Mar 20, 2020

I don't know why. But it behaves much better now.

Some optimization is always nice.

@stale
Copy link

stale bot commented May 19, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Thank you for your contributions.
We are doing this to be sure that the issue is still relevant. Anyone can comment to remove the stale state. (The issues marked with pinned, security, bug or EPIC label
are not considered stale.)

@stale stale bot added the stale Is the issue still valid? label May 19, 2020
@stale stale bot closed this as completed Jun 2, 2020
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/user-experience Usability issue deployment Related to our deployment. stale Is the issue still valid? triaged This issue was already processed by the team.
Projects
None yet
Development

No branches or pull requests

3 participants