Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: use infrahub image to run prefect server #5548

Merged
merged 1 commit into from
Jan 23, 2025

Conversation

fatih-acar
Copy link
Contributor

@fatih-acar fatih-acar commented Jan 22, 2025

This will avoid incompatibility issues between the Prefect client used by Infrahub and the Prefect server.
This also avoid dirty workarounds in CI to make jobs work when we have a mismatch between the Prefect version in docker compose and the client used.

Prefect's web UI is also working using this method.

Next step would be to have our own entrypoint to autoconfigure Prefect according to Infrahub's settings.

This will avoid incompatibility issues between the Prefect client used
by Infrahub and the Prefect server.
This also avoid dirty workarounds in CI to make jobs work when we have a
mismatch between the Prefect version in docker compose and the client
used.

Signed-off-by: Fatih Acar <[email protected]>
@fatih-acar fatih-acar requested a review from a team as a code owner January 22, 2025 14:09
@github-actions github-actions bot added the group/ci Issue related to the CI pipeline label Jan 22, 2025
Copy link

codspeed-hq bot commented Jan 22, 2025

CodSpeed Performance Report

Merging #5548 will not alter performance

Comparing fac-prefect-server-image (b1fefcb) with dga-20250119-prefect-3.1.13 (3f9d273)

Summary

✅ 10 untouched benchmarks

@fatih-acar
Copy link
Contributor Author

Note: doing this would add a dependency on Prefect's Python version support, e.g. if Prefect does not support 3.14 we wouldn't be able to upgrade Infrahub to use 3.14, etc.

@dgarros
Copy link
Collaborator

dgarros commented Jan 22, 2025

Note: doing this would add a dependency on Prefect's Python version support, e.g. if Prefect does not support 3.14 we wouldn't be able to upgrade Infrahub to use 3.14, etc.

I think this is already the case because we have it as a direct dependencies in python

@dgarros
Copy link
Collaborator

dgarros commented Jan 23, 2025

As a side note,
it looks like prefect doesn't support 3.13 yet because of pendulum doesn't support it
pydantic/pydantic-extra-types#239
python-pendulum/pendulum#844

Since we are also using pendulum directly, we need to look into this rather sooner than later

Base automatically changed from dga-20250119-prefect-3.1.13 to develop January 23, 2025 05:50
@dgarros dgarros merged commit 7a22217 into develop Jan 23, 2025
34 checks passed
@dgarros dgarros deleted the fac-prefect-server-image branch January 23, 2025 05:51
@ogenstad
Copy link
Contributor

As a side note, it looks like prefect doesn't support 3.13 yet because of pendulum doesn't support it pydantic/pydantic-extra-types#239 python-pendulum/pendulum#844

Since we are also using pendulum directly, we need to look into this rather sooner than later

Hm, we are using 3.13 in the SDK. I wonder if we should just get rid of pendulum. We talked about it in the past and the project seems fairly dead.

@dgarros
Copy link
Collaborator

dgarros commented Jan 23, 2025

I think we should get rid of pendulum but not sure how ...
As far as I can tell, there isn't a good library available today that could easily replace it ... I looked at a few issues earlier today and the migration path isn't obvious to me yet

@aaazzam
Copy link

aaazzam commented Jan 26, 2025

We're thinking through this problem, too.

We've identified https://github.com/ariebovenberg/whenever as a promising (albeit early) candidate. That said, since Prefect is >=3.9 we may just stoop to raw ZoneInfo (not sure we need a library for getting "now" in UTC).

@dgarros
Copy link
Collaborator

dgarros commented Jan 27, 2025

Thank you @aaazzam, whenever looks like a great alternative to pendulum

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
group/ci Issue related to the CI pipeline
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants