Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try using sqlite in production #32

Closed
wants to merge 1 commit into from
Closed

Conversation

major
Copy link
Member

@major major commented Jun 25, 2024

Switch over to using sqlite in production and add a step during the container build to pack the initial image data in the container. This speeds up the startup time and also makes it work better in air-gapped environments.

We're able to remove the threading/scheduling from the main module by using this method.

@major major force-pushed the try-sqlite branch 2 times, most recently from e1fc518 to 9698112 Compare June 25, 2024 21:45
Switch over to using sqlite in production and add a step during the
container build to pack the initial image data in the container. This
speeds up the startup time and also makes it work better in air-gapped
environments.

We're able to remove the threading/scheduling from the main module by
using this method.

Signed-off-by: Major Hayden <[email protected]>
@major
Copy link
Member Author

major commented Jun 25, 2024

@FKolwa This is kinda disgusting for now and could use some clean-up, but it's working. I can split it tomorrow if needed.

Copy link
Member

@F-X64 F-X64 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the approach.
You're touching on a lot of stuff in here but I can't see a good way to split this up.

In the long run I'd like to have an update function in place or at least an easy way to check whether your image data is out dated. Something that doesn't necessarily require you to pull a new image from our repository.



@repeat(every(24).hours)
def self_update_image_data() -> None:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would a regular update of image data still make sense?
This could be configurable in some way so that air gapped instances wouldn't try to reach our servers.

Or should we move to a "pull on demand"?
E.g. downloading data, mounting it into the container and restart / trigger a reload function.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know we've talked about having daily updated builds available with fresh data, but I could see a scenario in which it would be easier for customers to just inject new data instead of pulling a new container and moving it into an air gapped box. WDYT?

@major major marked this pull request as draft June 27, 2024 18:48
@major major closed this Jul 23, 2024
@major major deleted the try-sqlite branch July 23, 2024 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants