You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If the flowmachine query locker (redis) has somehow got out of sync with the state of the flowmachine query cache in FlowDB, so that a cached query is in known state, then query.invalidate_db_cache() will silently fail to remove the cache table (because query_state_machine.reset() has no effect on a query in known state).
A side effect of this is that flowmachine.core.cache.shrink_below_size reports (in log messages) that the cache is shrinking even if it isn't, because it assumes a successful invalidate_query_by_id run means that the cache table has been dropped and subtracts the table size from its internal record of the cache size, and does not check for cache table existence or re-fetch the cache size from FlowDB between query drops.
I think there are a few issues here:
query.is_stored (which does not touch redis, and directly checks the DB for the corresponding cache table) should always be the source of truth for checking whether a cache record exists
query.invalidate_db_cache() should raise an error if query.is_stored is still True at the end (if drop=True)
I can't think of a good reason not to allow QueryStateMachine.reset() to reset from known state as well, which would mean an existing cache entry could be cleared even if there's no corresponding query state in redis
flowmachine.core.cache.shrink_below_size shouldn't report the cache size as having shrunk if the cache table wasn't actually dropped - I think this should be covered by the above points (i.e. as long as query.invalidate_db_cache() raises an error if the cache table still exists at the end, then shrink_below_size can trust that the table has been removed).
The text was updated successfully, but these errors were encountered:
If the flowmachine query locker (redis) has somehow got out of sync with the state of the flowmachine query cache in FlowDB, so that a cached query is in
known
state, thenquery.invalidate_db_cache()
will silently fail to remove the cache table (becausequery_state_machine.reset()
has no effect on a query inknown
state).A side effect of this is that
flowmachine.core.cache.shrink_below_size
reports (in log messages) that the cache is shrinking even if it isn't, because it assumes a successfulinvalidate_query_by_id
run means that the cache table has been dropped and subtracts the table size from its internal record of the cache size, and does not check for cache table existence or re-fetch the cache size from FlowDB between query drops.I think there are a few issues here:
query.is_stored
(which does not touch redis, and directly checks the DB for the corresponding cache table) should always be the source of truth for checking whether a cache record existsquery.invalidate_db_cache()
should raise an error ifquery.is_stored
is stillTrue
at the end (ifdrop=True
)QueryStateMachine.reset()
to reset fromknown
state as well, which would mean an existing cache entry could be cleared even if there's no corresponding query state in redisflowmachine.core.cache.shrink_below_size
shouldn't report the cache size as having shrunk if the cache table wasn't actually dropped - I think this should be covered by the above points (i.e. as long asquery.invalidate_db_cache()
raises an error if the cache table still exists at the end, thenshrink_below_size
can trust that the table has been removed).The text was updated successfully, but these errors were encountered: