Bug description
LocalMemoryCacheManager memoizes file sizes in an unbounded ConcurrentHashMap<String, Long>; the only removal path is prefix invalidation when a CachingFileIO wrapper is fully released. In memory-cache mode (local-cache.enabled without a cache dir), a job reading many distinct whitelisted files (snapshots, manifests, index files — everything not excluded by FileType.isMutable) grows this map without limit: one path-string entry per distinct file for the lifetime of the shared cache manager.
Expected behavior
The memo should be size-bounded: entries are only a fast path to skip one getFileStatus call, so dropping the eldest entry merely costs a re-stat on the next open.
Bug description
LocalMemoryCacheManagermemoizes file sizes in an unboundedConcurrentHashMap<String, Long>; the only removal path is prefix invalidation when aCachingFileIOwrapper is fully released. In memory-cache mode (local-cache.enabledwithout a cache dir), a job reading many distinct whitelisted files (snapshots, manifests, index files — everything not excluded byFileType.isMutable) grows this map without limit: one path-string entry per distinct file for the lifetime of the shared cache manager.Expected behavior
The memo should be size-bounded: entries are only a fast path to skip one
getFileStatuscall, so dropping the eldest entry merely costs a re-stat on the next open.