Skip to content

Use user-specific cache directory by default#544

Open
MukundaKatta wants to merge 1 commit into
openai:mainfrom
MukundaKatta:codex/tiktoken-secure-cache-dir
Open

Use user-specific cache directory by default#544
MukundaKatta wants to merge 1 commit into
openai:mainfrom
MukundaKatta:codex/tiktoken-secure-cache-dir

Conversation

@MukundaKatta
Copy link
Copy Markdown
Contributor

Summary

  • move the implicit encoding download cache from the shared temp directory to a user-specific cache directory ($XDG_CACHE_HOME/tiktoken, %LOCALAPPDATA%/tiktoken, or the platform fallback)
  • create the implicit cache directory with 0700 permissions so other local users cannot pre-populate or read it
  • add a regression test that verifies the user cache path is used, cached reads avoid refetching, and POSIX permissions are private

Fixes #500.

Tests

  • /tmp/tiktoken-secure-cache-venv/bin/python -m pytest tests/test_misc.py -q
  • /tmp/tiktoken-secure-cache-venv/bin/python -m pytest tests/test_simple_public.py -q
  • /tmp/tiktoken-secure-cache-venv/bin/python -m pytest tests/test_encoding.py -q -k 'not hyp_roundtrip'\n\nNote: the full tests/test_encoding.py suite currently fails independently in test_hyp_roundtrip[cl100k_base] when Hypothesis generates the literal disallowed special token <|fim_prefix|>; the narrower command above excludes that pre-existing property-test case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Security: Cache Poisoning via Shared Temp Directory with Optional Hash Verification

1 participant