Skip to content

Commit

Permalink
Explicitly set the embedded timestamp in .pyc files
Browse files Browse the repository at this point in the history
  • Loading branch information
citrus-it committed Oct 3, 2023
1 parent bc6e4fd commit 2f57bec
Show file tree
Hide file tree
Showing 3 changed files with 41 additions and 3 deletions.
26 changes: 26 additions & 0 deletions build/python311/patches/pyc-timestamp.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@

When packaging python modules along with their compiled source, we
deliberately set the timestamp attributes for both the source
and compiled file so that they are not recompiled the first time
root uses them (via pkg, for example). Doing so causes `pkg validate`
errors and slows down the initial invocation.

However, the timestamp and size of the .py file is also embedded within
the .pyc header. This patch allows overriding that embedded timestamp
with a value of our choosing. See lib/functions.sh for how this is set.

diff -wpruN --no-dereference '--exclude=*.orig' a~/Lib/py_compile.py a/Lib/py_compile.py
--- a~/Lib/py_compile.py 1970-01-01 00:00:00
+++ a/Lib/py_compile.py 1970-01-01 00:00:00
@@ -159,6 +159,11 @@ def compile(file, cfile=None, dfile=None
pass
if invalidation_mode == PycInvalidationMode.TIMESTAMP:
source_stats = loader.path_stats(file)
+ if fpepoch := os.environ.get('FORCE_PYC_TIMESTAMP'):
+ try:
+ source_stats['mtime'] = int(fpepoch)
+ except ValueError:
+ raise ValueError("FORCE_PYC_TIMESTAMP is not a valid integer")
bytecode = importlib._bootstrap_external._code_to_timestamp_pyc(
code, source_stats['mtime'], source_stats['size'])
else:
1 change: 1 addition & 0 deletions build/python311/patches/series
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ encoding-alias.patch
locale-encoding.patch
cgiserver.patch
static-assert.patch
pyc-timestamp.patch
#
# Additional modules
module-ucred.patch
Expand Down
17 changes: 14 additions & 3 deletions lib/functions.sh
Original file line number Diff line number Diff line change
Expand Up @@ -1062,10 +1062,21 @@ prep_build() {

# Generate timestamps
typeset now=`TZ=UTC $DATE +%s`
typeset TS_SRC_EPOCH=$((now - 60))
typeset TS_OBJ_EPOCH=$((now - 30))
typeset TS_FMT="%Y%m%dT%H%M%SZ"
typeset TS_SRC=`$DATE -r $((now - 60)) +$TS_FMT`
typeset TS_OBJ=`$DATE -r $((now - 30)) +$TS_FMT`

typeset TS_SRC=`$DATE -r $TS_SRC_EPOCH +$TS_FMT`
typeset TS_OBJ=`$DATE -r $TS_OBJ_EPOCH +$TS_FMT`

# Python is patched to use the value of this variable as the timestamp that
# it embeds in .pyc files. We need to make sure that this embedded
# timestamp matches the timestamp that the packaging system will apply to
# the corresponding source .py file.
export FORCE_PYC_TIMESTAMP=$TS_SRC_EPOCH

# These tokens are used by rules in lib/mog/global-transforms.mog to
# automatically apply timestamp attributes to python modules and their
# compiled form. They can also be used by other packages in their local.mog
SYS_XFORM_ARGS+=" -DTS_SRC=$TS_SRC -DTS_OBJ=$TS_OBJ"

logmsg "--- Creating temporary installation directory"
Expand Down

0 comments on commit 2f57bec

Please sign in to comment.