Merge remote-tracking branch 'upstream/main' into clinic-param-aliases

Author: serhiy-storchaka

Doc/c-api/import.rst

@@ -146,7 +146,7 @@ Importing Modules
       alternatives.
 
    .. versionchanged:: 3.15
-      ``__cached__`` is no longer set.
+      The ``__cached__`` attribute is no longer set.
 
 
 .. c:function:: PyObject* PyImport_ExecCodeModuleEx(const char *name, PyObject *co, const char *pathname)
@@ -170,7 +170,7 @@ Importing Modules
       :class:`~importlib.machinery.ModuleSpec` for alternatives.
 
    .. versionchanged:: 3.15
-      ``__cached__`` no longer set.
+      The ``__cached__`` attribute no longer set.
 
 
 .. c:function:: PyObject* PyImport_ExecCodeModuleWithPathnames(const char *name, PyObject *co, const char *pathname, const char *cpathname)

Doc/library/runpy.rst

@@ -97,7 +97,7 @@ The :mod:`!runpy` module provides two functions:
       :class:`~importlib.machinery.ModuleSpec` for alternatives.
 
    .. versionchanged:: 3.15
-      ``__cached__`` is no longer set.
+      The global variable ``__cached__`` is no longer set.
 
 .. function:: run_path(path_name, init_globals=None, run_name=None)
 
@@ -175,7 +175,7 @@ The :mod:`!runpy` module provides two functions:
       ``__package__`` are deprecated.
 
    .. versionchanged:: 3.15
-      ``__cached__`` is no longer set.
+      The global variable ``__cached__`` is no longer set.
 
 .. seealso::
 

Doc/library/tarfile.rst

@@ -1112,6 +1112,10 @@ reused in custom filters:
     paths (in case the name is absolute
     even after stripping slashes, e.g. ``C:/foo`` on Windows).
     This raises :class:`~tarfile.AbsolutePathError`.
+  - Normalize filenames (:attr:`TarInfo.name`) that contain ``..`` components
+    using :func:`os.path.normpath`.
+    Note that this removes internal ``..`` components, which may change the
+    meaning of the name if it traverses symbolic links.
   - :ref:`Refuse <tarfile-extraction-refuse>` to extract files whose absolute
     path (after following symlinks) would end up outside the destination.
     This raises :class:`~tarfile.OutsideDestinationError`.
@@ -1120,6 +1124,10 @@ reused in custom filters:
 
   Return the modified ``TarInfo`` member.
 
+  .. versionchanged:: next
+
+     Filenames containing ``..`` components are now normalized.
+
 .. function:: data_filter(member, path)
 
   Implements the ``'data'`` filter.

Doc/reference/datamodel.rst

@@ -1099,14 +1099,9 @@ this approach.
    :ref:`import system <importsystem>` may opt to leave it unset if it
    has no semantic meaning (for example, a module loaded from a database).
 
-   .. deprecated-removed:: 3.13 3.15
-      Setting ``__cached__`` on a module while failing to set
-      :attr:`!__spec__.cached` is deprecated. In Python 3.15,
-      ``__cached__`` will cease to be set or taken into consideration by
-      the import system or standard library.
-
-   .. versionchanged:: 3.15
-      ``__cached__`` is no longer set.
+.. versionchanged:: 3.15
+   The ``__cached__`` attribute is no longer set on modules or taken into
+   consideration by the import system or standard library.
 
 Other writable attributes on module objects
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Doc/tutorial/stdlib.rst

@@ -36,7 +36,7 @@ aids for working with large modules like :mod:`os`::
    <returns an extensive manual page created from the module's docstrings>
 
 For daily file and directory management tasks, the :mod:`shutil` module provides
-a higher level interface that is easier to use::
+a higher-level interface that is easier to use::
 
    >>> import shutil
    >>> shutil.copyfile('data.db', 'archive.db')
@@ -63,7 +63,7 @@ wildcard searches::
 Command-line arguments
 ======================
 
-Common utility scripts often need to process command line arguments. These
+Common utility scripts often need to process command-line arguments. These
 arguments are stored in the :mod:`sys` module's *argv* attribute as a list.  For
 instance, let's take the following :file:`demo.py` file::
 
@@ -77,7 +77,7 @@ line::
    ['demo.py', 'one', 'two', 'three']
 
 The :mod:`argparse` module provides a more sophisticated mechanism to process
-command line arguments.  The following script extracts one or more filenames
+command-line arguments.  The following script extracts one or more filenames
 and an optional number of lines to be displayed::
 
     import argparse

Doc/whatsnew/3.15.rst

@@ -880,6 +880,14 @@ Other language changes
   other, as regular dynamic extensions do.
   (Contributed by Stefano Rivera in :gh:`122931`.)
 
+* The ``__cached__`` attribute on modules, which was deprecated since version
+  3.13, is no longer set or taken into consideration by the import system or
+  standard library.
+  Use :attr:`__spec__.cached <importlib.machinery.ModuleSpec.cached>` instead.
+  (Contributed by Brett Cannon in :gh:`97879`)
+
+  Note that the :attr:`~module.__loader__` and :attr:`~module.__package__`
+  attributes are also deprecated and scheduled for removal.
 
 
 Default interactive shell

Lib/tarfile.py

@@ -833,6 +833,13 @@ def _get_filtered_attrs(member, dest_path, for_data=True):
         # For example, 'C:/foo' on Windows.
         raise AbsolutePathError(member)
     # Ensure we stay in the destination
+    if '..' in name.replace(os.sep, '/').split('/'):
+        # Directories are created from the name as given, so a name that
+        # leaves the destination part-way through would create them
+        # outside it even if the resolved path stays inside.
+        normalized = os.path.normpath(name)
+        if normalized != name:
+            name = new_attrs['name'] = normalized
     target_path = os.path.realpath(os.path.join(dest_path, name),
                                    strict=os.path.ALLOW_MISSING)
     if os.path.commonpath([target_path, dest_path]) != dest_path:

Lib/test/support/__init__.py

@@ -1271,15 +1271,20 @@ def set_memlimit(limit: str) -> None:
 
 
 def _memory_watchdog(pid):
-    """Return a function printing the memory usage of process *pid*."""
+    """Return a function printing the memory usage of process *pid*.
+
+    The largest value it saw is kept in its ``peak`` attribute.
+    """
     # Imported here: test.support does not depend on test.libregrtest.
     from test.libregrtest.utils import get_process_memory_usage
 
     def watch():
         mem = get_process_memory_usage(pid)
         if mem is not None:
+            watch.peak = max(watch.peak, mem)
             print(f" ... process data size: {mem / (1024 ** 3):.1f} GiB",
                   flush=True)
+    watch.peak = 0
     return watch
 
 
@@ -1333,7 +1338,23 @@ def wrapper(self):
                 qualname = f'{cls.__qualname__}.{f.__name__}'
                 proc = isolation._start_test(cls.__module__, qualname)
                 watchdog = _memory_watchdog(proc.pid) if verbose else None
-                isolation._replay_test(self, *proc.wait(tick=watchdog))
+                payload, output, returncode = proc.wait(tick=watchdog)
+                if watchdog:
+                    # The subprocess measures its own peak exactly.  What the
+                    # parent sampled is only a lower bound.
+                    maxrss = payload and payload.get('maxrss')
+                    peak = maxrss or watchdog.peak
+                    if peak:
+                        print(f" ... peak memory use: "
+                              f"{peak / (1024 ** 3):.1f} GiB"
+                              f"{'' if maxrss else ' or more'}", flush=True)
+                    majflt = payload and payload.get('majflt')
+                    if majflt:
+                        # The test did not fit in memory, so its timing means
+                        # little.
+                        print(f" ... {majflt} major page faults: the test "
+                              f"waited for the disk", flush=True)
+                isolation._replay_test(self, payload, output, returncode)
                 return
 
             return f(self, maxsize)

Lib/test/support/subprocess_runner.py

@@ -72,7 +72,45 @@ def _outcome(kind, test, detail):
              for t, tb in result.expectedFailures]
 outcomes += [_outcome('skipped', t, reason) for t, reason in result.skipped]
 
-payload = {'outcomes': outcomes, 'durations': result.id_durations}
+def _usage():
+    """What this process used: peak resident set size in bytes, and the
+    number of major page faults it took, either of which can be None.
+
+    A major page fault is served from disk, so a non-zero count means swapping.
+
+    The modules are imported here, after the test has run, so that the test
+    does not see them.
+    """
+    try:
+        import resource
+    except ImportError:
+        pass
+    else:
+        usage = resource.getrusage(resource.RUSAGE_SELF)
+        # Solaris and illumos leave these fields at 0, which no live process
+        # has, so treat it as "not supported".
+        if not usage.ru_maxrss:
+            return {'maxrss': None, 'majflt': None}
+        # ru_maxrss is in bytes on macOS, in kilobytes on Linux and the BSDs.
+        maxrss = usage.ru_maxrss
+        return {'maxrss': maxrss if sys.platform == 'darwin' else maxrss * 1024,
+                'majflt': usage.ru_majflt}
+    try:
+        import os
+        import _winapi
+        handle = _winapi.OpenProcess(
+            _winapi.PROCESS_QUERY_LIMITED_INFORMATION, False, os.getpid())
+    except (ImportError, OSError):
+        return {'maxrss': None, 'majflt': None}
+    try:
+        info = _winapi.GetProcessMemoryInfo(handle)
+    finally:
+        _winapi.CloseHandle(handle)
+    # PageFaultCount counts all faults, not only the ones served from disk.
+    return {'maxrss': info['PeakWorkingSetSize'], 'majflt': None}
+
+
+payload = {'outcomes': outcomes, 'durations': result.id_durations, **_usage()}
 with open(outfile, 'wb') as f:
     marshal.dump(payload, f)