Skip to content

Commit 88ac995

Browse files
miss-islingtonaiskwillingcsharktide
authored
[3.14] gh-85222: Document the global start method side effect in multiprocessing (GH-136426) (#142770)
gh-85222: Document the global start method side effect in multiprocessing (GH-136426) * Document the ctx parameter in some types in multiprocessing. * Reduce duplication while still linking to the central explanation from API points with the side effect. (cherry picked from commit 0978b9a) Co-authored-by: AN Long <[email protected]> Co-authored-by: Carol Willing <[email protected]> Co-authored-by: R Chintan Meher <[email protected]>
1 parent 575174e commit 88ac995

File tree

1 file changed

+72
-14
lines changed

1 file changed

+72
-14
lines changed

Doc/library/multiprocessing.rst

Lines changed: 72 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -521,6 +521,21 @@ Reference
521521
The :mod:`multiprocessing` package mostly replicates the API of the
522522
:mod:`threading` module.
523523

524+
.. _global-start-method:
525+
526+
Global start method
527+
^^^^^^^^^^^^^^^^^^^
528+
529+
Python supports several ways to create and initialize a process.
530+
The global start method sets the default mechanism for creating a process.
531+
532+
Several multiprocessing functions and methods that may also instantiate
533+
certain objects will implicitly set the global start method to the system's default,
534+
if it hasn’t been set already. The global start method can only be set once.
535+
If you need to change the start method from the system default, you must
536+
proactively set the global start method before calling functions or methods,
537+
or creating these objects.
538+
524539

525540
:class:`Process` and exceptions
526541
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -910,6 +925,9 @@ For an example of the usage of queues for interprocess communication see
910925
locks/semaphores. When a process first puts an item on the queue a feeder
911926
thread is started which transfers objects from a buffer into the pipe.
912927

928+
Instantiating this class may set the global start method. See
929+
:ref:`global-start-method` for more details.
930+
913931
The usual :exc:`queue.Empty` and :exc:`queue.Full` exceptions from the
914932
standard library's :mod:`queue` module are raised to signal timeouts.
915933

@@ -1025,6 +1043,9 @@ For an example of the usage of queues for interprocess communication see
10251043

10261044
It is a simplified :class:`Queue` type, very close to a locked :class:`Pipe`.
10271045

1046+
Instantiating this class may set the global start method. See
1047+
:ref:`global-start-method` for more details.
1048+
10281049
.. method:: close()
10291050

10301051
Close the queue: release internal resources.
@@ -1055,6 +1076,9 @@ For an example of the usage of queues for interprocess communication see
10551076
:class:`JoinableQueue`, a :class:`Queue` subclass, is a queue which
10561077
additionally has :meth:`task_done` and :meth:`join` methods.
10571078

1079+
Instantiating this class may set the global start method. See
1080+
:ref:`global-start-method` for more details.
1081+
10581082
.. method:: task_done()
10591083

10601084
Indicate that a formerly enqueued task is complete. Used by queue
@@ -1167,8 +1191,8 @@ Miscellaneous
11671191
:mod:`multiprocessing` module.
11681192

11691193
If *method* is ``None`` then the default context is returned. Note that if
1170-
the global start method has not been set, this will set it to the
1171-
default method.
1194+
the global start method has not been set, this will set it to the system default
1195+
See :ref:`global-start-method` for more details.
11721196
Otherwise *method* should be ``'fork'``, ``'spawn'``,
11731197
``'forkserver'``. :exc:`ValueError` is raised if the specified
11741198
start method is not available. See :ref:`multiprocessing-start-methods`.
@@ -1179,10 +1203,9 @@ Miscellaneous
11791203

11801204
Return the name of start method used for starting processes.
11811205

1182-
If the global start method has not been set and *allow_none* is
1183-
``False``, then the start method is set to the default and the name
1184-
is returned. If the start method has not been set and *allow_none* is
1185-
``True`` then ``None`` is returned.
1206+
If the global start method is not set and *allow_none* is ``False``, the global start
1207+
method is set to the default, and its name is returned. See
1208+
:ref:`global-start-method` for more details.
11861209

11871210
The return value can be ``'fork'``, ``'spawn'``, ``'forkserver'``
11881211
or ``None``. See :ref:`multiprocessing-start-methods`.
@@ -1409,13 +1432,19 @@ object -- see :ref:`multiprocessing-managers`.
14091432

14101433
A barrier object: a clone of :class:`threading.Barrier`.
14111434

1435+
Instantiating this class may set the global start method. See
1436+
:ref:`global-start-method` for more details.
1437+
14121438
.. versionadded:: 3.3
14131439

14141440
.. class:: BoundedSemaphore([value])
14151441

14161442
A bounded semaphore object: a close analog of
14171443
:class:`threading.BoundedSemaphore`.
14181444

1445+
Instantiating this class may set the global start method. See
1446+
:ref:`global-start-method` for more details.
1447+
14191448
A solitary difference from its close analog exists: its ``acquire`` method's
14201449
first argument is named *block*, as is consistent with :meth:`Lock.acquire`.
14211450

@@ -1436,13 +1465,18 @@ object -- see :ref:`multiprocessing-managers`.
14361465
If *lock* is specified then it should be a :class:`Lock` or :class:`RLock`
14371466
object from :mod:`multiprocessing`.
14381467

1468+
Instantiating this class may set the global start method. See
1469+
:ref:`global-start-method` for more details.
1470+
14391471
.. versionchanged:: 3.3
14401472
The :meth:`~threading.Condition.wait_for` method was added.
14411473

14421474
.. class:: Event()
14431475

14441476
A clone of :class:`threading.Event`.
14451477

1478+
Instantiating this class may set the global start method. See
1479+
:ref:`global-start-method` for more details.
14461480

14471481
.. class:: Lock()
14481482

@@ -1458,6 +1492,9 @@ object -- see :ref:`multiprocessing-managers`.
14581492
instance of ``multiprocessing.synchronize.Lock`` initialized with a
14591493
default context.
14601494

1495+
Instantiating this class may set the global start method. See
1496+
:ref:`global-start-method` for more details.
1497+
14611498
:class:`Lock` supports the :term:`context manager` protocol and thus may be
14621499
used in :keyword:`with` statements.
14631500

@@ -1515,6 +1552,9 @@ object -- see :ref:`multiprocessing-managers`.
15151552
instance of ``multiprocessing.synchronize.RLock`` initialized with a
15161553
default context.
15171554

1555+
Instantiating this class may set the global start method. See
1556+
:ref:`global-start-method` for more details.
1557+
15181558
:class:`RLock` supports the :term:`context manager` protocol and thus may be
15191559
used in :keyword:`with` statements.
15201560

@@ -1574,6 +1614,9 @@ object -- see :ref:`multiprocessing-managers`.
15741614

15751615
A semaphore object: a close analog of :class:`threading.Semaphore`.
15761616

1617+
Instantiating this class may set the global start method. See
1618+
:ref:`global-start-method` for more details.
1619+
15771620
A solitary difference from its close analog exists: its ``acquire`` method's
15781621
first argument is named *block*, as is consistent with :meth:`Lock.acquire`.
15791622

@@ -1718,7 +1761,7 @@ processes.
17181761
attributes which allow one to use it to store and retrieve strings -- see
17191762
documentation for :mod:`ctypes`.
17201763

1721-
.. function:: Array(typecode_or_type, size_or_initializer, *, lock=True)
1764+
.. function:: Array(typecode_or_type, size_or_initializer, *, lock=True, ctx=None)
17221765

17231766
The same as :func:`RawArray` except that depending on the value of *lock* a
17241767
process-safe synchronization wrapper may be returned instead of a raw ctypes
@@ -1732,9 +1775,13 @@ processes.
17321775
automatically protected by a lock, so it will not necessarily be
17331776
"process-safe".
17341777

1735-
Note that *lock* is a keyword-only argument.
1778+
*ctx* is a context object, or ``None`` (use the current context). If ``None``,
1779+
calling this may set the global start method. See
1780+
:ref:`global-start-method` for more details.
17361781

1737-
.. function:: Value(typecode_or_type, *args, lock=True)
1782+
Note that *lock* and *ctx* are keyword-only parameters.
1783+
1784+
.. function:: Value(typecode_or_type, *args, lock=True, ctx=None)
17381785

17391786
The same as :func:`RawValue` except that depending on the value of *lock* a
17401787
process-safe synchronization wrapper may be returned instead of a raw ctypes
@@ -1747,19 +1794,27 @@ processes.
17471794
automatically protected by a lock, so it will not necessarily be
17481795
"process-safe".
17491796

1750-
Note that *lock* is a keyword-only argument.
1797+
*ctx* is a context object, or ``None`` (use the current context). If ``None``,
1798+
calling this may set the global start method. See
1799+
:ref:`global-start-method` for more details.
1800+
1801+
Note that *lock* and *ctx* are keyword-only parameters.
17511802

17521803
.. function:: copy(obj)
17531804

17541805
Return a ctypes object allocated from shared memory which is a copy of the
17551806
ctypes object *obj*.
17561807

1757-
.. function:: synchronized(obj[, lock])
1808+
.. function:: synchronized(obj, lock=None, ctx=None)
17581809

17591810
Return a process-safe wrapper object for a ctypes object which uses *lock* to
17601811
synchronize access. If *lock* is ``None`` (the default) then a
17611812
:class:`multiprocessing.RLock` object is created automatically.
17621813

1814+
*ctx* is a context object, or ``None`` (use the current context). If ``None``,
1815+
calling this may set the global start method. See
1816+
:ref:`global-start-method` for more details.
1817+
17631818
A synchronized wrapper will have two methods in addition to those of the
17641819
object it wraps: :meth:`get_obj` returns the wrapped object and
17651820
:meth:`get_lock` returns the lock object used for synchronization.
@@ -1877,8 +1932,9 @@ their parent process exits. The manager classes are defined in the
18771932
*serializer* must be ``'pickle'`` (use :mod:`pickle` serialization) or
18781933
``'xmlrpclib'`` (use :mod:`xmlrpc.client` serialization).
18791934

1880-
*ctx* is a context object, or ``None`` (use the current context). See the
1881-
:func:`get_context` function.
1935+
*ctx* is a context object, or ``None`` (use the current context). If ``None``,
1936+
calling this may set the global start method. See
1937+
:ref:`global-start-method` for more details.
18821938

18831939
*shutdown_timeout* is a timeout in seconds used to wait until the process
18841940
used by the manager completes in the :meth:`shutdown` method. If the
@@ -2371,7 +2427,9 @@ with the :class:`Pool` class.
23712427
the worker processes. Usually a pool is created using the
23722428
function :func:`multiprocessing.Pool` or the :meth:`Pool` method
23732429
of a context object. In both cases *context* is set
2374-
appropriately.
2430+
appropriately. If ``None``, calling this function will have the side effect
2431+
of setting the current global start method if it has not been set already.
2432+
See the :func:`get_context` function.
23752433

23762434
Note that the methods of the pool object should only be called by
23772435
the process which created the pool.

0 commit comments

Comments
 (0)