diff --git a/doc/source/components.rst b/doc/source/components.rst index fad7d32..321674d 100644 --- a/doc/source/components.rst +++ b/doc/source/components.rst @@ -675,6 +675,55 @@ Covariances ^^^^^^^^^^^ :py:class:`imagine.observables.Covariances ` +are a special type of +:py:class:`ObservableDict ` +which stores the set of covariance matrices associated with set of observables. +Its behavour is similar to the +:py:class:`Measurements `, +the main difference being that the :py:data:`global_data` attribute returns a +(N,N) covariance matrix instead of (1,N)-array. + +In practice, one rarely needs to initialize or work with an independent +:py:obj:`Covariances ` object. +This is because a `Covariances` object is *automatically generated* when one +initializes a :py:class:`Measurements ` +a :py:obj:`Dataset `, and stored in the +`Measurements` object +:py:data:`cov ` attribute:: + + # The covariances associated with a measurements object + covariances = measurements.cov + +When the original dataset only contained *variance* information (e.g. specified +using the :py:data:`error` keyword when preparing a :py:obj:`Dataset `) the full covariance matrix is *not* +constructed on initialization. Instead, the variance alone is stored internally +and the covariance matrix construction is delayed until the first time the +properties :py:data:`global_data` or :py:data:`data` +are requested. If :py:data:`imagine.rc['distributed_arrays'] ` +is set to :py:data:`True` (not the default), +the covariance matrix is spread among the available MPI processes. + +As many observational datasets are very large (i.e. very large N), and only +contain known variance information, it often unnecessary (or simply impractical) +to operate with full (sparse) covariance matrices. Instead, one can access +the variance stored in the `Covariances` object using the :py:data:`var` +property:: + + # An Observable object containing covariance information for a given key + cov_observable = covariances[('sync', 30, 32, 'I')] + # Reads the N-array containing the variances + variances = cov_observable.var + # Reads the (N,N)-array containing the covariance matrix + # (constructing it, if necessary) + cov_matrix = cov_observable.global_data + +Thus, when working with large datasets some care should be take when choosing +(or designining) the :py:class:`Likelihood ` +class which will manipulate the `Covariances` object: some of the likelihoods +(e.g. :py:class:`EnsembleLikelihoodDiagonal `) +only access the `var` attribute, keeping the RAM memory usage under control +even in the case of very large maps, while others will try construct the full +covariance matrix, potentially leading out-of-memory errors. @@ -686,7 +735,8 @@ Masks ^^^^^ :py:class:`imagine.observables.Masks ` - +which stores masks which can be applied to HEALPix maps. The mask itself is +specified in the form of an array of zeros and ones. .. _Simulators: