v2.0.0

(Release notes from all the 2.0.0 release candidates are summarized here for completeness.)

Supported platforms

• Deprecated support for Python 3.7 (will end of life in July).
• Added support for Python 3.10 and Python 3.11
• AWS-only client. For the time being, the AWS client can be used to communicate with the legacy GCP platform (e.g. DESCARTESLABS_ENV=gcp-production) but only supports those services that are supported on AWS (catalog and scenes). This support may break at any point in the future, so it is strictly transitional.

Dependencies

• Removed many dependencies no longer required due to the removal of GCP-only features.
• Added support for Shapely 2.X. Note that user code may also be affected by breaking changes in
  Shapely 2.X. Use of Shapely 1.8 is still supported.
• Updated requirements to avoid urllib3>=2.0.0 which breaks all kinds of things.

Configuration

• Major overhaul of the internals of the config process. To support other clients using namespaced packages within the descarteslabs package, the top level has been cleaned up, and most all the real code is down inside descarteslabs.core. End users should never have to import anything from descarteslabs.core. No more magic packages means that pylint will work well with code using descarteslabs.
• Configuration no longer depends upon the authorized user.

Catalog

• Added support for data storage. The Blob class provides mechanism to upload, index, share, and retrieve arbitrary byte sequences (e.g. files). Blobs can be searched by namespace and name, geospatial coordinates (points, polygons, etc.), and tags. Blobs can be downloaded to a local file, or retrieved directly as a Python bytes object. Blobs support the same sharing mechanisms as Products, with owners, writers, and readers attributes.
• Added support to Property for prefix filtering.
• The default geocontext for image objects no longer specifies a resolution but rather a shape, to ensure
  that default rastering preserves the original data and alignment (i.e. no warping of the source image).
• As with resolution, you can now pass a crs parameter to the rastering methods (e.g. Image.ndarray,
  ImageCollection.stack, etc.) to override the crs of the default geocontext.
• A bug in the code handling the default context for image collections when working with a product with a CRS based on degrees rather than meters has been fixed. Resolutions should always be specified in the units used by the CRS.

Compute

• Added support for managed batch compute under the compute module.

Raster Client

• Fixed a bug in the handling of small blocks (less than 512 x 512) that caused rasterio to generate bad download files (the desired image block would appear as a smaller sub-block rather than filling the resulting raster).

Geo

• The defaulting of align_pixels has changed slightly for the AOI class. Previously it always defaulted to
  True. Now the default is True if resolution is set, False otherwise. This ensures that when specifying
  a shape and a bounds rather than a resolution,the shape is actually honored.
• When assigning a resolution to an AOI, any existing shape attribute is automatically unset, since the
  two attributes are mutually exclusive.
• The validation of bounds for a geographic CRS has been slightly modified to account for some of the
  irregularities of whole-globe image products, correcting unintended failures in the past.
• Fixed problem handling MultiPolygon and GeometryCollection when using Shapely 2.0.

v2.0.0rc5

Catalog

• Loosen up the restrictions on the allowed alphabet for Blob names. Now almost any printable
  character is accepted save for newlines and commas.
• Added new storage types for Blobs: StorageType.COMPUTE (for Compute job results) and
  StorageType.DYNCOMP (for saved dynamic-compute operations).

Compute

• Added testing of the client.

v2.0.0rc4

Catalog

• The defaulting of the namespace value for Blobs has changed slightly. If no namespace is specified,
  it will default to <org>:<hash> with the user's org name and unique user hash. Otherwise, any other value,
  as before, will be prefixed with the user's org name if it isn't already so.
• Blob.get no longer requires a full id. Alternatively, you can give it a name and optionally a namespace
  and a storage_type, and it will retrieve the Blob.
• Fixed a bug causing summaries of Blob searches to fail.

Compute

• Function.map and Function.rerun now save the created Jobs before returning.
• Job.get return values fixed, and removed an extraneous debug print.

General

• Updated requirements to avoid urllib3>=2.0.0 which break all kinds of things.

v2.0.0rc3

Geo

• The defaulting of align_pixels has changed slightly for the AOI class. Previously it always defaulted to
  True. Now the default is True if resolution is set, False otherwise. This ensures that when specifying
  a shape and a bounds rather than a resolution,the shape is actually honored.
• When assigning a resolution to an AOI, any existing shape attribute is automatically unset, since the
  two attributes are mutually exclusive.
• The validation of bounds for a geographic CRS has been slightly modified to account for some of the irregularities
  of whole-globe image products, correcting unintended failures in the past.

Catalog

• The default geocontext for image objects no longer specifies a resolution but rather a shape, to ensure
  that default rastering preserves the original data and alignment (i.e. no warping of the source image).
• The Blob.upload and Blob.upload_data methods now return self, so they can be used in a fluent style.
• As with resolution, you can now pass a crs parameter to the rastering methods (e.g. Image.ndarray,
  ImageCollection.stack, etc.) to override the crs of the default geocontext.

Compute

• A bevy of fixes to the client.

v2.0.0rc2

Catalog

• Added support for data storage. The Blob class provides mechanism to upload, index, share, and retrieve arbitrary byte sequences (e.g. files). Blobs can be searched by namespace and name, geospatial coordinates (points, polygons, etc.), and tags. Blobs can be downloaded to a local file, or retrieved directly as a Python bytes object. Blobs support the same sharing mechanisms as Products, with owners, writers, and readers attributes.
• Added support to Property for prefix filtering.

Compute

• Added method to update user credentials for a Function.
• Added methods to retrieve build and job logs.

General

• Added support for Shapely=2.X.

v2.0.0rc1

[2.0.0rc1] - 2023-04-12

• This is an internal-only release. There is as of yet no updated documentation. However, the user-facing client APIs remain fully compatible with v1.12.1.

Compute

• Added support for managed batch compute under the compute module.

Auth and Configuration

• Removed the check on the Auth for configuration, since it is all AWS all the time.

Raster Client

• Fixed a bug in the handling of small blocks (less than 512 x 512) that caused rasterio to generate bad download files (the desired image block would appear as a smaller sub-block rather than filling the resulting raster).

v2.0.0rc0

• This is an internal-only release. There is as of yet no updated documentation. However, the user-facing client APIs remain fully compatible with v1.12.1.

Supported platforms

• Deprecated support for Python 3.7 (will end of life in July).
• Added support for Python 3.10 and Python 3.11
• AWS-only client. For the time being, the AWS client can be used to communicate with the legacy GCP platform (e.g. DESCARTESLABS_ENV=gcp-production) but only supports those services that are supported on AWS (catalog and scenes). This support may break at any point in the future, so it is strictly transitional.

Dependencies

• Removed many dependencies no longer required due to the removal of GCP-only features.

Configuration

• Major overhaul of the internals of the config process. To prepare for supporting other clients using namespaced packages within the descarteslabs package, the top level has been cleaned up, and most all the real code is down inside descarteslabs.core. However end users should never have to import anything from descarteslabs.core. No more magic packages means that pylint will work well with code using descarteslabs.
• GCP environments only support catalog and scenes. All other GCP-only features have been removed.

Catalog

• A bug in the code handling the default context for image collections when working with a product with a CRS based on degrees rather than meters has been fixed. Resolutions should always be specified in the units used by the CRS.

Client release v1.12.1

Workflows

• Fixed a bug causing descarteslabs.workflows.map.geocontext() to fail with an import error. This problem
  also affected the autoscaling feature of workflows map layers.

Catalog/Scenes/Raster

• Fixed a bug causing downloads of single-band images to fail when utilizing rasterio.

Client release v1.12.0

Catalog

• Catalog V2 is now fully supported on the AWS platform, including user ingest.
• Catalog V2 has been enhanced to provide substantially all the functionality of the Scenes API. The Image class now
  includes methods such as ndarray and download. A new ImageCollection class has been added, mirroring SceneCollection.
  The various Search objects now support a new collect method which will return appropriate Collection types
  (e.g. ProductCollection, BandCollection, and of course ImageCollection). Please see the updated Catalog V2
  guide and API documentation for more details.
• Previously, the internal implementation of the physical_range attribute on various band types was inconsistent with
  that of data_range and display_range. It has now been made consistent, which means it will either not be set,
  or will contain a 2-tuple of float values. It is no longer possible to explicitly set it to None.
• Access permissions for bands and images are now managed directly by the product. The readers, writers, and
  owners attributes have been removed from all the *Band classes as well as the Image class. Also the
  Product.update_related_objects_permissions and Product.get_update_permissions_status methods have been removed
  as these are no longer necessary or supported.
• All searches for bands (other than derived bands) and images must specify one or more product ids in the filtering.
  This requirement can be met by using the bands() and images() methods of a product to limit the search to that
  product, or through a filter(properties.product_id==...) clause on the search.
• Products have a new product_tier attribute, which can only be set or modified by privileged users.
• The Image.upload_ndarray will now accept either an ndarray or a list of ndarrays, allowing multiple files per image.
  The band definitions for the product must correspond to the order and properties of the multiple ndarrays.

Scenes

• With the addition of the Scenes functionality to Catalog V2, you are strongly encouraged to migrate your Scenes-based
  code to use Catalog V2 instead. Scenes will be deprecated in a future release. Some examples of migrating from Scenes
  to Catalog V2 are included in the Catalog V2 guide. In the meantime the Scenes API has been completely reimplemented
  to use Catalog V2 under the hood. From a user perspective, existing code using the Scenes API should continue to
  function as normal, with the exception of a few differences around some little-used dark corners of the API.
• The Scenes search_bands now enforces the use of a non-empty products= parameter value. This was previously
  documented but not enforced.

Metadata

• With the addition of the Scenes functionality to Catalog V2, you are strongly encouraged to migrate your Metadata-based
  code to use Catalog V2 instead. Metadata will be deprecated in a future release.
• As with Catalog and Scenes, one or more products must now be specified when searching for bands or images.

Raster

• The Raster client API now requires a bands= parameter for all rastering operations, such as raster, ndarray
  and stack. It no longer defaults to all bands defined on the product.

DLTile

• An off-by-1/2-pixel problem was identified in the coordinate transforms underlying
  DLTile.rowcol_to_latlon and DLTile.latlon_to_rowcol. The problem has been corrected,
  and you can expect to see slight differences in the results of these two methods.

REST Clients

• All the REST client types, such as Metadata and Raster, now support get_default_client() and set_default_client()
  instances. This functionality was previously limited to the Catalog V2 CatalogClient. Whenever such a client is required,
  the client libraries use get_default_client() rather than using the default constructor. This makes it easy to
  comprehensively redirect the library to use a specially configured client when necessary.

Geo package

• The GeoContext types that originally were part of the Scenes package are now available in the new descarteslabs.geo package,
  with no dependencies on Scenes. This is the preferred location from which to import these classes.

Utils package

• The descarteslabs.utils package, added in the previous release for the AWS client only, now exists in the GCP client
  as well, and is the preferred location to pick up the DotDict and DotList classes, the display and save_image functions,
  and the Properties class for property filtering in Catalog V2.
• The display method now has added support for multi-image plots, see the API documentation for the figsize, nrows,
  ncols and layout_direction parameters.

Property filtering

• The property_filtering.GenericProperties class has been replaced with property_filtering.Properties, but remains
  for back compatibility.
• Property filters now support isnull and isnotnull operations. This can be very useful for properties which may or
  may not be present, e.g. properties.cloud_fraction.isnull | properties.cloud_fraction <= 0.2.

Configuration and Authentication

• The Config exceptions RuntimeError and KeyError were changed to ConfigError exceptions
  from descarteslabs.exceptions.
• Auth now retrieves its URL from the Config settings. If no valid configuration can be found,
  it reverts to the commercial service (https://iam.descarteslabs.com).

General

• Dependencies for the descarteslabs library have been updated, but remain constrained to continue to support Python 3.7.
• Numerous bug fixes.

Client release v1.11.0

Installation

• The extra requirement options have changed. There are four extra requirement options now, visualization, tables,
  complete, and tests. visualization pulls in extra requirements to support operating in a Jupyter notebook or
  environment, enabling interactive maps and graphical displays. It is not required for operating in a "headless"
  manner. tables pulls in extra requirements to support the Tables client. complete is the combination of
  visualization and tables. tests pulls in extra requirements for running the tests. As always,
  pip install 'descarteslabs[complete]' will install a fully enabled client.

Configuration

• The Descartes Labs client now supports configuration to support operating in different environments. By default,
  the client will configure itself for standard usage against the GCP platform ("gcp-production"), except in the case of AWS Marketplace users, for whom
  the client will configure itself against the AWS platform ("aws-production").
  Alternate environments can be configured by setting the DESCARTESLABS_ENV environment variable before starting python, or by using a prelude like

  from descarteslabs.config import Settings
  Settings.select_env("environment-name")
  

  before any other imports of any part of the descarteslabs client package.
• The new AWS Enterprise Accelerator release currently includes only Auth, Configuration
  and the Scenes client.

Auth and Exceptions

• The descarteslabs.client.auth package has moved to descarteslabs.auth. It is now imported
  into the original location at descarteslabs.client.auth to continue to work with existing
  code, but new code should use the new location.
• The descarteslabs.client.exceptions module has moved to descarteslabs.exceptions. It is
  now imported into the original location at descarteslabs.client.exceptions to continue to
  work with existing code, but new code should use the new location.

Scenes

• Fixed an issue in scenes.DLTile.from_shape where there would be incomplete coverage of certain geometries. The function may now return more tiles than before.
• Added support for the new all_touched parameter to the different GeoContext types. Default behavior remains the same
  as always, but if you set all_touched=True this communicates to the raster service that you want the image(s) rastered
  using GDAL's CUTLINE_ALL_TOUCHED option which will change how source pixels are mapped to output pixels. This mode is
  only recommended when using an AOI which is smaller than the source imagery pixel resolution.
• The DLTile support has been fixed to avoid generating gaps when tiling regions that span
  a large distance north-to-south and straddle meridians which are boundaries between
  UTM zones. So methods such as DLTile.from_shape may return more tiles than previously,
  but properly covering the region.
• Added support for retrieving products and bands.
  • Methods added: get_product, get_band, get_derived_band, search_products,
    search_bands, search_derived_bands.
  • Disallows search without products parameter.
• Scaling support has been enhanced to understand processing levels for newer products. The
  Scene.scaling_parameters and SceneCollection.scaling_parameters methods now accept
  a processing_level argument, and this will be factored in to the determination of
  the default result data type and scaling for all rastering operations such as Scene.ndarray
  and SceneCollection.mosaic.
• If the user provides the rasterio package (which implies providing GDAL), then rasterio
  will be used to save any downloaded images as GeoTIFF, allowing for the use of compression.
  Otherwise, by default the tifffile support will be used to generate the GeoTIFF files
  but compression is not supported in this mode.
• As the Places client has been deprecated, so has any use of the place= parameter supported
  by several of the Scenes functions and methods.

Catalog

• (Core users only) Added support for specifying the image index to use when creating a new Product.
• Added support for defining per-processing-level data_type, data_range, display_range
  and physical_range properties on processing level steps.

Discover

• Added support for filtering Assets by type and name fields.
  • Supported filter types blob, folder, namespace, sym_link, sts_model, and vector. Specifying multiple types will find assets matching any given type.
  • The name field supports the following wildcards:
    • * matches 0 or more of any character.
    • ? matches 1 of any character.
  • Find assets matching type of blob and having a display name of file name.json or file2name.txt but not filename.json:
    • Discover().list_assets("asset/namespace/org:some_org", filters="type=blob&name=file?name.*")
    • Discover().list_assets("asset/namespace/org:some_org", filters=AssetListFilter(type=AssetType.BLOB, name="file?name.*"))
  • Find assets of type blob or vector:
    • Discover().list_assets("asset/namespace/org:some_org", filters="type=blob,vector")
    • Discover().list_assets("asset/namespace/org:some_org", filters=AssetListFilter(type=[AssetType.BLOB, AssetType.VECTOR], name="file?name.*"))

Metadata

• Metadata.products and Metadata.available_products now properly implement paging so that
  by default, a DotList containing every matching product accessible to the user is returned.

Raster

• If the user provides the rasterio package (which implies providing GDAL), then rasterio
  will be used to save any downloaded images as GeoTIFF, allowing for the use of compression.
  Otherwise, by default the tifffile support will be used to generate the GeoTIFF files
  but compression is not supported in this mode.

Tables

• Fixed an issue that caused a user's schema to be overwritten if they didn't provide a primary
  key on table creation.
• Now uses Discover backend filtering for list_tables() instead of filtering on the client to
  improve performance.
• list_tables() now supports filtering tables by name
  • Tables.list_tables(name="Test*.json")

Tasks

• New Tasks images for this release bump the versions of several dependencies, please see
  the Tasks guide for detailed lists of dependencies.

Workbench

• The new Workbench release bumps the versions of several dependencies.

Workflows

• Added support for the new all_touched parameter to the different GeoContext types.
  See description above under Scenes.

General

• The Places client has been deprecated, and use thereof will generate a deprecation warning.
• The older Catalog V1 client has been deprecated, and use thereof will generate a deprecation
  warning. Please use the Catalog V2 client in its place.
• Documentation has been updated to include the `AWS Enterprise Accelerator" release.
• With Python 2 far in the rearview mirror, the depedencies on the six python package have
  been removed throughout the library, the distribution and all tasks images.