Client release v1.10.0

Python Versions Supported

• Added support for Python 3.9.
• Removed support for Python 3.6 which is now officially End Of Life.

Workflows

• Added support for organizational sharing. You can now share using the Organization type:
  • workflows.add_reader(Organization("some_org"))

Discover

• Added support for organizational sharing. You can now share using the Organization type:
  • asset.share(with_=Organization("some_org"), as_="Viewer")
• Allow user to list their organization's namespace.
  • Discover().list_asset("asset/namespace/org:some_org")
• Allow user to list their organization's users.
  • Discover().list_org_users()

Tables - Added

• Added an alpha Tables client. The Tables module lets you organize, upload, and query tabular data and vector geometries. As an alpha release, we reserve the right to modify the Tables client API without any guarantees about backwards compatibility. See the Tables API and Tables Guide documentation for more details.

Scenes

• Added the progress= parameter to the various rastering methods such as Scene.ndarray,
  Scene.download, SceneCollection.mosaic, SceneCollection.stack, SceneCollection.download
  and SceneCollection.download_mosaic. This can be used to enable or disable the display
  of progress bars.

Client release v1.9.1

[1.9.1] - 2021-12-20

Raster

• Fixed a bug preventing retry-able errors (such as a 429) from being retried.

Client release v1.9.0

Catalog

• Allow retrieving Attribute as a class attribute. It used to raise an exception.

Scenes

• Fixed a bug preventing the user from writing JPEG files with smaller than 256x256 tiles.
• Allow specifying a NoData value for non-JPEG GeoTIFF files.
• Include band description metadata in created GeoTIFF files.
• Support scaling parameters as lists as well as tuples.
• Add caching of band metadata to drastically reduce the number of metadata queries when creating SceneCollections.
• DLTiles.from_shape was failing to handle shape objects implementing the __geo_interface__ API, most notably several of the Workflows GeoContext types. These now work as expected.
• Certain kinds of network issues could read to rastering operations raising an IncompleteRead exception. This is now correctly caught and retried within the client library.

Tasks

• Users can now use descarteslabs.tasks.update_credentials() to update their task credentials in case they became outdated.

Workflows

• We have introduced a hard limit of 120 as the number of outstanding Workflows compute jobs that a single user can have. This limit exists to minimize situations in which a user is unable to complete jobs in a timely manner by ensuring resources cannot be monopolized by any individual user. The API that backs the calls to compute will return a descarteslabs.client.grpc.exceptions.ResourceExhausted error if the caller has too many outstanding jobs. Prior to this release (1.9.0), these failures would be retried up to some small retry limit. With the latest client release however, the client will fail without retrying on an HTTP 429 (rate limit exceeded) error. For users with large (non-interactive) workloads who don’t mind waiting, we added a new num_retries parameter to the compute function; when specified, the client will handle any 429 errors and retry up to num_retries times.
• Workflows is currently optimized for interactive use cases. If you are submitting large numbers of long-running Workflows compute jobs with block=False, you should consider using Tasks and Scenes rather than the Workflows API.
• Removed ResourceExhausted exceptions from the list of exceptions we automatically catch and retry on for compute calls.

Documentation

• Lots of improvements, additions, and clarifications in the API documentation.

Client release v1.8.2

• Workflows client no longer validates processing_level parameter values, as these have been enhanced to support new products
  and can only be validated server side.
• Catalog V2 bands now support the vendor_band_name field (known as name_vendor in Metadata/Catalog V1).
• Scenes support for masking in version 1.8.1 had some regressions which have been fixed. For this reason, version
  1.8.1 has been pulled from PyPI.
• New task groups now default to a maximum_concurrency value of 5, rather than the previous 500. This avoids the common
  problem of deploying a task group with newly developed code, and having it scale up and turning small problems into
  big problems! You may still set values as large as 500.
• The Tasks client now provides an update_group() method which can be used to update many properties of an existing
  task group, including but not limited to name, image, minimum_concurrency, and maximum_concurrency.
• Improved testing across several sub-packages.
• Various documentation fixes.

Client release v1.8.1

General

• Added a new common.dltile library that performs geospatial transforms and tiling operations.
• Upgraded various dependencies: requests[security]>=2.25.1,<3,six>=1.15.0,blosc==1.10.2, mercantile>=1.1.3,Pillow>=8.1.1,protobuf>=3.14.0,<4,shapely>=1.7.1,<2,tqdm>=4.32.1,traitlets>=4.3.3,<6;python_version<'3.7',traitlets==5.0.5,<6;python_version>='3.7',markdown2>=2.4.0,<3,responses==0.12.1,freezegun==0.3.12,imagecodecs>=2020.5.30;python_version<'3.7',imagecodecs>=2021.5.20;python_version>='3.7',tifffile==2020.9.3;python_version<'3.7',tifffile==2021.4.8;python_version>='3.7'

Discover (alpha) - Added

• Added an alpha Discover client. Discover allows users to organize and share assets with other users. As an alpha release, we reserve the right to modify the Discover client API without any guarantees about backwards compatibility. See the Discover API documentation for more details.

Metadata/Catalog V1 - Changed

• breaking Image (Scene) metadata now accepts and returns the bucket and directory fields as lists of strings, of a length equal to that
  of the files fields. This allows the file assets making up an image to live in different locations. When creating new images,
  a simple string can still be provided for these fields. It will automatically be converted to a list of (duplicated) strings as
  necessary. As most users will never interact with these fields, the change should not affect user code.

Metadata/Catalog V1/Catalog V2 - Changed

• derived_params field for Image (scene) metadata now supported for product-specific service-implemented "native derived bands" which may
  only be created for core products.

Scenes - Changed

• Scenes now uses the client-side dltile library to make DLTiles. This improves performance when creating a large number of DLTile objects.
• Scenes DLTile from_shape now has a parameter to return tile keys only instead of full tile objects. Usage details can be found in the docs.
• Scenes DLTile now has new methods: iter_from_shape that takes the same arguments as from_shape but returns an iterator (from_shape docs), subtile that adds the ability to subdivide tiles (subtile docs), and rowcol_to_latlon and latlon_to_rowcol which converts pixel coordinates to spatial coordinates and vice versa (rowcol_to_latlon docs and latlon_to_rowcol docs).
• Scenes DLTile now has a new parameter tile_extent which is the total size of the tile in pixels including padding. Usage details can be found in the docs.
• breaking Removed the dependence on Raster for tiling. The raster_client parameter has been removed from the from_latlon, from_key, from_shape, and assignDLTile methods.
• Tiling using from_shape may return a different number of tiles compared to previous versions under certain conditions. These tiles are usually found in overlapping areas between UTM zones and should not affect the overall coverage.
• DLTile geospatial transformations are guaranteed to be within eight decimal points of the past implementation.
• DLTile errors now come from the dltile library and error messages should now be more informative.
• When specifying output bounds in a spatial reference system different from the underlying raster, a densified representation of the bounding box is used internally to ensure that the returned image fully covers the bounds. For certain methods (like mosaic) this may change the returned image dimensions, depending on the SRSs involved.
• breaking As with the Metadata v1 client changes, the bucket and directory fields of the Scene properties are now multi-valued lists.
• Scenes does not support writing GeoTiffs to file-like objects. Non-JPEG GeoTiffs are always uncompressed.

Raster - Changed

• dltiles_from_shape, dltiles_from_latlon, and dltile have been removed. It is
  strongly recommended to test any existing code which uses the Raster API when upgrading to this
  release.
• Fully masked arrays are now supported and are the default. Usage details can be found in the docs
• Added support to draw progress bar. Usage details can be found in the docs.
• The signature and return value of Raster.raster() have changed. The save= parameter has been removed as the resulting download is always saved
  to disk, to a file named by the outfile_basename= parameter. The method returns a tuple containing the name of the resulting file and the metadata
  for the retrieval, which is now an ordinary Python dictionary.
• As with Scenes, when specifying output bounds in a spatial reference system different from the underlying raster, a densified representation of the bounding box is used internally to ensure that the returned image fully covers the bounds. For certain methods (like mosaic) this may change the returned image dimensions, depending on the SRSs involved.

(Note: v1.8.0 was an internal-only release)

Client release v1.7.1

General

• Upgraded various dependencies: blosc==1.10.2, cachetools>=3.1.1, grpcio>=1.35.0,<2, ipyleaflet>=0.13.3,<1, protobuf>=3.14.0,<4, pyarrow>=3.0.0, pytz>=2021.1
• Upgraded from using Travis to GitHub Actions for CI.

Catalog

• Added support for the physical_range property on SpectralBand and MicrowaveBand.

Workflows (channel v0-19) - Added

• Workflows sharing. Support has been added to manage sharing of Workflow objects with other authorized users. The public option for publishing workflows
  has been removed now that Workflow.add_public_reader() provides the equivalent capability. See the Workflows Guide.
• Lots of improvements to API documentation and the Workflows Guide.

Workflows - Fixed

• Allow constructing Float instances from literal python integers.

Client release v1.7.0

This release was withdrawn due to a backwards compatility issue which arose.

Client release 1.6.1

[1.6.1] - 2021-01-27

Fixes a few buglets which slipped through. This release continues to use the workflows channel v0-18.

Workflows - Fixed

• Fixed a problem with the defaulting of the visual options when generating tile URLs, making it possible
  to toggle the checkerboard option on a layer and see the difference.
• Support axis=list(...) for Image.
• Corrected the results of doing arithmetic on two widgets (e.g. adding two IntSliders together should yieldanInt`).
• For single-band imagery VizOption will accept a single two-tuple for the scales= argument.

Client release 1.6.0

[1.6.0] - 2021-01-20

Python Version Support

• Python 3.6 is now deprecated, and support will be removed in the next version.

Catalog

• Added support to Bands for new processing levels and processing step specifications
  to support Landsat Collection 2.

Workflows (channel v0-18) - Added

• wf.widgets lets you quickly explore data interactively. Add widgets anywhere in your code just like normal values, and the widgets will display automatically when you call .visualize.
• View shared Workflows and XYZs in GIS applications using WMTS. Get the URL with wf.wmts_url(), XYZ.wmts_url(), Workflow.wmts_url().
  • Create publicly-accessible tiles and WMTS endpoints with wf.XYZ(..., public=True). Anyone with the URL (which is a cryptographically random ID) can view the data, no login required. Set days_to_expiration to control how long the URL lasts.
  • wf.XYZ.list() to iterate through all XYZ objects you've created, and XYZ.delete to delete them.
  • Set default vizualization options (scales, colormap, bands, etc.) in .publish or wf.XYZ with wf.VizOption. These viz_options are used when displaying the published object in a GIS application, or with wf.flows.
• ImageCollection.visualize(): display ImageCollections on wf.map, and select the reduction operation (mean, median, mosaic, etc.) interactively
• Image.reduction() and ImageCollection.reduction() (like ic.reduction("median", axis="images")) to reduce an Image/ImageCollection with an operation provided by name
• wf.map.controls is accessible (you had to do wf.map.map.controls before)
• Access the parameters used in a Job with Job.arguments and Job.geoctx.

Workflows - Fixed

• Errors like In 'or': : operand type(s) all returned NotImplemented from __array_ufunc__ when using the bitwise-or operator | are resolved.
• Errors when using computed values in the wf.Datetime constructor (like wf.Datetime(wf.Int(2019) + 1)) are resolved.
• wf.Timedelta can be constructed from floats, and supports all binary operations that Python does (support for /, //, %, * added)
• In .rename_bands, prohibit renaming a band to a name that already exists in the Image/ImageCollection. Previously, this would succeed, but cause downstream errors.
• .bandinfo.get("bandname", {}) now works---before, providing {} would fail with a TypeError
• Indexing an Any object (like wf.Any({"foo": 1})["foo"]) behaves correctly
• wf.Datetimes constructed from strings containing timezone information are handled correctly

Workflows - Changed

• .mask(new_mask) ignores masked pixels in new_mask. Previously, masked pixels in new_mask were considered True, not False. Note that this is opposite of NumPy's behavior.
• If you .publish an object that depends on wf.parameters or wf.widgets, it's automatically converted into a wf.Function.
• breaking .compute and .inspect no longer accept extra arguments that aren't required for the computation. If the object doesn't depend on any wf.parameters or wf.widgets, passing extra keyword arguments will raise an error. Similarly, not providing keyword arguments for all parameters the object depends on will raise an error.
• breaking The wf.XYZ interface has changed; construct an XYZ with wf.XYZ(...) instead of wf.XYZ.build(...).save()
• Set days_to_expiration on XYZ objects. After this many days, the object is deleted.
• Job metadata is deleted after 10 days; wf.Job.get(...) on a job ID more than 10 days old will fail. Note that Job results have always been deleted after 10 days; now the metadata expires as well.
• wf.Function has better support for named arguments. Now, f = wf.Function[{'x': wf.Int, 'y': wf.Str}, wf.Int] requires two arguments x and y, and they can be given positionally (f(1, "hi")), by name in any order(f(x=1, y="hi") or f(y="hi", x=1)), or both (f(1, y="hi")). wf.Function.from_callable will generate a Function with the same names as the Python function you decorate or pass in. Therefore, when using @wf.publish as a decorator, the published Function will automatically have the same argument names as your Python function.

Client release v1.5.0

Python Version Support

• Python 3.8 is now supported in the client.
• As Python 3.5 has reached End Of Life, it is no longer supported by the descarteslabs client.

Tasks Client

• Altered the behavior of Task function creation. Deprecation warnings will be issued when attempting to create
  a Task function for which support will be removed in the near future. It is
  strongly recommended to test any existing code which uses the Tasks client when upgrading to this
  release.
• New tasks public images for for use with Python 3.8 are available.

Workflows (channel v0-17) - Fixed

• .pick_bands supports proxy wf.Str objects; .unpack_bands supports wf.Str and wf.Tuple[wf.Str, ...].
• Better performance constructing a wf.Array from a List of numbers (like wf.Array(ic.sum(["pixels", "bands"])))
• No more error using @wf.publish as a decorator on a function without a docstring