Skip to content

v2.1.0
Compare
Choose a tag to compare
@stephencpope stephencpope released this 05 Oct 02:20
· 88 commits to master since this release
v2.1.0
af0d4fb

General

  • Following our lifecycle policy, client versions v1.11.0 and earlier are no longer supported. They may
    cease to work with the Platform at any time.

Catalog

  • The Catalog Blob class now has a get_data() method which can be used to retrieve the blob
    data directly given the id, without having to first retrieve the Blob metadata.

Compute

  • Breaking Change The status values for Function and Job objects have changed, to provide a
    better experience managing the flow of jobs. Please see the updated Compute guide for a full explanation.
    Because of the required changes to the back end, older clients (i.e. v2.0.3) are supported in a
    best effort manner. Upgrading to this new client release is strongly advised for all users of the
    Compute service.

  • Breaking Change The base images for Compute have been put on a diet. They are now themselves built
    from "slim" Python images, and they no longer include the wide variety of extra Python packages that were
    formerly included (e.g. TensorFlow, SciKit Learn, PyTorch). This has reduced the base image size by
    an order of magnitude, making function build times and job startup overhead commensurately faster.
    Any functions which require such additional packages can add them in as needed via the requirements=
    parameter. While doing so will increase image size, it will generally still be much smaller and faster
    than the prior "Everything and the kitchen sink" approach. Existing Functions with older images will continue
    to work as always, but any newly minted `Function`` using the new client will be using one of the new
    slim images.

  • Base images are now available for Python3.10 and Python3.11, in addition to Python3.8 and Python3.9.

  • Job results and logs are now integrated with Catalog Storage, so that results and logs can be
    searched and retrieved directly using the Catalog client as well as using the methods in the Compute
    client. Results are organized under storage_type=StorageType.COMPUTE, while logs are organized under
    storage_type=StorageType.LOGS.

  • The new ComputeResult class can be used to wrap results from a Function, allowing the user to
    specify additional attributes for the result which will be stored in the Catalog Blob metadata for
    the result. This allows the function to specify properties such as geometry, description,
    expires, extra_attributes, writers and readers for the result Blob. The use of
    ComputeResult is not required.

  • A Job can now be assigned arbitrary tags (strings), and searched based on them.

  • A Job can now be retried on errors, and jobs track error reasons, exit codes, and execution counts.

  • Function and Job objects can now be filtered by class attributes (ex.
    Job.search().filter(Job.status == JobStatus.PENDING).collect()).

  • The Job.cancel() method can now be used to cancel the execution of a job which is currently
    pending or running. Pending jobs will immediately transition to JobStatus.CANCELED status,
    while running jobs will pass through JobStatus.CANCEL (waiting for the cancelation to be
    signaled to the execution engine), JobStatus.CANCELING (waiting for the execution to terminate),
    and JobStatus.CANCELED (once the job is no longer executing). Cancelation of running jobs is
    not guaranteed; a job may terminate successfully, or with a failure or timeout, before it can
    be canceled.

  • The Job.result() method will raise an exception if the job does not have a status of
    JobStatus.SUCCESS. If Job.result() yields an None value, this means that there was no
    result (i.e. the execution returned a None).

  • The Job.result_blob() method will return the Catalog Storage Blob holding the result, if any.

  • The Job.delete() method will delete any job logs, but will not delete the job result unless
    the delete_results parameter is supplied.

  • The Function object now has attributes namespace and owner.

  • The Function.wait_for_completion() and new Function.as_completed() methods provide a richer
    set of functionality for waiting on and handling job completion.

  • The Function.build_log() method now returns the log contents as a string, rather than printing
    the log contents.

  • The Job.log() method now returns the log contents as a list of strings, rather than printing the log
    contents. Because logs can be unbounded in size, there's also a new Job.iter_log() method which returns
    an iterator over the log lines.

  • The requirements= parameter to Function objects now supports more pip magic, allowing the use
    of special pip controls such as -f. Also parsing of package versions has been loosened to allow
    some more unusual version designators.

  • Changes to the Function.map() method, with the parameter name change of iterargs changed to kwargs
    (the old name is still honored but deprecated), corrected documentation, and enhancements to support more
    general iterators and mappings, allowing for a more functional programming style.

  • The compute package was restructured to make all the useful and relevant classes available at the top level.

Utils

  • Property filters can now be deserialized as well as serialized.
Assets 2
Loading