v1.16.4

What's Changed

• Add support pyspark.sql.classic.dataframe.DataFrame transformer by @arbaobao in #3272
• Remove tmate session to release ci resource by @arbaobao in #3316
• [BUG]Invalid cron date schedule creates infinite loop in flytescheduler by @popojk in #3312
• [Fix] Issue when using FlyteFile with Elastic by @machichima in #3313
• [BUG] ImageSpec NoOpBuilder should always return True while calling should_build function by @popojk in #3311
• Create execution according to task config by @arbaobao in #3302
• Handle None in sorted bound_inputs conditionally by @pingsutw in #3317
• Sort subworkflows when creating workflows by @wild-endeavor in #3318
• feat: Add faulthandler registration for SIGUSR1 by @pingsutw in #3310
• fix(pyproject.toml): Exclude gcsfs 2024.2.0 due to bug by @pingsutw in #3309
• IDL upper bound by @wild-endeavor in #3321
• [Fix] add env var to prevent deprecation warning by @machichima in #3319
• Dynamic platform detection for ImageSpec to support cross-platform development by @BarryWu0812 in #3315
• [Fix] Issue when using FlyteDirectory with Elastic by @machichima in #3320
• [Fix] Add .dockerignore to reduce build context size by @machichima in #3323
• add short_description to Task and Workflow model by @cosmicBboy in #3224
• Restore behavior passing FlyteClient kwargs to gRPC channel creation by @clint-stripe in #3303

New Contributors

• @BarryWu0812 made their first contribution in #3315
• @clint-stripe made their first contribution in #3303

Full Changelog: v1.16.3...v1.16.4

v1.16.3

What's Changed

• docker build: make sure venvs are owned by flytekit by @cosmicBboy in #3299

(Recommended to use 1.16.3 instead of 1.16.2)

Full Changelog: v1.16.2...v1.16.3

v1.16.2

What's Changed

• support FlyteRemote.execute with no inputs by @cosmicBboy in #3266
• [breaking change] neptune scale plugin by @samhita-alla in #3260
• Ignore ray reinit errors in local execution by @Ziemin in #3277
• enable notebook task/workflow registration by @cosmicBboy in #3275
• Fix issue where pod template override pod spec was missing by @Sovietaced in #3270
• Feat: Allow setting k8s labels and annotations in task decorator by @fg91 in #3243
• make ngc image secret optional by @samhita-alla in #3280
• Reduce logging level for dataclass type structure by @wild-endeavor in #3283
• Support recursive ImageSpec build by DefaultBuilder by @amitani in #3284
• Fix launch plan CLI registration by @pingsutw in #3285
• Version hash should include the image from launchPlan.workflow.nodes by @pingsutw in #3290
• Specify pandas version for Python 3.9 for deck plugin by @pingsutw in #3289
• Fix click option parameter declaration to remove short flag conflict by @pingsutw in #3286
• sort input bound by @pingsutw in #3292
• add L40S, H100, H200 to pre-defined accelerators by @cosmicBboy in #3293
• fix name of nvidia H100/H200 accelerators by @cosmicBboy in #3295
• add ephemeral storage param to nim by @samhita-alla in #3291
• [BUG] Local testing using optional dicts with optional values does not work by @machichima in #3294
• image builder: give flytekit user ownership of /root after venv is cr… by @cosmicBboy in #3298

New Contributors

• @Ziemin made their first contribution in #3277

Full Changelog: v1.16.1...v1.16.2

v1.16.2b1

What's Changed

• Version hash should include the image from launchPlan.workflow.nodes by @pingsutw in #3290
• Specify pandas version for Python 3.9 for deck plugin by @pingsutw in #3289
• Fix click option parameter declaration to remove short flag conflict by @pingsutw in #3286
• sort input bound by @pingsutw in #3292

Full Changelog: v1.16.2b0...v1.16.2b1

v1.16.2b0

What's Changed

• support FlyteRemote.execute with no inputs by @cosmicBboy in #3266
• [breaking change] neptune scale plugin by @samhita-alla in #3260
• Ignore ray reinit errors in local execution by @Ziemin in #3277
• enable notebook task/workflow registration by @cosmicBboy in #3275
• Fix issue where pod template override pod spec was missing by @Sovietaced in #3270
• Feat: Allow setting k8s labels and annotations in task decorator by @fg91 in #3243
• make ngc image secret optional by @samhita-alla in #3280
• Reduce logging level for dataclass type structure by @wild-endeavor in #3283
• Support recursive ImageSpec build by DefaultBuilder by @amitani in #3284
• Fix launch plan CLI registration by @pingsutw in #3285

New Contributors

• @Ziemin made their first contribution in #3277

Full Changelog: v1.16.1...v1.16.2b0

v1.14.9

What's Changed

• [cherry-pick] KeyError: 'sparkConf' occurs when running a Databricks task without spark_conf by @pingsutw in #3282

Full Changelog: v1.14.8...v1.14.9

v1.16.1

What's Changed

• ci: Update version extraction in CI workflow by @pingsutw in #3261
• Add ability to specify default resource requests/limits for tasks via pyflyte run and pyflyte register by @redartera in #3229
• add FlyteRemote.deactivate_launchplan by @cosmicBboy in #3264
• Fix airflow tests by @pingsutw in #3262
• [BUG] KeyError: 'sparkConf' occurs when running a Databricks task without spark_conf by @machichima in #3263
• Bump jinja2 from 3.1.4 to 3.1.6 in /plugins/flytekit-airflow by @dependabot in #3258

Full Changelog: v1.16.0...v1.16.1

v1.16.1b0

What's Changed

• ci: Update version extraction in CI workflow by @pingsutw in #3261

Full Changelog: v1.16.0...v1.16.1b0

v1.16.0

What's Changed

• Eager failure task to share image with parent by @wild-endeavor in #3194
• Update documentation to generate site. by @nelsonjr in #3196
• Fix error code propagation from user defined errors by @DanBrunkow in #3197
• Rename agent to connector by @pingsutw in #3165
• [flyteagent] FileSensor Timeout (Remote Execution Only) Enhancement by @Future-Outlier in #3191
• Add GetTaskMetrics and GetTaskLogs to connector service by @pingsutw in #3200
• feat: add geopandas plugin by @ljstrnadiii in #3203
• Rename Slurm task and sbatch config for improved UX by @JiangJiaWei1103 in #3195
• Reuse get SSH connection method in Slurm function agent by @JiangJiaWei1103 in #3189
• Check script existence on the Slurm cluster for ShellTask by @JiangJiaWei1103 in #3193
• Add LOCAL_DYNAMIC_TASK_EXECUTION mode by @pingsutw in #3202
• fix: use get_random_remote_path in geopandas plugin by @ljstrnadiii in #3208
• Get and parse Slurm job status JSON object by @JiangJiaWei1103 in #3192
• support using cache policies in with_overrides by @pvditt in #3211
• Add support for IPv6 and iceberg with spark >= 3.4 by @julianStreibel in #3206
• Add new codeowner by @davidmirror-ops in #3212
• fix: set GRPC_VERBOSITY and remove grpcio upper bound by @cameronraysmith in #3213
• Update plugin metadata by @nelsonjr in #3214
• Jan/generic spark task by @fiedlerNr9 in #3216
• Bump apache-airflow-providers-common-sql from 1.17.1 to 1.25.0rc1 in /plugins/flytekit-airflow by @dependabot in #3217
• Check for extraneous arguments to task by @joe-polin in #3218
• feat: Add Xarray zarr persistence support by @ljstrnadiii in #3205
• update update_all_versions command by @pingsutw in #3219
• ci: Update version extraction in workflows by @pingsutw in #3220
• Handle structure dataset handler duplicated log by @arbaobao in #3221
• Fix Pythonpublish typo by @arbaobao in #3222
• fix: Fix async s3fs to handle latest changes to FlyteFile by @ggydush in #3223
• Bugfix for issue #6405: Fast registration runs into infinite loop by @rambrus in #3230
• Adds ImageSpec.with_runtime_packages by @thomasjpfan in #3231
• [connector] CI fix by @Future-Outlier in #3235
• [connector] CI fix by @Future-Outlier in #3237
• Peeter/docstring clean new by @ppiegaze in #3225
• Image spec builder options by @mhotan in #3233
• add user agent to bigquery connector by @samhita-alla in #3232
• Add user-defined builder image in image spec by @arbaobao in #3207
• Add an --interruptible option to pyflyte run --remote by @redartera in #3228
• fix(flytekit-airflow): pin apache-airflow version < 3.0.0 by @pingsutw in #3244
• [BUG] flytekit: Cache object "policies" kwarg typing accepts a list, but errors if provided a list by @popojk in #3242
• Replacing deprecated disable_deck with new enable_deck property by @Mathis-Z in #3245
• Set _F_IMG_ID env var all the way at the bottom during ImageSpec builds by @redartera in #3227
• Include hash of pod template when computing fast registration version by @dansola in #3247
• Make image_spec.tag a cached property by @amitani in #3246
• [FIX] prevent build-with-pandas CI fail by @machichima in #3250
• imagespec fast fail env variable by @dansola in #3251
• gcsfs must be lower than 2025.5.0 by @dansola in #3252
• fix file/directory download in a pytorch elastic task by @samhita-alla in #3254
• fix: remote-task doesn't support boolean False by @taieeuu in #3204
• Fix type annotation of poll_token_endpoint by @ZhouGongZiBBS in #3256
• [CI] Fix plugin errors by @pingsutw in #3257
• Fix dataclass deserialization for fields with Optional types by @redartera in #3253
• Bump jinja2 from 3.1.5 to 3.1.6 in /plugins/flytekit-onnx-pytorch by @dependabot in #3199

New Contributors

• @nelsonjr made their first contribution in #3196
• @DanBrunkow made their first contribution in #3197
• @ljstrnadiii made their first contribution in #3203
• @julianStreibel made their first contribution in #3206
• @davidmirror-ops made their first contribution in #3212
• @joe-polin made their first contribution in #3218
• @rambrus made their first contribution in #3230
• @popojk made their first contribution in #3242
• @Mathis-Z made their first contribution in #3245
• @ZhouGongZiBBS made their first contribution in #3256

Full Changelog: v1.15.3...v1.16.0

v1.14.8

What's Changed

• (backport) Fast registration runs into infinite loop by @pingsutw in #3238

Full Changelog: v1.14.7...v1.14.8