Release v0.43.0 · meltano/sdk

v0.43.0 (2024-12-10)

This release introduces a few improvements and changes to the SDK that should make it easier to work with SQL taps, REST streams, and targets.

A new SQL tap class attribute exclude_streams allows developers to exclude certain database schemas by default from discovery. This can speed up discovery and avoid issues when mapping types from database-internal schemas.
Discovery for SQL taps should be about 35% faster now, thanks to using SQLAlchemy 2.0+ reflection features.
A new target built-in setting process_activate_version_messages lets users disable processing of ACTIVATE_VERSION messages.
Support for other content-types in REST streams with the new RESTStream.payload_as_json attribute. This should make it easier to work with APIs that expect non-JSON payloads, such as XML.
The RESTStream.rest_method attribute is deprecated in favor of RESTStream.http_method.

Important

This release drops support for Python 3.8. Please make sure to upgrade your tap or target to support Python 3.9+.

Note

SQL taps now require SQLAlchemy 2.0+.

#2482 Allow SQL tap developers to auto-skip certain schemas from discovery
#2784 Added a new built-in setting activate_version for targets to optionally disable processing of ACTIVATE_VERSION messages
#2780 Numeric values are now parsed as decimal.Decimal in REST and GraphQL stream responses
#2775 Log a stream's bookmark (if it's avaiable) when its sync starts
#2703 Targets now emit record count from the built-in batch file processor
#2774 Accept a maxLength limit for VARCHARs
#2769 Add versioning-strategy to dependabot config of Cookiecutter templates
#2765 The last received Singer message is now logged when the target fails
#2762 Support other content-types in REST streams

#2790 Ensure the required global folder tap settings are merged into the concrete implementation settings
#2785 Use FS-specific listdir in folder tap
#2778 The path of the offending field is now printed for config validation errors
#2770 Respect standard Singer stream metadata table-key-properties, replication-key and forced-replication-method
#2755 Safely compare UUID replication keys with state bookmarks -- Thanks @nikzavada!

#2805 Rename setting activate_version to process_activate_version_messages
#2788 Fail early if input files to --catalog or --state do not exist
#2781 Added a class method to instantiate SQLToJSONSchema from the tap configuration
#2566 Standardize on JSON Schema Draft 2020-12 to validate stream schemas
#2751 Dropped support for Python 3.8

#2796 Document how to configure nested stream maps values with environment variables in Meltano