1.10.0
Core Library
- feat(paginator): enhance JSONResponseCursorPaginator to support cursor placement in request JSON body by @kang8 in #2446
- fix: better import exception for numpy by @zilto in #2397
- Fix read parquet chunk size by @diwu-sf in #2456
- rest_api: enhance placeholder expansion to preserve value types by @burnash in #2462
- list-destination flag added by @zilto in #2441
- Fixes bigquery, ibis tests by @anuunchin in #2470
- add ignore_unknown_values option to BigQuery destination by @xxntti3n in #2455
- Fix: insert-from-staging replace strategy incorrectly using staging-optimized pattern by @anuunchin in #2435
- fix: index invariant add_row_hash_to_table by @hagelborn in #2491
- adds condition to check XDG_DATA_HOME to set global_dir value by @goosethedev in #2361
- removes location tag from athena iceberg, fixes catalog name, allows for additional props @rudolfix in #2478
- adds default dataset/schema to ibis expressions, support for athena and databricks backends @rudolfix in #2478
- 2457-refactors iceberg and duckdb cache support by @rudolfix in #2430
- fixes wrong resolve for WithLocalFiles configuration by @rudolfix in #2430
- converts Iceberg fileio into dlt credentials by @rudolfix in #2430
- bumps and simplifies deltalake, enables streaming appends and upserts by @rudolfix in #2430
- fixes nullability warning on duckdb ALTER by @rudolfix in #2430
- adds replace strategy selector, internal x-replace-strategy hint, removes sql_params by @rudolfix in #2430
- borrows and returns sqlalchemy connections in destination and several other critical fixes to sqllite, connection management, merge operations by @rudolfix in #2430
- executes all sql jobs (replace, copy, merge) in a transaction on all destinations by @rudolfix in #2430
- shows info on locations for config providers when displaying exceptions, hides warnings when project context is present by @rudolfix in #2430
- Fix/add s3 region for redshift staging destination related to 2349 by @yannik207 in #2389
- Fix: sqlalchemy table names of type sqlalchemy.sql.elements.quoted_name causing deserialization error by @anuunchin in
#2496 - Preserve type of emtpy lists in incremental to allow to materialize empty resources by @sh-rp in #2511
- Fix clickhouse sql syntax for sentinel table creation by @sh-rp in #2510
- cli:
dlt ai setup $IDE
add cursor rules to REST API source to your dlt project (and more) @zilto in #2503 - "model" item_format: support for lazily evaluated tables/data frames by @anuunchin in #2423
- adds arrow ipc nested type encoding to json type by @rudolfix in #2519
- [chore] imports addon function when generating destination test cases to extend implemented destinations and test cases. allows for followup job in job client tests by @rudolfix in #2519
- allows iceberg to infer unified schema, allows csv files, uses explicit select list in sql_client filesystem by @rudolfix in #2519
- allows to use context manager on dataset to keep internal connection open + lifecycle tests by @rudolfix in #2519
- uses union by name when reading parquet via duckdb to see all columns when schema evolves by @rudolfix in #2519
- makes vault provider to check known toml fragments also for non-secrets by @rudolfix in #2519
- handles non existing iceberg tables in sql client and during registration by @rudolfix in #2519
- refactors how deterministic 'temp' tables are named for destinations that do not support temp tables by @rudolfix in #2519
- fixes open cursor not closed in mssql and athena sql_client by @rudolfix in #2519
Docs
- rest_api: docs: remove redundant authentication types section by @burnash in #2461
- docs: MCP installation docs page by @zilto in #2463
- Add snowflake plus source docs by @VioletM in #2450
- Docs: tip on explicitly disablint merg_key hint by @anuunchin in #2448
- Docs: BigQuery incorrect reference by @michelzurkirchen in #2490
- Added info about additional permissions to iceberg docs by @dat-a-man in #2453
- update dlt and dlt+ installations by @Soot3 in #2444
- docs: plus: snowflake_plus: added details on
catalog_sync
by @burnash in #2494 - docs: update incremental loading section in REST API documentation by @burnash in #2474
- Docs: fix example drop pipeline commands by @michelzurkirchen in #2506
- docs: extract Iceberg docs from delta-iceberg page by @burnash in #2472
- Update iceberg destination docs by @VioletM in #2538
- docs: snowflake_plus: document
iceberg_mode
andbase_location
templating by @burnash in #2524 - MCP sse-proxy docs by @djudjuu in #2504
Verified Sources (and rules)
We support dlt ai
command via https://github.com/dlt-hub/verified-sources/tree/master/ai
- adds cursor rules for building REST API Sources dlt-hub/verified-sources#608 @adrianbr @zilto @rudolfix
If you have better cursor rules or rules for other workflows contribute them here
New Contributors
- @diwu-sf made their first contribution in #2456
- @xxntti3n made their first contribution in #2455
- @Soot3 made their first contribution in #2444
- @hagelborn made their first contribution in #2491
- @goosethedev made their first contribution in #2361
- @yannik207 made their first contribution in #2389
Full Changelog: 1.9.0...1.10.0