Skip to content

Releases: apache/opendal

v0.38.0

28 Jun 05:00
v0.38.0
960a8de
Compare
Choose a tag to compare

Upgrade to v0.38

There are no public API changes.

Raw API

OpenDAL add the Write::sink API to enable streaming writing. This is a breaking change for users who depend on the raw API.

For a quick fix, users who have implemented opendal::raw::oio::Write can return an Unsupported error for Write::sink().

More detailes could be found at RFC: Writer sink API.

What's Changed

Added

  • feat(raw/http_util): Implement mixed multipart parser by @Xuanwo in #2430
  • feat(services/gcs): Add batch delete support by @wcy-fdu in #2142
  • feat(core): Add Write::sink API by @Xuanwo in #2440
  • feat(services/s3): Allow retry for unexpected 499 error by @Xuanwo in #2453
  • feat(layer): add throttle layer by @morristai in #2444
  • feat(bindings/haskell): init haskell binding by @silver-ymz in #2463
  • feat(core): add capability check by @unixzii in #2461
  • feat(bindings/haskell): add CONTRIBUTING.md by @silver-ymz in #2466
  • feat(bindings/haskell): add CI test for haskell binding by @silver-ymz in #2468
  • feat(binding/lua): introduce opendal lua binding by @oowl in #2469
  • feat(bindings/swift): add Swift binding by @unixzii in #2470
  • feat(bindings/haskell): support is_exist create_dir copy rename delete by @silver-ymz in #2475
  • feat(bindings/haskell): add Monad wrapper by @silver-ymz in #2482
  • feat(bindings/dotnet): basic structure by @tisonkun in #2485
  • feat(services/dropbox): Support create/read/delete for Dropbox by @Zheaoli in #2264
  • feat(bindings/java): support load system lib by @tisonkun in #2502
  • feat(blocking operator): add remove_all api by @infdahai in #2449
  • feat(core): adopt WebHDFS LISTSTATUS_BATCH for better performance by @morristai in #2499
  • feat(bindings/haskell): support stat by @silver-ymz in #2504
  • feat(adapters-kv): add rename and copy support to kv adapters by @oowl in #2513
  • feat: Implement sink for services s3 by @Xuanwo in #2508
  • feat(adapters-kv): add rename and copy support to non typed kv adapters by @oowl in #2515
  • feat: Implement test harness via libtest-mimic instead by @Xuanwo in #2517
  • feat(service/sled): introduce tree support by @oowl in #2516
  • feat(bindings/haskell): support list and scan by @silver-ymz in #2527
  • feat(services/redb): support redb service by @oowl in #2526
  • feat(core): implement service for Mini Moka by @morristai in #2537
  • feat(core): add Mini Moka GitHub Action workflow job by @morristai in #2539
  • feat(services): add cacache backend by @PsiACE in #2548
  • feat: Implement Writer::copy so user can copy from AsyncRead by @Xuanwo in #2552

Changed

  • refactor(bindings/C): refactor c bindings to call all APIs using pointer by @Ji-Xinyou in #2489

Fixed

  • fix(services/azblob): Fix azblob batch max operations by @A-Stupid-Sun in #2434
  • fix(services/sftp): change default root config to remote server setting by @silver-ymz in #2431
  • fix: Enable std feature for futures to allow futures::AsyncRead by @Xuanwo in #2450
  • fix(services/gcs): GCS should support create dir by @Xuanwo in #2467
  • fix(bindings/C): use copy_from_slice instead of from_static in opendal_bytes by @Ji-Xinyou in #2473
  • fix(bindings/swift): reorg the package to correct its name by @unixzii in #2479
  • fix: Fix the build for zig binding by @Xuanwo in #2493
  • fix(service/webhdfs): fix webhdfs config builder for disable_list_batch by @morristai in #2509
  • fix(core/types): add missing vercel artifacts for FromStr by @cijiugechu in #2519
  • fix(types/operator): fix operation limit error default size by @oowl in #2536

Docs

CI

  • ci: Fix append test should use copy_buf to avoid call times by @Xuanwo in #2436
  • build(bindings/ruby): fix compile rb-sys on Apple M1 by @tisonkun in #2451
  • ci: Use summary for zig test to fix build by @Xuanwo in #2480
  • ci(workflow): add lua binding test workflow by @oowl in #2478
  • build(deps): bump actions/setup-python from 3 to 4 by @dependabot in #2481
  • ci(bindings/swift): add CI for Swift binding by @unixzii in #2492
  • ci: Try to make webhdfs tests more stable by @Xuanwo in #2503
  • ci(bindings/java): auto release snapshot by @tisonkun in #2521
  • ci: Disable the stage snapshot CI by @Xuanwo in #2528
  • ci: fix opendal-java snapshot releases by @tisonkun in #2532
  • ci: Fix typo in binding java CI by @Xuanwo in #2534
  • ci(bindings/swift): optimize time consumption of CI pipeline by @unixzii in #2545
  • ci: Fix PR label not updated while edited by @Xuanwo in #2547
  • ci: automatic java binding release by @tisonkun in #2557

Chore

Read more

v0.37.0

06 Jun 09:19
v0.37.0
7a45800
Compare
Choose a tag to compare

Upgrade to v0.37

In v0.37.0, OpenDAL bump the version of reqsign to v0.13.0.

There are no public API and raw API changes.


What's Changed

Added

  • feat(services/webdav): support redirection when get 302/307 response during read operation by @Yansongsongsong in #2256
  • feat: Add Zig Bindings Module by @kassane in #2374
  • feat: Implement Timeout Layer by @Xuanwo in #2395
  • feat(bindings/c): add opendal_operator_blocking_delete method by @jiaoew1991 in #2416
  • feat(services/obs): add append support by @infdahai in #2422

Changed

  • refactor(bindings/zig): enable tests and more by @tisonkun in #2375
  • refactor(bindings/zig): add errors handler and module test by @kassane in #2381
  • refactor(http_util): Adopt reqwest's redirect support by @Xuanwo in #2390

Fixed

  • fix(bindings/zig): reflect C interface changes by @tisonkun in #2378
  • fix(services/azblob): Fix batch delete doesn't work on azure by @Xuanwo in #2382
  • fix(services/oss): Fix oss batch max operations by @A-Stupid-Sun in #2414
  • fix(core): Don't wake up operator futures while not ready by @Xuanwo in #2415
  • fix(services/s3): Fix s3 batch max operations by @A-Stupid-Sun in #2418

Docs

CI

Chore

  • chore(bindings/python): upgrade maturin to 1.0 by @messense in #2369
  • chore: Fix license headers for release/labler by @Xuanwo in #2371
  • chore(bindings/C): add one simple read/write example into readme and code by @Ji-Xinyou in #2421

New Contributors

Full Changelog: v0.36.0...v0.37.0

v0.36.0

30 May 08:14
v0.36.0
1eecbb7
Compare
Choose a tag to compare

Upgrade to v0.36

Public API

In v0.36, OpenDAL improving the xxx_with API by allow it to be called in chain:

After this change, all xxx_with alike call will be changed from

let bs = op.read_with(
  "path/to/file",
  OpRead::new()
    .with_range(0..=1024)
    .with_if_match("<etag>")
    .with_if_none_match("<etag>")
    .with_override_cache_control("<cache_control>")
    .with_override_content_disposition("<content_disposition>")
  ).await?;

to

let bs = op.read_with("path/to/file")
  .range(0..=1024)
  .if_match("<etag>")
  .if_none_match("<etag>")
  .override_cache_control("<cache_control>")
  .override_content_disposition("<content_disposition>")
  .await?;

For blocking API calls, we will need a call() at the end:

let bs = bop.read_with("path/to/file")
  .range(0..=1024)
  .if_match("<etag>")
  .if_none_match("<etag>")
  .override_cache_control("<cache_control>")
  .override_content_disposition("<content_disposition>")
  .call()?;

Along with this change, users don't need to call OpXxx anymore so we moved it to raw API.

More detailes could be found at [RFC: Chain Based Operator API][https://opendal.apache.org/docs/rust/opendal/docs/rfcs/rfc_2299_chain_based_operator_api/index.html].

Raw API

Migrated opendal::ops to opendal::raw::ops.


v0.36.0 - 2023-05-30

Added

  • feat(service/fs): add append support for fs (#2296)
  • feat(services/sftp): add append support for sftp (#2297)
  • RFC-2299: Chain based Operator API (#2299)
  • feat(services/azblob): add append support (#2302)
  • feat(bindings/nodejs): add append support (#2322)
  • feat(bindings/C): opendal_operator_ptr construction using kvs (#2329)
  • feat(services/cos): append support (#2332)
  • feat(bindings/java): implement Operator#delete (#2345)
  • feat(bindings/java): support append (#2350)
  • feat(bindings/java): save one jni call in the hot path (#2353)
  • feat: server side encryption support for azblob (#2347)

Changed

  • refactor(core): Implement RFC-2299 for stat_with (#2303)
  • refactor(core): Implement RFC-2299 for BlockingOperator::write_with (#2305)
  • refactor(core): Implement RFC-2299 for appender_with (#2307)
  • refactor(core): Implement RFC-2299 for read_with (#2308)
  • refactor(core): Implement RFC-2299 for read_with (#2308)
  • refactor(core): Implement RFC-2299 for append_with (#2312)
  • refactor(core): Implement RFC-2299 for write_with (#2315)
  • refactor(core): Implement RFC-2299 for reader_with (#2316)
  • refactor(core): Implement RFC-2299 for writer_with (#2317)
  • refactor(core): Implement RFC-2299 for presign_read_with (#2314)
  • refactor(core): Implement RFC-2299 for presign_write_with (#2320)
  • refactor(core): Implement RFC-2299 for list_with (#2323)
  • refactor: Move ops to raw::ops (#2325)
  • refactor(bindings/C): align bdd test with the feature tests (#2340)
  • refactor(bindings/java): narrow unsafe boundary (#2351)

Fixed

  • fix(services/supabase): correctly set retryable (#2295)
  • fix(core): appender complete check (#2298)

Docs

  • docs: add service doc for azdfs (#2310)
  • docs(bidnings/java): how to deploy snapshots (#2311)
  • docs(bidnings/java): how to deploy snapshots (#2311)
  • docs: Fixed links of languages to open in same tab (#2327)
  • docs: Adopt docusaurus pathname protocol (#2330)
  • docs(bindings/nodejs): update lib desc (#2331)
  • docs(bindings/java): update the README file (#2338)
  • docs: add service doc for fs (#2337)
  • docs: add service doc for cos (#2341)
  • docs: add service doc for dashmap (#2342)
  • docs(bindings/java): for BlockingOperator (#2344)

CI

  • build(bindings/java): prepare for snapshot release (#2301)
  • build(bindings/java): support multiple platform java bindings (#2324)
  • ci(binding/nodejs): Use docker to build nodejs binding (#2328)
  • build(bindings/java): prepare for automatically multiple platform deploy (#2335)
  • ci: add bindings java docs and integrate with website (#2346)
  • ci: avoid copy gitignore to site folder (#2348)
  • ci(bindings/c): Add diff check (#2359)
  • ci: Cache librocksdb to speed up CI (#2360)
  • ci: Don't load rocksdb for all workflows (#2362)
  • ci: Fix Node.js 12 actions deprecated warning (#2363)
  • ci: Speed up python docs build (#2364)
  • ci: Adopt setup-node's cache logic instead (#2365)

Chore

  • chore(test): Avoid test names becoming prefixes of other tests (#2333)
  • chore(bindings/java): improve OpenDALException tests and docs (#2343)
  • chore(bindings/java): post release 0.1.0 (#2352)
  • chore(docs): split docs build into small jobs (#2356)'
  • chore: protect branch gh-pages (#2358)

New Contributors

Full Changelog: v0.35.0...v0.36.0

v0.35.0

23 May 08:11
v0.35.0
40656bb
Compare
Choose a tag to compare

NOTE: This release is not yet an official ASF release.

Upgrade to v0.35

Public API

  • OpenDAL removes rarely used Operator::from_env and Operator::from_iter APIs
    • Users can use Operator::via_map instead.

Raw API

  • OpenDAL adds append support with could break existing layers. Please make sure append requests have been forward correctly.
  • After the merging of scan and list, OpenDAL removes the scan from raw API. Please use list_without_delimiter instead.

v0.35.0 - 2023-05-23

Added

  • feat(services/onedrive): Implement list, create_dir, stat and upload
    ing large files (#2231)
  • feat(bindings/C): Initially support stat in C binding (#2249)
  • feat(bindings/python): Enable abi3 to avoid building on different python
    version (#2255)
  • feat(bindings/C): support BDD tests using GTest (#2254)
  • feat(services/sftp): setup integration tests (#2192)
  • feat(core): Add trait and public API for append (#2260)
  • feat(services/sftp): support copy and rename for sftp (#2263)
  • feat(services/sftp): support copy and read_seek (#2267)
  • feat: Add COS service support (#2269)
  • feat(services/cos): Add support for loading from env (#2271)
  • feat(core): add presign support for obs (#2253)
  • feat(services/sftp): setup integration tests (#2192)
  • feat(core): add presign support for obs (#2253)
  • feat(core): public API of append (#2284)
  • test(core): test for append (#2286)
  • feat(services/oss): add append support (#2279)
  • feat(bindings/java): implement async ops to pass AsyncStepsTest (#2291)

Changed

  • services/gdrive: port code to GdriveCore & add path_2_id cache (#2203)
  • refactor: Minimize futures dependencies (#2248)
  • refactor: Add Operator::via_map to support init without generic type parameters (#2280)
  • refactor(binding/java): build, async and docs (#2276)

Fixed

  • fix: Fix bugs that failed wasabi's integration tests (#2273)

Removed

  • feat(core): remove scan from raw API (#2262)

Docs

  • chore(s3): update builder region doc (#2247)
  • docs: Add services in readme (#2251)
  • docs: Unify capabilities list for kv services (#2257)
  • docs(nodejs): fix some example code errors (#2277)
  • docs(bindings/C): C binding contributing documentation (#2266)
  • docs: Add new docs that available for all languages (#2285)
  • docs: Remove unlicensed svg (#2289)
  • fix(website): double active route (#2290)

CI

  • ci: Enable test for cos (#2270)
  • ci: Add integration tests for supabase (#2272)
  • ci: replace set-output for docs (#2275)
  • ci: Fix unit tests (#2282)
  • ci: Cleanup NOTICE file (#2281)
  • ci: Fix release not contains incubating (#2292)

Chore

  • chore(core): remove unnecessary path prefix (#2265)

New Contributors

Full Changelog: v0.34.0...v0.35.0

v0.34.0

09 May 08:41
v0.34.0
91611ca
Compare
Choose a tag to compare

NOTE: This release is not yet an official ASF release. We are waiting for the vote.

v0.34.0 - 2023-05-09

Added

  • feat(writer): configurable buffer size of unsized write (#2143)
  • feat(oay): Add basic s3 list_objects_v2 with start_after support (#2219)
  • feat: Add typed kv adapter and migrate moka to it (#2222)
  • feat: migrate service dashmap (#2225)
  • feat(services/memory): migrate service memory (#2229)
  • feat: Add assert for public types to ensure Send + Sync (#2237)
  • feat(services/gcs): Add abort support for writer (#2242)

Changed

  • refactor: Replace futures::ready with std::task::ready (#2221)
  • refactor: Use list without delimiter to replace scan (#2243)

Fixed

  • fix(services/gcs): checked_rem_euclid could return Some(0) (#2220)
  • fix(tests): Etag must be wrapped by " (#2226)
  • fix(services/s3): Return error if credential load fail instead skip (#2233)
  • fix(services/s3): Return error if region not set for AWS S3 (#2234)
  • fix(services/gcs): rsa 0.9 breaks gcs presign (#2236)

Chore

  • chore: change log subscriber from env_logger to tracing-subscriber (#2238)
  • chore: Fix build of wasabi (#2241)

New Contributors

Full Changelog: v0.33.3...v0.34.0

v0.33.3

09 May 07:32
v0.33.3
402aae7
Compare
Choose a tag to compare

NOTE: This release is not yet an official ASF release. We are waiting for the vote.

v0.33.3 - 2023-05-06

Added

  • feat(services/onedrive): Add read and write support for OneDrive (#2129)
  • test(core): test for read_with_override_cache_control (#2155)
  • feat(http_util): Implement multipart/form-data support (#2157)
  • feat(http_util): Implement multipart/mixed support (#2161)
  • RFC-2133: Introduce Append API (#2133)
  • feat(services/sftp): Add read/write/stat support for sftp (#2186)
  • feat(services/gdrive): Add read & write & delete support for GoogleDrive (#2184)
  • feat(services/vercel): Add vercel remote cache support (#2193)
  • feat(tests): Enable supabase integration tests (#2190)
  • feat(core): merge scan and list (#2214)

Changed

  • refactor(java): refactor java code for java binding (#2145)
  • refactor(layers/logging): parsing level str (#2160)
  • refactor: Move not initiated logic to utils instead (#2196)
  • refactor(services/memcached): Rewrite memecached connection entirely (#2204)

Fixed

  • fix(service/s3): set retryable on batch (#2171)
  • fix(services/supabase): Supabase ci fix (#2200)

Docs

  • docs(website): try to add opendal logo (#2159)
  • doc: update vision to be more clear (#2164)
  • docs: Refactor Contributing and add Developing (#2169)
  • docs: Merge DEVELOPING into CONTRIBUTING (#2172)
  • docs: fix some grammar errors in the doc of Operator (#2173)
  • docs(nodejs): Add CONTRIBUTING docs (#2174)
  • docs: Add CONTRIBUTING for python (#2188)

CI

  • ci: Use microsoft rust devcontainer instead (#2165)
  • ci(devcontainer): Install development deps (#2167)
  • chore: set workspace default members (#2168)
  • ci: Setup vercel artifacts integration tests (#2197)
  • ci: Remove not used odev tools (#2202)
  • ci: Add tools to generate NOTICE and all deps licenses (#2205)
  • ci: use Temurin JDK 11 to build the bindings-java (#2213)

Chore

  • chore(deps): bump clap from 4.1.11 to 4.2.5 (#2183)
  • chore(deps): bump futures from 0.3.27 to 0.3.28 (#2181)
  • chore(deps): bump assert_cmd from 2.0.10 to 2.0.11 (#2180)
  • chore: Refactor behavior test (#2189)
  • chore: update readme for more information that is more friendly to newcomers (#2217)

New Contributors

Full Changelog: v0.33.2...v0.33.3

v0.33.2

27 Apr 13:48
v0.33.2
e871a6f
Compare
Choose a tag to compare

NOTE: This release is not yet an official ASF release. We are still in the process of learning how to create a formal one.

v0.33.2 - 2023-04-27

Added

  • feat(core): add test for stat_with_if_none_match (#2122)
  • feat(services/gcs): Add start-after support for list (#2107)
  • feat(services/azblob): Add supporting presign (#2120)
  • feat(services/gcs): Add supporting presign support (#2126)
  • feat(java): connect rust async/await with java future (#2112)
  • docs: add hdfs classpath related troubleshoot (#2130)
  • fix(clippy): suppress dead_code check (#2135)
  • feat(core): Add cache-control to Metadata (#2136)
  • fix(services/gcs): Remove HOST header to avoid gcs RESET connection (#2139)
  • test(core): test for write_with_cache_control (#2131)
  • test(core): test for write_with_content_type (#2140)
  • test(core): test for read_with_if_none_match (#2141)
  • feat(services/supabase): Add read/write/stat support for supabase (#2119)

Docs

  • docs: add hdfs classpath related troubleshoot (#2130)

CI

  • ci: Mark job as skipped if owner is not apache (#2128)
  • ci: Enable native-tls to test gcs presign for workaround (#2138)

New Contributors

Full Changelog: v0.33.1...v0.33.2

v0.33.1

25 Apr 15:53
v0.33.1
8e12fed
Compare
Choose a tag to compare

NOTE: This release is not yet an official ASF release. We are still in the process of learning how to create a formal one.

v0.33.1 - 2023-04-25

Added

  • feat: Add behavior test for read_with_if_match & stat_with_if_match (#2088)
  • feat(tests): Add fuzz test for writer without content length (#2100)
  • feat: add if_none_match support for obs (#2103)
  • feat(services/oss): Add server side encryption support for oss (#2092)
  • feat(core): update errorKind PreconditionFailed to ConditionNotMatch (#2104)
  • feat(services/s3): Add start-after support for list (#2096)
  • feat: gcs support cache control (#2116)

Fixed

  • fix(services/gcs): set content length=0 for gcs initiate_resumable_upload (#2110)
  • fix(bindings/nodejs): Fix index.d.ts not updated (#2117)

Docs

  • chore: improve LoggingLayer docs and pub use log::Level (#2089)
  • docs(refactor): Add more detailed description of operator, accessor, and builder (#2094)

CI

  • chore(bindings/nodejs): update package.json repository info (#2078)
  • ci: Bring hdfs test back (#2114)

New Contributors

Full Changelog: v0.33.0...v0.33.1

v0.33.0

23 Apr 05:49
v0.33.0
fdc82f9
Compare
Choose a tag to compare

NOTE: This release is not yet an official ASF release. We are still in the process of learning how to create a formal one.

Upgrade to v0.33

Public API

OpenDAL 0.33 has redesigned the Writer API, replacing all instances of writer.append() with writer.write(). For more information, please refer to Writer.

Raw API

In addition to the redesign of the Writer API, we have removed append from oio::Write. Therefore, users who implement services and layers should also remove it.

After v0.33 landing, services should handle OpWrite::content_length correctly by following these guidelines:

  • If the writer does not support uploading unsized data, return a response of NotSupported if content length is None.
  • Otherwise, continue writing data until either close or abort has been called.

Furthermore, OpenDAL 0.33 introduces a new concept called Capability which replaces AccessorCapability. Services must adapt to this change.


v0.33.0 - 2023-04-23

Added

  • feat: Add OpenTelemetry Trace Layer (#2001)
  • feat: add if_none_match support for azblob (#2035)
  • feat: add if_none_match/if_match for gcs (#2039)
  • feat: Add size check for sized writer (#2038)
  • feat(services/azblob): Add if-match support (#2037)
  • feat(core): add copy&rename to error_context layer (#2040)
  • feat: add if-match support for OSS (#2034)
  • feat: Bootstrap new (old) project oay (#2041)
  • feat(services/OSS): Add override_content_disposition support (#2043)
  • feat: add IF_MATCH for http (#2044)
  • feat: add IF_MATCH for http HEAD request (#2047)
  • feat: add cache control header for azblob and obs (#2049)
  • feat: Add capability for operation's variant and args (#2057)
  • feat(azblob): Add override_content_disposition support (#2065)
  • feat(core): test for read_with_override_content_composition (#2067)
  • feat(core): Add start-after support for list (#2071)

Changed

  • refactor: Polish Writer API by merging append and write together (#2036)
  • refactor(raw/http_util): Add url in error context (#2066)
  • refactor: Allow reusing the same operator to speed up tests (#2068)

Fixed

  • fix(bindings/ruby): use rb_sys_env to help find ruby for building (#2051)
  • fix: MadsimLayer should be able to built without cfg (#2059)
  • fix(services/s3): Ignore prefix if it's empty (#2064)

Docs

  • docs(bindings/python): ipynb examples for users (#2061)

CI

  • ci(bindings/nodejs): publish support --provenance (#2046)
  • ci: upgrade typos to 1.14.8 (#2055)
  • chore(bindings/C): ignore the formatting of auto-generated opendal.h (#2056)

New Contributors

Full Changelog: v0.32.0...v0.33.0

v0.32.0

18 Apr 13:35
v0.32.0
7c129e3
Compare
Choose a tag to compare

NOTE: This release is not yet an official ASF release. We are still in the process of learning how to create a formal one.

Upgrade to v0.32

OpenDAL 0.32 doesn't have much breaking changes.

We changed Accessor::create into Accessor::create_dir. Only users who implement Layer need to change.

v0.32.0 - 2023-04-18

Added

  • feat: Add wasabi service implementation (#2004)
  • feat: improve the readability of oli command line error output (#2016)
  • feat: add If-Match Support for OpRead, OpWrite, OpStat (#2017)
  • feat: add behavioral test for Write::abort (#2018)
  • feat: add if-match support for obs (#2023)
  • feat: Add missing functions for trace layers (#2025)
  • feat(layer): add madsim layer (#2006)
  • feat(gcs): add support for gcs append (#1801)

Changed

  • refactor: Rename Create to CreateDir for its behavior changed (#2019)

Fixed

  • fix: Cargo lock not updated (#2027)
  • fix(services/s3): Ignore empty query to make it more compatible (#2028)
  • fix(services/oss): Fix env not loaded for oss signer (#2029)

Docs

  • docs: fix some typos (#2022)
  • docs: add dev dependency section (#2021)

New Contributors

Full Changelog: v0.31.1...v0.32.0