v0.19.0

Upgrade to v0.19

OpenDAL deprecate some features:

• serde: We will enable it by default.
• layers-retry: We will enable retry support by default.
• layers-metadata-cache: We will enable it by default.

Deprecated types like DirEntry has been removed.

---

What's Changed

• refactor: Move object to mod by @Xuanwo in #786
• feat: Implement object page stream for services like s3 by @Xuanwo in #787
• refactor: Implement azblob dir stream based on ObjectPageStream by @Xuanwo in #790
• RFC-0793: Generic KV Services by @Xuanwo in #793
• feat(services/kv): Implement Scoped Key by @Xuanwo in #796
• feat: Add scan in KeyValueAccessor by @Xuanwo in #797
• feat: Implement basic kv services support by @Xuanwo in #799
• refactor: Implement memory services by generic kv by @Xuanwo in #800
• feat: Introduce kv adapter for opendal by @Xuanwo in #802
• fix(services/redis): MATCH can't handle correctly by @Xuanwo in #803
• feat: Add integration test for redis by @Xuanwo in #804
• docs: Add docs for redis by @Xuanwo in #807
• fix: Disable ipfs redirection by @Xuanwo in #809
• fix(services/ipfs): Use ipfs files API to copy data by @Xuanwo in #811
• feat: Add OSS Service Support by @ClSlaid in #801
• feat: Add integration tests for OSS by @Xuanwo in #814
• fix(services/hdfs): Allow retrying would block by @Xuanwo in #815
• refactor: Don't expose backend to users by @Xuanwo in #816
• tests: allow running tests when env is true by @ClSlaid in #818
• refactor: Remove deprecated type aliases by @Xuanwo in #819
• test: list rich dir by @ClSlaid in #820
• Bump to version 0.19.0 by @Xuanwo in #821

Full Changelog: v0.18.2...v0.19.0

v0.18.2

What's Changed

• feat: Enable retry layer by default by @Xuanwo in #781
• ci: Enable IPFS NoFecth to avoid networking timeout by @Xuanwo in #780
• fix: Fix build error under release profile by @Xuanwo in #782
• ci: Build all feature in release to prevent build failure under release profile by @Xuanwo in #783
• Bump to version 0.18.2 by @Xuanwo in #784

Full Changelog: v0.18.1...v0.18.2

v0.18.1

What's Changed

• docs: Fix wrong link to object_store by @Xuanwo in #773
• fix(services/s3): Content MD5 not set during list by @Xuanwo in #775
• test: Add a test for ObjectEntry metadata cache by @Xuanwo in #776
• Bump to version 0.18.1 by @Xuanwo in #777

Full Changelog: v0.18.0...v0.18.1

v0.18.0

Upgrade to v0.18

OpenDAL v0.18 introduces the following breaking changes:

• Deprecated feature flag services-http has been removed.
• All DirXxx items have been renamed to ObjectXxx to make them more consistent.
  • DirEntry -> ObjectEntry
  • DirStream -> ObjectStream
  • DirStreamer -> ObjectStream
  • DirIterate -> ObjectIterate
  • DirIterator -> ObjectIterator

Besides, we also make a big change to our ObjectEntry API. Since v0.18, we can fully reuse the metadata that fetched during list. Take entry.content_length() for example:

• If content_lenght is already known, we will return directly.
• If not, we will check if the object entry is complete:
  • If complete, the entry already fetched all metadata that it could have, return directly.
  • If not, we will send a stat call to get the metadata and refresh our cache.

This change means:

• All API like content_length will be changed into async functions.
• metadata and blocking_metadata will not return errors anymore.
• To retrieve the latest meta, please use entry.into_object().metadata() instead.

---

What's Changed

• fix(http_util): Disable auto compress and enable http proxy by @Xuanwo in #731
• refactor: replace error::other with new_other_object_error by @STRRL in #738
• feat: Add Metadata Cache Layer by @Xuanwo in #739
• feat: Bump reqsign version to 0.5 by @ClSlaid in #741
• docs: Add comparison with object_store by @Xuanwo in #745
• docs: Add docs for list with prefix by @Xuanwo in #746
• feat: Derive Hash, Eq, PartialEq for Operation by @Xuanwo in #749
• feat: Make AccessorMetadata public so outer users can use by @Xuanwo in #750
• feat: Expose AccessorCapability to users by @Xuanwo in #751
• chore(compress): log with trace level instead of debug. by @youngsofun in #752
• feat: Expose opendal's http util to users by @Xuanwo in #753
• feat: Implement convert from PresignedRequest by @Xuanwo in #756
• feat: Make ObjectMetadata setter public by @Xuanwo in #758
• refactor: Rename DirXxxx to ObjectXxxx instead by @Xuanwo in #759
• docs: rename dir entry to object entry by @PsiACE in #760
• feat: Implement cached metadata for ObjectEntry by @Xuanwo in #761
• deps: Fix build after bump deps of oli and oay by @Xuanwo in #766
• feat: Assign unique name for memory backend by @Xuanwo in #769
• Bump to version 0.18.0 by @Xuanwo in #768

New Contributors

• @STRRL made their first contribution in #738
• @youngsofun made their first contribution in #752

Full Changelog: v0.17.4...v0.18.0

v0.17.4

What's Changed

• fix(http_util): Allow retry more errors by @Xuanwo in #724
• fix(services/ftp): Suffix endpoints with default port by @ClSlaid in #726
• Bump to version 0.17.4 by @Xuanwo in #727

Full Changelog: v0.17.3...v0.17.4

v0.17.3

What's Changed

• feat: Add SubdirLayer to allowing switch directory by @Xuanwo in #718
• fix: update metrics on result by @ClSlaid in #716
• fix: SubdirLayer should handle dir correctly by @Xuanwo in #720
• feat(layers/retry): Add warning log while retry happened by @Xuanwo in #721
• Bump to version 0.17.3 by @Xuanwo in #722

Full Changelog: v0.17.2...v0.17.3

v0.17.2

What's Changed

• fix(services/fs): Handle slash normalized false positives properly by @damooo in #702
• oli: Implement basic cp command by @eastfisher in #688
• chore: also parse 'FTPS' to Scheme::Ftp by @ClSlaid in #704
• chore: fix error message by @ClSlaid in #705
• fix: Tracing is too verbose by @Xuanwo in #707
• oli: Refactor copy implementation by @Xuanwo in #710
• refactor: remove enable_secure in FTP service. by @ClSlaid in #709
• Bump to version 0.17.2 by @ClSlaid in #712

Full Changelog: v0.17.1...v0.17.2

v0.17.1

What's Changed

• feat: redis service implement by @ClSlaid in #679
• feat: Implement AsyncBufRead for IntoReader by @Xuanwo in #690
• refactor: avoid unnecessary parent creating in Redis service by @ClSlaid in #692
• feat: expose security token of s3 by @ClSlaid in #693
• fix: Handle write data in async way for IPMFS by @xprazak2 in #694
• refactor: Refactor HTTP Client to split sending and incoming logic by @Xuanwo in #695
• ci: Try make ipfs test stable by @Xuanwo in #696
• Bump to version 0.17.1 by @Xuanwo in #698

Full Changelog: v0.17.0...v0.17.1

v0.17.0

Upgrade to v0.17

OpenDAL v0.17 refactor the Accessor to make space for future features.

We move path String out of the OpXxx to function args so that we don't need to clone twice.

- async fn read(&self, args: OpRead) -> Result<BytesReader>
+ async fn read(&self, path: &str, args: OpRead) -> Result<BytesReader>

For more information about this change, please refer to RFC-0661: Path In Accessor.

And since OpenDAL v0.17, we will use rustls as default tls engine for our underlying http client. Since this release, we will not depend on openssl anymore.

---

What's Changed

• RFC: Path In Accessor by @Xuanwo in #661
• feat: Implement RFC-0661: Path In Accessor by @Xuanwo in #664
• docs: Add how to implement service docs by @Xuanwo in #665
• fix: Immutable Index Layer could return duplicated pathes by @Xuanwo in #671
• feat: Hide http client internal details from users by @Xuanwo in #672
• feat: make rustls the default tls implementation by @sunng87 in #674
• fix: Remove not needed type parameter for immutable_layer by @Xuanwo in #677
• refactor: update redis support rfc by @ClSlaid in #676
• feat: Implement benches for layers by @Xuanwo in #681
• fix: Don't trace buf field in poll_read by @Xuanwo in #682
• docs: update metrics documentation by @ClSlaid in #684
• fix: List non-exist dir should return empty by @Xuanwo in #683
• fix: Add path validation for fs backend by @Xuanwo in #685
• Bump to version 0.17 by @Xuanwo in #686

Full Changelog: v0.16.0...v0.17.0

v0.16.0

Upgrade to v0.16

OpenDAL v0.16 refactor the internal implementation of http service. Since v0.16, http service can be used directly without enabling services-http feature. Accompany by these changes, http service has the following breaking changes:

• services-http feature has been deprecated. Enabling services-http is a no-op now.
• http service is read only services and can't be used to list or write.

OpenDAL introduces a new layer ImmutableIndexLayer that can add list capability for services:

use opendal::layers::ImmutableIndexLayer;
use opendal::Operator;
use opendal::Scheme;

async fn main() {
    let mut iil = ImmutableIndexLayer::default();

    for i in ["file", "dir/", "dir/file", "dir_without_prefix/file"] {
        iil.insert(i.to_string())
    }

    let op = Operator::from_env(Scheme::Http)?.layer(iil);
}

For more information about this change, please refer to RFC-0627: Split Capabilities.

---

What's Changed

• Bump to version 0.15 by @Xuanwo in #630
• feat: Implement tests for read-only services by @Xuanwo in #634
• refactor: Migrate all behavior tests to capability based by @Xuanwo in #635
• feat(services/ftp): Implemented multi connection by @ArberSephirotheca in #637
• refactor: Remove list support from http service by @Xuanwo in #639
• RFC-0623: Redis Service by @ClSlaid in #623
• refactor: Replace isahc with reqwest and ureq by @Xuanwo in #642
• deps: Bump reqsign to v0.4 by @Xuanwo in #643
• Finalize FTP read operation by @ArberSephirotheca in #644
• feat: Implement service for IPFS HTTP Gateway by @Xuanwo in #645
• docs: Add docs for ipfs by @Xuanwo in #649
• docs: Fix typo by @Xuanwo in #650
• feat: Add ImmutableIndexLayer by @Xuanwo in #651
• feat: derive Hash for Scheme by @damooo in #653
• feat(services/ftp): Setup integration tests by @Xuanwo in #648
• docs: Add docs for ftp services by @Xuanwo in #655
• chore(deps): Update criterion requirement from 0.3 to 0.4 by @dependabot in #656
• chore(deps): Update quick-xml requirement from 0.24 to 0.25 by @dependabot in #657
• deps: Remove not used features by @Xuanwo in #658
• Bump to version 0.16 by @Xuanwo in #659

New Contributors

• @damooo made their first contribution in #653

Full Changelog: v0.15.0...v0.16.0