v0.4.0

What's Changed

• refactor: Make read return BytesStream instead by @Xuanwo in #192
• feat(services/azblob): Implement list support by @D2Lark in #193
• refactor: Make write return BytesSink instead by @Xuanwo in #194
• feat: Implement io_util like into_sink and into_stream by @Xuanwo in #197
• feat(io_util): Implement observer for sink and stream by @Xuanwo in #198
• RFC-0191: Async Streaming IO by @Xuanwo in #191
• docs: Add docs for all newly added public functions by @Xuanwo in #199
• refactor: New public API design by @Xuanwo in #201
• deps: Bump to reqsign to fix s3 url encode issue by @Xuanwo in #202
• RFC-0203: Remove Credential by @Xuanwo in #203
• refactor: Adopt io::Result instead by @Xuanwo in #204
• refactor: Rollback changes around async streaming io by @Xuanwo in #205
• docs: Add docs for public types by @Xuanwo in #206
• refactor: Refactor behavior tests with macro_rules by @Xuanwo in #207
• Bump to version 0.4.0 by @Xuanwo in #208

Full Changelog: v0.3.0...v0.4.0

---

Refer to Upgrade From v0.3 to v0.4 section for more upgrade details.

From v0.3 to v0.4

OpenDAL introduces many breaking changes in v0.4.

Object::reader() is not AsyncSeek anymore

Since v0.4, Object::reader() will return impl BytesRead instead of Reader that implements AsyncRead and AsyncSeek. Users who want AsyncSeek please wrapped with opendal::io_util::seekable_read:

use opendal::io_util::seekable_read;

let o = op.object("test");
let mut r = seekable_read(&o, 10..);
r.seek(SeekFrom::Current(10)).await?;
let mut bs = vec![0;10];
r.read(&mut bs).await?;

Use RangeBounds instead

Since v0.4, the following APIs will be removed.

• Object::limited_reader(size: u64)
• Object::offset_reader(offset: u64)
• Object::range_reader(offset: u64, size: u64)

Instead, OpenDAL is providing a more general range_reader powered by RangeBounds:

pub async fn range_reader(&self, range: impl RangeBounds<u64>) -> Result<impl BytesRead>

Users can use their familiar rust range syntax:

let r = o.range_reader(1024..2048).await?;

Return io::Result instead

Since v0.4, all functions in OpenDAL will return std::io::Result instead.

Please check via std::io::ErrorKind directly:

use std::io::ErrorKind;

if let Err(e) = op.object("test_file").metadata().await {
    if e.kind() == ErrorKind::NotFound {
        println!("object not exist")
    }
}

Removing Credential

Since v0.4, Credential has been removed, please use the API provided by Builder directly.

builder.access_key_id("access_key_id");
builder.secret_access_key("secret_access_key");

Write returns BytesWriter instead

Since v0.4, Accessor::write will return a BytesWriter instead accepting a BoxedAsyncReader.

Along with this change, the old Writer has been replaced by a new set of write functions:

pub async fn write(&self, bs: impl AsRef<[u8]>) -> Result<()> {}
pub async fn writer(&self, size: u64) -> Result<impl BytesWrite> {}

Users can write into an object more easily:

let _ = op.object("path/to/file").write("Hello, World!").await?;

io_util replaces readers

Since v0.4, mod io_util will replace readers. In io_utils, OpenDAL provides helpful functions like:

• into_reader: Convert BytesStream into BytesRead
• into_sink: Convert BytesWrite into BytesSink
• into_stream: Convert BytesRead into BytesStream
• into_writer: Convert BytesSink into BytesWrite
• observe_read: Add callback for BytesReader
• observe_write: Add callback for BytesWrite

New type alias

For better naming, types that OpenDAL returns have been renamed:

• AsyncRead + Unpin + Send => BytesRead
• BoxedAsyncReader => BytesReader
• AsyncWrite + Unpin + Send => BytesWrite
• BoxedAsyncWriter => BytesWriter
• ObjectStream => ObjectStreamer

v0.3.0

What's Changed

• add azure blob support by @D2Lark in #165
• feat: Add tracing support via minitrace by @D2Lark in #175
• fix(service/s3): Endpoint template should be applied if region exists by @Xuanwo in #180
• chore: Level down some log entry to debug by @Xuanwo in #181
• feat(service/s3): Implement server side encryption support by @Xuanwo in #182
• Bump to version 0.3.0 by @Xuanwo in #186

New Contributors

• @D2Lark made their first contribution in #165

Full Changelog: v0.2.5...v0.3.0

v0.2.5

v0.2.5 - 2022-03-22

Added

• feat: Adopt quick_xml to parse xml (#164)
• test: Add behavior test for not exist object (#166)
• feat: Allow user input region (#168)

Changed

• feat: Improve error handle for s3 service (#169)
• feat: Read error response for better debugging (#170)
• examples: Improve examples for s3 (#171)

v0.2.4

v0.2.4 - 2022-03-18

Added

• feat: Add content_md5 and last_modified in metadata (#158)

Changed

• refactor: Say goodbye to aws-s3-sdk (#152)

v0.2.3

v0.2.3 - 2022-03-14

Added

• feat: Export BoxedObjectStream so that users can implement Layer (#147)

v0.2.2

v0.2.2 - 2022-03-14

Fixed

• services/fs: Refactor via tokio::fs (#142)
• fix: Stat root should return a dir object (#143)

v0.2.1

v0.2.1 - 2022-03-10

Added

• *: Implement logging support (#122)
• feat(service): Add service memory read support (#121)
• services: Add basic metrics (#127)
• services: Add full memory support (#134)

Changed

• benches: Refactor to support more read pattern (#126)
• services: Refactor into directories (#131)

Docs

• docs: Cover all public types and functions (#128)
• docs: Update README (#129)
• ci: Generate main docs to https://opendal.databend.rs (#132)
• docs: Enrich README (#133)
• Add examples for object (#135)

v0.2.0

v0.2.0 - 2022-03-08

Added

• RFC-112: Path Normalization (#112)
• examples: Add more examples for services and operations (#113)

Changed

• benches: Refactor to make code more readable (#104)
• object: Refactor ObjectMode into enum (#114)

v0.1.4

v0.1.4 - 2022-03-04

Added

• services/s3: Implement anonymous read support (#97)
• bench: Add parallel_read bench (#100)
• services/s3: Add test for anonymous support (#99)

v0.1.3

v0.1.3 - 2022-03-02

Added

• RFC and implementations for limited reader (#90)
• readers: Implement observe reader support (#92)

Changed

• deps: Bump s3 sdk to 0.8 (#87)
• bench: Improve logic (#89)

New RFCs

• limited_reader

New Contributors

• @sundy-li made their first contribution in #89