Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Streaming API #4

Merged
merged 47 commits into from
Dec 24, 2022
Merged

feat: Streaming API #4

merged 47 commits into from
Dec 24, 2022

Conversation

Crauzer
Copy link
Owner

@Crauzer Crauzer commented Dec 24, 2022

No description provided.

@Crauzer Crauzer added streaming Related to the Streaming API immediate Related to the Immediate API labels Dec 24, 2022
@Crauzer Crauzer self-assigned this Dec 24, 2022
@Crauzer Crauzer linked an issue Dec 24, 2022 that may be closed by this pull request
@Crauzer Crauzer changed the title feat: Streaming API [WIP] feat: Streaming API Dec 24, 2022
@Crauzer Crauzer changed the title [WIP] feat: Streaming API feat: Streaming API Dec 24, 2022
@Crauzer Crauzer merged commit 238e08e into main Dec 24, 2022
@Crauzer Crauzer deleted the feat/stream-api branch December 24, 2022 23:39
@Hassmann
Copy link

Hassmann commented Dec 26, 2022

Hello Crauzer,

and thanks a lot for the implementation!
I have one request, if you don't mind :)

Could you please make Update(...) and Digest64() public?

The reason is: I would like to use your work in a pipeline of streams. With a hashing stream acting as a filter or facade.
Your current implementation allows for an input stream, I need to initialize with output streams, though.

My implementation of such a Stream goes like this:

    public class HashingStream : Stream
    {
        readonly XXHash3 _hash = XXHash3.Create();

        private readonly Stream _destination;

        public HashingStream(Stream destination)
        {
            _destination = destination;
        }

        public Hash Hash { get; private set; }

        public override void Write(byte[] buffer, int offset, int count)
        {
            _destination.Write(buffer, offset, count);

            _hash.Update(new ReadOnlySpan<byte>(buffer, offset, count));
        }

        protected override void Dispose(bool disposing)
        {
            Hash = new Hash(_hash.Digest64());

            base.Dispose(disposing);
        }

And that's why I would like to have public access to these methods.

You have also ported the ZStd compression, which I use, too.
The pattern is something like:

Stream input = File.OpenRead(...);

var output = File.OpenWrite(...);
var compressing = new CompressingStream(output);
var hashing = new HashingStream(compressing);

using (hashing)
{
    input.CopyTo(hashing);
}

var hash = hashing.Hash;

If you would see such a stream in your repo, I'd be happy to propose a PR.

Cheerio,
Andreas

@Crauzer
Copy link
Owner Author

Crauzer commented Dec 27, 2022

This is a good request!

I initially had the methods as public but got led astray while looking at how the .NET standard library implements hashing interfaces and forgot that this is a valid use-case.

I will look into exposing those methods as public this or next week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
immediate Related to the Immediate API streaming Related to the Streaming API
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a streaming interface, please
2 participants