Add BufRead::consume

[yoshuawuyts]

Ref #131. This implements BufReader::consume. Thanks!

Note on fill_buf: I couldn't get the async fn fill_buf() to work, but tracked progress for it here: https://gist.github.com/yoshuawuyts/09bbdc7823225ca96b5e35cd1da5d581. Got some lifetimes isssues. If anyone wants to give this a shot, please do!

[yoshuawuyts]

I can't use poll_fill_buf manually at all, anywhere so I did some digging, and it seems we may need to do an unchecked projection here. From io::copy

    impl<R, W: Unpin + ?Sized> CopyFuture<'_, R, W> {
        fn project(self: Pin<&mut Self>) -> (Pin<&mut R>, Pin<&mut W>, &mut u64) {
            unsafe {
                let this = self.get_unchecked_mut();
                (
                    Pin::new_unchecked(&mut this.reader),
                    Pin::new(&mut *this.writer),
                    &mut this.amt,
                )
            }
        }
    }

///

                let buffer = futures_core::ready!(reader.as_mut().poll_fill_buf(cx))?;

[yoshuawuyts]

Nope, that doesn't work either. If someone who's good at lifetimes could take a shot at this that'd be fantastic! I'm currently stuck with not being able to use any of the BufReader variants (from both async-std and futures-rs) because I can't call fill_buf.

[yoshuawuyts]

Conversation with @tirr-c on whether to remove the Pin<&mut self> bounds for consume:

12:55 PM
]
Tirr
:
@yoshuawuyts the other methods in BufRead receives &mut self but this one is Pin<&mut Self>; is this because it's lower-level method?
[
12:55 PM
]
yoshuawuyts
:
oh shit
[
12:55 PM
]
yoshuawuyts
:
uhh
[
12:56 PM
]
yoshuawuyts
:
yeah good point :sweat_smile:
[
12:57 PM
]
yoshuawuyts
:
okay, this makes things more difficult
[
12:57 PM
]
yoshuawuyts
:
because the way I'm viewing it, fill_buf and consume very much seem like methods that should be called from userland; I have a legit use for it right now
[
12:58 PM
]
yoshuawuyts
:
but methods with Pin are not very user-friendly
[
12:58 PM
]
yoshuawuyts
:
as evidenced by the question of "is it safe to transmute a lifetime here"
[
12:58 PM
]
yoshuawuyts
:
so hmmm, I'm not sure what the answer is here?
[
1:02 PM
]
Tirr
:
well, the lifetime thing isn't the issue of Pin, it's because of the double reference caused by the wrapper type
[
1:03 PM
]
Tirr
:
anyway, Pin is more flexible but requires pinning before use (for Unpin types this is trivial though)
[
1:04 PM
]
yoshuawuyts
:
@Tirr could you say more about "pin is more flexible"?
[
1:07 PM
]
Tirr
:
umm, I meant Pin can receive already pinned value, but I noticed we can also use Pin to &mut self just now :smile:
[
1:08 PM
]
Tirr
:
Pin is unpin and AsyncBufRead if T::Target: AsyncBufRead
[
1:08 PM
]
yoshuawuyts
:
yeah, that's what I was wondering about
[
1:09 PM
]
Tirr
:
so &mut self is easier to use, yeah
[
1:09 PM
]
Tirr
:
hmm
[
1:09 PM
]
yoshuawuyts
:
so if consume(&mut self, amt: usize) where Self: Unpin then it shoudl always work right?
[
1:09 PM
]
yoshuawuyts
:
Not sure if the Self: Unpin bound is needed?
[
1:10 PM
]
yoshuawuyts
:
(probably not)
[
1:13 PM
]
Tirr
:
I'm not sure if it's necessary, but it's good to have to avoid the use of unsafe
[
1:14 PM
]
Tirr
:
without Unpin we need to use unsafe version of pinning before calling AsyncBufRead::consume
[
1:14 PM
]
yoshuawuyts
:
nods
[
1:14 PM
]
yoshuawuyts
:
that makes sense
[
1:15 PM
]
Tirr
:
Pin is always Unpin so that shouldn't bother users
[
1:15 PM
]
yoshuawuyts
:
ahh yeah
[
1:15 PM
]
yoshuawuyts
:
cool cool
[
1:16 PM
]
Tirr
:
well, it's Unpin for practical use cases; I checked the bounds, Pin
: Unpin where P: Unpin but we almost always use reference-like for P
[
1:17 PM
]
yoshuawuyts
:
updated!

[yoshuawuyts]

Related: #176!

feedback implemented!

[yoshuawuyts]

Implemented all feedback; bors r+

[ghost]

bors r+

[bors]

Build succeeded

• continuous-integration/travis-ci/push