Skip to content

[WIP] Make serializer interface streaming #20

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: master
Choose a base branch
from

Conversation

ntninja
Copy link
Collaborator

@ntninja ntninja commented Feb 1, 2020

Fixes #4, depends on #19.

Uses the JSON decoder from py-ipfs-api-client, but may switch to ijson in the future. The pickle implementation still reads the entire stream before beginning parsing, but it's more a proof-of concept anyways.

Copy link
Member

@AliabbasMerchant AliabbasMerchant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Alexander255 Can you please code some tests also?

…ods in adapters

Previously `.contains()` would always be forwarded, while `.get_all()` would *usually*
be emulated using `(await .get()).collect()`. Now both are emulated by default (simplifies
writing new adapters) with the option to set some constants on the class to forward them
instead.
Previously it was in the command-line switches in `tox.ini` and `mypy_run.py`.
@ntninja
Copy link
Collaborator Author

ntninja commented Feb 12, 2020

@AliabbasMerchant: But it does have tests?

(Only commit acd5245 is relevant, the rest is an old version of #19.)

@AliabbasMerchant
Copy link
Member

@Alexander255 I overlooked the tests file 🤦‍♂

@ntninja ntninja changed the title Make serializer interface streaming [WIP] Make serializer interface streaming Feb 19, 2020
@ntninja
Copy link
Collaborator Author

ntninja commented Feb 19, 2020

After some consideration I decided to revise the proposed serializer interface to decouple the serializers from the I/O framework and use a yield based API instead. Hopefully we can then release it as a separate library at some point and make it reusable in py-ipfs-http-client and maybe also other places.
Sorry for bothering you with a review already, @AliabbasMerchant!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[DISCUSSION] Merge datastore.core.serialize and ipfshttpclient.encoding?
2 participants