fix(fs): Use autocommit value from signature instead of computing it
#334
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
yup, this breaks our transaction expectations. I guess the play is to remove |
nicholasjng
force-pushed
the
fix-autocommit
branch
from
4511e7c to
8899597
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #334 +/- ##
==========================================
- Coverage 95.22% 95.21% -0.02%
==========================================
Files 6 6
Lines 440 439 -1
Branches 64 64
==========================================
- Hits 419 418 -1
Misses 15 15
Partials 6 6
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Unlike fsspec's base class, which infers the value of `autocommit` from the kwargs and, if necessary, from the current transaction state, we keep `autocommit` as an explicit named keyword argument in our signature of `LakeFSFileSystem.open()`. Previously, this value was unused, and the keyword arguments would never contain autocommit due to name collision, so we implicitly autocommitted files if and only if we were not in a transaction. This change makes the file committing strategy configurable, and defaults to `False`.
nicholasjng
force-pushed
the
fix-autocommit
branch
from
8899597 to
15e3b4d
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Unlike fsspec's base class, which infers the value of
autocommitfrom the kwargs and, if necessary, from the current transaction state, we keepautocommitas an explicit named keyword argument in our signature ofLakeFSFileSystem.open().Previously, this value was unused, and the keyword arguments would never contain autocommit due to name collision, so we implicitly autocommitted files if and only if we were not in a transaction. This change makes the file committing strategy configurable, and defaults to
True.Please review. In its current state, it is probably wrong, since this means we would also autocommit files in a transaction if the
autocommitkeyword is not explicitly set toFalse.The opposite choice, which is to remove
autocommitfrom the named keyword arguments and keeping the value computation, is probably better for our transaction logic, but would make the autocommit config option more opaque.