Releases: pfnet/pfio
Releases · pfnet/pfio
Release 2.8.0
What's Changed
- Fix document build fail and add status badge by @kuenishi in #330
- update: moto v5 by @yhrkw in #332
- Remove sparse file cache by @kuenishi in #335
- Support python 3.12 by @kuenishi in #334
- Path: more compatibility with pathlib.Path by @yhrkw in #331
- Pick out document build at CI by @kuenishi in #337
New Contributors
Full Changelog: 2.7.1...2.8.0
Download
3b70596f683cb316da385299d137de29fe972cfde448f495c60203de77c4dc5f pfio-2.8.0-py3-none-any.whl
7f609549fc688a82c416f1c9edd5b0fcbe1e6ab469b737d858f026fa5e4586cf pfio-2.8.0.tar.gz
Release 2.7.1
What's Changed
- S3 support for large files by @alexisVallet in #328
- Bump version by @kuenishi in #329
New Contributors
- @alexisVallet made their first contribution in #328
Full Changelog: 2.7.0...2.7.1
2.7.0
Incompatible Changes
- Fix Path.glob for compatibility with standard pathlib.Path by @niboshi in #314
- Deprecate sparse file cache in 2.7 (and will be deleted in 2.8) by @kuenishi in #324
New Features
What's Changed
- Bump version by @kuenishi in #309
- Refactor HTTPCache to introduce HTTPConnector by @y1r in #313
- Declare Python 3.11 in setup.py by @kuenishi in #319
- Fix flake8 v6.1.0 by @y1r in #320
- Rewrite HTTPConnector by http.client by @y1r in #318
- Avoid calling zip.namelist many times by @y1r in #322
- Add repr to FS classes by @niboshi in #323
- Bump version by @kuenishi in #327
New Contributors
Full Changelog: 2.6.0...2.7.0
2.6.0
What's Changed
- Fix broken sphinx reference by @himkt in #302
- Introduce GitHub Workflow testing by @kuenishi in #304
- Goodbye 3.7 and PFN in-house CI for OSS by @kuenishi in #305
- Make HDFS optional by @kuenishi in #306
- Clear the clutter on test race by @kuenishi in #307
- Introduce mypy type check by @kuenishi in #308
New Contributors
Full Changelog: 2.5.1...2.6.0
2.5.1
2.5.0 Release
Incompat note: that reset_on_fork
flag was removed at this release.
What's Changed
- Update documentation of sparse file cache by @kuenishi in #291
- Flake8 skip for Python 3.7 by @kuenishi in #295
- Refine pfio.cache.HTTPCache by @y1r in #293
- S3 + zipfile on fork-server multiprocessing by @y1r in #292
- Remove reset_on_fork parameter by @y1r in #297
- Bump version by @kuenishi in #298
Full Changelog: 2.4.0...2.5.0
2.4.0 Release
What's Changed
- Update Python, pyenv and migrate to GAR by @kuenishi in #283
- HTTPCache feature by @y1r in #282
- detail=True option for listing directory or prefix by @kuenishi in #284
- detail=True option for listing zip internals by @kuenishi in #285
- Support write mode in S3+ZIP by @kuenishi in #289
- Bump version by @kuenishi in #290
New Contributors
Full Changelog: 2.3.0...2.4.0
2.4.0rc1
2.3.0
What's Changed
- Fix
BufferedReader
error caused by opening an empty file in S3 in "rb" mode by @belltailjp in #270 - Sphinx 5 warns as language is None by @kuenishi in #274
- Control buffering for underlying file object for Zip(FS) by @kuenishi in #273
- Sparse file local cache by @kuenishi in #275
- Thread safety by @kuenishi in #277
- Multiprocessing-safe sparse file cache by @kuenishi in #278
- Limit the sparse cache size by @kuenishi in #279
- Release by @kuenishi in #280
- Bump version by @kuenishi in #281
Full Changelog: 2.2.0...2.3.0
2.3.0rc1
What's Changed
- Fix
BufferedReader
error caused by opening an empty file in S3 in "rb" mode by @belltailjp in #270 - Sphinx 5 warns as language is None by @kuenishi in #274
- Control buffering for underlying file object for Zip(FS) by @kuenishi in #273
- Sparse file local cache by @kuenishi in #275
- Thread safety by @kuenishi in #277
- Multiprocessing-safe sparse file cache by @kuenishi in #278
- Limit the sparse cache size by @kuenishi in #279
Sparse file cache example with multiprocessing would be like:
import os
from pfio.v2 import from_url
with from_url("s3://bucket/your.zip",
local_cache=True, reset_on_fork=True) as fs:
pid = os.fork()
if pid:
os.wait()
else:
with fs.open("file-in-zip.jpg", 'rb') as fp:
data = rp.read()
$ sha1sum dist/pfio-2.3.0rc1*
782f5ef08f08e4de64dac5175b300bce3dd46a38 dist/pfio-2.3.0rc1-py3-none-any.whl
4d21ede0d89f059830d681b74b8c464febd567e4 dist/pfio-2.3.0rc1.tar.gz
Full Changelog: 2.2.0...2.3.0rc1