Skip to content

Change write progress reporting to callback #315

@jreiberkyle

Description

@jreiberkyle
Contributor

Currently, models.StreamingBody.write() includes logic for reporting as well as downloading. Change this so that the function uses a callback provided as an argument. To help the user, implement the recommended callback in the reporting module.

Activity

jreiberkyle

jreiberkyle commented on Sep 8, 2021

@jreiberkyle
ContributorAuthor

Proposal for recommended callback:

class FileDownloadBar(ProgressBar):
    """Bar reporter of file download progress.

    Example:
        ```python
        from planet import reporting

        with reporting.FileDownloadBar('img.tif', 100000) as bar:
            bar.update(1000)
            ...
        ```
    """
    def __init__(
        self,
        filename: str,
        size: int,
        unit: int = None,
        disable: bool = False
    ):
        """Initialize the object.

        Parameters:
            filename: Name of file to display.
            size: File size in bytes.
            unit: Value to scale number of report iterations.
                (e.g. 1024*1024 scales to reporting in Megabytes.)
        """
        self.filename = filename
        self.size = size
        self.unit = unit or 1024*1024
        super().__init__()

    def open_bar(self):
        """Initialize and start the progress bar."""
        self.bar = tqdm(
            total=self.size,
            unit_scale=True,
            unit_divisor=self.unit,
            unit='B',
            desc=self.filename,
            disable=self.disable)

    def update(self, downloaded_amt, logger=None):
        self.bar.update(downloaded_amt)
        LOGGER.debug(str(self.bar))

and test:

def test_FileDownloadBar():
    with reporting.FileDownloadBar('fname', 2, unit=1) as bar:
        expected_init = r'fname: *0%\| *\| *0.00/2.00'
        assert re.match(expected_init, str(bar))

        expected_update = 'fname:  50%|█████     | 1.00/2.00'
        bar.update(1)
        assert(re.match(expected_update, str(bar)))

This can be called like so in write() (I haven't figured out how to make this a callback yet):

                with reporting.FileDownloadBar(
                        filename, self.size,
                        disable=not progress_bar) as progress:
                    previous = self.num_bytes_downloaded
                    async for chunk in self.aiter_bytes():
                        fp.write(chunk)
                        new = self.num_bytes_downloaded
                        progress.update(new-previous, logger=LOGGER.debug)
                        previous = new
self-assigned this
on Sep 8, 2021
removed their assignment
on Apr 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @cholmes@jreiberkyle@tbarsballe

        Issue actions

          Change write progress reporting to callback · Issue #315 · planetlabs/planet-client-python