Skip to content

Commit

Permalink
Rename package to fs-queue
Browse files Browse the repository at this point in the history
  • Loading branch information
costrouc committed Dec 9, 2022
1 parent 28c54ba commit 99a5c48
Show file tree
Hide file tree
Showing 14 changed files with 21 additions and 17 deletions.
16 changes: 10 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# File Queue
# Filesystem Task Queue

A task queue using the filesystem as the message queue. This project
was motivated by the use case where it is hard or near impossible to
Expand Down Expand Up @@ -27,13 +27,17 @@ chunking them or using plugins like
breaking it into many small tasks). Each task state modifications
results in 2-4 IOPS on the filesystem.

# Install

- `pip install fs-queue`

# API

Creating a queue is as simple as supplying a directory where the queue
will reside.

```python
from file_queue import Queue
from fs_queue import Queue

queue = Queue("path/to/queue")
```
Expand All @@ -42,7 +46,7 @@ Submitting jobs and monitoring over SSH is also supported via the same
interface. Workers currently cannot connect over SSH.

```python
from file_queue.plugins import SSHQueue
from fs_queue.plugins import SSHQueue

queue = SSHQueue("ssh://<username>:<password>@<hostname>:<port>/<path>")
```
Expand Down Expand Up @@ -75,10 +79,10 @@ Starting a worker is as simple as giving a filesystem directory where
the queue will reside.

```shell
file-queue-worker --path ./path/to/queue
fs-queue-worker --path ./path/to/queue
```

A `dask` worker is supported via `file_queue.plugin.dask.DaskWorker`
A `dask` worker is supported via `fs_queue.plugin.dask.DaskWorker`
for sending jobs to the dask cluster instead of executing locally.

A worker runs a continuous loop gathering tasks in the task queue. The
Expand All @@ -89,4 +93,4 @@ will:

# License

[BSD-3]()
[BSD-3](./LICENSE)
1 change: 0 additions & 1 deletion file_queue/__init__.py

This file was deleted.

1 change: 1 addition & 0 deletions fs_queue/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
from fs_queue.core import Queue, Job, Worker, JobStatus # noqa
2 changes: 1 addition & 1 deletion file_queue/__main__.py → fs_queue/__main__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from file_queue.cli import cli
from fs_queue.cli import cli


def main():
Expand Down
File renamed without changes.
2 changes: 1 addition & 1 deletion file_queue/core.py → fs_queue/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
import time
from typing import Union

from file_queue import utils
from fs_queue import utils


logger = logging.getLogger(__name__)
Expand Down
File renamed without changes.
2 changes: 1 addition & 1 deletion file_queue/plugins/dask.py → fs_queue/plugins/dask.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import logging

from file_queue.core import Worker, Queue, Job
from fs_queue.core import Worker, Queue, Job

import distributed

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

import filelock

from file_queue.core import DummyLock
from fs_queue.core import DummyLock


class FileLock(DummyLock):
Expand Down
2 changes: 1 addition & 1 deletion file_queue/plugins/ssh.py → fs_queue/plugins/ssh.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
import paramiko
from paramiko.client import SSHClient

from file_queue.core import Queue, JSONSerializer, DummyLock, Job, JobStatus
from fs_queue.core import Queue, JSONSerializer, DummyLock, Job, JobStatus


class SSHJob(Job):
Expand Down
File renamed without changes.
6 changes: 3 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ requires = ["setuptools", "setuptools-scm"]
build-backend = "setuptools.build_meta"

[project]
name = "file-queue"
description = "File based task queue"
name = "fs-queue"
description = "Filesystem based task queue"
readme = "README.md"
requires-python = ">=3.7"
license = {text = "BSD 3-Clause License"}
Expand All @@ -18,4 +18,4 @@ dev = ["pytest", "black", "flake8", "build", "twine"]
extra = ["filelock", "paramiko", "dask", "distributed"]

[project.scripts]
file-queue-worker = "file_queue.__main__:main"
fs-queue-worker = "fs_queue.__main__:main"
2 changes: 1 addition & 1 deletion tests/test_integration.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import file_queue
from file_queue.plugins.ssh import SSHQueue
from fs_queue.plugins.ssh import SSHQueue

import operator

Expand Down
2 changes: 1 addition & 1 deletion tests/test_unit.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from file_queue import Queue, JobStatus
from fs_queue import Queue, JobStatus


def add(a: int, b: int):
Expand Down

0 comments on commit 99a5c48

Please sign in to comment.