Skip to content

Commit

Permalink
0.3.0 version of the package, installation, readme, etc..
Browse files Browse the repository at this point in the history
  • Loading branch information
TonySchneider committed Aug 30, 2023
1 parent afe5fa4 commit 0e15fcf
Show file tree
Hide file tree
Showing 8 changed files with 375 additions and 140 deletions.
6 changes: 6 additions & 0 deletions .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[bumpversion]
current_version = 0.3.0
commit = True
tag = True

[bumpversion:file:setup.py]
169 changes: 139 additions & 30 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,44 +1,153 @@
*.pyc
*.pyo
*~
*.swp
*.swo
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/

# sqlite database created by django
# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# pycharm
.idea
*.iml
# Flask stuff:
instance/
.webassets-cache

# Visual Studio Code
.vscode
# Scrapy stuff:
.scrapy

# Special directories
tmp/*
.pytest_cache
.gradle/
# Sphinx documentation
docs/_build/

# Side effects of running the RPM builder localy.
installation/packaging/build_zone/*
# PyBuilder
target/

# Downloaded files from Artifactory
dev_utils/dev_common/artifactory/downloads/*
# Jupyter Notebook
.ipynb_checkpoints

# Links
.link
# IPython
profile_default/
ipython_config.py

# Unit Test Results Output
unit_tests_reports
# pyenv
.python-version

# Mac auto generated
.DS_Store
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

db_data/
ssh_known_hosts
# Celery stuff
celerybeat-schedule
celerybeat.pid

# for virtual env
# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# docker local volume
.docker
# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/

# VS Code directories
.vscode/

# Jetbrains IDE
.idea/

# User-specific stuff
.DS_Store
*.swp
*.swo
*.swn
*.suo
*.user
*.userprefs
*.pyc
*.pyo
*.exe
*.dll
*.so
*.dylib
*.db
*.db-journal
*.log
*.csv
*.json
*.asc
*.sublime-workspace
70 changes: 50 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,31 +1,61 @@
Celery Smartbase
=================
# Celery SmartBase

Celery Smartbase is a Python package that provides solutions for common problems that arise when working with the Celery distributed task queue. In particular, Celery Smartbase solves the problem of duplicate task execution, which can occur in certain scenarios such as when workers are restarted.
`celery-smartbase` is an extension and improvement of the popular `celery` PyPI package. It provides features to avoid duplicates for tasks that are pending or running and to see the "pending" tasks as well in the "jobs" Django model when you integrate Celery with Django. Although this package is improved to work with Django, users can modify the features slightly to make it work with any Python framework, not just Django.

Celery Smartbase achieves this by providing a custom task class that overrides the default Celery task class. This custom task class includes a unique task ID generator that ensures that each task is executed only once, even if it is queued multiple times.
## Features

In addition to solving the duplicate task problem, Celery Smartbase also provides a set of base classes that can be overridden to add custom code and improve the performance of Celery. These base classes include:
1. **Avoid Duplicate Tasks**: Prevents the creation of duplicate tasks that are either pending or running.
2. **View Pending Tasks**: Allows you to see the "pending" tasks in the "jobs" Django model when you integrate Celery with Django.

- Task: The base class for all Celery tasks.
- TaskSet: A task set is a group of tasks that are executed together.
- Canvas: A canvas is a group of tasks that are executed sequentially or in parallel.
- Group: A group is a set of tasks that are executed in parallel.
These features are developed inside an extension of the `BaseTask` class of the existing Celery Python package. The extension class is called `SmartBase`.

Celery Smartbase is designed to be used with Django and requires it as a dependency. To install Celery Smartbase, simply run `pip install celery-smartbase`. Once installed, you can use the custom task class provided by Celery Smartbase by importing it in your Celery configuration file:
## SmartBase Class

The `SmartBase` class is an extension of the `BaseTask` class from the Celery package. Here is the code for the `SmartBase` class:

```python
from celery import Celery
from celery_smartbase import SmartTask
class SmartBase(BaseTask):
...
```

[Include the full code of the SmartBase class here]

## Requirements

- Django
- djangorestframework
- django-model-utils
- django-celery-results
- django-celery-beat
- kombu
- celery

## Installation

To install the `celery-smartbase` package, run the following command:

app = Celery('tasks', broker='pyamqp://guest@localhost//')
app.task_cls = SmartTask
```
pip install celery-smartbase
```

## Usage

To use the `celery-smartbase` package, you need to...

[Include examples of how to use the package here]

## Contributing

If you would like to contribute to the `celery-smartbase` package, please...

[Include information on how others can contribute to your package]

## License

The `celery-smartbase` package is licensed under the...

[Include information about the license of your package]

Contributing
--------
Contributions to Celery Smartbase are welcome! If you find a bug or have an idea for a new feature, please open an issue or submit a pull request.
## Acknowledgments

License
--------
Celery Smartbase is licensed under the MIT License. See the LICENSE file for more information.
[Include any acknowledgments or credits you would like to include]
Loading

0 comments on commit 0e15fcf

Please sign in to comment.