📖 Search algorithms, transformations and check implementation details & examples in our documentation.
💭 In case you have doubts, ideas, want to ask for help or want to discuss different approach and usages, feel free to create a discussion.
💡 Do you have ideas for new features? Open a feature request on GitHub.
🚀 Want to find the available releases? Check our release notes on GitHub and PyPi.
- Git.
- Your IDE of choice with a Python 3 environment (e.g., virtualenv created from the requirements_cicd.txt file).
- Docker. Warning: The default spark driver memory limit for the tests is set at 2g. This limit is configurable but your testing docker setup MUST always have at least 2 * spark driver memory limit + 1 gb configured.
- GNU make.
- Fork the project.
- Clone the forked project into your working environment.
- Create your feature branch following the convention [feature|bugfix]/ISSUE_ID_short_name.
- Apply your changes in the recently created branch. It is mandatory to add tests covering the feature of fix contributed.
- Style, lint, test and test security:
make style make lint make test make test-security
Note: To use the make targets with another docker-compatible cli other than docker you can pass the parameter "container_cli". Example:
make test container_cli=nerdctl
Note: Most make target commands are running on docker. If you face any problem, you can also check the code of the respective make targets and directly execute the code in your python virtual environment.
- (optional) You can build the wheel locally with
make build
. - (optional) Install the wheel you have just generated and test it.
- If you have changed or added new requirements, you should run
make build-lock-files
, to rebuild the lock files. - If the transitive dependencies have not been updated for a while, and you want to upgrade them, you can use
make upgrade-lock-files
to update them. This will update the transitive dependencies even if you have not changed the requirements. - When you're ready with your changes, open a Pull Request (PR) to develop.
- Ping the team through the preferred communication channel.
- The team will come together to review it and approve it (2 approvals required).
- Your changes will be tested internally, promoted to master and included in the next release.
🚀🚀🚀
Pull Requests are welcome from anyone. However, before opening one, please make sure to open an issue on GitHub and link it. Moreover, if the Pull Request intends to cover big changes or features, it is recommended to first discuss it on a GitHub issue or Discussion.
🚀🚀🚀