Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions .github/workflows/_check_package.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: Package check

on:
# Runs when manually triggered from the GitHub UI.
workflow_dispatch:

# Runs when invoked by another workflow.
workflow_call:

permissions:
contents: read

jobs:
package_check:
name: Package check
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6

- name: Set up uv package manager
uses: astral-sh/setup-uv@v8.1.0
with:
python-version: "3.14"

- name: Build sdist and wheel
run: uv run poe build

- name: Verify built package
uses: apify/workflows/python-package-check@main
with:
package_name: crawlee
src_package_dir: src/crawlee
dist_dir: dist
python_version: "3.14"
extras: all
smoke_code: |
from crawlee.crawlers import (
HttpCrawler, BeautifulSoupCrawler, ParselCrawler,
PlaywrightCrawler, AdaptivePlaywrightCrawler,
)
from crawlee.storages import Dataset, KeyValueStore, RequestQueue
from crawlee.http_clients import HttpxHttpClient, ImpitHttpClient
from crawlee import Request
HttpCrawler()
BeautifulSoupCrawler()
ParselCrawler()
Comment on lines +29 to +47
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see this is repeated three times in the diff - is that necessary? Or a lesser evil than setting up a reusable workflow/action?

20 changes: 20 additions & 0 deletions .github/workflows/manual_release_beta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,26 @@ jobs:
version_number: ${{ needs.release_prepare.outputs.version_number }}
ref: ${{ needs.changelog_update.outputs.changelog_commitish }}

- name: Verify built package
uses: apify/workflows/python-package-check@main
with:
package_name: crawlee
src_package_dir: src/crawlee
dist_dir: dist
python_version: "3.14"
extras: all
smoke_code: |
from crawlee.crawlers import (
HttpCrawler, BeautifulSoupCrawler, ParselCrawler,
PlaywrightCrawler, AdaptivePlaywrightCrawler,
)
from crawlee.storages import Dataset, KeyValueStore, RequestQueue
from crawlee.http_clients import HttpxHttpClient, ImpitHttpClient
from crawlee import Request
HttpCrawler()
BeautifulSoupCrawler()
ParselCrawler()

# Publish the package to PyPI using PyPA official GitHub action with OIDC authentication.
- name: Publish package to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
Expand Down
21 changes: 21 additions & 0 deletions .github/workflows/manual_release_stable.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,27 @@ jobs:
is_prerelease: ""
version_number: ${{ needs.release_prepare.outputs.version_number }}
ref: ${{ needs.changelog_update.outputs.changelog_commitish }}

- name: Verify built package
uses: apify/workflows/python-package-check@main
with:
package_name: crawlee
src_package_dir: src/crawlee
dist_dir: dist
python_version: "3.14"
extras: all
smoke_code: |
from crawlee.crawlers import (
HttpCrawler, BeautifulSoupCrawler, ParselCrawler,
PlaywrightCrawler, AdaptivePlaywrightCrawler,
)
from crawlee.storages import Dataset, KeyValueStore, RequestQueue
from crawlee.http_clients import HttpxHttpClient, ImpitHttpClient
from crawlee import Request
HttpCrawler()
BeautifulSoupCrawler()
ParselCrawler()

# Publish the package to PyPI using PyPA official GitHub action with OIDC authentication.
- name: Publish package to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/on_pull_request.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ jobs:
name: Code checks
uses: ./.github/workflows/_check_code.yaml

package_check:
name: Package check
uses: ./.github/workflows/_check_package.yaml

tests:
name: Tests
uses: ./.github/workflows/_tests.yaml
Expand Down
Loading