Skip to content

Commit

Permalink
chore: updated README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
tikazyq committed Dec 18, 2024
1 parent 88faa10 commit 829c907
Showing 1 changed file with 24 additions and 8 deletions.
32 changes: 24 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,20 +26,36 @@ ITEM_PIPELINES = {
}
```

### Save Scraped Items

```python
from crawlab import save_item

scraped_items = [
{
'name': 'item1',
'value': 'value1'
},
{
'name': 'item2',
'value': 'value2'
}
]

for item in scraped_items:
save_item(item)
```

## Development

### Install dependencies
### Pre-requisites

```bash
pip install -r requirements.txt
pip install poetry
```

### Compile gRPC
### Install dependencies

```bash
# Set the environment variable CRAWLAB_PROTO_PATH to the path of the gRPC proto files
export CRAWLAB_PROTO_PATH=/path/to/grpc/proto/files

# Compile gRPC to Python code
./compile_grpc.sh
poetry install
```

0 comments on commit 829c907

Please sign in to comment.