Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions .dockerignore

This file was deleted.

1 change: 0 additions & 1 deletion examples/deepseek-ocr2/.gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1 @@
Dockerfile
uv.lock
1 change: 1 addition & 0 deletions examples/deepseek-ocr2/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
[project]
name = "deepseek-ocr2"
version = "1.0.0"
requires-python = ">=3.11"

[tool.jig.deploy]
# deploy an existing image instead of building
Expand Down
8 changes: 8 additions & 0 deletions examples/deepseek-ocr2/uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion examples/flux2-dev/.gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1 @@
Dockerfile
uv.lock
17 changes: 9 additions & 8 deletions examples/flux2-dev/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
[project]
name = "sprocket-flux2-dev"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = [
"diffusers>=0.33.1",
"transformers>=4.51.0",
"torch>=2.0.0",
"torchvision",
"pillow",
"accelerate",
"safetensors",
"sprocket",
"diffusers==0.37.1",
"transformers==4.57.1",
"torch==2.6.0",
"torchvision==0.21.0",
"pillow==12.2.0",
"accelerate==1.13.0",
"safetensors==0.7.0",
"sprocket==0.1.4",
]

[[tool.uv.index]]
Expand Down
8 changes: 3 additions & 5 deletions examples/flux2-dev/run.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,9 @@
class Flux2Sprocket(sprocket.Sprocket):
def setup(self) -> None:
model = "black-forest-labs/FLUX.2-klein-9B"
device = "cuda" if torch.cuda.is_available() else "cpu"

logging.info(f"Loading Flux2 pipeline from {model} on {device}...")
pipe = Flux2KleinPipeline.from_pretrained(model, torch_dtype=torch.bfloat16)
self.pipe = pipe.to(device)
self.pipe = Flux2KleinPipeline.from_pretrained(
model, torch_dtype=torch.bfloat16
).to("cuda")
logging.info("Pipeline loaded successfully!")

def predict(self, args: dict) -> dict:
Expand Down
1,246 changes: 1,246 additions & 0 deletions examples/flux2-dev/uv.lock

Large diffs are not rendered by default.

1 change: 0 additions & 1 deletion examples/hello-world/.gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1 @@
Dockerfile
uv.lock
3 changes: 2 additions & 1 deletion examples/hello-world/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
[project]
name = "hello-world"
version = "0.1.0"
dependencies = ["sprocket"]
requires-python = ">=3.11"
dependencies = ["sprocket==0.1.4"]

[[tool.uv.index]]
name = "together-pypi"
Expand Down
220 changes: 220 additions & 0 deletions examples/hello-world/uv.lock

Large diffs are not rendered by default.

1 change: 0 additions & 1 deletion examples/kitchen-sink/.gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1 @@
Dockerfile
uv.lock
51 changes: 51 additions & 0 deletions examples/kitchen-sink/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Kitchen Sink Example

A no-GPU Sprocket worker that exercises every platform feature: volume mounts, secrets, environment variables, file I/O, `FileOutput` uploads, and queued job processing.

See [`kitchen_sink.py`](./kitchen_sink.py) for the worker logic.

## How to Deploy

1. Generate a unique deployment name and update `pyproject.toml`:

```bash
sed -i '' "s/^name = \"kitchen-sink\"/name = \"kitchen-sink-$(date +%s)\"/" pyproject.toml
```

2. Create the `kitchen-sink-pantry` volume and upload recipe files (the worker reads `*.txt` from `/pantry` at startup):

```bash
together beta jig volumes create --name kitchen-sink-pantry --source ./pantry
```

3. Set the `SECRET_SPICE` secret (gets injected as an env var):

```bash
together beta jig secrets set --name SECRET_SPICE --value cumin
```

4. Deploy:

```bash
together beta jig deploy
```

5. Submit a request that exercises every code path:

```bash
together beta jig submit --payload '{"dish":"pasta","menu":true,"read_env":"SECRET_SPICE","read_file":"/pantry/curry.txt","receipt":true,"sleep":1}' --watch
```

Drop `--watch` to grab the request id and tail logs in parallel:

```bash
together beta jig logs --follow
together beta jig job-status --request-id <request-id>
```

The response includes:
- `plate` — string combining the requested `dish`, its recipe from the volume, and the secret spice
- `menu` — sorted list of recipe names found in `/pantry`
- `env_value` — value of the env var named by `read_env`
- `file_content` — text content at `read_file`
- `receipt` — uploaded `FileOutput` URL containing a generated receipt
43 changes: 43 additions & 0 deletions examples/kitchen-sink/kitchen_sink.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,59 @@
import os
import tempfile
import time
import traceback
from pathlib import Path

import sprocket
from sprocket import sprocket as _sp

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

PANTRY = Path("/pantry")


# --- diagnostics: instrument FileOutput uploads to pinpoint transient failures ---
# kitchen_sink.py is copied into the worker image directly (sprocket is
# pip-installed), so this ships on the next image build without a sprocket
# release. We wrap QueueClient.upload_file — which runs in THIS worker process,
# so it sees the *actual* failing request, not a re-connection — to record which
# hop broke (presigned POST vs Tigris PUT), at which layer (TCP connect vs TLS
# handshake), and the underlying cause, when uploads intermittently ConnectError.
_orig_upload_file = _sp.QueueClient.upload_file


def _cause_chain(exc: BaseException) -> str:
chain, cur = [], exc.__cause__ or exc.__context__
while cur is not None:
chain.append(f"{type(cur).__module__}.{type(cur).__name__}({cur})")
cur = cur.__cause__ or cur.__context__
return " <- ".join(chain) or "(none)"


async def _instrumented_upload_file(self, request_id, path):
start = time.time()
try:
return await _orig_upload_file(self, request_id, path)
except Exception as e:
# httpx attaches the in-flight request to the exception, so this is the
# exact request that failed (full presigned URL → host + which hop).
req = getattr(e, "request", None)
failing = f"{req.method} {req.url}" if req is not None else "(no request on exception)"
logger.error(
"FileOutput upload FAILED after %.1fs for %s-%s\n"
" failing request: %s\n"
" error: %r\n"
" cause chain: %s\n%s",
time.time() - start, request_id, path.name, failing,
e, _cause_chain(e), traceback.format_exc(),
)
raise


_sp.QueueClient.upload_file = _instrumented_upload_file


class KitchenSink(sprocket.Sprocket):
def setup(self) -> None:
self.secret_spice = os.environ.get("SECRET_SPICE", "love")
Expand Down
1 change: 1 addition & 0 deletions examples/kitchen-sink/pantry/curry.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
bloom onions in oil, add ginger garlic paste, toast spices, deglaze with tomato, simmer with coconut milk, finish with cilantro
1 change: 1 addition & 0 deletions examples/kitchen-sink/pantry/omelette.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
crack 3 eggs, whisk with salt, melt butter in pan, pour eggs, fold once edges set, slide onto plate
1 change: 1 addition & 0 deletions examples/kitchen-sink/pantry/pasta.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
boil water, salt heavily, drop spaghetti, cook 9 minutes, drain, toss with olive oil and garlic
1 change: 1 addition & 0 deletions examples/kitchen-sink/pantry/toast.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
slice sourdough, toast until golden, butter generously, eat while warm
3 changes: 2 additions & 1 deletion examples/kitchen-sink/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
[project]
name = "kitchen-sink"
version = "0.1.0"
dependencies = ["sprocket"]
requires-python = ">=3.11"
dependencies = ["sprocket==0.1.4"]

[[tool.uv.index]]
name = "together-pypi"
Expand Down
Loading