Quickstart

Deploy your first container with dedicated container inference and send it a job through the managed queue in 20 minutes.

Dedicated container inference runs your own Docker container on Together’s managed GPU infrastructure. You bring the container, and Together handles provisioning, autoscaling, networking, and observability. Use it when you need to run your own model or inference code, such as a custom architecture, a modified vLLM or SGLang engine, or an audio, image, or video model, without managing GPU servers yourself.

This quickstart deploys the hello-world example from the Sprocket repository, a minimal Sprocket worker that returns a greeting. You build and deploy the container with the Jig CLI, send the worker a job through the managed queue, read back the result, and delete the deployment.

Requirements

• Dedicated container inference access: Contact your account representative or the sales team to enable dedicated container inference for your organization.
• Together API key: The curl commands in steps 5 to 7 read your key from the TOGETHER_API_KEY environment variable, so export it before you start.
• Docker: Jig uses your local Docker to build and push the container image. See the Docker installation guide.
• uv: For installing the Together CLI. See uv’s installation guide.

Step 1: Install the Together CLI

Install the Together CLI, which includes the Jig commands (tg beta jig) that build and deploy your container:

uv tool install "together[cli]" --upgrade

Log in to authenticate the CLI:

tg login

Step 2: Clone the Sprocket examples

Clone the Sprocket repository, which contains the example workers:

Shell

git clone [e-mail]:togethercomputer/sprocket.git
cd sprocket

The worker’s code is in sprocket/examples/hello-world. Sprocket calls setup() once when the worker starts and predict() once per job. Here, predict() reads name from the job’s arguments and returns a greeting:

hello_world.py

import os
import sprocket


class HelloWorld(sprocket.Sprocket):
    def setup(self) -> None:
        self.greeting = "Hello"

    def predict(self, args: dict) -> dict:
        name = args.get("name", "world")
        return {"message": f"{self.greeting}, {name}!"}


if __name__ == "__main__":
    queue_name = os.environ.get("TOGETHER_DEPLOYMENT_NAME", "hello-world")
    sprocket.run(HelloWorld(), queue_name)

Step 3: Build and deploy

Jig reads the deployment configuration from the worker’s pyproject.toml. The example worker uses this one:

pyproject.toml

[project]
name = "hello-world"
version = "0.1.0"
dependencies = ["sprocket"]

[[tool.uv.index]]
name = "together-pypi"
url = "https://pypi.together.ai/"

[tool.uv.sources]
sprocket = { index = "together-pypi" }

[tool.jig.image]
python_version = "3.11"
cmd = "python3 hello_world.py --queue"
copy = ["hello_world.py"]

[tool.jig.deploy]
gpu_type = "none"
gpu_count = 0
cpu = 1
memory = 2
storage = 10
port = 8000
min_replicas = 1
max_replicas = 1

The [tool.jig.image] table tells Jig how to build the image, and [tool.jig.deploy] sets the deployment’s resources and replica bounds. This example runs a single CPU-only replica.

Jig uses name under [project] as the deployment name, which must be globally unique. Change hello-world to a name of your own in examples/hello-world/pyproject.toml, and use that name in place of hello-world in the commands in steps 5 to 7.

If you’re logged in with tg login, pass your API key explicitly on the first deploy: tg beta jig deploy --api-key "$TOGETHER_API_KEY". The first deploy stores an API key in the deployment, and it can’t use a tg login session for that. While you’re logged in, the CLI ignores TOGETHER_API_KEY.

Move into the example’s folder and deploy it:

Shell

cd examples/hello-world
tg beta jig deploy

The deploy command does three things:

1. Builds the Docker image from the example.
2. Pushes it to Together’s private registry.
3. Creates a deployment on Together’s infrastructure.

Step 4: Check that the deployment is running

Poll the deployment’s status with watch. macOS doesn’t include watch by default, so install it first with brew install watch or your package manager of choice:

Shell

watch 'tg beta jig status'

Wait until the deployment shows running and its replicas are ready, then press Ctrl+C to stop watching.

You can also check deployment status in the Together AI console.

Step 5: Test the health endpoint

Call the deployment’s health endpoint to confirm the worker is up:

Shell

curl https://api.together.ai/v1/deployment-request/hello-world/health \
  -H "Authorization: Bearer $TOGETHER_API_KEY"

A healthy worker responds with:

{"status": "healthy"}

Step 6: Submit a job

Submit a job to the deployment’s queue. The model field takes your deployment name, and predict() receives payload as its arguments:

Shell

curl -X POST "https://api.together.ai/v1/queue/submit" \
  -H "Authorization: Bearer $TOGETHER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hello-world",
    "payload": {"name": "Together"},
    "priority": 1
  }'

The response contains the job’s request_id. Copy it for the next step:

{
  "request_id": "req_abc123"
}

Step 7: Get the job result

Check the job’s status. Replace req_abc123 with the request_id from step 6, a UUIDv7 value such as 019ba379-92da-71e4-ac40-d98059fd67c7:

Shell

curl "https://api.together.ai/v1/queue/status?model=hello-world&request_id=req_abc123" \
  -H "Authorization: Bearer $TOGETHER_API_KEY"

When the job is complete, status is done and outputs holds the worker’s greeting:

{
  "request_id": "req_abc123",
  "model": "hello-world",
  "status": "done",
  "outputs": {"message": "Hello, Together!"}
}

Step 8: View the logs

Stream your deployment’s logs:

Shell

tg beta jig logs --follow

Step 9: Delete the deployment

The deployment keeps a replica running until you delete it, because min_replicas is 1. When you’re done, delete the deployment:

Shell

tg beta jig destroy

Next steps

Now that you’ve deployed your first container, explore the full platform:

Dedicated container inference overview

Read the architecture and core concepts.

Jig CLI

Build, push, and deploy containers, and manage secrets and volumes.

Sprocket SDK

Build queue-integrated inference workers.

API reference

Call the REST API for deployments, secrets, and queues.

Tutorials

Image generation with FLUX.2

Run single-GPU inference with 4-bit quantization.

Video generation with Wan 2.1

Run multi-GPU inference with torchrun.

Serve an OpenAI-compatible endpoint

Run in HTTP server mode behind the OpenAI Images API.

Deploy a prebuilt vLLM image

Serve Qwen2.5-VL from the stock vLLM image, with no Sprocket code and no Docker build.