Jig is in beta. Its commands, configuration schema, and supported hardware can change. To share feedback, reach out to your Together AI contact or contact sales.
Common flags and environment variables
All commands are subcommands oftg beta jig. To use a config file other than pyproject.toml, pass --config <path> after the subcommand. To debug a command, pass --debug, which prints each HTTP request and response and writes a full log to a temporary file.
Jig reads these environment variables:
Build
jig init
Create a starter pyproject.toml with default settings.
jig dockerfile
Generate a Dockerfile from your pyproject.toml configuration. Use it to debug the build.
jig build
Build the Docker image locally.
jig push
Push the built image to Together’s registry at registry.together.xyz.
Deployments
jig deploy
Build and push the image, then create or update the deployment.
jig status
Show deployment status and configuration.
jig list
List all deployments in your organization.
jig logs
Print the deployment’s logs.
jig destroy
Delete the deployment.
jig endpoint
Print the deployment’s endpoint URL.
Queue
jig submit
Submit a job to the deployment’s queue.
jig job-status
Get the status of a submitted job.
jig queue-status
Show queue backlog and worker status.
Secrets
Secrets are encrypted environment variables injected into your container at runtime.jig secrets set
Create a secret, or update it if it already exists.
jig deploy warns about any secret the deployment uses that expires within seven days. To renew a secret, set it again with --ttl, which restarts the countdown.
jig secrets list
List all secrets for the deployment.
jig secrets unset
Remove a secret from the local state without touching the deployment.
jig secrets delete
Delete a secret from the deployment and unset it locally.
Volumes
Volumes mount read-only data, such as model weights, into your container without baking them into the image.jig volumes create
Create a volume and upload files.
jig volumes update
Update a volume with new files.
version field always uses the initial version (0). To mount the new version, set version in your pyproject.toml:
jig volumes describe.
jig volumes describe
Show volume details and contents.
jig volumes list
List all volumes.
jig volumes delete
Delete a volume.
Configuration reference
Jig reads its configuration frompyproject.toml or a standalone jig.toml file. To keep a separate configuration for each environment, such as staging_jig.toml and production_jig.toml, pass the file with --config:
[tool.jig.image] for building the image, [tool.jig.deploy] for running the deployment, and [tool.jig.deploy.autoscaling] for scaling it.
The [tool.jig.image] section
These settings control how Jig builds your container image.
python_version
The Python version for the container. Jig uses it to select the base image.
"3.11"
system_packages
A list of APT packages to install in the container. Use it for system dependencies that your Python libraries need, such as FFmpeg for video processing or OpenGL for graphics.
[]
environment
Environment variables baked into the image as ENV directives. They’re available during the Docker build, the warmup step, and at runtime. Use them for build configuration such as CUDA architecture targets.
[tool.jig.deploy.environment_variables] instead, so you can change them without changing the image.
Default: {}
run
Additional shell commands to run during the Docker build. Each command becomes a separate RUN instruction. Use it for installation steps that you can’t express as Python dependencies.
[]
cmd
The default command the container runs at startup, written as the Docker CMD instruction.
--queue flag.
Default: "python app.py"
copy
A list of files and directories to copy into the container. Paths are relative to your project root.
[]
auto_include_git
When true, Jig copies every Git-tracked file into the container, in addition to the files listed in copy. Requires a clean Git repository with no uncommitted changes.
copy.
Default: false
The [tool.jig.deploy] section
These settings control how your container runs on Together’s infrastructure.
description
A human-readable description of your deployment. It appears in the Together AI console and in API responses.
""
gpu_type
The type of GPU to allocate for each replica.
"h100-80gb": NVIDIA H100 with 80 GB of memory, recommended for large models."b200-192gb": NVIDIA B200 with 192 GB of memory, for the largest models."none": CPU-only deployment.
"h100-80gb"
Other hardware is available on request. Contact sales to discuss options.
gpu_count
The number of GPUs to allocate per replica. For multi-GPU inference with tensor parallelism, set this above 1 and pass use_torchrun=True to sprocket.run() in your worker.
1
cpu
CPU cores to allocate per replica. Accepts fractional values for smaller workloads.
0.1 is 100 millicores, 1 is one core, and 8 is eight cores.
Default: 1.0
memory
Memory to allocate per replica, in gigabytes. Accepts fractional values. Set it high enough to hold your model weights plus inference overhead.
0.5 is 512 MB, 8 is 8 GB, and 64 is 64 GB. If you see out-of-memory (OOM) errors, increase this value.
Default: 8.0
storage
Ephemeral storage to allocate per replica, in gigabytes. Your container uses this disk space at runtime for temporary files, caches, and model artifacts.
100
min_replicas
The minimum number of replicas to keep running. Set it to 0 to scale to zero when idle, which saves cost but adds cold-start latency.
1
max_replicas
The maximum number of replicas the autoscaler can create. Set it based on your expected peak load and budget.
1
port
The port your container listens on. Sprocket uses port 8000 by default.
8000
health_check_path
The path Together uses to check whether your container is ready to accept traffic. It must return a 200 status when the container is healthy.
"/health"
termination_grace_period_seconds
How long, in seconds, a worker gets to finish its current job during shutdown or scale-down before it’s forcibly terminated. Set it higher for long-running inference jobs.
300
command
Overrides the container’s startup command at deploy time. It takes precedence over cmd in [tool.jig.image].
null (uses the image’s CMD)
image
An existing image to deploy instead of building one. When it’s set, jig deploy skips the build and push steps, and jig build returns an error. Pair it with command to run an off-the-shelf serving image, as in Deploy a prebuilt vLLM image.
--image flag on jig deploy overrides this setting.
Default: null (builds from [tool.jig.image])
environment_variables
Runtime environment variables injected into your container. For sensitive values like API keys, use secrets instead. Each name must be unique across environment variables and secrets.
{}
The [tool.jig.deploy.autoscaling] section
These settings control how your deployment scales with demand. For all supported metrics and scaling behavior, see Autoscaling.
metric
The metric the autoscaler scales on. QueueBacklogPerWorker is the recommended metric for queue-based workloads.
QueueBacklogPerWorker scales on queue depth relative to worker count. It adds replicas as the queue grows and removes them as workers go idle, down to min_replicas.
target
The target ratio for the autoscaler, which controls how aggressively the deployment scales.
desired_replicas = queue_depth / target. For example, with 100 jobs pending or running:
1.0: 100 workers, an exact match.1.05: 95 workers, 5% fewer than needed. This is the recommended setting.0.95: 105 workers, 5% more than needed, for lower latency.
Full configuration example
Related
- Dedicated container inference overview: How the platform works and when to use it.
- Jig CLI guide: An end-to-end walkthrough of building and deploying with Jig.
- Sprocket SDK reference: API reference for the worker SDK that Jig deploys.