Deploy a fine-tuned model

Serve your fine-tuned model on a dedicated endpoint or download it for local inference.

Once a fine-tuning job completes, your model is available for inference in two ways: hosted on a dedicated endpoint at Together AI, or downloaded as a standalone checkpoint.

Requirements

• A completed fine-tuning job.
• One of the job’s two registry identifiers for the trained weights. Look for them in the tg fine-tuning retrieve output on a completed job:
  • model_object_id (an ml_... value): The model-registry ID. The CLI takes it directly, and the SDK and API take it inside a projects/{project_id}/models/{model_id} path.
  • model_object_name (a <project_slug>/<model_name> value): The qualified registry name. The CLI accepts it in place of the ID. The SDK and API don’t.
• Together CLI version 3.0.0 or later. Check your version with tg --version. To install or upgrade, run uv tool install "together[cli]" --upgrade.

Deploy on a dedicated endpoint

Together AI serves fine-tuned models through dedicated model inference. A completed fine-tuning job is already a private model in your project, so you deploy it directly, with no upload step.

Dedicated model inference bills per minute per running replica, even when idle. Scale the deployment to zero or delete the endpoint when you’re done to stop billing.

CLI

Deploy the fine-tune by its model_object_name or its model_object_id. The deploy command creates the endpoint, attaches a deployment, and routes all traffic to it in one step:

CLI

# Retrieve the job to copy its identifiers (model_object_name and model_object_id)
tg fine-tuning retrieve "<JOB_ID>"

# Deploy the fine-tuned model to a new endpoint
# (pass <MODEL_OBJECT_ID> instead to deploy by ID)
tg endpoints deploy "<MODEL_OBJECT_NAME>" \
  --endpoint my-finetuned-endpoint

To deploy by name, pass it in full, including the project slug (acme-corp/my-model-abc123). The bare model name doesn’t resolve, and neither does model_output_name. The CLI resolves a name only when exactly one model in your project matches it.

If the model has more than one deployment profile, pass --config <cr_...> to choose one. List a model’s profiles with tg models configs "<MODEL_OBJECT_ID>".

Poll until the deployment reaches DEPLOYMENT_STATE_READY:

CLI

tg endpoints get "<ENDPOINT_ID>"

Once the deployment is READY, pass the endpoint string (your-project-slug/my-finetuned-endpoint, printed by the deploy output) as the model parameter and send the request to https://api-inference.together.ai:

cURL

curl -s https://api-inference.together.ai/v1/chat/completions \
  -H "Authorization: Bearer $TOGETHER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-project-slug/my-finetuned-endpoint",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

To stop billing when you’re done, delete the endpoint with rm --force, which also deletes its deployment:

CLI

tg endpoints rm "<ENDPOINT_ID>" --force

If you plan to use the endpoint again, scale the deployment to zero instead. This stops billing without deleting anything. For the full teardown order, see Delete resources.

SDK

The SDK has no single-call equivalent of the CLI’s deploy, so you run the same steps yourself: create the endpoint, bind the model and a config to a deployment, then route traffic to it. Reference the fine-tune by its Model Object ID (ml_..., the job’s model_object_id) inside a projects/{project_id}/models/{model_id} resource path. Only the CLI resolves model_object_name, so the SDK can’t take it.

Python

from together import Together

client = Together()
identity = client.whoami()
if identity.type != "project_api_key":
    raise RuntimeError("TOGETHER_API_KEY must be a project API key")
project_id = identity.project.id

# Reference the fine-tune by its Model Object ID (ml_...) and a config
# for the base model. List configs: tg models configs <model>.
model = f"projects/{project_id}/models/<MODEL_OBJECT_ID>"
config = f"projects/{project_id}/configs/<CONFIG_ID>"

# 1. Create the endpoint.
endpoint = client.endpoints.create(
    project_id=project_id,
    name="my-finetuned-endpoint",
)

# 2. Bind the model and config to a deployment.
deployment = client.endpoints.deployments.create(
    endpoint.id,
    project_id=project_id,
    name="prod",
    model=model,
    config=config,
    autoscaling={"min_replicas": 1, "max_replicas": 1},
)

# 3. Route 100% of traffic to the deployment.
client.endpoints.update(
    endpoint.id,
    project_id=project_id,
    traffic_split=[{"deployment_id": deployment.id, "weight": 1}],
)
print(endpoint.name)

Poll the deployment’s status.state until it reaches DEPLOYMENT_STATE_READY:

Python

deployment = client.endpoints.deployments.retrieve(
    deployment.id,
    project_id=project_id,
    endpoint_id=endpoint.id,
)
print(deployment.status.state)

Once the deployment is READY, pass the endpoint string (your-project-slug/my-finetuned-endpoint, returned as endpoint.name) as the model parameter and point the SDK’s base URL at https://api-inference.together.ai:

Python

from together import Together

# A dedicated client for inference: this base URL serves only
# inference, not the fine-tuning or files APIs.
inference_client = Together(base_url="https://api-inference.together.ai")

response = inference_client.chat.completions.create(
    model="your-project-slug/my-finetuned-endpoint",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(response.choices[0].message.content)

The SDK has no equivalent of the CLI’s rm --force either. To stop billing when you’re done, scale the deployment to zero, then delete the deployment and the endpoint once it reaches DEPLOYMENT_STATE_STOPPED:

Python

# Scale to zero to stop billing.
client.endpoints.deployments.update(
    deployment.id,
    project_id=project_id,
    endpoint_id=endpoint.id,
    autoscaling={"min_replicas": 0, "max_replicas": 0},
)

# Once it's DEPLOYMENT_STATE_STOPPED, drop its traffic weight, then
# delete the deployment and the endpoint.
client.endpoints.update(
    endpoint.id,
    project_id=project_id,
    traffic_split=[{"deployment_id": deployment.id, "weight": 0}],
)
client.endpoints.deployments.delete(
    deployment.id, project_id=project_id, endpoint_id=endpoint.id
)
client.endpoints.delete(endpoint.id, project_id=project_id)

If you plan to use the endpoint again, scale the deployment to zero and skip the deletes. This stops billing without deleting anything. For the full teardown order, see Delete resources.

UI

1. Open your models

   Go to My models. Your fine-tuned models appear here once training finishes.

2. Create the endpoint

   Select your fine-tuned model, then select Create dedicated endpoint.

3. Configure the deployment

   Choose a deployment configuration (hardware and GPU count) and set the replica bounds.

4. Deploy

   Select Deploy. Provisioning takes up to 10 minutes. You can navigate away while it runs.

5. Stop when done

   When you’re not using the endpoint, open it from the dashboard and stop or delete it to halt per-minute billing.

Run locally

To run your model outside Together, download the checkpoint by job ID:

CLI

tg fine-tuning download "<JOB_ID>"

Choose a checkpoint type

The checkpoint parameter selects what to download. If you omit it, the API picks a default automatically: merged for low-rank adapter (LoRA) jobs and model_output_path for full fine-tunes. You can also pass checkpoint_step, which downloads a specific intermediate step and overrides checkpoint. Valid values depend on how the job was trained.

Job type	Valid checkpoint values	What you get
LoRA fine-tune	merged, adapter, or model_output_path	merged combines the base model and adapter into self-contained weights, the usual choice for running the model locally or uploading it elsewhere. adapter returns only the LoRA adapter weights, so you can load them on top of the base model yourself (for example, with PEFT or vLLM).
Full fine-tune	model_output_path only	The full set of trained model weights. adapter returns an error for full fine-tunes.

model_output_path returns the raw training output directory before any merging. It works for both job types but is mainly useful for advanced workflows that need the unmodified artifacts. For LoRA jobs, prefer merged or adapter. For full fine-tunes, it’s the only option.

The output is a .tar.zst archive that uses ZStandard compression. On macOS, install zstd with Homebrew and decompress:

brew install zstd
cd my-model
zstd -d model.tar.zst
tar -xvf model.tar

You should see:

tokenizer_config.json
special_tokens_map.json
pytorch_model.bin
generation_config.json
tokenizer.json
config.json

Load the model with Hugging Face Transformers:

Python

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

tokenizer = AutoTokenizer.from_pretrained("./my-model")
model = AutoModelForCausalLM.from_pretrained(
    "./my-model",
    trust_remote_code=True,
).to(device)

input_ids = tokenizer.encode("Space robots are", return_tensors="pt").to(
    device
)
output = model.generate(input_ids, max_length=128, temperature=0.7)
print(tokenizer.decode(output[0], skip_special_tokens=True))

To download a specific checkpoint instead of the final one, pass --checkpoint-step <STEP_NUMBER> to tg fine-tuning download (or checkpoint_step=<STEP_NUMBER> to client.fine_tuning.content()). List checkpoints with tg fine-tuning list-checkpoints <JOB_ID>.

Model registry object IDs

When a job uploads its output artifacts to the Together model registry alongside the primary storage write, the job and checkpoint responses include the registry object and revision IDs, plus qualified object names in <project_slug>/<model_name> form. Use the IDs to reference the uploaded weights in downstream workflows, such as binding a fine-tuned adapter to a dedicated endpoint. The CLI’s deploy command also accepts the qualified name.

On a completed job in the fine-tuning jobs dashboard, the Output model field shows model_object_name and links to that model in the registry by model_object_id.

The object ID fields are omitted when the registry upload did not run or did not succeed. A name can still be present for a job whose upload never ran, but the CLI can’t resolve it, so treat the matching *_object_id as the signal that the artifact reached the registry. Object names are resolved on the fly on GET /fine-tunes/{id} and GET /fine-tunes/{id}/checkpoints only (not on list jobs). If the project slug cannot be resolved, the name fields fall back to the corresponding object ID.

On the job

GET /fine-tunes/{id} includes the final artifact IDs once the job reaches completed:

Field	Description
model_object_id	Registry object ID for the final model weights (for example, ml_…). Present when the job produced a model artifact and the upload succeeded.
model_object_revision_id	Registry revision ID for the final model weights (for example, rv_…).
model_object_name	Qualified registry name for the final model weights (for example, acme-corp/my-model-abc123). Pass it to tg endpoints deploy in place of model_object_id. The <model_name> part comes from model_output_name (the segment after / when namespaced, or the whole string otherwise).
adapter_object_id	Registry object ID for the final adapter weights on LoRA jobs.
adapter_object_revision_id	Registry revision ID for the final adapter weights on LoRA jobs.
adapter_object_name	Qualified adapter name on LoRA jobs (for example, acme-corp/my-model-abc123-adapter).

CLI

tg fine-tuning retrieve "<JOB_ID>"

On checkpoints

GET /fine-tunes/{id}/checkpoints returns the same ID pair on each checkpoint entry:

Field	Description
object_id	Registry object ID for that checkpoint’s artifact (for example, ml_…).
object_revision_id	Registry revision ID for that checkpoint’s artifact (for example, rv_…).
object_name	Qualified registry name for that checkpoint (for example, acme-corp/my-model-abc123-100 for an intermediate model checkpoint, or acme-corp/my-model-abc123-adapter for the final adapter).

Intermediate checkpoints carry the IDs from the upload at that step. Final model and adapter checkpoints in the list reuse the job-level IDs from the table above.

CLI

tg fine-tuning list-checkpoints "<JOB_ID>"

Troubleshooting

• The job has no model_object_id yet: The job hasn’t reached completed. Poll its status with client.fine_tuning.retrieve(id=...) until it’s done.
• endpoints_v1_create_access_disabled (HTTP 403): You’re calling the retired v1 endpoints API, for example with client.endpoints.create(...) in SDK 2.x or tg endpoints create in CLI 2.x. Upgrade together to 3.0.0 or later, then deploy with tg endpoints deploy.
• Multiple configs found for model when deploying: The model has more than one deployment profile. List its profiles with tg models configs "<MODEL_OBJECT_ID>", then re-run with --config <cr_...>.
• Model <MODEL_OBJECT_NAME> not found when deploying by name: Pass model_object_name in full, including the project slug, and check that the job reports a model_object_id. Without one, the weights never reached the registry, so neither identifier resolves. If the name still fails, deploy by model_object_id.
• Multiple models found for "<MODEL_OBJECT_NAME>": More than one model visible in your project has the same final name segment. Deploy by model_object_id, which is unambiguous.
• Either identifier fails with a key from another project: The model belongs to the project the job ran in, and both identifiers resolve only within your key’s project. Use a key scoped to the project that owns the model.
• Deploy fails because the base model isn’t supported: Not every base model can be hosted on dedicated model inference. Before training, confirm the base model appears in the supported models list. -Reference models often can’t be deployed.
• 404 on inference: Point the SDK’s base URL at https://api-inference.together.ai, or at https://api-inference.together.ai/v1 for OpenAI-compatible clients. Pass the endpoint string (your-project-slug/<endpoint_name>, printed by the deploy output) as the model parameter, not the Model Object ID.

Next steps

Upload a custom model

Upload your own model weights from outside the Together catalog.

Manage endpoints

Inspect, start, stop, update, and delete dedicated endpoints.

Configure autoscaling

Tune replica bounds and autoscale a deployment on the metric that fits your workload.