Deploy a checkpoint

Deploy a trained model or LoRA adapter from an inference checkpoint to dedicated model inference.

To serve a trained model, deploy one of its inference checkpoints to dedicated model inference. For production traffic, deploy it on its own deployment, optionally prepare it for FP4 inference before. A low-rank adapter (LoRA) can be merged into its base model when the deployment starts, so it serves at the base model’s speed and cost. It can also be served in multi-LoRA mode on an existing deployment for quicker experiments.

Find your model’s ID

Each inference checkpoint registers its weights in your project’s model registry, and you deploy by the registry ID. Read it from the checkpoint’s registration:

from together import Together

client = Together()
session = client.post_training.sessions.retrieve("<session_id>")
registration = session.inference_checkpoints[-1].registration
if registration is None:
    print("Not registered. Download the checkpoint instead.")
elif registration.model:
    print("Merged model:", registration.model.id)
elif registration.adapter:
    print("Adapter:", registration.adapter.id)
    print("Revision:", registration.adapter.revision_id)

To look up a checkpoint’s IDs without the session, see Find a checkpoint. From the terminal, tg training ls-checkpoints --type inference --session <session_id> lists each inference checkpoint’s registry ID in its Model column.

What you deploy depends on which ID your checkpoint registers:

• registration.adapter: A LoRA session registers only the adapter. Deploy the adapter on its own deployment, or attach it to a multi-LoRA deployment.
• registration.model: A full-weight session, or a LoRA session on custom base weights, registers a merged model. Deploy its ID directly.

Deploy a LoRA adapter in merged mode

To serve a LoRA adapter for production traffic, deploy it on its own deployment with --merge. Each replica merges the adapter into its base model’s weights when it starts, then serves the merged model at the base model’s speed and cost. There’s no merge job or upload.

Some models like GLM-5.3 and Kimi-K3 can only be served in FP4, so their adapters can’t deploy with --merge. Prepare them for FP4 inference instead.

Run tg beta endpoints deploy with the adapter’s ID:

tg beta endpoints deploy "<ADAPTER_ID>" \
  --endpoint my-trained-model \
  --merge \
  --config "<CONFIG_ID>" \
  --project "<PROJECT_ID>"

Parameter	Description
<ADAPTER_ID>	The adapter’s registry ID, registration.adapter.id.
--endpoint	A name for the new endpoint.
--merge	Serves the adapter merged into its base model. A LoRA adapter needs --merge, or --attach-adapter to attach it to a multi-LoRA deployment.
--config	A B200 profile of the serving base model whose adapter_mode is disabled. See Choose a config.
--project	Your project ID, from tg whoami. Pass it, or set TOGETHER_PROJECT_ID, even for your default project. Without either, the CLI resolves a private model ID to its base model and deploys that instead.

The merge adds to the time each replica takes to start, including replicas that autoscaling adds later. A new deployment can take 20 minutes or more to become ready. To deploy with the SDK, see Deploy the adapter.

Choose a config

List the serving base model’s profiles with tg beta models configs, passing the base model ID from this table:

Training base model	Serving base model	Base model ID
Qwen3.5 9B	Qwen/Qwen3.5-9B-BF16	ml_CbuqU8KKAtoGrowco4nqK
Qwen3.6 35B A3B	Qwen/Qwen3.6-35B-A3B-FP8	ml_CbSJsoXB4sHXmSRLUm9es
Qwen3.8 27B	Qwen/Qwen3.8-27B-FP8	ml_CeGmEZ22uUuWLHSKj43rJ

Pass --config the ID of a profile whose accelerator_type is nvidia-b200-180gb and adapter_mode is disabled. For Qwen3.6 35B A3B, cr_Ceh96pLTPcNatioB4YfHe is a single-B200 profile. The profile’s accelerator_count sets the GPUs per replica. If no B200 profile is listed, contact your account team for B200 serving support.

Adapters from a model’s single-session variant register against the same base model as the default model, so they deploy the same way.

Deploy LoRA adapters for FP4 inference

Dedicated model inference serves GLM-5.3 and Kimi-K3 in FP4. To serve an adapter trained on one of them on its own deployment, first run an FP4 preparation job. The job merges the adapter into the base model weights, then calibrates and quantizes the result to FP4. It registers the result as a private model in your project.

Estimate the job, then start it:

tg training fp4 estimate "<ADAPTER_ID>"
tg training fp4 create "<ADAPTER_ID>" --watch

estimate prints the job’s price and running time without creating anything. create --watch starts the paid job and prints the prepared model’s ID when it completes. A job runs for about 6 hours on GLM-5.3 and 18 hours on Kimi-K3, plus any wait for GPU capacity. Then deploy the prepared model by that ID.

For custom calibration data, job monitoring, cancellation, and the SDK, see Prepare for FP4 inference.

Deploy a merged model

Deploy a merged model, such as a full-weight checkpoint or an FP4-prepared model, with tg beta endpoints deploy:

tg beta endpoints deploy "<MODEL_ID>" \
  --endpoint my-trained-model \
  --config "<CONFIG_ID>" \
  --project "<PROJECT_ID>"

Parameter	Description
<MODEL_ID>	registration.model.id for a full-weight session or a LoRA session on custom base weights. For an uploaded or FP4-prepared model, that model’s ID.
--endpoint	A name for the new endpoint.
--config	A profile of the model whose adapter_mode is disabled. List the profiles with tg beta models configs "<MODEL_ID>". For a trained checkpoint, pick one whose accelerator_type is nvidia-b200-180gb. For an FP4-prepared model, use its FP4 profile. Required when the model has more than one profile.
--project	Your project ID, from tg whoami. Pass it, or set TOGETHER_PROJECT_ID, even for your default project. Without either, the CLI resolves a private model ID to its base model and deploys that instead.

deploy creates the endpoint and routes all traffic to the new deployment. The dedicated model inference quickstart walks through the rest of the flow, from choosing a config to calling the endpoint. Scale the deployment to zero or delete the endpoint when you’re done.

Serve the adapter on a multi-LoRA deployment

Instead of deploying a LoRA adapter on its own, you can attach it to a running deployment of its base model, if the base model supports multi-LoRA deployment. Attaching takes about a minute, with no extra job or upload. Each request does extra work for the adapter, and a deployment holds a limited number of adapters. Use it to compare candidates and to serve adapters with low or mixed traffic. For production traffic, deploy the adapter on its own deployment.

Multi-LoRA deployment covers attaching adapters with the CLI or the console, calling them, and comparing candidates.

Next steps

Multi-LoRA deployment

Attach candidate adapters to one deployment and compare them per request.

Prepare for FP4 inference

Merge and calibrate a GLM or Kimi-K3 adapter as one FP4 model.

Dedicated model inference quickstart

The CLI and SDK deploy flow, step by step.

Manage checkpoints

Save the inference checkpoints you deploy.