> ## Documentation Index > Fetch the complete documentation index at: https://docs.together.ai/llms.txt > Use this file to discover all available pages before exploring further. # Supervised fine-tuning > Train a model on example conversations with supervised fine-tuning. Supervised fine-tuning (SFT) trains a model to imitate example responses. Use it when you have high-quality demonstrations of the behavior you want: human-written answers, a target output format, or conversations from a system that already works. The [SFT quickstart](/docs/post-training/quickstart-sft) walks through a complete run. This page explains SFT on the post-training service, including data preparation, custom training loops, and continuing an adapter with GRPO. [Fine-tuning jobs](/docs/fine-tuning/supervised) also run SFT, with a fixed data format and no training loop to write. To choose between them, see [Post-training vs. standard fine-tuning](/docs/post-training/overview#post-training-vs-standard-fine-tuning). ## How post-training runs SFT SFT runs on a trainer-only model resource. Nothing samples from the model, so the resource runs with `num_generator_replicas=0`. Each step runs two operations: 1. `forward_backward` with `LOSS_TYPE_CROSS_ENTROPY`, which computes the weighted next-token cross-entropy over the batch and accumulates gradients. 2. `optim_step`, which applies the accumulated gradients with the step's learning rate. The next step trains directly on the updated weights. ## Run an SFT recipe The cookbook's `SFTConfig` and `train` function handle dataset loading, tokenization, loss masks, the training loop, and checkpoints. This example trains Qwen3.8 27B on the No Robots dataset, with every other setting at its default: ```python theme={null} from post_training_cookbook.sft.config import SFTConfig from post_training_cookbook.sft.train import train config = SFTConfig( model="Qwen/Qwen3.8-27B", enable_thinking=False, dataset_name="HuggingFaceH4/no_robots", max_steps=200, wandb_project="sft-no-robots", ) train(config) ``` `train` provisions a trainer-only model resource for `model`, attaches one session to it, and stops both when it returns. To train on a resource that's already running, set `model_resources_id` instead of `model`. See [Manage model resources](/docs/post-training/model-resources). The config sets `enable_thinking=False` because No Robots conversations contain no reasoning, as [Data for reasoning models](#data-for-reasoning-models) explains. The config also sets `wandb_project`, so `train` logs to Weights & Biases. Export `WANDB_API_KEY` before you run it, or set `WANDB_MODE=offline` to log locally. Without either, the run fails after it provisions the model resource, or, in an interactive terminal, waits at a W\&B login prompt while the resource bills. Settings worth adjusting first: * **`num_epochs` and `max_steps`:** `train` makes `num_epochs` passes over the dataset, one by default, and reshuffles the rows before each pass. `max_steps` caps the total number of steps, which keeps a test run short. Its default, `None`, sets no cap, so a config without it trains on every full batch: about 1,187 steps for No Robots at a `batch_size` of 8. The learning rate decays linearly toward zero over the run. * **`learning_rate`:** The right rate depends on the model and the task. For a low-rank adapter (LoRA) run, `1e-4` to `5e-4` is typical. Full-weight training needs a lower rate. The cookbook's No Robots recipes use `2e-5` for full-weight training and `1e-4` for LoRA on the same model. * **`train_on_what`:** `TRAIN_ON_ALL` trains on every assistant turn. `TRAIN_ON_LAST` trains on only the final one, which suits multi-turn datasets where only the last response matters. * **`batch_size`:** Larger batches (16 to 32) produce smoother loss curves. * **`use_lora=False`:** Trains the full weights instead of an adapter. Not every model supports full-weight training, so check [Supported models](/docs/post-training/supported-models) first. See [LoRA vs. full-weight training](/docs/post-training/training-methods#lora-vs-full-weight-training). The [cookbook reference](/docs/post-training/cookbook#sftconfig) lists every `SFTConfig` field. ## Prepare your data The post-training service receives tokenized `Sample` objects containing input tokens, next-token targets, and loss weights. Your code controls data loading, preprocessing, tokenization, and which tokens train, so any dataset schema works. For the cookbook recipe, select your data with `dataset_name`, `dataset_config`, and `dataset_split`. Its default mapping reads a `messages` column, a list of `{"role": ..., "content": ...}` dicts. For other schemas, pass `to_messages` to map each row to its messages. Put any instruction you want the model to learn, such as an answer format, in that mapping: ```python theme={null} def qa_messages(row): return [ {"role": "user", "content": row["question"]}, {"role": "assistant", "content": row["answer"]}, ] train(config, to_messages=qa_messages) ``` The recipe loads data with Hugging Face `datasets`, so local files work too. For example, save one row per line in `./my_data/train.jsonl` and set `dataset_name="./my_data"` with `dataset_split="train"`. Relative paths resolve from your working directory. `train` checks the first row with `to_messages` before provisioning a model resource or session, so schema errors fail before billing. ### Data for reasoning models When the cookbook renders your conversations, match the model's chat template and thinking settings to your data: * **Qwen data without reasoning:** Set `enable_thinking=False`, as the No Robots example does. Thinking is on by default for the catalog's Qwen models. Leaving it on makes prompt and full-conversation rendering disagree, so the recipe skips the rows and fails after provisioning the resource. The GLM-5.3 codec accepts these rows with thinking on and rejects `enable_thinking=False`. * **Data with reasoning:** Put assistant reasoning in `reasoning_content` alongside `content`. Qwen3.8 27B's template reads reasoning only from that field, so the recipe skips rows with `` tags inside `content`. `preserve_thinking` controls whether earlier turns retain reasoning. See [Thinking settings](/docs/post-training/cookbook#thinking). * **Consecutive GLM assistant turns:** Merge or split them before training. GLM closes each turn with the next message's role marker, so consecutive assistant turns have no stop token and the recipe skips those rows with a warning. ## Loss masks Choose which tokens contribute to cross-entropy with per-target `float32` weights: `0` excludes a token, `1` trains it fully, and fractional values reduce its contribution. The cookbook's `render_conversation` uses `0` for prompt content and role headers, including the assistant header, and `1` for selected assistant content and end-of-turn tokens. Training on end-of-turn tokens teaches the model when to stop. `train_on_what` selects all assistant turns or only the last one. For custom weighting, supply your own mask to `build_cross_entropy_sample`. [Data and samples](/docs/post-training/concepts#data-and-samples) explains token alignment. ## Write your own SFT loop To customize data loading, preprocessing, tokenization, batching, or loss weights, write your own training loop and submit samples to the post-training service. This example renders an in-memory list of conversations with the cookbook's chat-template codec. Replace that preparation with your own sample construction as needed: ```python theme={null} from post_training_cookbook.codecs import get_codec from post_training_cookbook.samples import ( build_cross_entropy_sample, is_trainable, ) from post_training_cookbook.sft.train import TRAIN_ON_ALL, render_conversation from together.post_training import ( AdamParams, ComputeConfig, LoraConfig, LossConfig, ModelResourcesClient, ) MODEL = "Qwen/Qwen3.8-27B" BATCH_SIZE = 8 conversations = load_conversations() # a list of message lists # Load the model's chat template, with thinking off for data without reasoning codec = get_codec(MODEL, enable_thinking=False) # Provision a trainer-only model resource (SFT never samples) with ModelResourcesClient.create( base_model=MODEL, compute_config=ComputeConfig( gpu_type="B200-SXM", num_generator_replicas=0 ), ) as resources: # Start a training session that trains a LoRA adapter with resources.create_session( lora_config=LoraConfig(rank=16, alpha=16) ) as session: # Each step trains on the next BATCH_SIZE conversations for step, start in enumerate(range(0, len(conversations), BATCH_SIZE)): # Render each conversation into tokens and a loss mask # that trains only the assistant turns samples = [] for messages in conversations[start : start + BATCH_SIZE]: for seq in render_conversation( messages, codec=codec, train_on_what=TRAIN_ON_ALL, max_length=4096, ): # Skip sequences that truncation left with no trained token if not is_trainable(seq.loss_mask): continue samples.append( build_cross_entropy_sample( tokens=seq.tokens, loss_mask=seq.loss_mask ) ) # Compute the cross-entropy loss and accumulate gradients result = session.trainer.forward_backward( samples=samples, loss=LossConfig(type="LOSS_TYPE_CROSS_ENTROPY"), ) # Update the adapter's weights session.trainer.optim_step( adam_params=AdamParams(learning_rate=1e-4, weight_decay=0.1), ) # The loss is summed over the trained tokens, not averaged print(f"[step {step}] loss_sum={result.loss:.1f}") # Save the adapter as an inference checkpoint you can deploy checkpoint = session.create_inference_checkpoint() print(checkpoint.id) ``` `render_conversation` returns a list of sequences, not a single one, because it splits a conversation wherever the chat template [strips earlier reasoning](/docs/post-training/concepts#reasoning-in-conversation-history). `build_cross_entropy_sample` shifts the tokens into input and target positions for you. For cross-entropy, `result.loss` is the sum over all trained positions, not a mean. Divide it by the batch's total weight to get the per-token mean the recipes log. To train with a variant of cross-entropy, such as one with an added regularization term, [write a custom loss function](/docs/post-training/custom-loss-functions). Each step then uses `forward` and `custom_forward_backward` instead of `forward_backward`. ## Chain SFT and GRPO To continue an SFT adapter with [GRPO](/docs/post-training/grpo), hand it over with a training checkpoint. [Combine methods](/docs/post-training/training-methods#combine-methods) explains when this helps. The handoff takes three settings: 1. Set `training_checkpoint_interval` so the SFT run writes a training checkpoint at its last step. The cookbook writes one only when the interval fires, and `train` returns the latest checkpoint ID (or `None`). 2. Start the GRPO run with `resume_from_checkpoint_id` set to that ID and `load_optimizer=False`. The new session loads the adapter weights with a fresh optimizer and its step counter at zero. 3. Use the same LoRA rank and alpha in both phases. A checkpoint with a different rank fails to load, and a different alpha silently changes how strongly the adapter applies. Because SFT is trainer-only and GRPO needs a generator, the two phases run in separate sessions, typically on a new model resource for the same base model: ```python theme={null} from dataclasses import replace from post_training_cookbook.datasets.registry import DATASETS from post_training_cookbook.grpo.config import GRPOConfig from post_training_cookbook.grpo.train import train_single_turn from post_training_cookbook.sft.config import SFTConfig from post_training_cookbook.sft.train import train as sft_train MODEL = "Qwen/Qwen3.8-27B" def gsm8k_messages(row): # Pose each question the way the GRPO phase does, so SFT teaches # the "#### " format that the GRPO reward reads. return [ {"role": "user", "content": DATASETS["gsm8k"].format_question(row)}, {"role": "assistant", "content": row["answer"]}, ] sft_config = SFTConfig( model=MODEL, dataset_name="openai/gsm8k", dataset_config="main", max_steps=20, enable_thinking=False, lora_rank=16, lora_alpha=16, training_checkpoint_interval=20, ) grpo_config = GRPOConfig( model=MODEL, dataset="gsm8k", enable_thinking=False, num_steps=20, max_model_length=1536, max_prompt_length=768, max_sample_tokens=768, lora_rank=16, lora_alpha=16, ) checkpoint_id = sft_train(sft_config, to_messages=gsm8k_messages) train_single_turn( replace( grpo_config, resume_from_checkpoint_id=checkpoint_id, load_optimizer=False, ) ) ``` GSM8K rows have `question` and `answer` columns, not `messages`, so the SFT phase maps each row with `to_messages`. The mapping poses each question in the GRPO prompt format, so SFT teaches the answer format that GRPO then scores. The cookbook's `recipes/sft_then_grpo.py` is the complete worked example. Unlike `SFTConfig`, `GRPOConfig` logs to Weights & Biases by default, so export `WANDB_API_KEY` or `WANDB_MODE=offline`, or pass `wandb_project=None` to `GRPOConfig`, before you run this example. ## Next steps Train on preference pairs with a custom loss function. Continue the adapter with reinforcement learning. Every `SFTConfig` field and its default. Deploy the trained model on dedicated model inference. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.