Amazon SageMaker AI Serverless Model Customization adds full fine-tuning (FFT); AWS reports 90.0% LLM-judge accuracy with 100,000+ examples
AWS has announced that SageMaker AI Serverless Model Customization now supports full fine-tuning (FFT), with no GPUs or clusters to manage. AWS compared FFT with LoRA on a code-generation task and warns that FFT is very sensitive to hyperparameters.
News date: About 6 min read
Contents
Fine-tuning means taking an already-trained large language model and continuing to train it on a company's own data so it performs better in a specific domain or task. In its official blog, AWS announced that Amazon SageMaker AI Serverless Model Customization now supports Full Fine-Tuning (FFT), letting teams update all of a model's parameters without managing infrastructure. LoRA (low-rank adaptation), the more common approach until now, trains only a small portion of the parameters and uses fewer GPU resources. The post does not say whether this is general availability or a preview, and gives no pricing or region information, so check AWS documentation. For teams that want a model to absorb domain knowledge without worrying about GPU capacity planning, this offers an option beyond LoRA. All figures below come from AWS's own experiments and have not been independently verified.
What actually changed? How do FFT and LoRA differ?
AWS explains that LoRA freezes the pretrained weights and inserts small low-rank matrices; when tasks become complex and data exceeds tens of thousands of examples, LoRA hits an accuracy bottleneck, which AWS attributes to insufficient rank. FFT updates the model's parameters all at once, which AWS says unlocks the full expressive capacity of the weight matrices.
AWS says that with this service you only specify the base model, the dataset and hyperparameters (values set before training, such as learning rate, batch size and number of epochs); compute provisioning and checkpoint management (saving snapshots of the model during training) are handled automatically.
| Dimension | LoRA | FFT (per AWS) |
|---|---|---|
| How it updates | Freezes original weights, trains only low-rank matrices | Updates the model's parameters all at once |
| Performance on large data | AWS says complex tasks with tens of thousands+ examples hit an accuracy bottleneck | With 100,000+ examples, 90.0% semantic accuracy under LLM-as-Judge |
| One LLM-as-Judge comparison | 81.5% | 79.5% |
| Risk | — | Too high a learning rate overwrites pretrained knowledge, causing catastrophic forgetting |
How did AWS test it, and how far can the results be trusted?
AWS used supervised fine-tuning (SFT, training on example instructions paired with target responses) on the Qwen3-4B base model offered in SageMaker AI Studio, with LoRA and FFT using the same prompts, targets and evaluation code. The dataset was Magicoder-Evol-Instruct-110K, 110,000 code instruction–response pairs, split 90/5/5 into 99,000 training, 5,500 validation and 5,500 held-out test examples. The Hugging Face dataset page also lists a train split of about 111k rows.
Evaluation used three methods built into the service: LLM-as-Judge with Amazon Nova Pro as the judge; PRIMECODE, which measures code similarity with BLEU and ROUGE; and MMLU, a general-knowledge benchmark covering 57 subjects, which AWS used to detect forgetting. AWS also says it ran dozens of experiments on Qwen3-4B, DeepSeek-R1-1.5B, Nemotron-30B and Gemma-4-26B, each taking roughly a few hours from submission to results, with the resulting models registered automatically in SageMaker Model Registry.
What should you watch for with hyperparameters and forgetting?
- Learning rate: in AWS's experiments on large datasets of over 100,000 examples, 2e-5 was in the safe zone.
- Epochs (one full pass over the training data): LLM-as-Judge rose to 90.0% at 3 epochs, while going beyond 3 epochs made every test variant worse.
- Batch size: at a fixed learning rate, smaller batches produce noisier updates, similar to raising the learning rate, and increase forgetting risk; larger batches may underfit.
- Preserving general knowledge: AWS says the baseline of 5% warmup, batch size 8 and learning rate 2e-5 best preserved general knowledge on MMLU.
How do you get started?
AWS recommends using the SageMaker Python SDK to configure and launch training and evaluation jobs programmatically; see the AWS documentation for the list of base models that support FFT. Below is AWS's FFT example from the post (content taken from the original, formatting lightly tidied):
from sagemaker.train.sft_trainer import SFTTrainer
from sagemaker.train.common import TrainingType
trainer = SFTTrainer(
model=base_model_id,
training_type=TrainingType.FULL,
model_package_group=model_package_group_name,
training_dataset=training_dataset,
validation_dataset=val_dataset,
s3_output_path=output_path,
sagemaker_session=sess,
role=role,
)
# Configure hyperparameters as needed
trainer.hyperparameters.learning_rate = 2e-5
trainer.hyperparameters.global_batch_size = 8
trainer.hyperparameters.max_epochs = 2
trainer.hyperparameters.lr_warmup_steps_ratio = 0.1
trainer.train(wait=False)AWS says that to use LoRA instead, you only change TrainingType.FULL to TrainingType.LORA; the rest of the code stays the same.
Frequently asked questions
Is this feature generally available, and what does it cost or where can I use it?
The AWS post only says SageMaker AI Serverless Model Customization "now supports" FFT. It does not say whether it is a preview and lists no pricing or available regions, so check the official AWS documentation before planning around it.
Is FFT always better than LoRA?
No. AWS says FFT performs better on complex tasks and large, high-quality datasets, but in one comparison LoRA scored slightly higher under LLM-as-Judge (81.5% vs. 79.5%), and FFT is very sensitive to hyperparameters.
What is catastrophic forgetting?
AWS explains that FFT modifies all of a model's parameters, so pushing too aggressively toward a narrow domain can make the model forget its broader original capabilities. AWS uses MMLU to track how much general knowledge is retained before and after fine-tuning.
Do I need to prepare my own GPUs?
According to AWS, no: using this service requires no GPU provisioning, cluster setup or distributed-training configuration, and the service handles compute and checkpoints automatically.
Browse the latest news in this topic
Found this guide helpful? Share it with someone who needs it
Claude Code 2.1.329: Code Mode Works With Telemetry, /tasks Lists Scheduled Tasks, macOS Computer Use Gets Stricter Safety Rules
Anthropic's changelog lists Claude Code 2.1.329, covering Code Mode behavior changes, new diagnostics in /tasks and /mcp, /code-review changes, and new safety rules for macOS computer use.
Kimi Code CLI v2.7.0 Adds a Native Amazon Bedrock Provider: Use Your AWS Credentials Instead of a Separate API Key
AWS Rounds Up 13 SageMaker Inference Features: Managed NVIDIA Dynamo on HyperPod, MIG GPU Partitioning, Cross-Region Routing
BMW Group Shares How It Governs AI Agents: 12,500+ Developers, Three Autonomy Levels and a Four-Stage CI/CD Gate
Sources
- Full Fine-Tuning (FFT) on SageMaker AI Serverless Model Customization | Artificial IntelligenceChecked:
- ise-uiuc/Magicoder-Evol-Instruct-110K · Datasets at Hugging FaceChecked:
Compiled from the references and verification dates listed above; fares, schedules, product features, and terms can change, so please confirm the latest details on the relevant official website before travelling, booking, or purchasing.