Skip to content

Fine-tuning LLMs Locally with Unsloth Studio: A Complete Guide

Introduction

Fine-tuning allows you to take a small open-source language model and specialize it for your specific use case, often outperforming models 100 times larger. However, until recently, fine-tuning came with two major challenges: creating a quality dataset was time-consuming, and the training process itself was difficult to set up on local hardware.

This guide covers Unsloth Studio, an open-source tool that solves both problems. You'll learn how to:

  • Install and configure Unsloth Studio on your local machine
  • Choose and download a base model for fine-tuning
  • Prepare a dataset (either from Hugging Face or your own custom data)
  • Run a fine-tuning session locally
  • Create your own dataset from a PDF document

What is Unsloth Studio?

Unsloth Studio is a free, open-source desktop application developed by former NVIDIA engineers. It provides:

  • A graphical interface for fine-tuning LLMs
  • Built-in dataset creation tools (called "recipes")
  • The ability to chat with AI models locally
  • Support for running models on consumer hardware (M-series Macs, Windows/Linux PCs with GPUs)
  • Integration with Hugging Face for model and dataset access

Prerequisites

Before beginning, ensure you have:

  • A computer with at least 16 GB RAM (32 GB+ recommended for 7B-9B models)
  • Python 3.8 or higher installed
  • At least 20 GB free disk space for models and datasets
  • (Optional) A Hugging Face account for accessing models and datasets
  • (Optional) An Open Router or similar API key if you want to create datasets using large cloud models

Installation Guide

Step 1: Run the Installer

  1. Visit the Unsloth Studio quickstart guide
  2. Copy the one-liner installer command for your operating system
  3. Open a terminal (Command Prompt on Windows, Terminal on macOS/Linux)
  4. Paste and execute the command

Example (macOS/Linux): 

pip install unsloth-studio

Example (Windows): 

pip install unsloth-studio

The installer will download all dependencies automatically.

Step 2: Launch the Studio

After installation completes, you'll see output indicating the server has started. Run:

unsloth-studio

The terminal will display something like: 

Starting Unsloth Studio on localhost:8888

Open your web browser and navigate to: 

http://localhost:8888

Step 3: Set a Local Password (First Run Only)

The first time you access the interface, you'll be prompted to set a local password. This secures your instance from other devices on the same network. Enter any password you'll remember—this is not an online account.

Understanding Model Formats

Before fine-tuning, it's crucial to understand the difference between model formats:

Format Purpose Use for
SafeTensors Full, uncompressed model Fine-tuning (training)
GGUF Compressed, single-file model Inference (chatting) only

For fine-tuning, you must select the SafeTensors version of a model. GGUF models are optimized for running locally but cannot be trained.

Choosing a Base Model

Option A: Browse Hugging Face

  1. Go to huggingface.co/models
  2. Search for "unsloth" to find optimized versions of popular models
  3. Recommended models for local fine-tuning:
  4. Small (4B-9B): unsloth/Qwen3.5-9B (runs on most modern laptops)
  5. Medium (12B-27B): unsloth/Qwen3.6-27B (requires 32 GB+ RAM)
  6. Large (32B+): Requires dedicated GPU or cloud instance

Option B: Use Artificial Analysis

  1. Visit artificialanalysis.ai
  2. Filter by "Open Source" models
  3. Look at the "Small Models" category (4B-40B parameters)
  4. Compare intelligence scores and hardware requirements

Step-by-Step Fine-Tuning Process

Step 1: Navigate to the Train Tab

In the Unsloth Studio interface, click the Train tab on the left sidebar.

Step 2: Select a Base Model

  1. Under "Model Type," choose Hugging Face Model
  2. Enter the model name in the format unsloth/model-name
  3. Example: unsloth/Qwen3.6-27B
  4. (Optional) Use "Local Model" if you already have a SafeTensors model downloaded

Step 3: Choose a Dataset

You have two options:

Option A: Use a Hugging Face Dataset

  1. Search for datasets at huggingface.co/datasets
  2. Recommended for beginners: Finance-Alpaca or similar domain-specific datasets
  3. Copy the dataset name (e.g., lucasmorin/finance-alpaca)
  4. Paste into the dataset field in Unsloth Studio

Option B: Use Your Own Custom Dataset (see "Creating Custom Datasets" below)

Step 4: Configure Training Parameters

  1. Context Length: Start with 1024 to reduce compute requirements
  2. Training Hyperparameters:
  3. Batch Size: 1 (for small runs, increase for larger)
  4. Steps: 20-100 (quick test), 1000+ (serious fine-tuning)
  5. Learning Rate: Default is usually fine
  6. Method: Keep on "CULARA" (optimized for local hardware)

Step 5: Start Training

  1. Click Start Training in the right panel
  2. Monitor progress in the Current Run tab
  3. Watch the Training Loss—it should decrease over time, indicating learning
  4. To check history, click the History tab

[!NOTE] Training loss decreasing means the model is learning the dataset. If loss is not decreasing or increases, check your dataset quality or reduce the learning rate.

Step 6: Save and Use Your Fine-Tuned Model

  1. After training completes, the new model is saved locally
  2. You can now chat with it using the Chat tab
  3. To reuse it, select "Local Model" in future fine-tuning sessions

Creating Custom Datasets from PDFs

This is one of Unsloth Studio's most powerful features—creating datasets from a single PDF document.

Step 1: Navigate to Recipes

Click the Recipes tab on the left sidebar.

Step 2: Create a New Recipe

  1. Click New Recipe
  2. Select Start from Learning Recipe
  3. Choose PDF Document QA

This opens a workflow with four main nodes: - Provider 1: API endpoint configuration - Provider Column LM Structure: Output format settings - Document File: Your PDF source - Output: Generated dataset

Step 3: Configure the Provider

  1. Click Provider 1 to configure the API
  2. Enter the API endpoint (e.g., https://openrouter.ai/api/v1)
  3. Ensure you include /v1 at the end
  4. Enter your API key
  5. For Open Router: Get key from their dashboard
  6. For local models: Use http://localhost:8080/v1 with no key

Step 4: Select the Model for Dataset Generation

  1. Click Provider Column LM Structure
  2. Under "Model," enter the model you want to use
  3. Recommended: deepseek/deepseek-v4-pro (good balance of quality and cost)
  4. Alternatives: gemini-3.5-pro or claude-sonnet-4.6
  5. Click Save in the top right

[!WARNING] Using large cloud models (like GPT-4 or Opus) for dataset generation can be expensive. DeepSeek V4 Pro offers excellent quality at a fraction of the cost.

Step 5: Upload Your PDF

  1. Click Document File > Configure
  2. Click Select File and choose your PDF
  3. Any domain-specific document works: financial reports, legal documents, SOPs, or business records
  4. Click Download to confirm upload

Step 6: Test the Recipe

  1. Click Check in the top toolbar
  2. Set Number of Records to 5 (test run)
  3. Click Start Test Run
  4. Monitor progress in the Logs tab
  5. When complete, click Chunk Test to view the generated Q&A pairs

Step 7: Run Full Dataset Generation

  1. Return to the recipe editor
  2. Set a Name (e.g., "Nvidia Financials 2026")
  3. Set Number of Records to 100, 500, or 1000
  4. Click Start Full Run
  5. Monitor progress—this may take several minutes depending on PDF length and chosen model

Step 8: Use Your Custom Dataset for Fine-Tuning

  1. Go back to the Train tab
  2. Under Dataset, select Local
  3. Your newly created dataset should appear in the list
  4. Select it and proceed with fine-tuning as described above

Best Practices

Dataset Size Guidelines

Use Case Minimum Records Recommended
Quick test 50 100
Domain specialization 300 1000+
High-quality production 1000 5000+

Model Selection Tips

  • First time: Use a 7B-9B model (runs on most hardware)
  • Better quality: Use a 27B model (requires 32+ GB RAM)
  • Best performance: Use a 70B+ model (requires cloud GPU)

Cost Optimization for Dataset Creation

  • Use DeepSeek V4 Pro or Flash models for cheap generation
  • Start with a test run of 5-10 records to verify quality
  • Batch requests to minimize API calls
  • Consider using a local model if privacy is critical

Troubleshooting

Common Issues

Issue Solution
Model fails to load on Apple Silicon Switch to a smaller model (7B-9B)
Training loss is not decreasing Reduce learning rate or check dataset quality
API call fails during dataset generation Verify API key and endpoint URL (include /v1)
"Metal allocation" error This is a known macOS bug; use a smaller model
Out of memory Reduce batch size or context length

Error Messages

Metal Allocation Error (Apple Silicon) 

This is a known bug in the MLX framework on Apple Silicon M-series chips.
Workaround: Use a 7B-9B model instead of 27B+

API Key Invalid 

Verify your API key is correct and has sufficient credits.
Check that the endpoint URL includes "/v1" at the end.

Conclusion

You now have the complete workflow for fine-tuning open-source LLMs locally:

  1. Install Unsloth Studio with a single command
  2. Select a base model from Hugging Face
  3. Choose or create a dataset (using public datasets or your own PDFs)
  4. Train your model locally with customized parameters
  5. Use your fine-tuned model immediately for domain-specific tasks

This skill—creating custom datasets and fine-tuning models on your own hardware—puts you in a small percentage of AI practitioners who can truly customize AI for their specific needs. Start with a small test run to get comfortable, then scale up with larger datasets and more training steps for production-quality results.

Resources