Local AI on Mini PCs: A Simple Guide for AMD Systems

This article explains how to set up powerful AI models that run directly on affordable AMD mini PCs, without needing cloud services. It compares different hardware, gives clear instructions for installing all the necessary software, and shows how to start and use your own local AI server.

System Comparison

There are three main AMD mini PC configurations suitable for running large language models locally:

Budget System: AMD Ryzen 7 8745HS

• 8 CPU cores, 16 threads (up to 5.1 GHz)
• Integrated AMD Radeon 780M GPU (12 compute units)
• 64 GB DDR5-5600 RAM
• 8-16 GB BIOS-configurable VRAM; dynamically scales up to 32 GB for AI workloads
• Price: $600–$800 USD

Mid-Range System: AMD Ryzen AI 9 HX 370

• 12 CPU cores, 24 threads (up to 5.1 GHz)
• AMD Radeon 890M GPU (16 compute units)
• 128 GB DDR5-5600 RAM
• Configurable up to 64 GB VRAM through BIOS (options: 0.5 GB, 32 GB, or 64 GB)
• 2 TB PCIe 4.0 SSD storage
• Built on Zen 5 architecture with 4nm technology
• Price: $1,200–$1,600 USD

Premium System: AMD Ryzen AI Max+ 395

• 16 CPU cores, 32 threads (up to 5.1 GHz)
• AMD Radeon 8060S GPU (40 compute units)
• 128 GB LPDDR5X-8000 RAM
• Up to 96 GB shared VRAM through AMD Variable Graphics Memory (VGM)
• 2 TB SSD storage
• Price: $1,800–$2,500 USD

Here's a quick comparison:

Summary: The budget system is great for learning and personal use. The mid-range system with AMD Ryzen AI 9 HX 370 and 128 GB RAM offers excellent value for those who need more power than the budget option but don't want to spend premium prices—it's ideal for small businesses, content creators, and developers working with moderately large AI models.

The premium system with AMD Ryzen AI Max+ 395 is best for professionals or those needing to run very large AI models up to 120B parameters with maximum performance and 96 GB VRAM.

Hardware Setup

You need:

• Ubuntu 25.04 or 25.10 (for latest Vulcan drivers support)
• At least 50 GB of free disk space
• Internet connection for downloading files
• Terminal or command line access

For Ubuntu 25.04:

1. Update your system:
   bash

   sudo apt update && sudo apt upgrade -y
   

2. Install required tools:
   bash

   sudo apt install -y build-essential cmake git libvulkan-dev vulkan-tools mesa-vulkan-drivers python3 python3-pip jq curl
   

3. Check GPU support:
   bash

   vulkaninfo | grep deviceName
   

   You should see your AMD GPU listed.

Software Installation

Step 1: Download llama.cpp

cd ~
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp

Step 2: Build llama.cpp with GPU support

• For the Budget System:

  bash

  cd ~
  cd llama.cpp
  mkdir build && cd build
  cmake .. -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON -DGGML_NATIVE=ON -DCMAKE_C_FLAGS="-march=native -O3 -ffast-math" -DCMAKE_CXX_FLAGS="-march=native -O3 -ffast-math"
  cmake --build . --config Release -j$(nproc)
  

• For the Mid-Range System (AMD Ryzen AI 9 HX 370):

  bash

  cd ~
  cd llama.cpp
  mkdir build && cd build
  cmake .. -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON -DGGML_NATIVE=ON -DCMAKE_C_FLAGS="-march=native -O3 -ffast-math" -DCMAKE_CXX_FLAGS="-march=native -O3 -ffast-math"
  cmake --build . --config Release -j$(nproc)
  

• For the Premium System (AMD Ryzen AI Max+ 395):

  bash

  cd ~
  cd llama.cpp
  mkdir build && cd build
  cmake .. -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON -DGGML_NATIVE=ON -DGGML_AVX512=ON -DCMAKE_C_FLAGS="-march=native -O3 -ffast-math" -DCMAKE_CXX_FLAGS="-march=native -O3 -ffast-math"
  cmake --build . --config Release -j$(nproc)
  

If everything works, you'll see a message that says the server has been built.

Step 3: Download AI Models

• On Budget System (choose one model):

  bash

  cd ~/llama.cpp/models
  # Fast 20B model
  wget https://huggingface.co/unsloth/gpt-oss-20b-GGUF/resolve/main/gpt-oss-20b-Q4_K_M.gguf
  # OR Smart 30B model
  wget https://huggingface.co/unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF/resolve/main/Qwen3-30B-A3B-Instruct-2507-Q4_K_M.gguf
  

• On Mid-Range System (AMD Ryzen AI 9 HX 370):

  bash

  cd ~/llama.cpp/models
  # Start with 30B model
  wget https://huggingface.co/unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF/resolve/main/Qwen3-30B-A3B-Instruct-2507-Q4_K_M.gguf
  # Optional: Try a 50B model if you have 128GB RAM
  wget https://huggingface.co/unsloth/Qwen3-50B-Instruct-GGUF/resolve/main/Qwen3-50B-Instruct-Q4_K_M.gguf
  

• On Premium System (AMD Ryzen AI Max+ 395, you can get several):

  bash

  cd ~/llama.cpp/models
  wget https://huggingface.co/unsloth/gpt-oss-20b-GGUF/resolve/main/gpt-oss-20b-Q4_K_M.gguf
  wget https://huggingface.co/unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF/resolve/main/Qwen3-30B-A3B-Instruct-2507-Q4_K_M.gguf
  # For the biggest model (only for premium 395)
  wget https://huggingface.co/ggml-org/gpt-oss-120b-GGUF/resolve/main/gpt-oss-120b-Q4_K_M.gguf
  

Downloading can take between 20 and 60 minutes.

Starting the AI Server

Start the server to make the AI model available:

• On Budget System:

  bash

  cd ~/llama.cpp/build/bin
  ./llama-server -m ~/llama.cpp/models/Qwen3-30B-A3B-Instruct-2507-Q4_K_M.gguf -ngl 99 -cmoe -fa auto -c 16384 -ub 2048 -b 2048 -t 8 --host 0.0.0.0 --port 8080
  

• On Mid-Range System (AMD Ryzen AI 9 HX 370):

  bash

  cd ~/llama.cpp/build/bin
  ./llama-server -m ~/llama.cpp/models/Qwen3-30B-A3B-Instruct-2507-Q4_K_M.gguf -ngl 99 -cmoe -fa auto -c 24576 -ub 3072 -b 3072 -t 12 --host 0.0.0.0 --port 8080
  

• On Premium System (AMD Ryzen AI Max+ 395):

  bash

  cd ~/llama.cpp/build/bin
  ./llama-server -m ~/llama.cpp/models/Qwen3-30B-A3B-Instruct-2507-Q4_K_M.gguf -ngl 99 -cmoe -fa auto -c 32768 -ub 4096 -b 4096 -t 16 --host 0.0.0.0 --port 8080
  

Wait for a message like: llama_server: server listening on http://0.0.0.0:8080

• Use your web browser: http://localhost:8080
• Or API: http://localhost:8080/v1/chat/completions

Using the OpenAI-Compatible API

Here are some example queries you can try using the API from your terminal.

Simple Question

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
      "messages": [
        {"role": "user", "content": "Explain quantum computing in simple terms"}
      ],
      "max_tokens": 100
    }' | jq '.choices[0].message.content'

Budget system answers in ~8 seconds; mid-range system in ~5 seconds; premium in ~3.5 seconds.

Structured JSON Output

You can ask it to reply in a specific format:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
      "messages": [
        {"role": "system", "content": "Respond only in valid JSON format."},
        {"role": "user", "content": "Analyze sentiment: \"The product is amazing but expensive.\" Schema: {\"sentiment\": \"positive|negative|neutral\", \"score\": 0-1, \"reason\": \"string\"} JSON only:"}
      ],
      "max_tokens": 100,
      "temperature": 0.1
    }' | jq -r '.choices[0].message.content' | jq '.'

It will reply with something like:

{
  "sentiment": "neutral",
  "score": 0.6,
  "reason": "Positive product quality but negative price perception"
}

Performance Benchmarks

Testing setup:

• Using the 30B model, average speeds over 3 runs.

Budget system is good for smaller models.
Mid-range system with AMD Ryzen AI 9 HX 370 offers the best price-to-performance ratio for medium workloads and 30-50B models.
Premium system with AMD Ryzen AI Max+ 395 is needed for the heaviest workloads and biggest models (up to 120B parameters) with professional-grade speed.

Optimization Tips

• On Budget systems, use smaller models, set context to 12-16k, only run one model at a time.
• On mid-range systems with AMD Ryzen AI 9 HX 370, you can comfortably run 30B models with 24k context, or try 50B models with 128 GB RAM. Configure BIOS VRAM allocation to 64 GB for best results.
• On Premium systems with AMD Ryzen AI Max+ 395, you can use larger context windows (32k+), run multiple models, and even try parallel requests. The 395 can handle 120B models efficiently with 96 GB VRAM.
• For fastest responses set temperature low (e.g. 0.1) and reduce max_tokens.
• For higher quality, increase max_tokens and temperature.

Note about VRAM allocation: AMD integrated GPUs allocate shared memory through BIOS settings. The budget system can dynamically access up to 32 GB for AI workloads. The mid-range HX 370 can be configured to 64 GB through BIOS (with options: 0.5 GB, 32 GB, or 64 GB). The premium 395 supports up to 96 GB through AMD Variable Graphics Memory settings. You can configure these values in your system BIOS before deploying models.

Troubleshooting

• If your GPU is not detected, check drivers and Vulkan installation.
• If you get out-of-memory errors, use a smaller model or reduce the context length. You can also adjust BIOS VRAM allocation lower to free up system RAM.
• If responses are slow, check that the server uses your GPU (the terminal output should show layers loaded onto the GPU).
• If you get a port error, change the port number or end the process using it.
• If you can't connect from other devices, ensure the firewall allows port 8080.

Quick Setup Checklist

• Install Ubuntu 25.04
• Configure BIOS VRAM allocation (set to maximum for your system)
• Install build tools and Vulkan
• Clone and build llama.cpp
• Download a 20B or 30B model
• Start llama-server
• Test replies with curl or browser
• (Optional) Set up the server to start automatically when your computer boots

Final Recommendations

• The budget AMD mini PC (~$700) is best for beginners, learners, and personal use.
• The mid-range system with AMD Ryzen AI 9 HX 370 and 128 GB RAM (~$1,400) is the sweet spot for most users who want serious AI capabilities without breaking the bank—perfect for developers, content creators, and small teams. Set BIOS VRAM to 64 GB for optimal performance.
• The premium system with AMD Ryzen AI Max+ 395 (~$2,100) is for those who need professional speed, can afford more, or want to use the largest AI models up to 120B parameters with maximum performance and 96 GB VRAM.

With this guide, you have everything needed to set up and run local AI models on AMD mini PCs, using software that works just like OpenAI's API. Start experimenting and see what you can build!

Published on 11/9/2025