Running an LLM Locally on an AMD Ryzen AI NPU with FastFlowLM

2 min
Notes

Update (September 2026): FastFlowLM added Linux support in March 2026. This post documents my Windows test from August 2025; check the current FastFlowLM project for supported platforms and installation instructions.

I use a Copilot+ PC with an AMD Ryzen processor and an NPU. I had previously tried running an LLM with Lemonade Server, but it couldn’t use the NPU by itself; it only supported a mixed mode that still used the integrated GPU.

Recently, I saw a Reddit post about running LLMs exclusively on the AMD Ryzen AI NPU: Running LLMs exclusively on AMD Ryzen AI NPU. That prompted me to try FastFlowLM.

Like Lemonade, it only supported smaller models in my test. If you need to run a larger model, this may not be the right option. See the FastFlowLM model list for the models available there.

Installation

Download FastFlowLM

Download the ready-to-run installer from the FastFlowLM GitHub repository. At the time of my test, it appeared to support Windows only, and I used flm-setup.exe. The latest Windows installer is available from the official project.

Run the installer and follow its prompts.

Install the NPU driver

Make sure the NPU appears in Windows Task Manager or as a “Neural Processor” in Device Manager. If it doesn’t, you may need to install the AMD NPU driver.

Download AMD drivers

Set the NPU performance mode

For better performance, you may want to enable performance mode for the AMD NPU. Open a terminal and run:

cd C:\Windows\System32\AMD\
.\xrt-smi configure --pmode turbo

See the AMD XRT SMI documentation for more about NPU power modes.

Usage

FastFlowLM works much like Ollama. To check that it runs, open a command prompt and try:

flm run llama3.2:1b

Start with a small model. That way, you won’t waste time downloading a large one only to find out that your device can’t run it.

The CLI also supports these commands:

Usage: C:\Program Files\flm\flm.exe <command: run <model_tag> <file_name>
Usage: C:\Program Files\flm\flm.exe <command: serve <model_tag>
Usage: C:\Program Files\flm\flm.exe <command: pull <model_tag> [--force]
Commands:
  run     - Run the model interactively
  serve   - Start the Ollama-compatible server
  pull    - Download model files if not present
  help    - Show the help
  remove  - Remove a model
  list    - List all the models
  version - Show the version
Options:
  --force - Force re-download even if the model already exists

Further reading

I translated parts of the installation instructions from the FastFlowLM documentation and simplified them, adding a few notes of my own.

See the FastFlowLM documentation for more details.