9 Best Local AI Tools for Raspberry Pi (Run LLMs on Pi 5)
It’s now possible to run more AI models on the Raspberry Pi than ever before. All you need are the right software tools that support the Pi’s architecture. I’ve gathered a few of the best ones, and today, I’ll share them with you.
AI tools can be installed on the Raspberry Pi 5 to run large language models (LLMs). They can run completely local, without an internet connection, to keep data private. These AI models can be used for text reasoning, audio processing, or image generation.
To run these models efficiently, you’ll probably want a Raspberry Pi 5 with at least 4GB of memory; the 8GB and 16GB boards open up even more options. Without further ado, let’s take a tour of what’s possible right now.
If you’re looking for inspiration for your next Raspberry Pi project, I’ve put together a list of 75+ ideas with full descriptions, difficulty ratings, and links to tutorials. Whether you’re a beginner or more advanced, there’s something here for you. Grab the list for free here!
Ollama
The most user-friendly AI tool.
If you’re new to self-hosting AI, you have to check out Ollama. It can do a little of everything, and it’s the most accessible tool to install and use on this list.
Ollama is an open-source platform that lets you run LLMs locally. You can download AI models for a wide range of general-purpose tasks, including text reasoning, code assistance, and even role-playing. Ollama preconfigures sane defaults for you, so you can instantly start playing with AI without having to tinker with code or manage complex configurations.
With just a few commands on your Raspberry Pi, you’ll have it up in no time. You can then chat with your AI via a web browser, just like you’re used to, except locally on your Pi. Check out my full guide here: How to Run ChatGPT Locally on Raspberry Pi with Ollama.
Models supported:
- gpt-oss
- DeepSeek
- Gemma
- OLMo
- Gemini
- Ministral
- Qwen
- Phi
- +over 200 models in the Ollama library.
OnnxStream
Create AI images.
You can get answers from real experts in minutes.
Get help with your setup
AI image generation tools typically require a GPU and a substantial amount of VRAM, making it something that didn’t seem feasible with Raspberry Pi hardware. That is, until now.
Bonus tip: When I started, the hardest part wasn’t the setup, it was figuring out what to build next. I ended up creating a list of 75+ project ideas with links and difficulty, so it’s easier to just pick something and try it. Grab the list here.
If you want a clearer path than jumping from tutorial to tutorial, my book gives you a step-by-step roadmap to really understand your Raspberry Pi.
Check the book here
OnnxStream is an open-source project that can perform image generation on a Raspberry Pi Zero 2. That means it can create AI images locally with only the CPU and RAM. It’s quite slow on the Pi Zero 2, but you can speed things up by using a Raspberry Pi 5 instead.
To install OnnxStream, you’ll probably have to git clone the repository and build the project manually. This guide from Pye Sone Kyaw may be helpful if you want more details.
After that, if you want to interact with the image creator from a web browser, there’s an unofficial project called OnnxStreamGui that you can install to make things more user-friendly.
You might also like: Turn your Raspberry Pi into a money-making machine
Models supported:
- Stable Diffusion 1.5
- Stable Diffusion XL 1.0 Base
- Stable Diffusion XL Turbo 1.0
Faster Whisper
Convert speech-to-text (STT).
Automatic speech recognition can be quite handy when you need to transcribe recorded meetings, add subtitles to videos, or translate audio to different languages. For this use case, the most popular tool at the moment is Whisper, a speech-to-text (STT) model.
There are a few Whisper variants, so let’s clear things up real quick. The original Whisper was made by OpenAI, whisper.cpp is an optimized version based on C/C++, and Faster Whisper is an optimized version coded in Python.
Faster Whisper is the highest performing STT model among this bunch on the Raspberry Pi 5. It can batch multiple audio inputs at the same time while still maintaining lower latency and using fewer resources than the others we mentioned.
I recommend installing Faster Whisper with a Docker image like this one, using a Docker Compose configuration that you can modify on the fly.
Models Supported:
- Whisper
Piper
AI text-to-speech (TTS) has never sounded so good.
We covered speech-to-text, but what about the other way around, converting the written word to audio? If you have a project in mind where you’d like your Raspberry Pi to speak out loud, then you’ll want an AI text-to-speech (TTS) model.
Piper is a TTS model that includes over 100 voices in more than 40 languages. It’s Python-based, which means you can run it on a Raspberry Pi using a virtual environment. You can also train Piper by feeding it your own voices to get something more unique or more realistic-sounding.
I think the best thing about Piper is the many different ways it’s supported and extended. You can run it through a web server or an API. It has a Home Assistant integration to give a voice to your smart home. There’s even a plugin for Old School RuneScape.
Models Supported:
- Piper
Bitnet.cpp
Ultra-efficient AI models on edge devices.
The 1-bit LLM emerged after research was published by Microsoft in 2024. The researchers found that a 1-bit design can run huge models (100b parameters) on CPUs while still getting nearly the same performance as traditional methods that use GPU processing.
You might also like: 15 Easy Projects for Raspberry Pi Beginners
This is a game-changer: it could lead to running complex AI on edge devices like the Raspberry Pi in the future, which wasn’t possible before with its hardware limitations.
Bitnet.cpp is the tool to use to load 1-bit LLMs on the Raspberry Pi 5. It’s rather new, so the installation is somewhat involved. You’ll have to build from source, compile with Clang, and then run from a Python virtual environment. This guide from Adafruit might help.
We’re in cutting-edge territory here, so there are only a few AI models supported at the moment. But I’m betting there are many more in development as we speak.
Models supported:
- bitnet_b1_58-large
- bitnet_b1_58-3B
- Llama3-8B-1.58-100B-tokens
- Falcon3 (1B / 3B / 7B / 10B)
- Falcon Edge (1B / 3B)
LocalAI
An AI agent platform.
The generic name of this project (LocalAI) makes for a terrible time searching for guides. But this tool deserves a mention because it’s the only one on this list that’s both geared toward producing AI agents and can run on the Raspberry Pi.
LocalAI is a framework to build AI agents that operate locally. It’s like making multiple bots that work together to accomplish a bigger goal. For example, if one of your customers emails you about a common problem, you can have one AI agent identify what’s wrong, a second research possible answers from your FAQ, and a third agent automatically reply with the solution.
Sounds too good to be true? Yep, I would add some caveats:
- Agentic AI performance on the Raspberry Pi will probably be quite slow.
- Creating AI agents is an advanced project meant for experienced AI users only.
- At the moment, no clear guides exist yet for the Raspberry Pi.
Even with these obstacles to overcome, I still think LocalAI is moving what’s possible on the Raspberry Pi forward. To install LocalAI, you’ll probably want to run it via Docker image here. It’s also built to support Kubernetes (read our brief intro here), so you can run it on a Raspberry Pi cluster to speed things up.
Models Supported:
- vLLM
- transformers
- langchain-huggingface
- bark
- coqui
- kokoro
- chatterbox
- kitten-tts
- vibevoice
- stablediffusion.cpp
- transformers-musicgen
- local-store
- +complete model compatibility list here.
Related: Can Raspberry Pi Run an AI Assistant? I Tested OpenClaw
llama.cpp
Optimized AI performance through customization.
We put Ollama at the top of this list earlier because it’s the easiest and has user-friendly defaults. But what if you need your AI model to perform at maximum capability? In that case, you might turn to llama.cpp for advanced customization.
Bonus tip: When I started, the hardest part wasn’t the setup, it was figuring out what to build next. I ended up creating a list of 75+ project ideas with links and difficulty, so it’s easier to just pick something and try it. Grab the list here.
Llama.cpp is an open-source C/C++ library that can run AI models cross-platform—stripped down without a GUI—so you can interact with it via code. We’ve covered llama.cpp before as a way to make your own local chatbot without fees: 5 Easy AI Projects on Raspberry Pi for Beginners.
The llama.cpp project supports over 150,000 AI models and counting. Which of these you can actually run depends on your Pi’s hardware capabilities, but the point is that llama.cpp is the most expansive tool on this list, if you’re willing to put in the work.
Models Supported:
- LXTV2
- LFM
- Qwen
- Z-Image
- Nemotron
- Youtu-LLM
- +over 150,000 models here.
Vosk
Lightning-fast AI speech recognition.
We’ve already covered AI audio tools for STT and TTS above, but this one’s got a niche use on the Raspberry Pi that’s worth mentioning.
Vosk can do offline speech recognition for over 20 spoken languages. Vosk excels with voice commands, making it perfect for projects like voice assistants on the Raspberry Pi.
It achieves this speed by using small models (~50MB parameters) that are low latency with very little CPU/RAM usage. Other AI audio models may be more accurate overall, but Vosk is much faster, making it perfect for these kinds of use cases.
The most straightforward way you can install Vosk is to use Python with pip on the Raspberry Pi. More details can be found on their official documentation.
Models supported:
- vosk-model-small
- vosk-model
- vosk-model-lgraph
- vosk-model-gigaspeech
- +support for over 20 languages here.
Want even more ideas? I put together a free resource with over 75 Raspberry Pi project ideas, each with a quick description, tutorial link, and hardware requirements. Whether you’re just starting out or looking for something to do this weekend, this list will keep you busy for a while. Just click here to get instant access.
Something not working? Don't waste hours going in circles. Ask in the RaspberryTips Community and get help from people who've already figured it out. Get unstuck now.
llamafile
Portable single-file LLMs.
Is it me, or are there too many AI projects with ‘llama’ in the name? However, this particular tool, llamafile, does have a unique purpose from the others I mentioned earlier.
The goal of llamafile is to repackage a specific AI model into a single executable file. It simplifies using models across different environments without needing complex setups. All you need to do is copy/paste the one file over and use it. Think of the llamafile software as an LLM factory.
If you don’t want to package your own model, there are already pre-made ones for you to try. Just download one in .llamafile format from here, and you can launch the AI directly on your Raspberry Pi like it’s just a regular program.
After running the llamafile, it will create a web interface for you to chat with the AI locally.
You might also like: 7 Hidden Raspberry Pi Features You Should Be Using
Models Supported:
- WizardCoder
- Rocket
- Gemma
- Qwen3
- LLaVA
- Mistral
- Granite
- Phi
- Mixtral
- +more pre-made llamafiles here.
Ready to try this for real?
Reading is easy. Doing is different.
Once you start your own setup, things don’t always go as planned. That’s where people lose hours.
Get help right away instead of figuring everything out alone.
- Avoid the usual beginner mistakes
- Get guidance on your setup
- Access step-by-step videos for most projects
