How to Make a Voice Assistant Using Raspberry Pi (Full Guide)

The first time I used a voice assistant was with the Amazon Echo, which had Alexa built in. It could play music, give weather updates, and even control a few smart devices. Over the years, voice assistants have become much smarter and more accessible. In this post, I’ll show you how to create your own voice assistant using a Raspberry Pi, step by step.

One can build a voice assistant on a Raspberry Pi by installing a suitable voice assistant, such as Google Assistant, setting up a microphone and speaker, and configuring it to process and respond to voice commands.

This tutorial will be more hands-on. I’ll guide you through the entire process of installing and setting up a microphone for sound input and speakers for sound output. After that, we’ll configure a voice assistant software to bring your Raspberry Pi to life, allowing it to respond to voice commands and perform various tasks.

If you’re new to Raspberry Pi or Linux, I’ve got something that can help you right away!
Download my free Linux commands cheat sheet – it’s a quick reference guide with all the essential commands you’ll need to get things done on your Raspberry Pi. Click here to get it for free!

Project Overview

Before I dive into the technical details, let me give you a quick overview of what we’ll be building. To do that, let’s first understand what a voice assistant actually is.

A voice assistant is basically a smart program that listens to your voice, understands what you say, and responds or performs actions accordingly.

Think of it like Google Assistant, Alexa, or Siri.
You say, “What’s the weather like today?” and it gives you an answer without you touching a thing.

In this project, we’ll be building a simple version of that experience using a Raspberry Pi. You’ll be able to communicate with your Raspberry Pi and have it respond to basic commands, such as playing music, displaying the time, or even controlling other devices around your home (depending on how far you take it).

Now, besides the hardware we’ll connect to the Raspberry Pi, we also need to talk about the software that will give it these capabilities. There are several options available:

• OpenVOS: An open-source voice assistant option you can fully customize and even run offline. I have actually tried OpenVOS and have shared my experience with it below.
• ElevenLabs: I have not yet tried ElevenLabs. However, from the reviews I have gathered online, it’s great for generating realistic, human-like speech if you want your assistant to sound more natural.
• Mycroft AI: Another popular open-source project for building your own private, privacy-focused assistant.

In this tutorial, we’ll be using the Google Assistant SDK. The same technology that powers Google Home devices. It’s reliable, smart, and connects seamlessly with Google’s services.

That means once we’re done, you’ll be able to say things like “Hey Google, what’s the weather?” or “Play my favorite playlist,” and your Raspberry Pi will respond just like a real Google smart speaker.

You might be wondering why use a Raspberry Pi for this instead of just buying a Google Nest Mini?

Well, the Raspberry Pi gives you complete control and flexibility. You can customize how your assistant behaves, integrate it with your own projects (such as controlling lights or sensors), and even keep it running 24/7 with minimal power consumption.

Plus, it’s a fun and affordable way to learn how real-world voice assistants actually work under the hood.

Hardware Requirements

• Raspberry Pi: Any modern Raspberry Pi will work for this project, but I recommend using a Raspberry Pi 4 or Raspberry Pi 3 Model B+ for smoother performance.
  When I first tested this project, I tried working with the Raspberry Pi Zero. Everything was fine, but it seemed to struggle a bit with audio processing and the Google Assistant’s requirements.
• Power supply: You’ll need a stable power source for this project. A common mistake many people make is assuming that since the Raspberry Pi uses a USB Type-C cable, a regular phone charger will work.
  That’s not the case! I highly recommend checking out our article on how to power a Raspberry Pi, where we explain the right way to do it.
• A USB Microphone: The Raspberry Pi does not have a built-in microphone. And for the Raspberry Pi to hear your voice commands, you will need one. You can use any plug-and-play USB microphone. Even a cheap desktop or clip-on mic will do fine for testing.
  If you have a webcam, you can also check if it comes with an in-built mic. Some webcams, like the NexiGo N60, have built-in mics that work out of the box.
• Speakers (for sound output): You’ll need speakers so the assistant can talk back. You can use:
  • If available on your model, the Raspberry Pi’s 3.5mm audio jack allows for wired audio output. This is actually the option I recommend when testing your project. You can even plug in your phone’s earphones.
  • USB-powered desktop speakers.
  • A Bluetooth speaker.
    You can go through our article on how to configure audio on the Raspberry Pi, which gives more details and options for setting up audio on your Pi.
• A Keyboard and a Mouse: You’ll need these for the initial setup. Once everything’s configured, you can run your voice assistant completely headless (without screen or keyboard). Did you know that you can also use a wireless keyboard and mouse on your Raspberry Pi?
• LEDs (optional): We’ll use LEDs to make the project a bit more interactive. For example, lighting up when the assistant is listening or processing your command. Even a single red or blue LED connected to a GPIO pin can make the project more fun and visual.

Software Requirements

• Raspberry Pi OS: We’ll be using the official Raspberry Pi OS. It’s the most stable and compatible operating system for projects like this. But don’t feel limited. You can also use other Linux-based operating systems like Ubuntu.
• Python: We will use Python to set up and configure the Google Assistant SDK on the Raspberry Pi. The good news is that Python comes preinstalled on Raspberry Pi OS, but it’s a good idea to make sure you have the latest version:
  python3 --version
• Google Cloud Project with Google Assistant API enabled: You need a Google Cloud project where the Google Assistant API is activated. This is essential because the Assistant SDK communicates with Google’s servers through this API.
  Don’t worry about this part, as I will give you a step-by-step guide below.

How to Make a Voice Assistant Using Raspberry Pi

At this point, I believe you’re ready to get hands-on and start building your very own Raspberry Pi voice assistant. We’ve already covered what the project is about, what you’ll need, and why the Raspberry Pi is such a great platform for this.

Now, let’s bring everything to life step by step.

Tip: From this point onwards, I assume you already have your Raspberry Pi running with the official Raspberry Pi OS installed.

Step 1. Connect and Set up the USB Mic

The first thing you need to do is configure the USB microphone and ensure that it’s working perfectly.

Here you have many options to choose from. If you already use a microphone for content creation or online meetings, you can use that one. But if you want something small that won’t take up much space on your desk, you can go for the SunFounder USB 2.0 Mini Microphone. That’s the one I’ll be using for this project.

Start by plugging your USB microphone into one of the Raspberry Pi’s USB ports. Most USB mics are plug-and-play, so the system should detect them automatically.

To confirm your Raspberry Pi recognizes the mic, open the terminal and run the command below:
arecord -l

As you can see in the screenshot above, our USB mic is attached to card 1 and assigned the device ID 0. If that is not the case for you, try unplugging it and reconnecting, or test a different USB port.

Now, you might wonder why it’s not connected to card 0 or card 2.

That’s because the Raspberry Pi reserves card 0 for its internal sound card. When you connect additional audio devices, they’re assigned incrementally – card 2, card 3, card 4, and so on.

Tip: If you’re using a webcam with a built-in mic, it will also show up here. Just make sure you select it as your input source.

Once you have confirmed the USB mic is well connected, it’s time you do some audio configs to match your project needs.

First, run the command below to create a configuration file. You will notice that I am using the nano editor. However, you can use any other text editor of your choice.
sudo nano /etc/asound.conf

Next, copy the configurations below and paste them into this new file. Save the file (Ctrl + S) and Exit (Ctrl + X).

pcm.!default {
  type asym
  capture.pcm "mic"
  playback.pcm "speaker"
}
pcm.mic {
  type plug
  slave {
    pcm "hw:1,0"
  }
}
pcm.speaker {
  type hw
  card 0
}
ctl.!default {
 type hw card 0
}

This configuration is doing one main thing. It sets your default audio device (pcm.!default) to an asymmetric one – meaning:

• Capture (input) will use “mic”.
• Playback (output) will use “speaker”.

So when any program records or plays audio without specifying a device, the system will automatically use these devices. And that is what we need for our project.

Step 2. Connect and Set up the Speakers (Sound Output)

After you’re done setting up the mic, the next thing you need to do is set up the sound output.

To keep things simple (and avoid those annoying audio configuration issues), I recommend starting with the 3.5mm audio jack on your Raspberry Pi. Just plug in your earphones or a small speaker — that’s the easiest way to check if everything’s working.

Once you’ve confirmed the audio is fine, feel free to experiment a bit. Try USB speakers, Bluetooth audio, or even sound over HDMI if you want to take it up a notch later.

Now, to set up the 3.5mm audio jack as the default audio output device, you won’t need to write any configurations or execute any commands. You can easily do it using the Raspberry Pi Configuration Tool.

First, launch the terminal and run the command below:
sudo raspi-config

This will open the Raspberry Pi Configuration Tool. Use the arrow keys to select “Advanced Options” and hit “Enter. Then go to: Audio -> Choose your output (HDMI or Headphones).

Once you’ve selected the correct device, we’ll test both input and output in the next step.

Step 3. Test the Mic and Speakers

Now you need to make sure both your microphone and speakers are working correctly.

Let’s start with the speaker, as it is the easiest to test. Launch the terminal and run the command below. If everything is okay, you should hear a sound like “Front Right” – “Front Left.” To terminate the test, press Ctrl + C.
speaker-test -t wav

In my case, as you can hear in the video above, the “front-left” sound plays just fine. Don’t worry about the background noise. That’s just the fan from one of the setups on my desk.

If you don’t hear any sound, don’t panic. First, double-check that your speakers are plugged in properly, the volume isn’t muted, and the power is on.

Next, test your microphone.

Go ahead and run the command below in your terminal. It’ll record a short 5-second audio clip using your mic, so feel free to say something into it – even the classic “Mic testing, one, two.”
arecord --format=S16_LE --duration=5 --rate=16000 --file-type=raw out.raw

Once you’re done recording, go ahead and play the audio using the command below. This will let you hear what your mic picked up.
aplay --format=S16_LE --rate=16000 out.raw

If you can hear your voice clearly, congratulations! Your audio setup is working perfectly. If not, don’t worry. Just double-check your input and output device settings or try reconnecting your hardware.

Step 4. Download Required Packages and Configure Python Environment

Up to this point, our hardware is set and we are now ready to jump into the sweet stuff.

In this part, I will guide you on downloading the required packages and configuring the Python environment on your Raspberry Pi.

The first step is updating your system to ensure you are running the latest packages. Launch the terminal and execute the commands below:
sudo apt update
sudo apt upgrade

Next, we need to configure Python. Before we continue, please ensure that you are running Python 3. You can check that by executing the command:
python --version

If you see an output like Python 2.x.x, please go through our article and learn how to install the latest Python version on your Raspberry Pi.

If you are running the latest Python version, then we can proceed. First, you need to install the tools that allow you to create a virtual environment and build Python packages that require compilation. You can do that by running the command below:
sudo apt install python3-dev python3-venv

Next, create a virtual environment where we’ll install everything we need for this project. This keeps your setup clean and prevents any conflicts with system-wide Python packages.
python3 -m venv env

Once that’s done, activate the environment and upgrade pip and setup tools to the latest versions:
env/bin/python -m pip install --upgrade pip setuptools

This makes sure your environment is ready for any dependencies we’ll install later.

Once that’s done, activate the environment using the command below. This will bring a “(env)” text in front of your Pi’s command terminal:
source env/bin/activate

With your virtual environment now active, we can install the packages needed for the project.

Next, let’s install the Google Assistant SDK. This package gives your Raspberry Pi everything it needs to run Google Assistant.
python3 -m pip install google-assistant-sdk[samples]

This will download and set up the Google Assistant Library, which is the core of your voice assistant.

Step 5. Enable the Google Assistant Cloud Project

Now it’s time to set up Google Assistant access from your account.

• Open the Google Cloud Console and create a new project. You can name it anything you like. The account you use here will be linked to your Assistant, so it’ll send queries and get personalized responses through it.
  
• After creating the project, look for “APIs & Services” under the Quick Access menu. Click it.
  
• On the new window that appears, select “Library” on the left panel and proceed to enable the Google Assistant API.
  
• Next, open your Activity Controls and make sure the following are turned on for your account:
  • Web & App Activity.
  • Device Information.
  • Voice & Audio Activity.
• Head over to Credentials -> Configure Consent Screen, and set it up (you can keep most defaults for now).
  
• Then go to Credentials -> Create Credentials -> OAuth Client ID.
• Select Application type: Other (or Desktop app if “Other” isn’t available) and give it a name.
  
• Once created, download the JSON file. This contains your OAuth key information. From what I experienced, this file is slightly hidden, and you will need to do some digging to get it. First, you will see a screen like this, which lists all your OAuth 2.0 Client IDs.
  You can start by copying the Client ID, as you will need it somewhere. To download the json file, click the “pencil” icon, highlighted in the screenshot above. On the new screen that appears, scroll down, and on the far right under Client Secrets, you will see an option to download the json file.
  Keep this json file somewhere safe, as you will need it later.

Step 6. Authenticate the Raspberry Pi to Google Assistant

Now that you’ve set up your credentials, it’s time to authenticate your Raspberry Pi with Google Assistant.

First, install the authorization tool by running:

python3 -m pip install --upgrade google-auth-oauthlib[tool]

Next, run the tool using the command below. Make sure to replace "JSON_FILE_PATH" with the actual path to the JSON file you downloaded in the previous step.

google-oauthlib-tool --client-secrets "client_secret_xxxxxxxxxxx.json" \
--scope https://www.googleapis.com/auth/assistant-sdk-prototype \
--save

You’ll then see a message like this:

Please visit this URL to authorize this application: https://accounts.google.com/o/.....

Copy the link, open it in your browser, and sign in with the same Google account you used earlier. Once the authentication flow completes, you’ll see a message saying it’s done. You can then return to the terminal and continue.

Step 7: Start Google Assistant Automatically on Boot

Let’s make sure your Google Assistant starts automatically whenever your Raspberry Pi boots up. We’ll do this by creating a small Python script and a shell script to launch it.

First, install the RPi.GPIO package to enable GPIO support:
pip3 install RPi.GPIO

Now, let’s set up the Python script:
cd ~
sudo nano main.py

Paste the script below into the file, then save and exit (Ctrl + X, then Y, then Enter):

#!/usr/bin/env python3
import os
import json
import grpc
import sounddevice as sd
import numpy as np
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from google.assistant.embedded.v1alpha2 import embedded_assistant_pb2_grpc, embedded_assistant_pb2

CREDENTIALS_PATH = os.path.expanduser("~/.config/google-oauthlib-tool/credentials.json")
DEVICE_ID = 'rpitips-pi'
DEVICE_MODEL_ID = 'rpitips-pi-model'
LANGUAGE_CODE = 'en-US'

SAMPLE_RATE = 16000
CHUNK_SIZE = 1024

def load_credentials():
    with open(CREDENTIALS_PATH, 'r') as f:
        data = json.load(f)
        creds = Credentials(
            token=data.get('token'),
            refresh_token=data['refresh_token'],
            token_uri=data['token_uri'],
            client_id=data['client_id'],
            client_secret=data['client_secret'],
            scopes=data['scopes']
        )
    try:
        creds.refresh(Request())
    except Exception as e:
        print(f"[Warning] Failed to refresh token: {e}")
    return creds

def create_assistant_channel(creds):
    def metadata_callback(context, callback):
        callback([('authorization', 'Bearer ' + creds.token)], None)

    auth_interceptor = grpc.metadata_call_credentials(metadata_callback)
    composite_credentials = grpc.composite_channel_credentials(
        grpc.ssl_channel_credentials(),
        auth_interceptor
    )

    grpc_channel = grpc.secure_channel(
        'embeddedassistant.googleapis.com:443',
        composite_credentials
    )

    return embedded_assistant_pb2_grpc.EmbeddedAssistantStub(grpc_channel)

def record_audio(duration=5):
    print("Listening...")
    audio = sd.rec(int(duration * SAMPLE_RATE), samplerate=SAMPLE_RATE, channels=1, dtype='int16')
    sd.wait()
    return audio.flatten().tobytes()

def play_audio(audio_data):
    audio_np = np.frombuffer(audio_data, dtype=np.int16)
    sd.play(audio_np, samplerate=SAMPLE_RATE)
    sd.wait()

def main():
    creds = load_credentials()
    assistant = create_assistant_channel(creds)

    print("Voice Assistant ready. Say something!\n(Type 'exit' to quit)")

    while True:
        text_input = input("Press Enter to speak or type 'exit': ").strip()
        if text_input.lower() in ("exit", "quit"):
            break

        audio_in = record_audio(duration=5)  # 5 seconds of audio

        request = embedded_assistant_pb2.AssistRequest(
            audio_in=audio_in,
            config=embedded_assistant_pb2.AssistConfig(
                audio_out_config=embedded_assistant_pb2.AudioOutConfig(
                    encoding=embedded_assistant_pb2.AudioOutConfig.LINEAR16,
                    sample_rate_hertz=SAMPLE_RATE,
                    volume_percentage=100
                ),
                dialog_state_in=embedded_assistant_pb2.DialogStateIn(language_code=LANGUAGE_CODE),
                device_config=embedded_assistant_pb2.DeviceConfig(
                    device_id=DEVICE_ID,
                    device_model_id=DEVICE_MODEL_ID
                )
            )
        )

        try:
            for resp in assistant.Assist(iter([request])):
                if resp.speech_results:
                    print("You said:", ' '.join(r.transcript for r in resp.speech_results))
                if resp.audio_out.audio_data:
                    play_audio(resp.audio_out.audio_data)
                if resp.dialog_state_out.supplemental_display_text:
                    print("Assistant:", resp.dialog_state_out.supplemental_display_text)
        except grpc.RpcError as e:
            print(f"[Error] {e.code().name}: {e.details()}")

if __name__ == "__main__":
    main()

Next, let’s create a shell script to run this Python file on startup:
sudo nano google-assistant-init.sh

Paste this inside and remember to replace “/home/pat” with your actual home directory:

#!/bin/sh
/home/pat/env/bin/python3 -u /home/pat/main.py

Save the file, then make it executable:
sudo chmod +x google-assistant-init.sh

Now, you can start Google Assistant anytime by running:
./google-assistant-init.sh

If you’re having trouble with audio, you can test your assistant using a text-based interface. This way, you can type commands in the terminal and receive text-based responses from Google Assistant.

But first, you will need to tweak your Python script. Or you can create a new file using the command below
nano text-based-assistant.py

Then, proceed to run the copy and paste the script below.

#!/usr/bin/env python3
import os
import json
import grpc
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from google.assistant.embedded.v1alpha2 import embedded_assistant_pb2_grpc, embedded_assistant_pb2

# Path to OAuth2 credentials (generated by google-oauthlib-tool)
CREDENTIALS_PATH = os.path.expanduser("~/.config/google-oauthlib-tool/credentials.json")

def load_credentials():
    """Load and refresh OAuth2 credentials."""
    with open(CREDENTIALS_PATH, 'r') as f:
        data = json.load(f)
        creds = Credentials(
            token=data.get('token'),
            refresh_token=data['refresh_token'],
            token_uri=data['token_uri'],
            client_id=data['client_id'],
            client_secret=data['client_secret'],
            scopes=data['scopes']
        )
    try:
        creds.refresh(Request())
    except Exception as e:
        print(f"[Warning] Failed to refresh token: {e}")
    return creds

def create_assistant_channel(creds):
    """Create gRPC channel with Google Assistant API using credentials."""
    # Add authorization metadata
    def metadata_callback(context, callback):
        callback([('authorization', 'Bearer ' + creds.token)], None)

    auth_interceptor = grpc.metadata_call_credentials(metadata_callback)
    composite_credentials = grpc.composite_channel_credentials(
        grpc.ssl_channel_credentials(),
        auth_interceptor
    )

    grpc_channel = grpc.secure_channel(
        'embeddedassistant.googleapis.com:443',
        composite_credentials
    )

    return embedded_assistant_pb2_grpc.EmbeddedAssistantStub(grpc_channel)

def main():
    print("Assistant ready! Type your message and press Enter.")
    print("(Type 'exit' to quit)\n")

    creds = load_credentials()
    assistant = create_assistant_channel(creds)

    while True:
        text_input = input("You: ").strip()
        if text_input.lower() in ("exit", "quit"):
            print("Goodbye!")
            break

        request = embedded_assistant_pb2.AssistRequest(
            config=embedded_assistant_pb2.AssistConfig(
                audio_out_config=embedded_assistant_pb2.AudioOutConfig(
                    encoding=embedded_assistant_pb2.AudioOutConfig.LINEAR16,  # we won't play audio
                    sample_rate_hertz=16000,
                    volume_percentage=0
                ),
                dialog_state_in=embedded_assistant_pb2.DialogStateIn(language_code='en-US'),
                text_query=text_input,
                device_config=embedded_assistant_pb2.DeviceConfig(
                    device_id='rpitips-pi',
                    device_model_id='rpitips-pi-model'
                )
            )
        )

        try:
            responses = assistant.Assist(iter([request]))
            for resp in responses:
                if resp.dialog_state_out.supplemental_display_text:
                    print("Google Assistant:", resp.dialog_state_out.supplemental_display_text)
        except grpc.RpcError as e:
            print(f"[Error] {e.code().name}: {e.details()}")

if __name__ == "__main__":
    main()

How To Use the Voice Assistant on the Raspberry Pi

After all that setup, it’s finally time to try out our Raspberry Pi voice assistant!

By now, I believe it’s already running, as we covered in the previous step on how to start it. This is the fun part where all that coding, configuration, and patience finally pay off. Let’s see what our little Pi can actually do!

I will run the text-based assistant since my audio has so much voice as you can hear in the previous tests above. Always remember to run the scripts inside your virtual environment.

From the image above, you can see my text-based assistant worked perfectly. I asked a few questions, and it returned the answers accurately. I hope you’re getting the same smooth experience on your end as you try out your own assistant.

I believe this tutorial has given you a complete guide on building your own voice assistant with the Raspberry Pi.

Honestly, the only tricky part is setting up Google Assistant itself — and that’s mostly because Google keeps updating how its Assistant SDK works.

For example, older versions used simpler REST-based APIs, but now most communication happens through gRPC, which is more secure and efficient, but also a bit more complex to set up.

Still, once you get past the setup stage, the rest is straightforward and quite exciting. You can now ask your Raspberry Pi questions, get real-time answers, and even expand it to perform actions like controlling lights, sensors, or other connected devices.

FAQ

How do I control real devices with Google Assistant?

To control real devices, you need to connect your Raspberry Pi to external hardware such as a relay module or smart plug. The relay connects to the Pi’s GPIO pins and allows it to switch devices like lights or fans on and off. You can then create simple Python scripts or use automation platforms like Home Assistant or Node-RED to link these actions to Google Assistant commands.

For example, when you say “Hey Google, turn off the lights,” Google Assistant can send a signal to your Raspberry Pi, which then triggers the relay to cut power to the light.

Can I use LEDs?

Yes, LEDs are a perfect idea since they make the project feel more interactive and lively. For example, you can make an LED blink while the assistant is “listening” or light up when it’s “speaking.” It’s a simple way to give your voice assistant a visual response.

Whenever you’re ready, here are other ways I can help you:

Test Your Raspberry Pi Level (Free): Not sure why everything takes so long on your Raspberry Pi? Take this free 3-minute assessment and see what’s causing the problems.

The RaspberryTips Community: Need help or want to discuss your Raspberry Pi projects with others who actually get it? Join the RaspberryTips Community and get access to private forums, exclusive lessons, and direct help (try it for just $1).

Master your Raspberry Pi in 30 days: If you are looking for the best tips to become an expert on Raspberry Pi, this book is for you. Learn useful Linux skills and practice multiple projects with step-by-step guides.

Master Python on Raspberry Pi: Create, understand, and improve any Python script for your Raspberry Pi. Learn the essentials step-by-step without losing time understanding useless concepts.

You can also find all my recommendations for tools and hardware on this page.