Overview
Google Assistant is an artificial intelligence-powered virtual assistant developed by Google that is primarily available on mobile and smart home devices.
The ReSpeaker 4-Mics Pi HAT is a quad-microphone array expansion board for Raspberry Pi. This means you can build a more powerful and flexible voice product that integrates Amazon Alexa Voice Service, Google Assistant, and can also be used as voice recognition, voice recording, and many more
In this series of posts, there will be four tutorials on setting up Google Assistant on Raspberry Pi. This post is the first in the series and basically we'll learn how to integrate Google Assistant to the Raspberry Pi with ReSpeaker 4-Mics Pi HAT.
Hardware
- Raspberry Pi 3B+ (Compatible on Raspberry Pi Zero and Zero W, Raspberry Pi B+, Raspberry Pi 2 B, Raspberry Pi 3 B, Raspberry Pi 3 B+, Raspberry Pi 3 A+)
Software
- RPI terminal
- Internet Browser (Example Chrome)
Application Discussion
Google Assistant is an artificial intelligence-powered virtual assistant developed by Google. With the integration of the Raspberry Pi and ReSpeaker device, we can make a portable Google Assitant device that enables voice-activated searches, playing music, and so on.
A ReSpeaker 4-Mics Pi HAT is a quad-microphone expansion board for Raspberry Pi. This board is developed based on AC108, a highly integrated quad-channel ADC with I2S/TDM output transition for high definition voice capture. Besides, it provides a super cool LED ring, which contains 12 APA102 programmable LEDs.
ReSpeaker 4-Mics Pi Hat Expansion Board:
Other than its audio feature, here are the other components included on the board.
ReSpeaker 4-Mics Pi Hat components:
- MIC: 4 analog microphones
- LED: 12 APA102 programable RGB LEDs, connected to SPI interface
- Raspberry Pi 40-Pin Headers: support Raspberry Pi Zero, Raspberry Pi 1 B+, Raspberry Pi 2 B, Raspberry Pi 3 B, and Raspberry Pi 3 B+
- AC108: highly integrated quad-channel ADC with I2S/TDM output transition
- I2C: Grove I2C port, connected to I2C-1
- GPIO12: Grove digital port, connected to GPIO12 & GPIO13
Setup the Hardware
Mount the ReSpeaker 4-Mics Pi HAT to Raspberry pi like this.
Setup up the Software
Google Assistant Setup
- With your Google Account ready, go to the Google Console Actions dashboard, or go to the URL indicated below.
- Log-in using your google account. The screen below will appear. Next, click the "New Project" button.
- The below screen will appear. Set the terms and services, country, and answer the questions asked by Google. Then click "Accept and Continue" to proceed.
- After setting up, you will be asked to fill in the Project Name, language, and country. Then click the "Create Project" button to proceed.
- In a new tab, go to Google developers console to enable the Google Embedded Assistant API, or go to this URL.
https://console.developers.google.com/apis/library/embeddedassistant.googleapis.com
- The below screen will appear. Select the project if not already selected and click the "Enable" button to proceed.
- Now go back to the other tab where you created the project, scroll down to the bottom of the screen. Click the Custom category then you should see at the bottom part "Are you looking for device registration? Click here". Proceed by clicking the "Click here" text as shown in the screenshot below.
- You will now be taken to the following screen, click the "Register Model" button to continue.
- You need to set a "Product name", "Manufacturer name", and a "Device type". The "Product name" is the name of the product you desire. For this tutorial, we set it as "Pi Google Assistant". Next is the "Manufacturer Name" is the name of the manufacturer you desire. For this tutorial, we set it as "Createlabz". Next is the "Device type", for this we set it as Speaker. Take note or copy the "Model ID" as we will need it later. Finally, once everything is set click the button "Register Model".
- After the model has been registered, you will now be taken to the next screen as shown in below. This part is crucial as the provided credentials file is what we need for our Raspberry Pi to communicate with the server. To get this file click the “Download OAuth 2.0 credentials” button as shown below. Store this somewhere safe, as we will need this later (Of course, unless you downloaded it directly to your Pi). Once you have the credentials safely stored on your computer or Raspberry Pi, you need to click the “Next” button.
- Finally, you can specify any traits that you might need. In our case, we don't need any of these so we just click the "Save Traits" button. Once everything is done, below screen will appear.
- We're not yet done yet. We also need to configure the OAuth consent screen. Without this, Google won’t let us authorize our Raspberry Pi Google Assistant device. To do this, you will need to go to the API Credentials OAuth consent screen settings page as shown in the screenshot below. On this first page, make sure that your project is selected in the top dropdown box. Secondly, you will need to select “External” as the user type. This option will allow anyone with a Google account to utilize the Google Assistant software that we are setting up. Once all are configured, click the “CREATE” button to continue to set up the auth screen.
- This page will allow you to set up the details of the auth screen. As we don’t intend this to be publicly accessible there is only one option we are required to set. Click the "Support email" dropdown to select your accounts email address, selecting this will allow us to continue with the guide. Then fill up "Developer contact information" with your account email address. Then click the “Save” button at the bottom of the page.
- For the "Optional info" and "Summary" just click save and continue. After completing next, click the button "BACK TO DASHBOARD" as shown in the screenshot below.
- On Testing click "PUBLISH APP" to put it on production as shown in the screenshot below. This will be needed for the Raspberry Pi later.
- Finally, go to Google My Account activity controls. Here you will need to activate the following activity controls to ensure that the Google Assistant API works correctly.
- Web & App Activity
- Location History
- Device Information
- Voice & Audio Activity
Install ReSpeaker 4-Mics Pi HAT driver
- Install the seeed voice card driver for ReSpeaker 4-Mics Pi HAT by inputting these commands on the RPI terminal.
$ sudo apt-get update $ sudo apt-get upgrade $ git clone https://github.com/respeaker/seeed-voicecard.git $ cd seeed-voicecard $ sudo ./install.sh $ reboot
$ sudo apt-get update $ sudo apt-get upgrade $ git clone https://github.com/createlabz/seeed-voicecard $ cd seeed-voicecard $ sudo ./install.sh $ reboot
- Check if ReSpeaker 4-Mics Pi HAT is recognized by using this command
$ arecord -l
- Check the list of capture hardware devices. You should see the ReSpeaker 4-Mics Pi HAT recognized like below response:
**** List of CAPTURE Hardware Devices ****
card 2: seeed4micvoicec [seeed-4mic-voicecard], device 0: bcm2835-i2s-ac10x-codec0 ac10x-codec.1-003b-0 [bcm2835-i2s-ac10x-codec0 ac10x-codec.1-003b-0] Subdevices: 1/1 Subdevice #0: subdevice #0
- Then, modify raspberry pi configuration to select the headphone jack on Raspberry Pi for audio output.
$ sudo raspi-config
- On raspi-config, set the following:
- Select 7 Advanced Options
- Select A4 Audio
- Select 1 Force 3.5mm ('headphone') jack
- Select Finish
- Save and Exit
- Test Speaker Audio Output by using this command.
$ speaker-test -t wav
- Call alsamixer to change the volume of the speaker.
$ alsamixer
- Store alsamixer new settings.
$ sudo alsactl store
Google Assistant integration to Raspberry Pi
- For this tutorial, we are using the GassistPi library. First, clone the library
$ git clone https://github.com/shivasiddharth/GassistPi
$ git clone https://github.com/createlabz/GassistPi
- Next, save the "OAuth 2.0 credentials" we downloaded earlier and save it into the Raspberry Pi. Use this command.
$ sudo nano /home/${USER}/google-assist-credentials.json
- nano is a text editor, it will open a new file ready for editing. We need to copy the contents of the credentials file that we downloaded earlier. You can open the .json file in any text editor then press CTRL+A then CTRL+C to copy the contents. On the terminal, for SSH connection, right-click to paste it. Once you have copied the contents of your credentials over to our nano session, we can then save the file by pressing CTRL+X then press Y to save it.
- Now we need to take note of "Credential File Path", "Project ID", and "Model ID" to be used later. First, take note of the the credential file path which is this:
/home/${USER}/google-assist-credentials.json
- Take note change the ${USER} to your own user directory, the default value for it is "pi". It could be like these "/home/pi/google-assist-credentials.json". Next is to get the Project ID, go to the Google Console Action dashboard, or go to this URL.
- Next, click your Project then you can see this screen.
- Click the "Project Settings" then you can take note of your Project ID.
- To get the Model ID data from earlier, click "Develop", then click "Device Registration". Then click the existing model in your project as shown in the screenshot below.
- Afterwards, take note of the Model ID as seen on screenshot below
- After the credentials are saved, and Project ID and Model ID are noted, we proceed to install the GassistPi library by using these commands
$ cd /home/${USER}/
$ sudo chmod +x ./GassistPi/scripts/gassist-installer.sh
$ sudo ./GassistPi/scripts/gassist-installer.sh
- The screen below will appear. Comply with the needed data by supplying the credential file name path we took note of earlier.
- Wait for it to install. Then, copy the URL given on the terminal and visit it on a browser to get the authorization code. On the browser, allow everything. After allowing, copy the code and paste it on the terminal, for SSH windows paste it by right-clicking on it.
- Now to test it manually input this command on the terminal. Check and replace the needed value as the command indicated.
$ /home/${USER}/env/bin/python -u /home/${USER}/GassistPi/src/main.py --project_id 'replace this with the project id' --device_model_id 'replace this with the model id'
- Congratulations! You can now use Google Assistant on Raspberry Pi with ReSpeaker 4-Mics Pi Hat. To make a query, make sure to say "Ok Google" then your question or commands. An example would be saying first "Ok Google" then "weather today".
Libraries Used
https://github.com/respeaker/seeed-voicecard.git
https://github.com/shivasiddharth/GassistPi
Demo Video
Conclusion
Google is helpful in day to day use, by checking the weather, time, or any inquiry to google, play music, and so on. Integrated to Raspberry with the ReSpeaker 4-Mics Pi you can have a portable device or a low-cost Google Assistant device installed at your home. This can also be a smart home device to control your lights, check your power rating, check room temperature, and many more.
Check other tutorials in this Series: Integrating Google Assistant to Raspberry Pi
Integration of Youtube and Spotify with ReSpeaker 4-Mics Pi Hat on Raspberry Pi
Voice-activated Lights Control with IFFFT, Adafruit IO, NodeMCU, and Raspberry Pi
References
https://wiki.seeedstudio.com/ReSpeaker_4_Mic_Array_for_Raspberry_Pi/
