Google-Assistant-SDK
Contents
Google Assistant SDK
Introduction
This guide shows how to enable Google Assistant SDK on your Tinker Borad and combine with external voice card (audio card) for some applications.
The Google Assistant can be regarded as a part of AI, it can detect keywords, understand natual language and give response provided by Google via Internet.
For more details, please visit the official website https://developers.google.com/assistant/sdk/
Outlines
a. Bring up external voice card, ReSpeaker 2-Mics Pi HAT
b. Install & run Google Assistant SDK
c. Interactive application example, using RGB leds
Porting steps
a. Bring up external voice card, ReSpeaker 2-Mics Pi HAT
ReSpeaker 2-Mics Pi HAT is a expansion board with 2 mics and 1 audio jack interface designed for voice applications.
With it, we can do a great extension of receiving user speaking range for Google assistant.
In above diagram, what we used is the L/R mics and audio jack for voice I/O.
Base on v2.0.10 image release to bring up external voice card, first enable vendor driver by adding a line in /boot/hw_intf.conf as following:
intf:dtoverlay=seeed-2mic-voicecard
After re-booting, we have to configure the audio settings.
First, copy config files File:Seeed-2mic-voicecard.zip to responding locations as following:
cp libasound_module_pcm_ac108.so /usr/lib/arm-linux-gnueabihf/alsa-lib/
cp asound.conf /etc/
cp wm8960_asound.state /var/lib/alsa/asound.state
Set default source/sink of alsa device:
pacmd set-default-source alsa_input.platform-sound-ext-card.analog-stereo
pacmd set-default-sink alsa_output.platform-sound-ext-card.analog-stereo
Turn on "PulseAudio Volume Control" in menu -> Sound & Video
You can see all audio device here, check our new voice card is enabled.
Modify Seeed-2mic-voicecard output device to headphones
Change PulseAudio -> Output Devices -> Seeed-2mic-voicecard Analog Stereo -> Port to Headphones
The bring up and configuration are done now!
b. Install & run Google Assistant SDK
In this section, we will install Google Assistant SDK and configure the voice settings for the SDK.
After you done, you can communicate with Tinker Board by speaking & listening nature language.
Warning: This step requires:
- network available
- headphone with mic (3.5mm audio jack)
To enable access to the Google Assistant API, do the following steps on your development machine:
https://console.cloud.google.com/project
https://console.developers.google.com/apis/api/embeddedassistant.googleapis.com/overview.
Click Enable
3.1 Create the client ID. https://console.developers.google.com/apis/credentials/oauthclient
3.2 You may need to set a product name for the product consent screen. On the OAuth consent screen tab, give the product a name and click Save.
3.3 Click Other and give the client ID a name.
3.4 Click Create. A dialog box appears that shows you a client ID and secret. (No need to remember or save this, just close the dialog.)
client_secret_<client-id>.json).client_secret_<client-id>.json file from your development machine to your device. This step is not necessary if you downloaded this file directly to your device.
Use a https://docs.python.org/3/library/venv.html Python virtual environment to isolate the SDK and its dependencies from the system Python packages.
sudo apt-get install python-dev python-virtualenv
virtualenv env --no-site-packages
env/bin/python -m pip install --upgrade pip setuptools
source env/bin/activate
Use pip to install the latest version of the https://pypi.python.org/pypi/google-assistant-library Python package in the virtual environment:
(env) python -m pip install --upgrade google-assistant-library
Install the authorization tool & run it:
(env) python -m pip install --upgrade google-auth-oauthlib[tool]
(env) google-oauthlib-tool --client-secrets /path/to/client_secret_client-id.json --scope https://www.googleapis.com/auth/assistant-sdk-prototype --save
After above, the webpage will jump to your Google account page, please login & allow the machine to use Google service.
Finally, start the Google Assistant SDK:
(env) google-assistant-demo
Then, say Ok Google or Hey Google, followed by your query. The Assistant should respond.
c. Interactive application example, using RGB leds
Install utilities before starting:
apt-get update
apt-get install python-setuptools python-pip python-wheel
pip install spidev
Download the sample code to Tinker Board as following:
git clone https://github.com/respeaker/mic_hat.git
Set the init function parameters of rgb leds in mic_hat/apa102.py (remove red words and add green words):
order='rgb', bus=0, device=1, max_speed_hz=8000000):
order='rgb', bus=2, device=0, max_speed_hz=8000000):
Test rgb leds as following:
python mic_hat/pixels.py
If the modification is correct, you will see the blink colorful leds:
Finally, run google_assistant.py as following:
python mic_hat/google_assistant.py
You will find the leds change status in the conversation~
Reference
https://developers.google.com/assistant/sdk/
http://wiki.seeed.cc/Respeaker_2_Mics_Pi_HAT/
https://aiyprojects.withgoogle.com/voice/?_sm_au_=iVVJNmfRLZ5W30km#project-overview
Known issues
1. The virtual device, simple-audio-card, can only run in slave mode so far.
2. HDMI will play even you are using other output device.
3. It can only play stereo format .wav file once, but can repeat for mono format .wav file.
