Controlling the R22 robot over Wi-Fi

Example showing R22 Robot using GStreamer to send real-time video to the Mobile Command Center

This article provides an example of how to control the Pi in the R22 robot from the keyboard of another Pi or computer using a pair of Python scripts and a GStreamer pipeline.

The Python scripts are based on the RemoteKeyBorg example from Piborg (www.piborg.org). However, the original scripts have been significantly modified to add the following features:

• Additional GPIO drive channels to control the L298-based motor driver PCB

• Bidirectional feedback between controller and robot

• Integration of real-time GStreamer video pipeline in Pygame

Both scripts establish a data socket connection through a client-server architecture, where the script running on the R22 robot is configured as the server (listener) and the script running on the other Pi is the client (controller). From this point forward, the Pi running the client (controller) script will be referred to as the "base station Pi" and the Pi running the server (listener) script will be referred to as the "R22 Pi".

In addition to receiving motor control commands from base station, the R22 Pi uses GStreamer to send real-time video back to the base station Pi.

The R22 robot kit has been verified to work with the Pi Zero 2 W, Pi 3, and Pi 4.

For this example, the R22 Pi is a Pi 4 with 4 GB of RAM. This example will work with either the 32-bit or 64-bit Trixie operating system.

The base station Pi will send movement commands and receive real-time video from the R22 robot. For the base station Pi, I use the Mobile Command Center, which currently has an embedded Pi 4 with 8GB of RAM, but either a Pi 4 or Pi 5 can be used. For this example, the base station Pi will run the 64-bit version of the Trixie or Bookworm Raspberry Pi operating system.

The overall process to set up Wi-Fi control is organized into four parts.

• Part 1: Install the combined Python and GStreamer script onto the base station Pi

• Part 2: Install the Python and GStreamer scripts onto the R22 Pi.

• Part 3: Test the scripts to verify functionality

• Part 4: Set the R22 Pi Python and GStreamer scripts to automatically start upon bootup

Part 1: Install the Python and GStreamer script onto the base station Pi

This part is comprised of six steps:

1. Install the 64-bit Trixie operating system onto a micro SD card.

2. Install the missing GStreamer libraries and plug-ins.

3. Run ifconfig to determine the IP address and set it as a static IP address in your Wi-Fi router.

4. Enable ssh on the base station Pi.

5. Download the BaseStation.tar file to the /home/<user name> directory.

6. Edit the KEYctlgst.py script to point to the R22 Pi IP address

   
   

Step 1 (base station Pi) - Install the 64-bit Trixie operating system onto a micro SD card

Use the Raspberry Pi imager tool to install the 64-bit Trixie operating system onto a micro SD card with at least 32GB of storage.

Upon first boot, you'll need to complete the setup process, which will prompt you to select your language, keyboard layout.

After completing the setup, open a terminal window and update the operating system

sudo apt update && sudo apt full-upgrade

After the update is complete, reboot the system.

sudo reboot

Step 2 (base station Pi) - Install the missing GStreamer libraries and plug-ins.

GStreamer is used to broadcast video from the R22 Pi to the base station Pi, so it is required on both Pi computers. GStreamer is already installed on the Trixie operating system, however additional libraries are required for this the script to work. Open a terminal window and run the following command to install the necessary plug-ins.

# install a missing dependency

sudo apt-get install libx264-dev libjpeg-dev

Running this command will inform you that 2,884 kB of additional disk space will be used and will ask if you want to continue, answer yes, or y.

# install the remaining plugins

sudo apt-get install libgstreamer1.0-dev \
libgstreamer-plugins-base1.0-dev \
libgstreamer-plugins-bad1.0-dev \
gstreamer1.0-plugins-ugly \
gstreamer1.0-tools \
gstreamer1.0-gl \
gstreamer1.0-gtk3

Step 3 (base station Pi) - Run ifconfig to determine the IP address and set it as a static IP.

At some point the R22 Pi will be set to automatically start its GStreamer script upon startup.

In order to guarantee that it can reliably broadcast video to the base station Pi, you will need to assign an IP address that stays the same every time the base station Pi is powered up. This is referred to as a static IP address and you can set this up by first learning the IP address of the base station Pi, then adding a DHCP reservation in your Wi-Fi router configuration.

To determine the IP address, open a terminal window and run the "ifconfig" command.

ifconfig

In the image below, the IP address is 192.168.0.106 and the MAC (ether) address is d8:3a:dd:5f:1a:43

You will need this IP address for part two, where you edit the GStreamer script to broadcast video to the base station. Both the IP address and MAC address are required for setting up a static IP address using the DHCP reservation setting in your Wi-Fi router.

Refer to the instruction manual for your Wi-Fi router to set up a DHCP reservation.

Step 4 (base station Pi) - Enable SSH on the base station Pi.

At some point, you may need to edit, troubleshoot, create scripts, as well as reboot and shut down the R22 Pi from the base station. This will be done by using SSH from the base station to the R22 Pi. For this to happen, you will need to enable SSH by running the raspi-config utility on the R22 Pi.

In a terminal window, run the following command to start the raspi-config utility:

sudo raspi-config

Use the down arrow key to highlight "Interface Options", then use the TAB key to highlight the "<Select>" button, then press <ENTER>

Use the TAB key to highlight the "<Select>" key, then press enter. The utility may ask if you wish to enable SSH, answer Y or YES.

This will enable SSH, which should produce a confirmation message as shown below:

With the <Ok> option highlighted as shown above, press <ENTER>. This may bring you back to the original menu. Use the TAB key to highlight the <Finish> option, then press <ENTER> to exit the configuration utility.

Step 5 (base station Pi) - Download the BaseStation.tar file to the /home/<user name> folder

Open a terminal window a download a tar file containing the python and gstreamer scripts by running the following command:

wget -O "$HOME/BaseStation.tar" "https://09224636-6dc8-4937-beba-f9d4dfaee62b.filesusr.com/archives/322f50_2c3c7b486d5c4ec4ab664b27368e8e26.tar?dn=BaseStation.tar"

In the same terminal window, change to the /home/<user name> directory and untar the file:

tar -xvf BaseStation.tar

This will create a "solvebotics" folder under /home/<user name> and will extract:

• KEYctlgst.py to the /home/<user name>/solvebotics directory

• R22receive.sh to the /home/<user name>/solvebotics/gstreamer directory.

Step 6 (base station Pi) - Edit the KEYctlgst.py script to point to the R22 Pi IP address

From the /home/<user name> directory, change to the /solvebotics directory:

cd solvebotics

Edit the KEYctlgst.py script and change the IP address to match the static IP address of the R22 Pi.

(*The static IP address of the R22 Pi may not be known at this point, but make sure to return to this step after the R22 Pi IP address has been set up as a static IP address in Part 2)

You can use nano to edit the KEYctlgst.py script:

nano KEYctlgst.py

Or, you can use a Python editor to make the change. In either case, the IP address is on line 98 of the KEYctlgst python script.

Part 2: Install the Python and GStreamer scripts onto the R22 Pi.

This part is comprised of seven steps and steps 1 through 4 are going to look very similar to steps 1-4 that were done for the base station Pi.

1. Install the 32-bit or 64-bit Trixie operating system onto a micro SD card.

2. Install the missing GStreamer libraries and plug-ins.

3. Run ifconfig to determine the IP address and set it as a static IP in your Wi-Fi router.

4. Enable ssh on the R22 Pi.

5. Download the R22robot.tar file to the /home/<user name> directory.

6. Change to the /home/<user name>/solvebotics/gstreamer directory and edit the R22sendvid.sh script to broadcast to the static IP address of the base station Pi.

7. Make the R22sendvid.sh script executable.

Step 1 (R22 Pi) - Install the Operating System onto an SD card

Use the Raspberry Pi imager tool to Install the 32-bit or 64-bit Raspberry Pi operating system onto a micro SD card with at least 32GB of storage.

Upon first boot, you'll need to complete the setup process, which will prompt you to select your language, keyboard layout.

After completing the setup, open a terminal window and update the operating system

sudo apt update && sudo apt full-upgrade

After the update is complete, reboot the system.

sudo reboot

Step 2 (R22 Pi) - Install gstreamer plug-ins

For this application, I use gstreamer to broadcast video from the robot to the base station. GStreamer is already installed on the Trixie operating system, however additional libraries are required for this the script to work. Open a terminal window and run the following command to install the necessary plug-ins. These installation commands are also in the README file under the /home/<user>/solvebotics directory.

# install a missing dependency

sudo apt-get install libx264-dev libjpeg-dev

Running this command will inform you that 2,884 kB of additional disk space will be used and will ask if you want to continue, answer yes, or y.

# install the remaining plugins

sudo apt-get install libgstreamer1.0-dev \
libgstreamer-plugins-base1.0-dev \
libgstreamer-plugins-bad1.0-dev \
gstreamer1.0-plugins-ugly \
gstreamer1.0-tools \
gstreamer1.0-gl \
gstreamer1.0-gtk3

Step 3 (R22 Pi) - Determine the IP address and set it as a DHCP reservation in your Wi-Fi router

Eventually, you'll be running the R22 Pi headless (without a monitor, keyboard, or mouse). So, in order to guarantee you can connect to it reliably from the base station, you will need to assign an IP address that stays every time the R22 is powered up. This is known as a static IP address and you can set this up by first learning the IP address of the R22 Pi, then adding a DHCP reservation in your Wi-Fi router configuration.

To determine the IP address, open a terminal window and run the "ifconfig" command.

ifconfig

In the image below, the IP address is 192.168.0.107 and the MAC (ether) address is dc:a6:32:e8:ba:f7

In part one, this address was needed for the KEYctlgst.py script. After setting this IP address as a static IP address make sure the update the KEYctlgst.py script to use this address on line 98.

Both the IP address and MAC address are required for setting up a static IP address using the DHCP reservation setting in your Wi-Fi router.

Refer to the instruction manual for your Wi-Fi router to set up a DHCP reservation.

Step 4 (R22 Pi) - Enable SSH on the R22 Pi

At some point, it will be necessary to edit, troubleshoot, or create scripts, as well as reboot and shut down the R22 Pi from the base station. This will be done by using SSH from the base station to the R22 Pi. For this to happen, you will need to enable SSH by running the raspi-config utility on the R22 Pi. This is the exact same process that was done for the base station Pi, but I will go ahead and repeat it here.

In a terminal window, run the following command to start the raspi-config utility:

sudo raspi-config

When the configuration utility opens, use the down arrow key to highlight "Interface Options", then use the TAB key to highlight the <Select> option and press <ENTER>.

With SSH highlighted, use the TAB key to highlight the <Select> option again, then press <ENTER>

The following prompt will appear - Use the TAB key to highlight the <Yes> option, then press <ENTER>

You will then receive confirmation that SSH has been enabled, as shown below. Use the TAB key to highlight the <Ok> option, then press <ENTER>

This will return you to the main configuration menu. Use the TAB key to highlight the <FInish> option, then press <ENTER> to exit the configuration utility.

Step 5 (R22 Pi) - Download the python and gstreamer scripts to the R22 Pi

Open a terminal window a download a tar file containing the python and gstreamer scripts by running the following command:

wget -O "$HOME/R22robot.tar" "https://09224636-6dc8-4937-beba-f9d4dfaee62b.filesusr.com/archives/322f50_f014c9a0427449a4a8a35b111aa4b60e.tar?dn=R22robot.tar"

In the same terminal window, change to the /home/<user name> directory and untar the file:

tar -xvf R22robot.tar

This will create a "solvebotics" folder under /home/<user name> and will extract:

• R22robot_fdbk.py to the /home/<user name>/solvebotics directory

• R22sendvid.sh to the /home/<user name>/solvebotics/gstreamer directory.

Step 6 (R22 Pi) - Edit the R22sendvid.sh script to point to the IP address of the base station

From the /home/<user name> directory, change to the /solvebotics/gstreamer directory:

cd solvebotics/gstreamer

Make the R22sendvid.sh script executable by running the following command:

chmod +x R22sendvid.sh

Edit the R22sendvid.sh script and change the host IP address to match the static IP address of the base station Pi.

In part 1, the IP address of the base station Pi was determined to be 192.168.0.106, so that will be the value that is set in the R22sendvid.sh script as shown below:

Part 3: Test the Python and GStreamer scripts for functionality

After the above steps have been performed on the R22 Pi and base station Pi, it's time to test the python and GStreamer scripts. This is done by first running the R22robot_fdbk.py and R22sendvid.sh scripts on the R22 Pi, then running the KEYctlgst.py script on the base station Pi.

The R22robot_fdbk.py script uses the following pin assignments on the Pi GPIO header:

DRIVE_1 = 5 # ENA, ENB of L298 driver PCB, connect to Pin #29 of Pi GPIO header

DRIVE_2 = 4 # IN1 of L298 PCB, connect to Pin #7 of Pi GPIO header

DRIVE_3 = 18 # IN4 of L298 PCB, connect to Pin #12 of Pi GPIO header

DRIVE_4 = 7 # IN2 of L298 PCB, connect to Pin #26 of Pi GPIO header

DRIVE_5 = 8 # IN3 of L298 PCB, connect to Pin #24 of Pi GPIO header

DRIVE_6 = 9 # Extra signal, not currently assigned, Pin #21

Notice that pins ENA and ENB of the L298 motor driver PCB have pre-installed jumpers that pull these pins high. As long as these jumpers are in place, Pin#29 of the Pi GPIO header doesn't need to be connected pins ENA or ENB on the L298 board.

Use 8 inch (20cm) female-female jumper leads to connect the Pi GPIO header pins to the correct pins of the L298 motor driver PCB.

IN1 -> Pin#7

IN2 -> Pin#26

IN3 -> Pin#24

IN4 -> Pin#12

On the R22 Pi, open a terminal window and change to the solvebotics directory:

cd solvebotics

Run the R22robot_fdbk.py script by issuing the following command:

python R22robot_fdbk.py

Open a second terminal window and change to the solvebotics/gstreamer directory:

cd solvebotics
cd gstreamer

Run the R22sendvid.sh script with the following command:

./R22sendvid.sh

This should display the camera feed on the R22 monitor

Head over to the base station Pi and open a terminal window, then change to the solvebotics directory:

cd solvebotics

Run the KEYctlgst.py script by issuing the following command:

python KEYctlgst.py

If the IP addresses have been set up correctly on the R22 Pi and base station Pi, then you will see the camera feed from the R22 Pi on the display of the base station Pi. At this point, you should also be able to control the movement of the R22 robot using the arrow keys on the base station keyboard.

Part 4: Set the R22 Pi Python and GStreamer scripts to start upon bootup

If the base station is able to control the R22 robot from the keyboard and receive video by executing the Keyctlgst.py script, then it's time to enable the R22 Pi to automatically start the R22robot_fdbk.py and R22sendvid.sh by adding service unit files to the /etc/systemd/system directory.

When the R22robot.tar file was downloaded and extracted in step 5 of Part2, it created a /solvebotics/service_unit_files_startup directory.

Open a terminal window and change to this directory.

cd solvebotics
cd service_unit_files_startup

Copy the two *.service files over to the /etc/systemd/system directory by running the following commands:

sudo cp R22keycontrol.service /etc/systemd/system/
sudo cp R22keycontrol.service /etc/systemd/system/

Change to the /etc/systemd/system directory:

cd /etc/systemd/system

Edit the R22keycontrol.service file to have the correct username for your system.

sudo nano R22keycontrol.service

Locate lines 6-8 in the file and change them to match the username of your system. In the example below, the current username is "r22b", so lines 6-8 has been changed to use "r22b". When finished press <CTRL>-O to save the changes.

The R22sendvideo.service file also needs to be updated. Edit it with the command:

sudo nano R22sendvideo.service

This is the same process of updating the user name in lines 6-8 to reflect the current user name. The current user name of the system is "r22b", so lines 6-8 have been updated to use "r22b". Press <CTRL>-o to save the file when finished making the updates.

Next, these service unit files need to be enabled so they will start the R22robot_fdbk.py and R22sendvid.sh scripts automatically on startup.

Enable the R22keycontrol.service file by executing the following command:

sudo systemctl enable R22keycontrol.service

Next, enable the R22sendvideo.service file by executing the following command:

sudo systemctl enable R22sendvideo.service

You can check the operation of the service unit file by starting it with the start command:

sudo systemctl start R22keycontrol.service

You can also check the status by issuing the status command:

sudo systemctl status R22keycontrol.service

The status command shows that the service unit file is active and running, and there are no issues. It's time to test the full system by issuing a reboot command.

sudo reboot

If, after the R22 robot boots up, you are able to run the KEYctlgst.py script on the base station Pi and successfully receive video and control the R22 robot, then congratulations—you now have an operational robot!