Make your Raspberry Pi accessible from anywhere

After having setup your Pi on your WiFi and figured out its local IP address, it is probably not reachable from the public internet. Why? Since you are likely behind a NAT or a firewall, it can’t act as a server.

On a home network it is often possible to open a port in your home router (firewall) and let traffic through the Pi. That’s great! But, you will have to do it. Also if you have more than one Pi you will have to start mapping router ports to your different devices, since they share the same public IP.
However, some Internet Service Providers (ISP) will not let you do this. Especially if you have your Pi on a mobile network, like a 3G/4G/LTE dongle.

A solid solution to this problem is tunneling.

Tunnels could be used to encrypt traffic between two parties and also to route encrypted traffic between two parties via a third party.
This third party would typically be a small virtual GNU/Linux server on a cheap cloud provider.

In short the Pi connects to the third party server, creating what nerds call a reverse tunnel. This reverse tunnel makes it so that you (or others) can connect to the Pi.
So in practice you are connecting to the third party server, and the third party will route the connection through the reverse tunnel back to the Pi where the connection you initiated will be answered, just as you would have connected directly to it. Only difference is that you always connect to the third party and all traffic between you and the Pi will go through that third party.
This makes it so that your Pi will always be accessible even though it’s buried behind a NAT or a firewall.

In this tutorial we will show you how to set up your Raspberry Pi for tunneling so you can reach it over the internet wherever it is located.

To help us, we use Space.sh (see the previous post for the shameless self promotion).
In short Space.sh is our command line chaperon to help us build server apps and create automation using Bash and POSIX shell. It is the perfect assistant when dealing with Raspberry Pi and Linux.

1. Preparing your tunnel machine

You will need a virtual machine (VM) somewhere on the internet. Any cheap Linux VM will do.
On this machine you will need SSH running, a user for the Pi to connect as and this user must use ssh keys to login, not password login.

See this post for how Space.sh can swiftly configure such a machine for you (remember Space.sh is your Number #1 command line chaperon).

Now you have:

  • IP address of tunnel machine, e.g. “1.2.3.4”.
  • User to connect as, e.g. “spaceagent”
  • SSH private key which is setup on machine for user spaceagent, e.g. “mykey_id_rsa”.

2. Creating the SpaceAgent

The SpaceAgent is a Space.sh Server Application, which simply is a shell script assembled by Space.sh, that runs on your Pi. This server app will be responsible for connecting to the third party server creating and upholding the reverse tunnel.

If you haven’t already installed Space.sh, do so now:

curl https://get.space.sh | sudo sh

We will be using the Space.sh module gitlab.com/rpi-shell/agent to create the agent:

space -m gitlab.com/rpi-shell/agent /open/ \
      -e SSHHOST=1.2.3.4 \
      -e SSHFLAGS=-ostricthostkeychecking=no \
      -e SSHUSER=spaceagent \
      -e SSHKEYFILE='$0' \
      -e agentid=Green \
      -e bindaddress=0.0.0.0:9333 \
      -e localaddress=127.0.0.1:22 -d > spaceagent.sh

Let me explain what these parameters stand for:
-m gitlab.com/rpi-shell/agent tells Space.sh where the module we are using is located.
/open/ is the node in the modules YAML file which we are running.
-e SSHHOST is an environment variable which is used by the SSH module to let it know what server to connect to.
-e SSHFLAGS is to tell SSH not to check the host key, because the agent is meant to be run in the background.
-e SSHUSER is the user created on the VM for our Agent.
-e SSHKEYFILE the filename of the private key file for the agent to use (note on this below).
-e agentid The unique ID we want to give our Agent.
-e bindaddress the address on the third party server which the client will connect to. If you put 0 as port, a random port will be assigned when the agent connects, which is useful when setting up many Pis with the same process.
-e localaddress the address on the Pi which is the destination of the connecting client.
-d (dump) tells Space.sh that we do not want to run this now, we want the script output to stdout instead.
> spaceagent.sh tells your shell to redirect (save) the output to the file spaceagent.sh.

So now you have the two files of interest the agent script and the private key file. To make things easier for us we will bundle these two files together, by appending the key file to the script. This is why SSHKEYFILE is '$0' above. In shell programming $0 refers back to the script file itself, which also will contain our key file, isn’t that neat!

echo "exit" >> spaceagent.sh
cat mykey_id_rsa >> spaceagent.sh
chmod 700 spaceagent.sh  # Executable and owner only permissions.

First we append an “exit” to the bottom of the file, so we then can append the key file contents without causing any syntax errors.

The file spaceagent.sh will now look something like this:

#!/usr/bin/env sh
# Script assembled and exported by:
#   ____  ____   __    ___  ____  ___   __   __
#  / ___)(  _ \ / _\  / __)(  __)/ __) / _\ (  )
#  \___ \ ) __//    \( (__  ) _)( (_ \/    \/ (_/\
#  (____/(__)  \_/\_/ \___)(____)\___/\_/\_/\____/
#                                         space.sh

[ ... some neat shell code here ... ]

main
exit
-----BEGIN RSA PRIVATE KEY-----
[ ... key contents here ... ]
-----END RSA PRIVATE KEY-----

So this is your Space Agent, ready to rock!

You could test your agent on your laptop before copying it over to the Pi.

Preemptive troubleshooting

To not get ahead of our selves, we should test some things first to make sure everything works before deploying the agent onto the Pi.

Test SSH connection to server

First we’ll test so that the server is responding and we can connect to it as expected.

space -m ssh -e SSHHOST=1.2.3.4 -e SSHUSER=spaceagent \
      -e SSHKEYFILE=mykey_id_rsa

If this does not work then you have to skid back to configuring the server.

Now close the connection.

Test reverse tunnel from server

Now that we know the SSH connection works we want to test that the server allows us to create a reverse tunnel.

space -m ssh -e SSHHOST=1.2.3.4 \
      -e SSHUSER=spaceagent \
      -e SSHKEYFILE=mykey_id_rsa
      -e SSHTUNNEL=0.0.0.0:9333:127.0.0.1:9333

If it did not work then the server does not allow reverse tunnels, so that has to be configured by setting GatewayPorts yes in the sshd_config on the server and issue a systemctl restart ssh or service ssh restart.

If the connection stays open the tunnel is alive, leave it open and proceed to next step to test it.

Test tunnel using Transfer module.

Now we will use two other terminal windows to test the communication through the tunnel back and forth to our laptop. We will use the Space.sh module Transfer to do this.

In the first terminal window:

space -m transfer /listen/ \
      -e host=127.0.0.1 -e port=9333
[WARN]  TRANSFER_LISTEN: STDOUT is a terminal, if you are expecting a file transfer you might want to redirect stdout to a file.
[INFO]  TRANSFER_LISTEN: Listening on 127.0.0.1:9333.

If you get the above output, goodie, now we are posing as the Pi listening on the port 9333 on our laptop.

Now we open the second terminal window to pose as the client connecting to the Pi via the tunnel.

echo "Hello Pi, from Client!" | \
        space -m transfer /connect/ \
        -e host=1.2.3.4
        -e port=9333

In the first window you should now see:

Hello Pi, from Client!
[OK]    Script exited with success (status 0) for node /listen/

Great! This means that the tunneling is working.
Exit the tunnel using ctrl-c. Now let’s resume the Agent installation.

3. Install Agent on Pi

Now we want to copy over the Agent we created (spaceagent.sh) to the Pi and make it into a service which runs automatically.

Mounting the SD card

If your device is named something else than /dev/mmcblk0 apply that for the -e device= flag.
For more details about mounting the SD card check out the previous tutorial._

space -m gitlab.com/rpi-shell/manage /sdcard/mount/ \
      -e device=/dev/mmcblk0 -s sudo

Copying over the Agent to the SD card

space -m gitlab.com/rpi-shell/manage /sdcard/root/cpbin/ \
        -e file=spaceagent.sh
        -s sudo

Great, now we have put the Agent script into the Pi’s /bin directory, so now one could run the script manually from the device to create the tunnel, but we of course want the tunnel to start automatically at boot.

Creating a systemd service

Systemd is an init system which is used on many GNU/Linux distributions, including Raspbian and HypriotOS.

We will use the Space.sh module Systemd to help us.

space -m systemd /service/create/ \
      -e description="The SpaceAgent" \
      -e root=/mnt/sdcard/root
      -e service=SpaceAgent \
      -e execstart=/bin/spaceagent.sh \
      -s sudo

The above creates the systemd service on the SD card for us, but we will also need to enable it to be run on boot:

space -m systemd /service/enable/ \
      -e root=/mnt/sdcard/root \
      -e service=SpaceAgentGreen \
      -s sudo

Unmounting the SD card

space -m gitlab.com/rpi-shell/manage /sdcard/umount/ \
      -e device=/dev/mmcblk0 \
      -s sudo

4. Congratulations

Pop out the SD card and pop it into the Pi, power it up, wait some moments then try to connect to it.

ssh pi@1.2.3.4 -p 9333

You can also list all the Pi’s connected to the server using the agent module:

space -m gitlab.com/rpi-shell/agent /list/ \
      -e SSHHOST=1.2.3.4
      -e SSHUSER=spaceagent \
      -e SSHKEYFILE=mykey_id_rsa

… and get information about a specific connected Agent:

space -m gitlab.com/rpi-shell/agent /get/ \
      -e SSHHOST=1.2.3.4
      -e SSHUSER=spaceagent \
      -e SSHKEYFILE=mykey_id_rsa
      -e fullagentid=0.Green.0

5. Power saving extras

If your Pi is running on battery power, or has a very limited connection, you might want to restrict the time it keeps the tunnel open.
To have the Pi open the tunnel for five minutes every full hour do these changes to the above installation:

Set a timeout on the tunnel

When creating the Agent script, add the -e timeout parameter so that the Agent automatically closes the tunnel after five minutes (300 seconds).
Note that any ongoing connection will not forcefully be closed.

Creating the agent

space -m gitlab.com/rpi-shell/agent /open/ \
      -e SSHHOST=1.2.3.4 \
      -e SSHFLAGS=-ostricthostkeychecking=no \
      -e SSHUSER=spaceagent \
      -e SSHKEYFILE='$0' \
      -e agentid=Green \
      -e bindaddress=0.0.0.0:9333 \
      -e localaddress=127.0.0.1:22 \
      -e timeout=300 -d > spaceagent.sh

Bundle the key file into the script and copy it to the SD card as showed above.

Creating a systemd timer

Make sure the SD card is mounted.

Above we created and enabled the systemd service. But to use a timer we do not want the service enabled.

if it is enabled you have to disable it:

space -m systemd /service/disable/ \
      -e root=/mnt/sdcard/root \
      -e service=SpaceAgentGreen \
      -s sudo

Create and enable a systemd timer which will start the Agent service for us:

space -m systemd /timer/create/ \
      -e description="The Green SpaceAgent timer" \
      -e root=/mnt/sdcard/root
      -e timer=SpaceAgentGreen \
      -e oncalendar=hourly -s sudo

space -m systemd /timer/enable/ \
      -e timer=SpaceAgentGreen \
      -e root=/mnt/sdcard/root
      -s sudo

Now the Agent will connect every full hour, stay connected for five minutes waiting for a client connection, if none, it closes the tunnel and waits until the next full hour.

6. Adding security

The Agent above is listening on the interface 0.0.0.0 on the remote ssh server, meaning that it will accept connections from the public internet.

For a SSH service or web server this is often what we want, but for a Virtual Network Computing (VNC) service or other similar sensitive connections we could tighten the security by listening to the local interface 127.0.0.1 instead.

This would mean that to be able to connect to the reverse tunnel port on the remote server we would first have to create a forward tunnel from our laptop to the server, and then make a VNC connection to our local port (forward tunnel->reverse tunnel->Pi).

The Space module rpi-shell/agent has the node /connect/ that could help you set that up for a specific Agent.

Thanks a lot for following along this tutorial. As you might have guessed, an Agent could be connected to a HTTP server or any other service you want to expose.

This tutorial was written by @ThomasBacklund.
Please share this tutorial if you find it useful. Check us out on GitLab , Twitter and other social media following the links at the bottom of this page.