Beyond Cloudflare Tunnels: A High-Bandwidth Solution for Home Lab Remote Access

For many, a Home Lab is a useful tool for playing around with and learning about deployment and development. There are many useful tools you can host for yourself to lessen your reliance on Big Tech, including apps like Immich for photo management or Mealie for recipe sharing.

My personal Home Lab started as a Minecraft Server running on a copy of Windows 7. Over the years, I learned how to set up a Hypervisor to ensure each application is isolated, a backup system to quickly fix changes that break, a NAS (Network Attached Storage) to store large datasets, and much more.

Sharing A Home Lab

From the beginning, my goal was to share these services with my friends and family. The main hurdle to this was a Domain Name and having 1-to-1 Nat Back. The domain name was easy enough. Buy a domain with a vendor, then configure it to point to your server. The main issue is, where is your server?

The 1-to-1 Nat Back is how the IP address assigned to your house is mapped. When a message comes in, it goes to the IP plus a port. Some internet service providers assign a single IP address per customer. When I started, this is what my ISP did. This allowed me to treat the IP address as a Dynamic IP and to route to that. Others will have many customers under the same IP address and split the traffic using a subdomain. Doing this allows the ISP to reduce the number of IPs they need, but this prevents me from accessing my home lab!

Luckily, when I moved, I was able to purchase a static IP address from my ISP. This prevented any issues with routing. But this solution may not work the next time that I move. Now that my home lab has become part of my everyday life, I need a backup in case my ISP changes its services or I move.

A Partial Solution - Cloudflare Tunnels

A large portion of my Home Lab's services are various websites (including this one) and low-bandwidth applications. Luckily for us, the solution is simple: Cloudflare Tunnels.

Why this works

My domains are already defined using Cloudflare's DNS service. Enabling Cloudflare tunnels lets me route traffic to specific (and some wildcard) addresses directly to my Home Lab.

Doing so allows my Home Lab to act as the initiator for the connection, meaning that Cloudflare doesn't need to know my Home Lab's IP address and all traffic can be sent through this tunnel without needing any port forwarding and without needing any configuration.

When the tunnel data comes in, we reroute to my reverse proxy (since all items are web data), which then determines where the request should go based on the request's URI.

Best of all, this is all available with the free account that Cloudflare offers.

Why this doesn't work

Unfortunately, Cloudflare is very strict about high-bandwidth applications. While my recipe site, Mealie, will work, heavy data sites, like Immich, that may transfer gigabytes of data in a single day, may cause Cloudflare to force me onto a paid account.

Not wanting this, I have to look for a different solution similar to Cloudflare, but without the bandwidth limits.

A High Bandwidth Solution - VPS + WireGuard

What we need is a 'Gateway' server that can accept traffic and forward it to our local network. The requirements are to have something super inexpensive, physically close to my location (to lower latency), and unlimited (or very generous) data caps.

Selecting a VM Provider

I ended up looking at several providers who offered these requirements (or very close to these):

My initial choice was BuyVM due to their reputation I saw on Reddit. Unfortunately, when I went to sign up, they were out of space in all regions!

After more research, I settled on IONOS for my host. Their smallest offering is a 1 vCore CPU, 1 GB of RAM, and 10 GB of NVMe Storage. This should be enough to run a VPN and forward any incoming traffic.

The Plan

We will treat our new VM as the Gateway for our traffic. In our DNS, any traffic that is high-bandwidth will point to the VM's IP address.

From here, we will host a WireGuard server and an Nginx server. The Nginx Server will listen for any incoming connections we are about and forward them to the appropriate service using the IP address of the VM that handles it in the Home Lab.

In the Home Lab, we also launch a tiny VM instance that only runs a WireGuard client. This will connect to the WireGuard server (the Gateway) and keep that connection alive. We will also configure it to allow connections from the server to our local subnet, thus allowing connections from the server to any service in the subnet.

Setting Up The Gateway and the Client

On both the Gateway and the Local Client, I installed the latest headless Ubuntu. This is mostly because I am most familiar with the OS.

The Gateway is the VM we just purchased that has a public IP address. Typically, VMs like this use root as the default user to login as.

The Local Client will be our 'tunnel' exit for the WireGuard VPN. In my case, I set up a Proxmox Container for this and have it a static IP on my subnet. Adding the static IP address means I always know where the container is in the future. I had the container configured with 1 CPU, 512MB of RAM, and 8GB of SSD storage. The default user is also root here.

Once both VMs are up and running, SSH into both machines. I did this in two different terminal windows. (Gateway and Local Client)

ssh root@<vm-ip-address>

As usual, once I set up the accounts and log in, I update everything.

sudo apt update && sudo apt upgrade -y

This will ensure that all packages are up to date and that we have a clean baseline on both machines.

It is typically good to set up your SSH keys here, too. This will make it easier to log in and work on the machine in the future.

# Example on adding a key to allow quicker logins
echo "<public ssh key>" >> ./.ssh/authorized_keys

On both machines, we need to install WireGuard. We want to keep the installation light since we have limited drive space.

sudo apt install wireguard-tools --no-install-recommends -y

The --no-install-recommends will ensure we only install WireGuard and nothing extra.

Once this finishes, we will need to generate keys for both the Gateway and the Local Client, allowing the two to identify each other and preventing a third party from sniffing our data.

Setting Up WireGuard

On both VMs, in the home directory, run this:

umask 077
wg genkey | tee privatekey | wg pubkey > publickey

This will generate our keys and dump our private and public keys in the home directory as privatekey and publickey, respectively.

To set up WireGuard on the server, we need to define a WireGuard configuration file. To create and edit the file, we'll use nano:

sudo nano /etc/wireguard/wg0.conf

[Interface]
PrivateKey = <Gateway PrivateKey>
Address = 10.0.0.1/24
ListenPort = 51820

[Peer]
PublicKey = <Local Client PublicKey>
AllowedIPs = 10.0.0.2/32, 192.168.67.0/24

To note, we have 2 main subnets we care about.

10.0.0.0/24 is the subnet defining our WireGuard instance. The first 3 numbers are static, and the third is not. In our configuration above, we also set the Local Client to 10.0.0.2 and lock it by setting the configurable bits to '32', which means the IP address must match exactly.

192.168.67.0/24 is the subnet of the local network where all my services live. We let the last number be modifiable and assign this subnet to the Local Client so that any connections to the Gateway server within that subnet are redirected to the Local Client.

On the Local Client, we also need to define a WireGuard configuration file. To create and edit the file, we'll use nano again:

sudo nano /etc/wireguard/wg0.conf

[Interface]
PrivateKey = <Local Client PrivateKey>
Address = 10.0.0.2/24
PostUp = sysctl -w net.ipv4.ip_forward=1; iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 >
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE

[Peer]
PublicKey = <Gateway PublicKey>
Endpoint =  <Gateway Public IPv4 Address>:51820
AllowedIPs = 10.0.0.0/24
PersistentKeepalive = 25

We can see on the local client that an address is defined and that we have the additional values of PostUp and PostDown which add modifiers to the WireGuard connection.

The PostUp commands will run when the WireGuard instance is initiated.

• sysctl -w net.ipv4.ip_forward=1: This enables IP forwarding. This will ensure that packets from the VPN are allowed to reach the local network.
• iptables -A FORWARD -i wg0 -j ACCEPT: This will add a rule to the firewall to explicitly allow traffic from our WireGuard connection (wg0) to be forwarded.
• iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE: This is the most important part of these commands. Since we aren't getting data from the gateway, we need to masquerade or pretend that the request is coming from the local client directly. This will ensure that the service will return the request to this VM so the response can be sent back through the VPN connection to the Gateway before being returned to the actual requester.

Similarly, the PostDown commands will run immediately after the WireGuard instance is disabled. This will essentially clean up all of the PostUp commands.

• iptables -D FORWARD -i wg0 -j ACCEPT: The -D stands for Delete. It'll remove the similar rule from the PostUp command.
• iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE: This will also Delete the masquerade rule.

We don't need to worry about deleting the IP forwarding, so we'll leave that as is.

We can also see that we now define the Endpoint in the Peer, allowing the Local Client to know where the Gateway is.

We also set PersistentKeepalive to ensure the WireGuard connection isn't closed prematurely.

Once these are setup, we can run the following command to ensure that WireGuard is running on both VMs.

sudo wg-quick up wg0

Modify VPS Firewall Settings

Before the Local Client can see the Gateway, we need to update the default ports so that the Gateway is allowed to accept connections.

I will give the instructions for IONOS, but you should be able to look up instructions for any other service that you may be using.

Overall, we want to keep the firewall and not remove it completely. This prevents attack vectors from unmonitored ports.

Once you login to the IONOS dashboard, be sure to navigate to the Servers & Cloud section. From here, in the left-hand menu, navigate to Network > Firewall Policies. You'll see a default policy there, but IONOS only allows one policy per VPS, so we'll recreate the default policy and modify it to our own needs.

Select Create and name the policy something like 'Gateway Server'. We can then add ports allowed for the incoming section. All of these will be the 'Allow' action with 'All' set for the Allowed IPs.

For now, we'll keep SSH (how we talk to the server), Web Server Ports, and add WireGuard (so our Local Client can find the Gateway).

Once these are added, create the Policy then go to edit it. When editing, add your VM under the 'Assigned IP' section. Once assigned, it'll take a few moments for the configuration to update, then the ports should be opened.

Test Your WireGuard Connection

To confirm that the WireGuard connection is working, on the Local Client, try running ping 10.0.0.1 which will ping the Gateway. If the ping is successful, the WireGuard connection is established.

If it doesn't work, you can try rebooting WireGuard on the client (it may have given up while we modified the firewall for the Gateway).

sudo wg-quick down wg0 && sudo wg-quick up wg0

If things are still not working at this point, try checking firewalls and checking your configuration. Ensure you set the right keys in the right spot.

Test Your Forwarding of Data

If the WireGuard Connection is working, we next need to check that the Gateway can connect to specific services on your home lab subnet.

I personally run my Immich server at 192.168.67.33 so on the Gateway, I want to test its connection by running:

ping 192.168.67.33

If successful, this means that the Gateway can connect to the local network.

If not, try checking on your Local Client VM. If that can't reach the service, there's likely something wrong with the service. If it can, then there is likely an error in the WireGuard configuration. Be sure that the packets are being masqueraded and that IP forwarding is enabled on the Local Client.

Setting Up the Proxy

Next, configure the Gateway to listen for connections and forward them. To start, install Nginx and the Stream package:

sudo aput install nginx libnginx-mod-stream -y

The stream package is required as it allows us to forward data at Layer 4 (the transport layer). This prevents us from needing any special configuration for web traffic and enables us to forward data for non-web applications.

We will then modify the nginx configuration file:

sudo nano /etc/nginx/nginx.conf

user www-data;
worker_processes auto;
pid /run/nginx.pid;
error_log /var/log/nginx/error.log;
include /etc/nginx/modules-enabled/*.conf;

events {
        worker_connections 768;
        # multi_accept on;
}

stream {
    # Disable Logging. Lowers Disk Usage
    access_log off;

    # Forward Web Traffic
    upstream backend_80 {
        server 192.168.67.99:80;
    }
    server {
        listen 80;
        proxy_pass backend_80;
    }
    upstream backend_443 {
        server 192.168.67.99:443;
    }
    server {
        listen 443;
        proxy_pass backend_443;
    }
}

We will replace the whole file with the above. The http block will be removed and replaced with the stream block. We explicitly forward our traffic on 80 and 443 to another service.

In my case, I host an instance of Traefik on 192.168.67.99, which will redirect any web traffic to the appropriate service. You can always use the http block if you'd like to use this as your reverse proxy.

Once the configuration is updated, we need to validate and restart nginx:

sudo nginx -t && sudo systemctl restart nginx

Once done, our traffic should now be redirected from the Gateway to the correct host.

Testing Packet Forwarding

On the Gateway, we can test that our data is being forwarded by nginx by sending a web request.

curl -I localhost:443

Assuming the nginx http block has been removed, we should expect this message to be forwarded to the service that handles it. In my case, Traefik will respond to the request when the request is successful.

I get HTTP/1.1 404 Not Found back from Traefik, confirming that the connection was successful.

If you get errors or an empty response, check that the correct service is set in your nginx configuration and that that service works locally as well. You can curl directly to that service on the Local Client to see the expected response.

All Set Up!

At this point, you're all set up. You can add additional ports to nginx to ensure they're forwarded. Just remember to add them to the Firewall so that the Gateway server can see them.

Keep the VMs Chugging

Additional steps can be taken to help reduce the maintenance of the VMs. Lowering disk usage and setting file links to make the configuration easier to find.

Reducing Disk Usage

Since we have very small drives for both our VMs, we need to update some configurations to keep disk usage low. The primary killer of our services is logs, so we should reduce the number of logs that are saved.

First up, we want to clean up any unused packages from the system plus any downloads that are not used anymore:

sudo aput autoremove -y
sudo apt clean

Next, we want to update the nginx log rotation to store logs for only 2-3 days. Any more is unnecessary.

sudo nano /etc/logrotate.d/nginx

Edit the line

rotate 7

so that it says

rotate 3

Once done, reboot nginx:

sudo systemctl restart nginx

Next, we'll update the system logs so that they have an explicit total size cap.

sudo nano /etc/systemd/journald.conf

We'll see this:

[Journal]
# ...
#SystemMaxUse=
# ...

Uncomment SystemMaxUse and set it to 50M

[Journal]
# ...
SystemMaxUse=50M
# ...

This will prevent the system logs from getting way too large. Save this file and be sure to edit the system logs limit on both the Gateway and the Local Client.

Luckily, WireGuard runs silently, so no log modifications are needed there.

Setting File Links

Personally, I never remember where any configuration files are. As a result, I f how to update the services on the VMs.

To solve this, I typically add syslinks to configuration files in my home drive so that when I log in, I can see the stuff I want to update right away.

To do this, we'll update the mount points via editing the fstab file:

sudo nano /etc/fstab

We'll add in links for the WireGuard Config on both VMs, but on the Gateway, we will also add in a link for the nginx config file. Add the following lines to the fstab:

/etc/wireguard /root/config/wireguard none bind 0 0
/etc/nginx/nginx.conf /root/config/nginx.conf none bind 0 0

Note: Only add the lines for the files you want to mount. I also am setting these to the root's home folder which is at /root and not at /home/<user>.

Save these changes, then let's create the folders needed:

sudo mkdir /root/config
sudo mkdir /root/config/wireguard

Now, we mount the file by running:

sudo mount -a && sudo systemctl daemon-reload

Once done, we can see our files under ~/config and can now easily find them next time we log in.

Conclusion

Having a home lab can be challenging. Many times we encounter problems that can feel overwhelming or impossible to fix. However, with time, practice, and a lot of Googling, we can figure things out.

Cloudflare Tunnels and our custom Gateway ensure we don't need to expose our server to the public internet. Services like IONOS also offer DDOS protection, ensuring that our home network can't be taken down even if one of our gateways is attacked.

These options are not always free, but being able to choose the cheaper ones lets us continue to grow and explore while still making mistakes.

Enjoy Building!