Self-hosting, trouble in paradise

August 15th 2022 | ~ 5 minute read

Introduction

Ever wanted to host services from the comfort of your home? I know I did. There's a plethora of awesome self-hosted applications you can install on something as cheap as a raspberry pi. It can give you a real sense of control and let's be honest It's every tinkerer's dream come true.

But, something's not right. After countless attempts at deploying the app of your dreams, you lay in the gutter, utterly defeated. As it turns out your problems are caused by every network administrator's worst enemy, DNS.

It's not DNS

The heading says it all. You set up your service expecting it to work. You got all those pesky DNS records in order. All A, AAAA, CNAME and especially all MX records are there and are correct. You fire up your browser, type in the domain to the address bar expecting it to work. No dice. The page loads forever before failing.

At this moment, depending on your sensibilities, you either reach for your intoxicant of choice to drown your sorrows or, proceed to pull every strand of hair from your head in frustration. You desperately reach for the terminal. A quick dig command confirms your suspicions.

; <<>> DiG 9.18.5 <<>> mysuperawesome.app
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19567
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;mysuperawesome.app.			IN	A

;; ANSWER SECTION:
mysuperawesome.app.		10	IN	A	93.184.216.34

;; Query time: 3 msec
;; SERVER: 192.168.1.3#53(192.168.1.3) (UDP)
;; WHEN: Mon Aug 15 23:45:26 CEST 2022
;; MSG SIZE  rcvd: 48

Wait!? That's not even the right IP address!

-- You, probably

Somehow, somewhere, instead of resolving the application to your local, private IP address, DNS decides to resolve it to your public IP address instead.

And why should it not? It was, in fact, you who told it to do precisely that. The A records don't lie.

Right about this time, as you inevitably pour another glass of your favorite intoxicant, you realize that you're hosting your services behind NAT. Ouch.

It's NAT, probably

NAT is a peculiar beast. Without form, it's but a vicious animal, but tamed, it single-handily solves the IPv4 address shortage.

You calm down, carefully weighing in your options. You decide to try loading your app outside your home network. As expected, it loads instantly and everything seems to be working fine.

You see, after resolving the domain name to your public IP, your computer sends packets with its private IP address in the source field, but the destination field lists your public IP address as the recipient. When this packet leaves the network, the source field is replaced with your public IP, as NAT usually does. The problem occurs when the packet returns to your home network, both fields contain the same IP address. Unless specifically configured to do otherwise, the router considers this configuration invalid and drops the packet.

Before NAT

After NAT

How to actually solve this mess

Use a NAT loopback

If your router supports it, this is the most straightforward solution you can employ. This will make it so that the router doesn't simply drop the packet, but redirect it to its proper destination. It'll replace the destination IP address with the appropriate private IP of the machine that hosts the requested service.

Buy a router that supports NAT loopback

Most residential routers unfortunately don't support NAT loopback. You can simply buy a new router that does. For this you'll need to create what's known as a network bridge, connecting your old and your new router. This requires using something called PPPoE (Point-To-Point over Ethernet). Unless you wanted a new router already, buying one specifically for this is not recommended.

Configure split DNS

A bit more involved than enabling NAT loopback, but still better than buying a new router, you can leverage your host machine to manage DNS in your local network. I recommend using something like AdGuard Home or PiHole, simply because they streamline the process and also offer network-wide ad blocking as well.

The idea is to rewrite the resolved DNS queries to point to your private IP address instead of your public one. So, instead of myawesome.app resolving to 93.184.216.34, It'll instead resolve to 192.168.1.2. All you've got to do next is configure your router's DHCP server to point all DNS queries to your host machine. Problem solved.

Use your host machine as a DHCP server to facilitate split DNS

In a perfect world, your router would offer a way to tell all its clients where to send their DNS queries. But what if it doesn't? Fear not, there is still a drastic measure you can undertake to win this battle. Turn off your router's DHCP server and configure your own.

It helps that both AdGuard Home and PiHole offer this out of the box. 😉

Afterwards you just need to configure split DNS as normal.

What did we learn here anyway?

The last drops of that drink from earlier enter your mouth. You're calm and confident. You solved your networking issues. All your services are working like a finely tuned orchestra.

I guess the moral of the story is the age old maxim: Computers do what we tell them, not what we want them to do.

It's perfectly normal for DNS to resolve to the address you configured it to resolve to.

It's perfectly normal for NAT to replace that source IP address.

It's perfectly normal for your router to drop a malformed packet.

We like to think of computers as these magnificent machines, capable of almost anything. The reality is that computers are in fact pretty dumb. They just follow orders. It helps when those orders are actually correct too!