Document how to configure K3S to resolve custom domains to the host

[butonic]

When I tried to deploy the ocis-charts to k3s running on a remote machine I had to tell CoreDNS how to resolve domains like ocis.kube.owncloud.test to the host raspbian-bullseye-arm64. To authenticate users via OIDC, the proxy service has to be able to resolve the extarnalDomain:ocis.kube.owncloud.test. While I found out how to solve it, the below snippet should be added to documentation. But where?

Looking at the K3S CoreDNS config we can see that it imports *.override files.

Googling how to provide them we can find MS Docs on how to customize CoreDNS.

Since we want to point all *.kube.owncloud.test domains to the host, we can use CorDNS rewrite plugin like this:

apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns-custom
  namespace: kube-system
data:
    rewritehost.override: |
          rewrite name regex (.*).kube.owncloud.test raspbian-bullseye-arm64

Replace raspbian-bullseye-arm64 with your hostname and (.*).kube.owncloud.test with a regex for the domains you want to point back to the host.

Apply the config and restart CoreDNS:

$ kubectl apply -f corednsms.yaml
$ kubectl -n kube-system rollout restart deployment coredns

Finally, test that you can now resolve the domain:

$ kubectl -n ocis get pods | grep proxy
proxy-76bdf4bdb6-j5rmp               1/1     Running   0             25h
$ kubectl -n ocis exec proxy-76bdf4bdb6-j5rmp -- ping ocis.kube.owncloud.test
PING ocis.kube.owncloud.test (192.168.1.208): 56 data bytes
64 bytes from 192.168.1.208: seq=0 ttl=42 time=0.243 ms
64 bytes from 192.168.1.208: seq=1 ttl=42 time=0.359 ms

Profit!

[dragotin]

Please do add to developer docs, but not the "official" documentation. For that, it is out of scope.

[d7oc]

Just for completeness: host aliases might come in handy here as well. For k3d see k3d-io/k3d#938, but there is a caveat: k3d-io/k3d#973.

Maybe we should consider solving this in Kubernetes via https://kubernetes.io/docs/tasks/network/customize-hosts-file-for-pods/.