Setting up custom DNS routing on EKS Cluster

Our one of the third party API URL was failing to resolve, so we figured out the solution to route through Google Public DNS, thus changing the routing of a particular domain from EKS Default DNS ( 10.100.0.10 ) to resolve using Google Public DNS.
We used 8.8.8.8, the primary DNS server for Google DNS, in order to function it correctly.

Configure Conditional Forwarder with CoreDNS in Amazon EKS cluster

What is CoreDNS?

• CoreDNS is a DNS server that is modular and pluggable, and each plugin adds new functionality to CoreDNS. This can be configured by maintaining a Corefile, which is the CoreDNS configuration file.
• As a cluster administrator, you can modify the ConfigMap for the CoreDNS Corefile to change how DNS service discovery behaves for that cluster.
• CoreDNS uses negative caching whereas kube-dns does not (this means CoreDNS can cache failed DNS queries as well as successful ones, which overall should equal better speed in name resolution).

You can use CoreDNS to configure conditional forwarding for DNS queries that are sent to the domains resolved by a customized DNS server(like Google DNS Server).

How Amazon EKS uses CoreDNS?

Pods running inside the Amazon EKS cluster use the CoreDNS service’s cluster IP as the default name server for querying internal and external DNS records.

You can follow the mentioned steps to modify the CoreDNS ConfigMap and add the conditional forwarder configuration.

1. Run the following command:

$ kubectl -n kube-system edit configmap coredns

Output of the command should be:

apiVersion: v1 
kind: ConfigMap 
metadata: 
  annotations: 
  labels:
    eks.amazonaws.com/component: coredns 
    k8s-app: kube-dns 
  name: coredns 
  namespace: kube-system 
data: Corefile: | 
        .:53 { 
            errors 
            health 
            kubernetes cluster.local in-addr.arpa ip6.arpa { 
              pods insecure 
              upstream 
              fallthrough in-addr.arpa ip6.arpa 
            } 
           prometheus :9153 
           proxy . /etc/resolv.conf 
           cache 30 
           loop 
           reload
           loadbalance 
       } 
       domain-name:53 { 
           errors
           cache 30 
           forward . custom-dns-server 
           reload 
     }

Note: We have customized the above configMap with the domain-name “plapi.ecomexpress.in. Replace it with your domain name.

The custom-DNS-server IP address for Google DNS is used, that is (8.8.8.8). Replace the custom DNS server IP address with your custom DNS server IP address.

2.The final CoreDNS ConfigMap will look like:

apiVersion: v1
data:
    Corefile: |
         .:53 {
               errors
               health
               kubernetes cluster.local in-addr.arpa ip6.arpa {    
                   pods insecure
                   upstream
                   fallthrough in-addr.arpa ip6.arpa
                 }
                 prometheus :9153
                 forward . /etc/resolv.conf
                 cache 30
                 loop
                 reload
                 loadbalance
}
plapi.ecomexpress.in:53 {
       errors
       cache 30
       forward . 8.8.8.8
       reload
}
kind: ConfigMap

3. To verify that domain-name resolution works, run the following command:

prod@ip-192-168-X-XXX:/home/devtron$ kubectl exec busybox -- nslookup domain-name.in

Note: Replace the domain-name with your domain name.

The output before updating custom route for CoreDNS:

prod@ip-192-168-X-XXX:/home/devtron$ kubectl exec busybox -- nslookup plapi.ecomexpress.in

Server:    10.100.0.10
Address 1: 10.100.0.10 kube-dns.kube-system.svc.cluster.local
Name:      plapi.ecomexpress.in
Address 1: 172.20.92.37 ip-172-20-92-37.ap-south-1.compute.internal
Address 2: 172.20.54.52 ip-172-20-54-52.ap-south-1.compute.internal

The output after updating custom route for CoreDNS:

prod@ip-192-168-X-XXX:/home/devtron$ kubectl exec busybox -- nslookup plapi.ecomexpress.in

Server:    10.100.0.10
Address 1: 10.100.0.10 kube-dns.kube-system.svc.cluster.local
Name:      plapi.ecomexpress.in
Address 1: 35.154.40.19 ec2-35-154-40-19.ap-south-1.compute.amazonaws.com
Address 2: 3.6.218.14 ec2-3-6-218-14.ap-south-1.compute.amazonaws.com