Application Cannot Resolve a Hostname (no such host, Temporary Failure in Name Resolution)

Test DNS lookups from a pod when your application cannot resolve a hostname, match the result to the cause, and know what to send if the platform is at fault.

If your application cannot find a hostname from inside the cluster — an internal service, a managed database endpoint, an external API or a host across a VPN — or DNS lookups fail or time out only some of the time, this page shows you how to test DNS from a pod yourself and what to send us if the problem is on the platform side.

Symptoms

Your application logs one of these, or something close to it. The exact wording depends on the language and library:

You seeTypical source
dial tcp: lookup db.example.com on 192.0.2.10:53: no such hostGo
Temporary failure in name resolutionPython, curl, apt, anything using the system resolver
java.net.UnknownHostException: db.example.comJava
getaddrinfo ENOTFOUND db.example.com or getaddrinfo EAI_AGAIN db.example.comNode.js
lookup db.example.com: i/o timeoutGo, when the DNS server does not answer in time
Unknown MySQL server host 'db.example.com'MySQL clients
No resolvable bootstrap urls given in bootstrap.serversKafka clients

no such host, ENOTFOUND and UnknownHostException usually mean the name does not exist where the pod looked for it. Temporary failure in name resolution, EAI_AGAIN and i/o timeout usually mean the DNS server did not answer — that points more towards a network or platform problem, especially when it happens only sometimes.

Step 1 — test the lookup from a pod

Testing from your own laptop tells you nothing about what the pod sees: the cluster has its own DNS, and private names inside the environment do not resolve from outside it.

  1. Get namespace access and log in to the cluster as described in Namespace Access. Running commands in a pod needs a role that includes exec.

  2. Open a shell in a pod of the affected application:

    kubectl exec -it <pod-name> -n <namespace> -- /bin/sh
  3. Look the hostname up. Use whichever tool the image has:

    nslookup db.example.com
    dig db.example.com
    getent hosts db.example.com

    getent hosts is present in most images and uses the same resolver path as your application, so it is the closest match to what your code sees.

If your application image has no shell or no DNS tools, start a throwaway debug pod in the same namespace instead. It carries nslookup, dig and database clients, and deletes itself when you exit. Starting a pod needs a role that can create pods in the namespace; if yours cannot, request a higher role through Access Requests.

kubectl run -it --rm dns-debug -n <namespace> --image=cto2b/debug:latest --restart=Never -- /bin/bash

Use the same namespace as the failing application, because the DNS search domains a pod uses depend on its namespace.

Step 2 — match the result to the cause

An internal service name does not resolve

Services inside the cluster are reachable by these names:

NameWorks from
<service>Pods in the same namespace only
<service>.<namespace>Any namespace
<service>.<namespace>.svc.cluster.localAny namespace — the full name
  1. Check the Service exists and the name is spelled correctly:

    kubectl get svc -n <namespace>
  2. If the caller is in a different namespace, use <service>.<namespace> or the full name. A short name only works inside the same namespace.

  3. If the full name does not resolve from a debug pod either, and the Service exists, the problem is on the platform side — raise a request (see below).

If the name resolves but the connection then fails or hangs, DNS is working. Check Network Policies for traffic between namespaces instead.

A managed database endpoint or other private name does not resolve

  1. Compare the hostname your application actually uses with the endpoint the platform injects. If the database is linked through a dependency block, the injected endpoint is in the pod's environment — for example env | grep DB_HOST from the pod shell. A typo or a hand-typed hostname is the most common cause.
  2. After a migration, a restore or a move to a new Environment, the endpoint often changes. Check that every place the old hostname was configured — variables, secrets, connection strings in code — now uses the new one. The old name may no longer exist, or may still resolve to the old instance. Taking the endpoint from the dependency block, as described in Attach a Database to Your Application, avoids this.
  3. Restart the application after changing the hostname. Some runtimes cache DNS answers for the life of the process: the JVM, for example, can keep an old answer until it restarts. See Restart an Application or a Single Pod.

An external name fails, or resolves slowly

  1. Check the name resolves publicly, from your own machine, with nslookup api.example.com. If it does not resolve there either, the problem is with that domain's DNS, not the platform.
  2. If it resolves outside but not from the pod, send us both results.

Kubernetes tries the cluster's search domains before it tries the name as written, for any name with fewer than five dots. A lookup for api.example.com is first attempted as api.example.com.<namespace>.svc.cluster.local and similar, and each of those must fail before the real name is tried. This is normal and usually invisible, but it multiplies the number of DNS queries, and it makes timeouts more likely under load.

You can skip the search domains by adding a trailing dot, which marks the name as complete:

api.example.com.

Not every client library accepts a trailing dot — HTTP clients sending TLS in particular can reject it or send it on in the Host header — so test it before using it in production.

A host across a VPN or on a partner network does not resolve

Private hostnames on your office network, a data centre, a partner's cloud or a managed service in someone else's network — for example a partner's private Kafka bootstrap address — are not in public DNS. A pod can only resolve them if the environment has been set up to send lookups for that domain to the DNS servers on the other side.

  1. Confirm the network connection itself works first, by testing the IP address rather than the name: nc -vz <ip-address> <port>. If that fails, see Site-to-Site VPN and On-Premises Connectivity.
  2. If the IP works but the name does not resolve, raise a request with the private domain (for example internal.example.com), the environment, and — if you know them — the addresses of the DNS servers that answer for that domain on the other side.

As a short-term workaround you can point your application at the IP address directly, but only where TLS certificates and the remote service allow it.

Lookups fail only some of the time

Intermittent failures (Temporary failure in name resolution, EAI_AGAIN, i/o timeout) are usually not caused by your configuration. Before raising a request, measure it so we can locate it quickly:

  1. Run the same lookup repeatedly from a failing pod:

    for i in $(seq 1 20); do getent hosts db.example.com > /dev/null && echo "$i ok" || echo "$i FAIL"; done
  2. Run the same loop from a pod of a different application, ideally one that does not show the problem.

  3. Note which node each pod runs on:

    kubectl get pods -n <namespace> -o wide

Failures on some pods and not others, or on one node and not others, are exactly the detail that helps us.

Still having issues?

Raise a request in your dedicated Slack channel with:

  • The Environment, namespace and pod name (or names) where the lookup fails
  • The exact hostname being looked up
  • The exact error message, copied from the logs
  • When it started, and whether it is constant or intermittent
  • The output of your nslookup, dig or getent test, and of the repeated loop if the problem is intermittent
  • Any recent change: a migration, a new endpoint, a new VPN or partner connection

If production traffic is failing, follow How to Raise a Sev-1 Incident.

Related articles


Did this page help you?