Friday, April 22, 2022

[SOLVED] Docker on CentOS 7 - DNS not workin within containers

April 22, 2022 centos, centos7, dns, docker

Issue

I installed Docker on a CentOS 7 machine and DNS is not working within containers.

So, if I run nslookup google.com on my host, it resolves correctly. However, if I do docker container run busybox nslookup google.com I get:

Server:         8.8.8.8
Address:        8.8.8.8:53

Non-authoritative answer:
Name:   google.com
Address: 142.250.72.46

*** Can't find google.com: No answer

root@XXX [~]# docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
  scan: Docker Scan (Docker Inc.)

Server:
 Containers: 57
  Running: 0
  Paused: 0
  Stopped: 57
 Images: 10
 Server Version: 20.10.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 05f951a3781f4f2c1911b05e61c160e9c30eaa8e
 runc version: 12644e614e25b05da6fd08a38ffa0cfe1903fdec
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 3.10.0-1160.25.1.el7.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 3.7GiB
 Name: cgh.cgh.net
 ID: X2YB:LILA:S5Q3:CLRJ:ACY7:ST4T:YRDD:3D3E:PLXV:DHGS:3IE2:QMIP
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: akhtariev
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

root@xxx [~]# docker version
Client: Docker Engine - Community
 Version:           20.10.6
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        370c289
 Built:             Fri Apr  9 22:45:33 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.6
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       8728dd2
  Built:            Fri Apr  9 22:43:57 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.4
  GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
 runc:
  Version:          1.0.0-rc93
  GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

root@XXX [~]# docker network inspect bridge
[
    {
        "Name": "bridge",
        "Id": "609282cb44a298bb8f69b424d4ba2b1e3ab391b37fc171d2ec032f592dc3afbb",
        "Created": "2021-05-23T14:52:38.789641794-05:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.17.0.0/16",
                    "Gateway": "172.17.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {},
        "Options": {
            "com.docker.network.bridge.default_bridge": "true",
            "com.docker.network.bridge.enable_icc": "true",
            "com.docker.network.bridge.enable_ip_masquerade": "true",
            "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
            "com.docker.network.bridge.name": "docker0",
            "com.docker.network.driver.mtu": "1500"
        },
        "Labels": {}
    }
]

I already tried setting the default DNS server to 8.8.8.8 and following various other troubleshooting guides but nothing has worked. It's worth mentioning that the server is provisioned by hostgator and has cPanel installed. I requested hostgator to disable anti virus and firewall. They do not provide support with Docker and could not help resolve my issue.

Any help is greatly appricated and please let me know if there is more information needed to help solve this issue.

UPDATE: This does not only happen on busybox but on other platforms as well. My main problem is that I cannot run a container of the image that is built from the following Dockerfile:

FROM python:3.9.5-buster

ENV HOME_PATH=/root
ENV PATH="${HOME_PATH}/.local/bin:${PATH}"
WORKDIR ${HOME_PATH}

COPY requirements.txt cron-job ${HOME_PATH}/
RUN apt-get update
RUN apt-get install -y cron
RUN useradd --create-home service
RUN pip install --user -r ${HOME_PATH}/requirements.txt

COPY . ${HOME_PATH}
RUN ls \
    && mv cron-job /etc/cron.d/cron-job \
    && chmod 0644 /etc/cron.d/cron-job \
    && crontab /etc/cron.d/cron-job \
    && touch ${HOME_PATH}/agl_history.log

CMD cron && tail -f ${HOME_PATH}/agl_history.log

docker-compose.yml:

version: "3.9"
services:
  agl-history:
    depends_on:
        - mariadb
    build: .
    restart: on-failure
    networks:
        - main
  mariadb:
    image: "mariadb:10.5"
    restart: on-failure
    environment:
        MYSQL_ROOT_PASSWORD: ${MARIADB_PASSW}
    ports:
        - 3306:3306
    volumes:
        - /var/lib/docker/volumes/add3-data:/var/lib/mysql
    networks:
        - main
networks:
    main:
        driver: bridge

This service works well and tested on my personal machine running the latest Ubuntu

UPDATE 2:

So, the recommendation in the comments worked to solve the problem with nslookup by doing docker run --rm busybox nslookup -type=a google.com

However, I still have the issue running the mentioned-above buster container. This is what happens:

Building agl-history
Sending build context to Docker daemon  918.5kB
Step 1/12 : FROM python:3.9.5-buster
 ---> a6a0779c5fb2
Step 2/12 : ENV HOME_PATH=/root
 ---> Using cache
 ---> 7aea834621d4
Step 3/12 : ENV PATH="${HOME_PATH}/.local/bin:${PATH}"
 ---> Using cache
 ---> dc16e6178688
Step 4/12 : WORKDIR ${HOME_PATH}
 ---> Using cache
 ---> 950712b33f21
Step 5/12 : COPY requirements.txt cron-job ${HOME_PATH}/
 ---> Using cache
 ---> 6923c1410731
Step 6/12 : RUN apt-get update
 ---> Using cache
 ---> aa4c4a4d25c4
Step 7/12 : RUN apt-get install -y cron
 ---> Running in 26c7517a05d0
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package cron
The command '/bin/sh -c apt-get install -y cron' returned a non-zero code: 100
ERROR: Service 'agl-history' failed to build : Build failed

Solution

You seems to be hitting a caching issue due to those two lines in your Dockerfile:

RUN apt-get update
RUN apt-get install -y cron

As pointed in the best practices for writing a Dockerfile, having apt-get update and apt-get install in two separate RUN instructions might make you face caching issues:

Using apt-get update alone in a RUN statement causes caching issues and subsequent apt-get install instructions fail. For example, say you have a Dockerfile:
FROM ubuntu:18.04
RUN apt-get update
RUN apt-get install -y curl
After building the image, all layers are in the Docker cache. Suppose you later modify apt-get install by adding extra package:
FROM ubuntu:18.04
RUN apt-get update
RUN apt-get install -y curl nginx
Docker sees the initial and modified instructions as identical and reuses the cache from previous steps. As a result the apt-get update is not executed because the build uses the cached version. Because the apt-get update is not run, your build can potentially get an outdated version of the curl and nginx packages.

Using RUN apt-get update && apt-get install -y ensures your Dockerfile installs the latest package versions with no further coding or manual intervention. This technique is known as “cache busting”.

^{Source: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#run}

So your best bet would be to rather do:

RUN apt-get update && apt-get install -y \
        cron

^{Having the packages, one per line, on subsequent line, alphabetically sorted is another recommandation of the best practices.}

Answered By - β.εηοιτ.βε

Answer Checked By - David Marino (WPSolving Volunteer)

This Answer collected from stackoverflow and tested by PythonFixing community admins, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0

Friday, April 22, 2022

Issue

Solution

Popular Posts

Labels