docker 容器正常运行，但不能 exec

# docker ps | grep mq 5d0e262527cfrabbitmq:3-management"docker-entrypoint..."12 months agoUp 3 months10.168.93.209:4369->4369/tcp, 10.168.93.209:5671-5672->5671-5672/tcp, 10.168.93.209:15671-15672->15671-15672/tcp, 10.168.93.209:25672->25672/tcpmq01 # docker exec -it mq01 /bin/bash rpc error: code = 2 desc = oci runtime error: exec failed: container_linux.go:247: starting container process caused "process_linux.go:75: starting setns process caused \"fork/exec /proc/self/exe: no such file or directory\""

docker 版本信息

# docker version Client: Version:1.13.1 API version:1.26 Package version: Go version:go1.8.3 Git commit:774336d/1.13.1 Built:Wed Mar7 17:06:16 2018 OS/Arch:linux/amd64Server: Version:1.13.1 API version:1.26 (minimum version 1.12) Package version: Go version:go1.8.3 Git commit:774336d/1.13.1 Built:Wed Mar7 17:06:16 2018 OS/Arch:linux/amd64 Experimental:false # docker info Containers: 20 Running: 20 Paused: 0 Stopped: 0 Images: 215 Server Version: 1.13.1 Storage Driver: devicemapper Pool Name: docker-202:17-4703339-pool Pool Blocksize: 65.54 kB Base Device Size: 107.4 GB Backing Filesystem: xfs Data file: /dev/loop0 Metadata file: /dev/loop1 Data Space Used: 20.17 GB Data Space Total: 107.4 GB Data Space Available: 87.2 GB Metadata Space Used: 17.29 MB Metadata Space Total: 2.147 GB Metadata Space Available: 2.13 GB Thin Pool Minimum Free Space: 10.74 GB Udev Sync Supported: true Deferred Removal Enabled: true Deferred Deletion Enabled: true Deferred Deleted Device Count: 0 Data loop file: /data/docker/devicemapper/devicemapper/data WARNING: Usage of loopback devices is strongly discouraged for production use. Use `--storage-opt dm.thinpooldev` to specify a custom block storage device. Metadata loop file: /data/docker/devicemapper/devicemapper/metadata Library Version: 1.02.140-RHEL7 (2017-05-03) Logging Driver: json-file Cgroup Driver: systemd Plugins: Volume: local Network: bridge host macvlan null overlay Swarm: inactive Runtimes: docker-runc runc Default Runtime: docker-runc Init Binary: docker-init containerd version:(expected: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1) runc version: N/A (expected: 9df8b306d01f59d3a8029be411de015b7304dd8f) init version: N/A (expected: 949e6facb77383876aeff8a6944dde66b3089574) Security Options: seccomp WARNING: You're not using the default seccomp profile Profile: /etc/docker/seccomp.json Kernel Version: 3.10.0-693.21.1.el7.x86_64 Operating System: CentOS Linux 7 (Core) OSType: linux Architecture: x86_64 Number of Docker Hooks: 3 CPUs: 16 Total Memory: 31.25 GiB Name: docker02 ID: 74LF:KJUT:GI6B:VKVC:OGWX:GBIB:C3WQ:W2ON:Y54T:YHL3:5TBJ:ATUD Docker Root Dir: /data/docker Debug Mode (client): false Debug Mode (server): false Registry: https://index.docker.io/v1/ Experimental: false Insecure Registries: 127.0.0.0/8 Registry Mirrors: https://edavcczl.mirror.aliyuncs.com/ Live Restore Enabled: false Registries: docker.io (secure)

具体解决过程

1、查找出 docker 的容器 id 5d0e262527cf # docker ps | grep rabbitmq 5d0e262527cfrabbitmq:3-management"docker-entrypoint..."12 months agoUp 3 months10.168.93.209:4369->4369/tcp, 10.168.93.209:5671-5672->5671-5672/tcp, 10.168.93.209:15671-15672->15671-15672/tcp, 10.168.93.209:25672->25672/tcpmq012、根据 docker 容器 id 5d0e262527cf 找到对应的 libcontainerd 的运行pid 7309 # ps -ef|grep libcontainerd | grep 5d0e262527cf root730911360 May07 ?00:00:09 /usr/bin/docker-containerd-shim-current 5d0e262527cf7c3c7f459104ac542a45b050d42817d07026e3ce0cd20b7c5459 /var/run/docker/libcontainerd/5d0e262527cf7c3c7f459104ac542a45b050d42817d07026e3ce0cd20b7c5459 /usr/libexec/docker/docker-runc-current3、用 nsenter 进入 docker 容器 5d0e262527cf 的 namespace # nsenter -m -t 7309 bash4、查看 docker 容器 5d0e262527cf 的 DeviceName # docker inspect --format='{{.GraphDriver.Data.DeviceName}}' 5d0e262527cf docker-202:17-4703339-5d0e262527cf7c3c7f459104ac542a45b050d42817d07026e3ce0cd20b7c54595、 可以看到 docker 容器 5d0e262527cf 的 /data/docker/devicemapper/mnt/docker容器ID 这个目录不存在 # ll /data/docker/devicemapper/mnt/5d0e262527cf7c3c7f459104ac542a45b050d42817d07026e3ce0cd20b7c5459 ls: cannot access /data/docker/devicemapper/mnt/5d0e262527cf7c3c7f459104ac542a45b050d42817d07026e3ce0cd20b7c5459: No such file or directory只看到 /data/docker/devicemapper/mnt/docker容器ID-init 这个目录，所以执行 docker exec 时，会报错 \"fork/exec /proc/self/exe: no such file or directory\" # ll /data/docker/devicemapper/mnt/5d0e262527cf7c3c7f459104ac542a45b050d42817d07026e3ce0cd20b7c5459-init total 06、对比正常 docker 容器 aa4416c1f1e8 的目录 # ll /data/docker/devicemapper/mnt/aa4416c1f1e8fb192e72b2cf60aae8507cc4bf7bbe69ef2b96d81e29640f7a4a total 8 -rw-------1 root root64 Dec 112017 id drwxr-xr-x 21 root root 4096 Apr3 15:35 rootfs7、创建 docker 容器 5d0e262527cf 对应的目录(去掉 -init 后缀) # mkdir /data/docker/devicemapper/mnt/5d0e262527cf7c3c7f459104ac542a45b050d42817d07026e3ce0cd20b7c54598、重新挂载 docker 容器 5d0e262527cf 的 /data/docker/devicemapper/mnt/容器ID 目录 // 用法：mount /dev/mapper/docker容器的DeviceName -o rw,relatime,nouuid,attr2,inode64,sunit=512,swidth=1024,noquota -t xfs /Dockerd服务的数据目录/devicemapper/mnt/容器ID # mount /dev/mapper/docker-202:17-4703339-5d0e262527cf7c3c7f459104ac542a45b050d42817d07026e3ce0cd20b7c5459 -o rw,relatime,nouuid,attr2,inode64,sunit=512,swidth=1024,noquota -t xfs /data/docker/devicemapper/mnt/5d0e262527cf7c3c7f459104ac542a45b050d42817d07026e3ce0cd20b7c5459# ll /data/docker/devicemapper/mnt/5d0e262527cf7c3c7f459104ac542a45b050d42817d07026e3ce0cd20b7c5459 total 8 -rw-------1 root root64 Aug 102017 id drwxr-xr-x 17 root root 4096 Dec 102017 rootfs9、退出 docker 容器 5d0e262527cf 的 namespace # exit exit10、测试此 docker 容器 5d0e262527cf 可以正常执行 docker exec # docker exec -it 5d0e262527cf /bin/bash root@mq01:/# ls binbootdevdocker-entrypoint.shetchomeliblib32lib64libx32mediamntoptpluginsprocrootrunsbinsrvsystmpusrvar root@mq01:/# exit exit

【Docker|Docker exec 出现 "fork/exec /proc/self/exe: no such file or directory" 问题】docker exec 出现问题时另一种折衷解决办法，通过 nsenter 进入容器

# docker exec -it 86ffcb615a74 /bin/bash rpc error: code = 2 desc = oci runtime error: exec failed: container_linux.go:247: starting container process caused "process_linux.go:75: starting setns process caused \"fork/exec /proc/self/exe: no such file or directory\""# docker inspect -f {{.State.Pid}} 86ffcb615a74 6670 # nsenter -t 6670 -m -u -i -n -p -bash: /var/log/usermonitor/usermonitor.log: No such file or directory root@86ffcb615a74:/# ps -ef UIDPIDPPIDC STIME TTYTIME CMD mysql100 May07 ?02:50:32 mysqld root6000 14:32 ?00:00:00 -bash root67600 14:33 ?00:00:00 ps -ef -bash: /var/log/usermonitor/usermonitor.log: No such file or directory

推荐阅读

• 

  男频小说都有哪些梗？
• 

  宜昌动物大世界在哪里 宜昌野生动物园在哪里
• 

  手机|年度最保值的手机排行榜：苹果12pro仅排第二名，华为夺冠！
• 

  文言文春节祝福语 有关文言文的春节祝福语有哪些
• 

  2022年女孩名字库 女孩有涵养的名字
• 

  怎么判别是否为安置房 安置房便宜为什么没人买
• 

  世界上有100多种性别 如何确定人类最初有几男几女，人类共同的祖先
• 

  如何使用奔图6556清零软件，让电脑状态更加稳定
• 

  沈阳市机动车违章查询_沈阳交通违章查询官网
• 

  有人说陈瑞、孙露等网络歌手的歌很好听且流传广,那么为什么看不到宣传和上电视综艺节目呢？
• 

  TCL,海信科龙空调哪个好,怎么解决呢？
• 

  word中字数统计在哪个功能区 字数统计在哪里
• 

  汕头金银岛介绍 汕头金银岛在哪里
• 

  乳酸菌补充过量的危害
• 

  哈弗H6三代口碑 第三代哈弗H6车主真实口碑
• 

  你是被谁的哪首歌带进民谣的？
• 

  mini4和mini5区别（mini4和mini5有哪些区别）
• 

  牡鼠粪
• 

  redis持久化开启 Redis怎么持久化命令
• 

  微信销售的开场白怎么说？