Issue
running a Centos 7.1.1503 docker container, when adding a few lines of code (node.js) it crashes with the error:
/bin/sh: line 1: 6 Segmentation fault (core dumped) node --inspect server.js
the file /proc/sys/kernel/core_pattern contains the following:
|/usr/libexec/abrt-hook-ccpp %s %c %p %u %g %t e
There's no /var/spool/abrt directory within the container. The /var/spool/abrt directory on the server running the containers doesn't get anything. I can't change the /proc/sys/kernel/core_pattern to point to another directory/program because of the read-only fs thing. Can't run the container in privileged, either :-(
I've read through tonnes of docker/stackexchange and other docs and can't figure out where/how to get the core dump?
In the olden days I'd play with the settings and wreck a replica of the machine, but this is a production container and I'm very limited in what I can do and when/how many times I can crash it :-(
Host is RHEL 7.1, docker version is 1.7
EDIT: On my laptop, running the same container (with docker 1.12 though), I sometimes get core dumps on the host /var/spool/abrt by running sleep 60 & in the container, then running (still in the container) kill -ABRT <pid of the sleep 60> . By "sometimes" I mean that trying again doesn't always work... I'm not sure why, but about 2 out of 3 tries succeed. I figure this might have to do with a privileged run or something..? I run the container with docker run -it centos bash. If I can understand this I might replicate this behavior in the production env.
Solution
I ended up skipping the abrt and changing the core_pattern file to out to a directory on the host. Here's my two bytes on getting a core dump out of a crashing docker instance:
On the host:
docker run --privileged -it -v /tmp:/core image-name bash
(you can do this with docker exec, but my machine didn't have the flags available to exec)
--privileged = required to be able to edit the /proc/sys/kernel/core_pattern file
-v = to mount the /tmp directory of the host in a /core directory in the container
In the instance:
set the location of the core dumps to /core (which is a mount of the /tmp dir in the host machine):
echo "/core/core-%e-%s-%u-%g-%p-%t" > /proc/sys/kernel/core_pattern
test it:
sleep 60 &
kill -SEGV <pid of that sleep process>
Should be able to see a core file in the /tmp dir on the host. When my instance crashed, I finally got the dump in the host machine.
Answered By - Nahshon paz Answer Checked By - Clifford M. (WPSolving Volunteer)
