[SOLVED] Linux background task maximum process time

Issue

Is there any fixed time duration as how long a background task can run?

This is how I run the script (background task) manually:

php /var/www/html/app_v2/console.php massbulkinsert app.example.com 10 > /dev/null &

this script process huge data set, it takes about 1 hour to complete.

First time it stopped at 10100th record. second time it stopped at 9975th record. There is no pattern of it terminating.

top command and the mysql pid was at 98% and 100% and 130% most of the time and the free memory had about 200 MB. There is enough disk space.

Solution

Its a bit of a wild guess, but usually when you succeed with a smaller amount of data - and then gets crashes with larger amounts, it has to do with memory issues.

You should have a look at /etc/php5/cli. There is probably also a folder named cgi inthere - depending how your framework executes the background script i would expect either of these two configurations are used.

Files with extensions called 'ini' are configurations for PHP scripting, and these are among the values that you're interested in (values are defaults on debian 8):

; Maximum execution time of each script, in seconds
; http://php.net/max-execution-time
; Note: This directive is hardcoded to 0 for the CLI SAPI
max_execution_time = 30


; Maximum amount of memory a script may consume
; http://php.net/memory-limit
memory_limit = -1

Note, that there is also a timeout for how long the script can spend, reading the data sent to it through, say a pipe (max_input_time). But seeing your command, youre not piping values to it via stdin - but most likely reading a file already on the disk.

Hope it helps

Answered By - mschr