[SOLVED] rsync succeeds from the shell but complains of "syntax or usage error" from crontab

Issue

The following crontab line (on CentOS 7.6)

@daily rsync -rav --remove-source-files --files-from=<(find /home/qa/buildbot/master/packages/rhel7 -type f -mtime +30 -exec basename {} \;) /home/qa/buildbot/master/packages/rhel7 192.168.1.31:/local/raid0/buildbotpackages/packages/rhel7

fails, and sends me the following email message:

From: (Cron Daemon) <[email protected]>
Subject: Cron <qa@docker> rsync -rav --remove-source-files --files-from=<(find /home/qa/buildbot/master/packages/rhel7 -type f -mtime +30 -exec basename {} \;) /home/qa/buildbot/master/packages/rhel7 192.168.1.31:/local/raid0/buildbotpackages/packages/rhel7


rsync: failed to open files-from file <(find /home/qa/buildbot/master/packages/rhel7 -type f -mtime +30 -exec basename {} ;): No such file or directory
rsync error: syntax or usage error (code 1) at main.c(1567) [client=3.1.2]

I looked at the source code of rsync's main.c, but could not see the relevance of line 1567:

   1557     SIGACTMASK(SIGINT, sig_int);
   1558     SIGACTMASK(SIGHUP, sig_int);
   1559     SIGACTMASK(SIGTERM, sig_int);
   1560 #if defined HAVE_SIGACTION && HAVE_SIGPROCMASK
   1561     sigprocmask(SIG_UNBLOCK, &sigmask, NULL);
   1562 #endif
   1563 
   1564     /* Ignore SIGPIPE; we consistently check error codes and will
   1565      * see the EPIPE. */
   1566     SIGACTION(SIGPIPE, SIG_IGN);
   1567 #ifdef SIGXFSZ
   1568     SIGACTION(SIGXFSZ, SIG_IGN);
   1569 #endif
   1570 
   1571     /* Initialize change_dir() here because on some old systems getcwd
   1572      * (implemented by forking "pwd" and reading its output) doesn't
   1573      * work when there are other child processes.  Also, on all systems
   1574      * that implement getcwd that way "pwd" can't be found after chroot. */
   1575     change_dir(NULL, CD_NORMAL);
   1576 
   1577     init_flist();

Furthermore, when I run that exact crontab line from the shell, it works:

qa@docker /tmp$ rsync -rav --remove-source-files --files-from=<(find /home/qa/buildbot/master/packages/rhel7 -type f -mtime +30 -exec basename {} \;) /home/qa/buildbot/master/packages/rhel7 192.168.1.31:/local/raid0/buildbotpackages/packages/rhel7
sending incremental file list

sent 18 bytes  received 12 bytes  20.00 bytes/sec
total size is 0  speedup is 0.00
qa@docker /tmp$

Any ideas on how to debug this issue?

Solution

Broken cron jobs are frequently caused by testing the command in a different shell from the one cron uses. Most of the popular interactive shells like bash and zsh and the standard /bin/sh (used by cron) have similar basic syntax, because they're all descended from the Bourne shell. The similarity between them is strong enough that you can get by, for a while, thinking that cron's command syntax is the same as your login shell.

When you put more complex commands into a crontab, you find that there are differences. In your example, I suspect the command substitution operator <(...). This type of substitution didn't exist in Bourne shell, and I don't think POSIX has adopted it either, so I wouldn't trust it in a cron job.

Test your command in /bin/sh on your system by simply running one within your other shell:

BIG-FANCY-PROMPT%>$ sh
$ rsync -rav --remove-source-files --files-from=<(find /home/qa/buildbot/master/packages/rhel7 -type f -mtime +30 -exec basename {} \;) /home/qa/buildbot/master/packages/rhel7 192.168.1.31:/local/raid0/buildbotpackages/packages/rhel7
... probably some error message here...
$ exit
BIG-FANCY-PROMPT%>$ _

If you can't think of a way to rewrite the command in sh syntax, you can always explicitly invoke bash or any other shell you like:

@daily bash -c '...'

This way involves an extra layer of quoting so that sh will pass the whole command to bash without trying to parse it. If that becomes too difficult, then put your command in a script with a nice #! line and run that from cron.

Answered By - user2404501

Answer Checked By - Timothy Miller (WPSolving Admin)