[SOLVED] Crontab not sending Sendgrid emails from Python script on AWS Linux 2

Issue

I have an AWS Amazon Linux 2 box. I am using a python script (Python 3.7) to send an email using Sendgrid as the SMTP service. I can send the email using $ python3 send_email.py but, when I use crontab ($ crontab -e then * * * * * python3 ~/apps/send_email.py), the error in the log file /var/log/cron is (CRON) EXEC FAILED (/usr/sbin/sendmail): No such file or directory. The crontab is working as expected (I've tested other cron commands and they work fine), but the email part is what's not working.

Here's what I've tried to fix it:

1. Run the command as a root crontab (ie $ sudo crontab -e)
2. Run the the crontab as a user crontab (ie $crontab -e)), but with sudo python3 ... in the crontab command.
3. Add the path directory at the top of the crontab file in case those directories couldn't be resolved
4. Installed Postfix to install something in the /usr/bin/sendmail directory but, since I don't need it to run the Sendgrid-powered email using $ python3 send_email.py, I'm not sure why I would need it through cron. I could be totally wrong on this, though. With Postfix installed, it resolves the (CRON) EXEC FAILED (/usr/sbin/sendmail): No such file or directory error in the cron log - the log entry in that case is (ec2-user) CMD (python3 ~/apps/send_email.py) - but I don't receive an email. Probably because Postfix isn't configured for the SMTP I'm using (Sendgrid).
5. (EDIT) I have configured Sendgrid to work with Postfix via the Sendgrid docs but it still won't send me an email although it looks like nothing is erroring out in the Postfix logs...

Jun  1 20:27:02 ip-[my-ip] postfix/pickup[25506]: 101769641F3: uid=1000 from=<ec2-user>
Jun  1 20:27:02 ip-[my-ip] postfix/cleanup[25508]: 101769641F3: message-id=<20210601202702.101769641F3@ip-[my-ip].us-east-2.compute.internal>
Jun  1 20:27:02 ip-[my-ip] postfix/qmgr[25507]: 101769641F3: from=<ec2-user@ip-[my-ip].us-east-2.compute.internal>, size=1165, nrcpt=1 (queue active)
Jun  1 20:27:02 ip-[my-ip] postfix/local[25510]: 101769641F3: to=<ec2-user@ip-[my-ip].us-east-2.compute.internal>, orig_to=<ec2-user>, relay=local, delay=0.01, delays=0/0/0/0, dsn=2.0.0, status=sent (delivered to mailbox)
Jun  1 20:27:02 ip-[my-ip] postfix/qmgr[25507]: 101769641F3: removed

My thought is that there is something in the cron call of the Sendgrid python lib (installed with sudo pip install sendgrid) that is failing out that doesn't happen when python calls it directly using the python3 interpreter from the CLI. I don't know why those would be different.

Solution

I was able to finally resolve this after much troubleshooting. Here's what worked.

1. I kept the original setup the same: Sendgrid for SMTP using their python lib (no Postfix or smtplib), using user crontab (using $ crontab -e not $ sudo crontab -e), and using Python 3.7.
2. Apparently the word 'email' in the python script name can cause interpreter issues so I renamed the file to remove the word 'email'. For example, send_noti.py instead of send_email.py
3. I used absolute paths for every call to a file in the python script. I was reading, writing, and executing files in the user directory in my python script as well as sending an email. Those R,W,X commands started erroring out in cron but not when called from outside of cron for some reason, even after renaming the file. Using absolute paths for all references to files fixed that.
4. I also used the absolute path for the python file in the crontab file. For example, * * * * * python3 /home/ec2-user/apps/send_noti.py instead of * * * * * python3 ~/apps/send_noti.py
5. I removed the unnecessary items in the crontab file I had put in before such as redefining the PATH directory.

I hope this helps someone because this took me about 3 weeks to troubleshoot and figure out.

Answered By - dbadness