Issue
I use the python tqdm package for showing the progress of the execution of my code. Heres a small sample for explaining my use case better
progress = tqdm(range(2))
for epoch in progress:
loss = []
for i in range(2):
time.sleep(0.5)
loss_ = random.random()
progress.set_description("EPOCH: {}, LOSS: {}".format(epoch, loss_))
loss.append(loss_)
loss_mean = sum(loss) / len(loss)
print("EPOCH: {}, LOSS: {}".format(epoch, loss_mean))
When i run this code without nohup, it runs as i expect. it refreshes tqdm descriptions and keeps only the last one on terminal. sample output
EPOCH: 0, LOSS: 0.9789279434307936: 0%| | 0/2 [00:01<?, ?it/s]
EPOCH: 0, LOSS: 0.5650528466113542
EPOCH: 1, LOSS: 0.2959674437384582: 50%|████████████████████▌ | 1/2 [00:02<00:01, 1.00s/it]
EPOCH: 1, LOSS: 0.39633745424290057
EPOCH: 1, LOSS: 0.2959674437384582: 100%|█████████████████████████████████████████| 2/2 [00:02<00:00, 1.00s/it]
however when i run the same code using nohup for running the process in background, all the tqdm descriptions from the inner loop get printed and the outputs of the print statement are printed at the end. Heres the output when using nohup.
0%| | 0/2 [00:00<?, ?it/s]
EPOCH: 0, LOSS: 0.6247515429741374: 0%| | 0/2 [00:00<?, ?it/s]
EPOCH: 0, LOSS: 0.531051885429166: 0%| | 0/2 [00:01<?, ?it/s]
EPOCH: 0, LOSS: 0.531051885429166: 50%| ^v^h ^v^h ^v^h ^v^h ^v^h | 1/2 [00:01<00:01, 1.00s/it]
EPOCH: 1, LOSS: 0.4399544030856224: 50%| ^v^h ^v^h ^v^h ^v^h ^v^h | 1/2 [00:01<00:01, 1.00s/it]
EPOCH: 1, LOSS: 0.6654644291991813: 50%| ^v^h ^v^h ^v^h ^v^h ^v^h | 1/2 [00:02<00:01, 1.00s/it]
EPOCH: 1, LOSS: 0.6654644291991813: 100%| ^v^h ^v^h ^v^h ^v^h ^v^h ^v^h ^v^h ^v^h ^v^h ^v^h| 2/2 [00:02<00:00, >
EPOCH: 1, LOSS: 0.6654644291991813: 100%| ^v^h ^v^h ^v^h ^v^h ^v^h ^v^h ^v^h ^v^h ^v^h ^v^h| 2/2 [00:02<00:00, >
EPOCH: 0, LOSS: 0.5779017142016517
EPOCH: 1, LOSS: 0.5527094161424018
So I wanted to know if its possible to flush/erase out the previous tqdm outputs from files(keep only the last description) thus mimicking the final output from when executed without nohup
Solution
No.
Please understand that under nohup you don't have a pty, a terminal,
it just connects sys.stdout to the file nohup.out.
Normally tqdm does not exactly "erase", rather it prints
a line of progress output followed by \r RETURN without \n NEWLINE.
Doing that repeatedly gives the visual effect of overwriting same line.
But many characters were output, and they all appear in the text output file.
Your complaint is that "noisy" tqdm output interferes with other output,
and you're hoping to see less noise.
To do that, either add an option to your app so it doesn't call tqdm at all,
or ensure that your app writes its output to some other file descriptor.
Then when you cat that file you won't see any progress output.
Currently you have
from tqdm import tqdm
...
for item in tqdm(items):
do_stuff(item)
You can create your own def tqdm(...): wrapper
which conditionally calls the "real" tqdm.
If a user-supplied option or isatty()
indicate that we don't wish to see progress output,
then your wrapper can choose to silently return
instead of passing through to a tqdm call.
Answered By - J_H Answer Checked By - Cary Denson (WPSolving Admin)
