Issue
I will download some files from a server via this shell script. If I download the files with this command wget -Nc $url/$languages/$year/{1..53}$url_suffix -P $dir
, then likewise the calendar weeks > 43 are downloaded for the date 20/26/2023. Unfortunately wget
does not abort the download by itself because the file does not exist on the server. It creates files with the calendar weeks 43 - 53. How do I solve the problem?
#!/bin/bash
# The URL is a combination of the languages, year and week of the year.
# Example URL https://example.com/greek/2020/29.gz
url=https://example.com
url_suffix=.gz
# List of intruments
greek=(Alpha Beta Gamma Delta)
german=(Auto Haus Boot)
english=(Hello)
# Year
year=(2019 2020 2021 2022 2023)
for languages in "${greek[@]}" "${german[@]}" "${english[@]}"; do
for t in "${year[@]}"; do
dir="ftp/$languages/$t"
# Check if the directories exits
if [[ ! -d $dir ]]; then
mkdir -p $dir
fi
# Downlaod the files
wget -Nc $url/$languages/$year/{1..53}$url_suffix -P $dir
done
done
Solution
If you want to break the script at the first 404
(Not found) you can do something like this:
#!/bin/bash
# The URL is a combination of the languages, year and week of the year.
# Example URL https://example.com/greek/2020/29.gz
url=https://example.com
url_suffix=.gz
# List of intruments
greek=(Alpha Beta Gamma Delta)
german=(Auto Haus Boot)
english=(Hello)
# Year
year=(2019 2020 2021 2022 2023)
for languages in "${greek[@]}" "${german[@]}" "${english[@]}"; do
for t in "${year[@]}"; do
dir="ftp/$languages/$t"
mkdir -p $dir
for week in "${url}/${languages}/${t}/"{1..53}"${url_suffix}"
do
wget -Nc $week -P $dir || break
done
done
done
Answered By - Alberto Fecchi Answer Checked By - Clifford M. (WPSolving Volunteer)