Tuesday, October 4, 2022

[SOLVED] Argument list too long for ls while moving files from one dir to other in bash shell

Issue

Below is the command I am using for moving files from dir a to dir b 

  ls /<someloc>/a/* | tail -2000 | xargs -I{} mv {} /<someloc>/b/
    -bash: /usr/bin/ls: Argument list too long

folder a has files in millions ..

Need your help to fix this please.


Solution

If the locations of both directories are on the same disk/partition and folder b is originally empty, you can do the following

$ rmdir /path/to/b
$ mv /other/path/to/a /path/to/b
$ mkdir /other/path/to/a

If folder b is not empty, then you can do something like this:

find /path/to/a/ -type f -exec mv -t /path/to/b {} +

If you just want to move 2000 files, you can do

find /path/to/a/ -type f -print | tail -2000 | xargs mv -t /path/to/b

But this can be problematic with some filenames. A cleaner way would be is to use -print0 of find, but the problem is that head and tail can't process those, so you have to use awk for this.

# first 2000 files (mimick head)
find /path/to/a -type f -print0        \
  | awk 'BEGIN{RS=ORS="\0"}(NR<=2000)' \
  | xargs -0 mv -t /path/to/b

# last 2000 files (mimick tail)
find /path/to/a -type f -print0        \
  | awk 'BEGIN{RS=ORS="\0"}{a[NR%2000]=$0}END{for(i=1;i<=2000;++i) print a[i]}' \
  | xargs -0 mv -t /path/to/b


Answered By - kvantour
Answer Checked By - Robin (WPSolving Admin)
at