This isn’t really new, but it’s sure worth knowing… The fact that there’s a 128K buffer used to pass arguments to child processes in the Linux Kernel can have the following effect on shell external commands:

When you run commands such as rm spam* the shell (e.g. bash) will expand the “spam*” to a list of names of all files in the working directory that start with “spam”. That list is then passed to the executable /bin/rm using the 128K buffer mentioned above. If you have a large number of files, or the list gets long because of long file names, you may get an error message saying /bin/rm: Argument list too long.

To work around this you can get the list of filenames using find and pipe it to xargs which in turn invokes rm for every single file. There is no limit on the size of a pipe (or at least none that I am aware of for this practical purpose). Here’s the full command:

find . -name 'spam*' -print0 | xargs -0 rm

This differs from similar commands you might find in the -print0 and -0 arguments: These are needed in case you have spaces in your filenames.

For an in-depth explanation of both the 128K buffer and the spaces in filenames issue you may also want to read the May 2004 update in this blog post. (See, I told you it’s not really new.)