For each File
For each File is a Bash script wrapper for repetitive file manipulation tasks such as mass file renaming. It lets you run arbitrary commands (recursively if you like) upon files, directories, and more.
| Tags | Utilities Systems Administration |
|---|---|
| Licenses | GPL |
| Operating Systems | POSIX Unix Windows Windows Cygwin |
| Implementation | Unix Shell |
Recent releases
- 26 Jan 2006 00:28
Changes: In extract.ff, a defect preventing the output directory from being relocated to its proper location was fixed. The "dev" and "user" mailing lists were replaced with a new "friends" mailing list to better suit the size of the user community.
- 02 Jan 2006 01:17
Changes: The project has relocated from SourceForge to Gna!. Information about Gna! services, contributing, getting help, and examples of usage have been added to the user's manual. Removal of empty output directory and logic for adjusting of output directory in extract.ff has been fixed.
- 18 Dec 2005 12:23
Changes: For each File has been made into a GNU BASH shell function (as opposed to a shell script). A comprehensive user's manual, written in Docbook-XML and complete with explanations and examples, has been added. Full support for file names with spaces, newlines, Unicode, non-printable, and other weird characters has been added. The "For each Line" script has been removed.
- 16 Sep 2004 23:00
Changes: Several minor enhancements and bugfixes were made, including improved user query interfaces and the addition of manuals for the example scripts.
- 17 Aug 2004 01:12
Changes: Some important functionality, such as the '--args' option for passing command-line arguments to user scripts and the '-- lang' option for specifying an arbitrary language code, has been added in this release. Also, the preset scripting variables have been renamed to better reflect their purpose.
- All comments
Recent commentsRe: Advantages over find?
> This is a limitation of the GNU Bash
> shell as there is currently no way to
> express the ASCII NUL character '\0' in
> its scripting language. Also, such
> arbitrary filenames with newlines in
> them are a rather special case in most
> POSIX systems. Thus, in accordance with
> the common Engineering design rule:
> "make the common case fast", the
> handling of newlines in filenames is
> best left to the 'xargs' tool.
>
The above statement is false.
This tool can handle file names that contain new-line characters, as was proven in a comp.unix.shell newsgroup post (Message-ID:
<2m5ifuFjctd8U1@uni-berlin.de>). Furthermore, we can generate the NUL character in BASH scripts: echo $'NUL is \000'.
Re: Advantages over find?
My previous response to this question contains errors. Below is an updated and correct response from the FAQ (located at: http://ff-bash.sourceforge.net/docs/faq.html).
There is one advantage, which can be observed under the following conditions. If find -exec is used to invoke an intermediate shell: $ find . -exec sh -c '...' \;, then it will expend more system resources than For each File. This is because, for each file it handles, find -exec will fork a child process (the intermediate shell) and thereby cause the operating system to perform a context switch. In contrast, For each File will invoke a GNU BASH function that is already loaded into memory and thus does not cause a context switch. This behavior reduces the number of process forks and thereby expends less system resources as compared to find -exec under said conditions.
Re: Advantages over find?
> For example, renaming .tgz to .tar.gz
> can be done with a simple command line:
>
> find $some_path -type f -name \*.tgz | sed 's/\(.*\)\.tgz/mv & \1.tar.gz/' | sh
You could also use this command:
tgz -> tar.gz
for i in *tgz; do mv $i `echo $i | sed s/tgz/tar\.gz/`; done
tar.gz -> tgz
for i in *tar.gz; do mv $i `echo $i | sed s/tar\.gz/tgz/`; done
or, if you want to use more complicated mechanisms to find files and therefore want to use the `find' command, use this:
for i in `find . -type f [--further-options] -name "*tgz"`; do mv $i `echo $i | sed s/tgz/tar\.gz/`; done
Re: Advantages over find?
> Why such -- rather complicated -- bash
> magic?
Making use of the shell's builtin constructs yeilds better performance than invoking external programs.
> What's so bad on just teaching people
> clever use of
> standard tools as sh(1), find(1),
> xargs(1), sed(1) etc?
I agree. Please note that this tool is not meant to replace any existing tools; it simply aims to facilitate alternative approaches to performing filesystem manipulation tasks.
> For example, renaming .tgz to .tar.gz
> can be done
> with a simple command line:
>
> find $some_path -type f -name \*.tgz |
> sed
> 's/\(.*\)\.tgz/mv & \1.tar.gz/' | sh
That's a good solution and notice that you also have utilized an intermediate shell 'sh' in solving the example problem. This observation is detailed in my response to the question at the root of this thread.
> When unsure, run that command without
> the trailing |
> sh to see what would happen. Or run it
> with | sh -x to
> see *what* actually happens.
>
> Note that this example is a *oneliner*,
> so with a little
> bit more magic, you could easily make it
> robust
> against weird filenames (containing
> whitespace,
> quotes and thelike). However, neither ff
> nor a
> combination of find(1), sed(1) and sh(1)
> as above
> can handle filenames with newlines.
> You've to use
> find ... -print0 | xargs -0 for this,
> which unfortunately is
> a GNU extension.
This is a limitation of the GNU Bash shell as there is currently no way to express the ASCII NUL character '\0' in its scripting language. Also, such arbitrary filenames with newlines in them are a rather special case in most POSIX systems. Thus, in accordance with the common Engineering design rule: "make the common case fast", the handling of newlines in filenames is best left to the 'xargs' tool.
> Ciao,
>
> Kili
>
Thanks.
Re: Advantages over find?
Why such -- rather complicated -- bash magic?
What's so bad on just teaching people clever use of
standard tools as sh(1), find(1), xargs(1), sed(1) etc?
For example, renaming .tgz to .tar.gz can be done
with a simple command line:
find $some_path -type f -name \*.tgz | sed
's/\(.*\)\.tgz/mv & \1.tar.gz/' | sh
When unsure, run that command without the trailing |
sh to see what would happen. Or run it with | sh -x to
see *what* actually happens.
Note that this example is a *oneliner*, so with a little
bit more magic, you could easily make it robust
against weird filenames (containing whitespace,
quotes and thelike). However, neither ff nor a
combination of find(1), sed(1) and sh(1) as above
can handle filenames with newlines. You've to use
find ... -print0 | xargs -0 for this, which unfortunately is
a GNU extension.
Ciao,
Kili