Subversion Repositories sysadmin_scripts

# find greatest savings after fdupes run

Using fdupes to report on duplicate files, or even remove them
automatically, is excellent. However, sometimes just working on a subset of
the files can be done in 10% of the time and result in a 90% cleanup.

fdupesGreatestSavings is a perl script that takes the output of fdupes (on
stdin) and determines which entries will result in the greatest savings,
whether it is 100 copies of a 1 Meg file or 2 copies of a 20G file. The
output is sorted by greatest savings to least.

fdupesGreatestSavings takes one parameter, the number of entries to display.
It accepts input from stdin and sends output to stdout, so it is a filter.

> fdupes must be run with only the flags --recurse and --size, though
> --recurse is optional.

The following command will look through the entire file system on a Unix
machine and report the top 10 duplicates it finds.

    fdupes --recurse --size / | ./fdupesGreatestSavings 10

If you want to save the results (for other procesing), you could do
something like

    fdupes --recurse --size  /path/to/be/checked > /tmp/duplicate_files
    fdupesGreatestSavings 100 < /tmp/duplicate_files > /tmp/fdupe.savings

## Bugs

The only bug I have found so far is that the count of the number of files is
incorrect (always 2), and I haven't tracked it down. Total space used by an
entry is correct, however.