Deduplicator

Find, Sort, Filter & Delete duplicate files

Usage

Usage: deduplicator [OPTIONS]

Options:
  -t, --types <TYPES>    Filetypes to deduplicate (default = all)
  -d, --delete <DELETE>  Delete files by algorithm (default = oldest) [options = oldest | newest]
      --dir <DIR>        Run Deduplicator on dir different from pwd
  -n, --nocache          Don't use cache for indexing files (default = true)
  -h, --help             Print help information
  -V, --version          Print version information

Performance

Deduplicator uses fxhash (a non-cryptographic hashing algorithm) which is extremely fast. As a result, deduplicator is able to process huge amounts of data in a couple of seconds.

While testing, Deduplicator was able to go through 8.6GB of pdf files and detect duplicates in 2.9 seconds

5.99s user 6.16s system 411% cpu 2.950 total

Screenshots