Deduplicator

Find, Sort, Filter & Delete duplicate files

NOTE: This project is still being developed. At the moment, as shown in the screenshot below, deduplicator is able to scan through and list duplicates with and without caching. Contributions are welcome.

Usage

Usage: deduplicator [OPTIONS]

Options:
  -t, --types <TYPES>  Filetypes to deduplicate (default = all)
      --dir <DIR>      Run Deduplicator on dir different from pwd
  -n, --nocache        Don't use cache for indexing files (default = true)
  -h, --help           Print help information
  -V, --version        Print version information

Performance

Deduplicator uses fxhash (a non-cryptographic hashing algorithm) which is extremely fast. As a result, deduplicator is able to process huge amounts of data in a couple of seconds.

While testing, Deduplicator was able to go through 8.6GB of pdf files and detect duplicates in 2.9 seconds

Screenshots