mirror of
https://github.com/sreedevk/deduplicator.git
synced 2026-09-01 13:06:43 +00:00
removed old hashing lib
This commit is contained in:
7
Cargo.lock
generated
7
Cargo.lock
generated
@@ -252,7 +252,6 @@ dependencies = [
|
||||
"sqlite",
|
||||
"thiserror",
|
||||
"tokio",
|
||||
"xxhash-rust",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -962,9 +961,3 @@ name = "windows_x86_64_msvc"
|
||||
version = "0.42.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f40009d85759725a34da6d89a94e63d7bdc50a862acf0dbc7c8e488f1edcb6f5"
|
||||
|
||||
[[package]]
|
||||
name = "xxhash-rust"
|
||||
version = "0.8.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "735a71d46c4d68d71d4b24d03fdc2b98e38cea81730595801db779c04fe80d70"
|
||||
|
||||
@@ -17,6 +17,15 @@ Options:
|
||||
-h, --help Print help information
|
||||
-V, --version Print version information
|
||||
```
|
||||
<h2 align="center">Performance</h2>
|
||||
|
||||
<p align="center">
|
||||
Deduplicator uses fxhash (a non-cryptographic hashing algorithm) which is extremely fast. As a result, deduplicator is able to process huge amounts of data in a couple of seconds.</p>
|
||||
|
||||
<p align="center">
|
||||
While testing, Deduplicator was able to go through 8.6GB of pdf files and detect duplicates in 2.9 seconds
|
||||
</p>
|
||||
<pre>5.99s user 6.16s system 411% cpu 2.950 total</pre>
|
||||
|
||||
<h2 align="center">Screenshots</h2>
|
||||
|
||||
|
||||
Reference in New Issue
Block a user