removed old hashing lib

This commit is contained in:
sreedev
2022-12-30 00:30:52 -05:00
parent 2cfb298468
commit 68286afab3
2 changed files with 9 additions and 7 deletions

7
Cargo.lock generated
View File

@@ -252,7 +252,6 @@ dependencies = [
"sqlite",
"thiserror",
"tokio",
"xxhash-rust",
]
[[package]]
@@ -962,9 +961,3 @@ name = "windows_x86_64_msvc"
version = "0.42.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f40009d85759725a34da6d89a94e63d7bdc50a862acf0dbc7c8e488f1edcb6f5"
[[package]]
name = "xxhash-rust"
version = "0.8.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "735a71d46c4d68d71d4b24d03fdc2b98e38cea81730595801db779c04fe80d70"

View File

@@ -17,6 +17,15 @@ Options:
-h, --help Print help information
-V, --version Print version information
```
<h2 align="center">Performance</h2>
<p align="center">
Deduplicator uses fxhash (a non-cryptographic hashing algorithm) which is extremely fast. As a result, deduplicator is able to process huge amounts of data in a couple of seconds.</p>
<p align="center">
While testing, Deduplicator was able to go through 8.6GB of pdf files and detect duplicates in 2.9 seconds
</p>
<pre>5.99s user 6.16s system 411% cpu 2.950 total</pre>
<h2 align="center">Screenshots</h2>