Show HN: Hashsync – Deduplicate SSH streams against the receiver's files

Category: devtools

Tags: ssh, deduplication, data-transfer, compression, rsync, backup

Score: 8.0/10 (Innovation: 8, Technical: 9, Documentation: 8, Utility: 7)

Hashsync is a Rust tool that transparently deduplicates SSH streams by matching data blocks at any offset against an index of files already present on the receiving host, sending references instead of repeated content. It integrates with existing tools like rsync and tar by acting as a drop-in ssh replacement, compressing non-deduplicated data with zstd and verifying all references cryptographically. The approach is novel in its flexibility and tool-agnostic design, enabling large bandwidth savings for backups, file transfers, and synchronization scenarios.

Target audience: devops engineers, system administrators, backend developers

Repository: https://github.com/someplaceguy/hashsync · Rust · GPL-3.0 · 3 stars

View on Hacker News