Show HN: 500 TB internet index in ClickHouse, with congestion pricing

Category: database

Tags: clickhouse, internet-index, sql, analytics, embeddings

Score: 7.3/10 (Innovation: 8, Technical: 8, Documentation: 4, Utility: 6)

Scry is a massive internet-scale index (500 TB) built in ClickHouse that includes social media archives, forums, citations, and prediction markets, exposed as SQL queries with custom search predicates and vector embedding functions. It is interesting because it enables novel analytical queries like finding 'saved but not liked' tweets, cross-platform drift analysis, and follow graph intersections, all with sub-second response times over hundreds of millions of rows.

Target audience: data engineers, backend developers, data scientists

Repository: https://scry.io/

View on Hacker News