Show HN: The RAG ingestion layer problem: what it takes to build one

Category: devtools

Tags: rag, file-conversion, web-scraping, api, markdown

Score: 6.0/10 (Innovation: 5, Technical: 6, Documentation: 7, Utility: 6)

EnConvert is an HTTP API that unifies file conversion (46 formats) and web page rendering into Markdown, JSON, PDF, or screenshots, targeting RAG ingestion pipelines. It offers endpoints for crawling sites into chunks, structured extraction, and change monitoring, with a self-hostable open-source core under AGPL-3.0 while advanced rendering and quality scoring remain in the hosted cloud service.

Target audience: backend developers and data engineers building RAG or content ingestion pipelines

Repository: https://www.enconvert.com/blog/how-to-build-a-unified-ingestion-layer-for-your-rag-pipeline-from-scratch · Python · AGPL-3.0 · 7 stars

View on Hacker News