zstd

Benchmark Python libraries implementing Zstandard

small shell script to do some compression, arching and deduplication of files in bulk

🗜️ Framework-independant compression middleware for ASGI apps, supporting Gzip, Brotli and Zstandard algorithms

Efficiently store FASTA sequences in sqlite compressed with sidecar zstd dictionary

Extreme-density data vault for LLM training sets. MsgPack + Zstd-L22 + AES-256-GCM + SHA3-256 Merkle. 39× compression with full encryption and tamper detection.

The simplest index-and-search engine for huge multiline text files. Focused primarily on bioinformatics. Inspired by tabix, but isn't its replacement. Written in Python. Works on top of Zstandard Seekable & pyzstd SeekableZstdFile.

CLI to archive logs into timestamped tarballs with retention, incremental mode, gzip/zstd compression, SHA256 and optional GPG, Prometheus metrics; cron/systemd friendly

asyncio client for kafka - HARD FORK from aio-libs/aiokafka

Findata Knowledge Graph

This is a program to compress and decompress files, using Zstandard.

Automated, encrypted, and compressed backups for your OpenClaw installation.

Single-box lakehouse on Citi Bike data. Convert monthly CSVs once, store Parquet with ZSTD, query with DuckDB and Polars, compact files, and stream batches into a small PyTorch model. Grows from one laptop to a future cluster with the same data layout.

A library implementing composable and stackable reversible processor objects for data. !!! Migrated to Codeberg 🏔️ !!!

Smart content-aware file compression utility with automatic codec selection (Zstd, Brotli, Gzip, BZ2, LZMA), streaming compression, integrity verification, dictionary training, and CLI support.

微信聊天记录 → 人格画像与关系洞察：本地解密、行为统计、语言模式、抢功检测、单文件 HTML 报告 | Local-first personality & relationship insights from decrypted WeChat (macOS) chat history — zero network, privacy-first, agent-skill compatible

双密码「分块打乱加密压缩」归档工具 — dual-password chunked shuffle-encrypt archive tool (zstd + AES-256-GCM + keyed permutation)

The Adaptive Tournament Multi-Engine Compression Tool & Next-Generation Container Format

Project built for scraping Reddit datasets.

Read every clip marker out of a DaVinci Resolve Project.db

A lightweight, pure-Python tool for comparing large JSONL/NDJSON datasets without loading them entirely into memory.