This is a repo with links to everything you'd ever want to learn about data engineering
-
Updated
Aug 3, 2026 - Jupyter Notebook
#
bigdata
Here are 2,422 public repositories matching this topic...
Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.
mysql sql database bigdata postgresql shard distributed-database encrypt data-pipeline data-encryption database-cluster distributed-transaction read-write-splitting database-middleware distributed-sql-database database-gateway
-
Updated
Oct 7, 2026 - Java
High-performance, scalable time-series database designed for Industrial IoT (IIoT) scenarios
iot sql database monitoring time-series metrics cluster bigdata scalability distributed cloud-native financial-analysis tsdb time-series-database connected-vehicles industrial-iot tdengine
-
Updated
Oct 6, 2026 - C
GridDB is a next-generation open source database that makes time series IoT and big data fast,and easy.
-
Updated
Mar 19, 2026 - C++
A curated list of awesome big data frameworks, ressources and other awesomeness.
data-science data awesome database data-stream bigdata series-database data-visualization data-warehouse stream-processing data-analytics awesome-list distributed-database visualize-data streaming-data
-
Updated
Jul 31, 2026
Upserts, Deletes And Incremental Processing on Big Data.
bigdata stream-processing data-integration datalake apachespark hudi apachehudi incremental-processing apacheflink
-
Updated
Oct 7, 2026 - Java
RustFS is an open-source, S3-compatible high-performance object storage system supporting migration and coexistence with other S3-compatible platforms such as MinIO and Ceph.
rust filesystem bigdata s3 minio cloud-native objectstorage object-storage multi-cloud amazon-s3 ai-storage ai-native
-
Updated
Oct 7, 2026 - Rust
A Cloud Native Batch System (Project under CNCF)
-
Updated
Oct 7, 2026 - Go
100+套大数据可视化炫酷大屏Html5模板；包含行业：社区、物业、政务、交通、金融银行等，全网最新、最多，最全、最酷、最炫大数据可视化模板。陆续更新中
-
Updated
Oct 24, 2025 - JavaScript
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
go redis golang distributed-systems storage filesystem bigdata s3 posix hdfs cloud-native object-storage
-
Updated
Sep 29, 2026 - Go
Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks
python data-science machine-learning big-data spark notebook ipython bigdata ipython-notebook pyspark mllib data-analysis
-
Updated
Mar 16, 2024 - Jupyter Notebook
Data Agent Ready Warehouse : One for Analytics, Search, AI, Python Sandbox. — rebuilt from scratch. Unified architecture on your S3.
rust elasticsearch sql database ai serverless geospatial bigdata snowflake olap cloud-native vector-search vector-database lakehouse
-
Updated
Oct 7, 2026 - Rust
🔨 用 JSON 来生成结构化的 SQL 语句，基于 Vue3 + TypeScript + Vite + Ant Design + MonacoEditor 实现，项目简单（重逻辑轻页面）、适合练手~
-
Updated
Jan 17, 2024 - Vue
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
-
Updated
Oct 6, 2026 - Scala
Add this topic to your repo
To associate your repository with the bigdata topic, visit your repo's landing page and select "manage topics."