dataengineering

Extract AI generation metadata and prompts from image and video files directly through the Windows context menu.

🚀 Build a containerized data engineering workflow for NYC Yellow Taxi Trip Data using Apache Spark, Airflow, MinIO, and DuckDB, fully reproducible with Docker.

My Data Engineering Master's project involved increasing the number of features in a dataset of R515B thermodynamic refrigerant using symbolic transformer and evolutionary forest methods to improve performance.

🚀 Migrate legacy mainframe data to a modern Hadoop ecosystem, automating ingestion, transformation, and validation for scalable storage and analytics.

JS SDK for Heimdahl onchain data platform

Open Standard for Metadata. A Single place to Discover, Collaborate and Get your data right.

Let's be honest - most data pipeline frameworks treat types as suggestions. Config files are strings. Schemas are "validated" at runtime. Data quality is an afterthought. So, let's do differently

Technical Portfolio - Shrikant Lambe

Projeto end-to-end de Análise de Dados de Continuidade Elétrica e Previsão de Risco Regulatório (ANEEL) | Programa AI Talent Academy 2026

Track Hong Kong city data in real time with an open-source dashboard for transport, weather, air quality, and public info

healthcare-sql-to-snowflake-migration - Demonstrates migration discovery, SQL translation, target modeling, and control-total validation — the exact conversation a client needs to have before trusting a migration.

Scalable and efficient data transformation framework - backwards compatible with dbt.

A full-stack personal portfolio with HoodaAgents AI assistant, live Strava training analytics, real-time flight tracking, network analysis tools, interactive snow & hike photo albums, live weather, and a secure AI gateway — all deployed on Vercel.

An end-to-end data engineering climate analytics pipeline comparing long-term temperature trends between 13 stations: Houston, Newark, Delhi, London, Dallas, Denver, Helsinki, Belgium, Paris, Amsterdam, Rome, Chicago, and Los Angeles using NOAA GHCN Daily data (1970–Present).

Companion repository that goes along with Snowflake's "Introduction to Modern Data Engineering with Snowflake" course on Coursera

The open-source context layer for your AI. Catalog your tables, topics, queues and APIs then expose real metadata to your AI agents.

Give your AI coding agent access to curated ML/AI knowledge.

End-to-end data engineering pipeline for Kommineni Automotive - Bronze/Silver/Gold medallion architecture with role based access control and live Streamlit dashboard

Open-source data lakehouse on your laptop: Iceberg 1.12 with the Lakekeeper REST catalog, RustFS (or SILO), DuckDB, Polars, PyIceberg, Spark 4.1 and Trino. Retail medallion and renewal-features pipelines.

🌊 Build a modern data lakehouse with ease, leveraging tools like MinIO, MariaDB, and Dremio for efficient data management and analytics.