Learn PySpark through practical examples. This repository includes Spark fundamentals, DataFrame operations, Spark SQL, transformations, actions, and real-world practice for aspiring Data Engineers.
Restaurant Analysis using Apache Spark
Pyspark data analytics & ML models
Projeto PySpark orientado a objetos relatório de pedidos com pagamentos recusados e legítimos (2025)
The Forex Data Pipeline is a comprehensive solution designed to collect, process, and prepare currency exchange rate data for downstream machine-learning pipelines. This repository showcases the creation of a data pipeline that fetches currency rates from an external API and performs data transformation using PySpark.
Mini projet realisé au sein de la Faculté de Sciences de Kenitra pour le cours de Technologies du Big Data(Master Big Data et Cloud Computing)
The IPL Data Analysis project aims to explore and analyze the Indian Premier League (IPL) data using PySpark for data processing and Matplotlib and Seaborn for data visualization. The goal is to derive actionable insights into player performances, match trends, and overall league dynamics.
This Repo contains details about leveraging GCP Airflow DAGS and Dataproc Cluster to Load the data from GCS bucket to Snowflake, Thanks
Loading different types of dataset files using Flume and pyspark
Databricks notebook for loading Ambient Weather station data into Delta Lake
This is done in pyspark. Please take the CSV file from Kaggle. Link: https://www.kaggle.com/mlg-ulb/creditcardfraud
PySpark is a Python API for support Python with Spark. Whether it is to perform computations on large datasets or to just analyze them
A comparative study to understand the computing efficiencies of Pyspark architectures vs python based distributed programming methodologies such as MPI, multi-threading or multi-processing on the Yelp kaggle dataset.
An end-to-end Batch ETL Pipeline implemented on Azure Databricks using PySpark and SQL, processing over 11 million rows of real-world urban data. The project uncovers hidden correlations between New York City transit patterns and historical weather conditions.
Real-time oil well production monitoring pipeline with Kafka, PySpark, Grafana, and AWS (S3, SNS, SQS, ECR). Infrastructure managed by Terraform with CI/CD via GitHub Actions.
Finding frequent itemsets using Apriori and FP Growth algorithm on Spark
Creating a logistic model that predicts which passengers survived the Titanic shipwreck using pyspark
To associate your repository with the pyspark-python topic, visit your repo's landing page and select "manage topics."