Namazone: A Massively Scalable Distributed Microservices E-Commerce Application

1. Project Overview
2. System Design
3. Components
   • Microservices, Databases, Caching & Design Patterns
     • Users Microservice
     • Transactions Microservice
     • Merchants Microservice
     • Notifications Microservice
     • Search Microservice
   • API Documentation
   • Message Queues
   • Synchronous Communication
   • API Gateway
4. Observability
   • Dashboards
   • Metrics
   • Logging
   • Tracing
5. CI/CD
6. Developing with Docker Compose
7. Deployment to Kubernetes Local Cluster (Minikube)
8. Deployment to DigitalOcean
9. Testing
10. Contributors & Teams
11. License
12. Credits

Namazone is a distributed microservices E-commerce application inspired as an Amazon Replica that allows merchants to list products and customers to purchase products using wallet, credit card (Stripe), or cash on delivery (COD), generate invoices, and admins to view sales reports for the system. The system is designed to be massively scalable, with each microservice handling a specific domain of the application. The architecture is built using Spring Boot for the backend, with various databases and caching mechanisms to ensure high availability and performance, and uses Kubernetes for a highly available and scalable deployment, and is designed to be observability-first, with built-in support for metrics, logging, and tracing.

This project uses OpenAPI 3.0 for API documentation for each microservice. The API documentation is generated automatically from the code using the Springdoc OpenAPI library.

For example, the API documentation for the Users microservice is available at:

http://localhost:8085/swagger-ui/index.html

This project uses RabbitMQ 🐰 as the message queue for asynchronous communication between microservices. Each microservice that needs to send or receive messages from the message queue has a dedicated RabbitMQ exchange and queue.

This project uses OpenFeign for synchronous communication between microservices. Each microservice that needs to communicate with another microservice declares an OpenFeign client to make HTTP requests to the other microservice's exposed API controller endpoints.

This project uses Spring Cloud Gateway as the API Gateway for routing requests to the appropriate microservice. The API Gateway is responsible for handling incoming requests, routing them to the appropriate microservice, and returning the response to the client.

It also handles cross-cutting concerns such as authentication and request/response transformation. It verifies the JWT token for authenticated requests, and forwards the request to the appropriate microservice based on the request path after attaching the extracted User ID and User Role from the JWT to the request headers, to be used by the microservices downstream.

This project uses Grafana for visualizing metrics, logs, and traces. It provides a unified way for monitoring the entire system. The dashboards are configured to display metrics from Prometheus, logs from Loki, and traces from Tempo.

This project uses Prometheus for collecting and storing metrics from the microservices. Each microservice exposes a /actuator/prometheus endpoint that Prometheus scrapes to collect metrics.

This project uses Grafana Loki for logging. In the Docker Compose environment, each microservice is configured to use the Loki Docker driver for logging, which sends logs to the Loki server. The logs can be viewed in Grafana. In the Kubernetes environment, the Loki docker driver is replaced with Promtail, which is a log collector that sends logs to Loki.

This project uses OpenTelemetry for distributed tracing. Each microservice is instrumented with OpenTelemetry to collect traces and send them to the Tempo server. The traces can be viewed in Grafana.

This project uses GitHub Actions for continuous integration and continuous deployment (CI/CD). The CI/CD pipeline is defined in the .github/workflows directory.

All changes to the main branch require at least one approval from a code owner of the part(s) of the codebase that were modified, as defined in the CODEOWNERS file in the .github directory.

The CI pipeline is triggered on every push to the main branch and on pull requests. It checks out the code, sets up the JDK, and builds the project using Maven to ensure that the project builds successfully. Finally, it applies formatting to the code using Maven Spotless plugin to ensure that the code adheres to the project's coding standards.

The CD pipeline is triggered on every push to the main branch. It does the following:

1. Builds the project using Maven.

2. Builds Docker images for each microservice that was changed and pushes them to our Docker Hub registry/repository.

3. Deploys the updated microservices to the Kubernetes cluster using the deployment files in the k8s directory.

4. Notifies the team via our Discord server about the deployment status, timestamp, commit, author and jobs.

   

• Docker Desktop
• Maven 3.9.10+
• Java 23

This project uses Docker Compose to run the entire system locally for development purposes.

In each microservice folder (e.g. services/users, services/transactions, etc.), there is a docker-compose.yml file that defines the service and its dependencies (e.g. its database, cache). This can be used to run each microservice independently for development and testing purposes if the microservice does not depend on other microservices or on the message queue.

To run the entire system locally, you can use the provided docker-compose.yml file in the root directory of the project. This file defines all the services, databases, and message queue needed to run the application, also including the API Gateway and the observability stack (Prometheus, Grafana, Loki, Tempo).

The following commands are generally required for either case.

1. First, install the Docker plugin for Loki, which is used for logging.

   docker plugin install grafana/loki-docker-driver:2.9.2 --alias loki --grant-all-permissions

2. Then, build the project and start the services using Docker Compose.

   mvn clean install -DskipTests
   docker compose up -d --build

3. Add an email and its app password to the application.yml file of the Notifications microservice in services/notifications. This is necessary to enable the email notification functionality in the Notifications microservice.

   • You can use any email service provider, such as Gmail, Outlook, etc.
   • This email will be used to send notifications to users, such as order confirmations, product stock updates, etc.

4. Add your stripe secret key to the application.yml file of the Transactions microservice in services/transactions.

5. Install stripe cli from this link and add its path to your path environment variable on your operating system. This is necessary to enable the Stripe payment functionality in the Transactions microservice, through a local Stripe sandbox.

6. After that, you can run the following command to start listening to Stripe events and forward them to the Transactions microservice webhook endpoint.

   stripe login
   stripe listen --forward-to localhost:8084/stripe/webhook
   

7. Copy the webhook secret that is printed in the terminal after running the previous command, and paste it in the application.yml file of the Transactions microservice in services/transactions.

8. Download the Postman collection from the docs/postman directory and import it into Postman.

9. Update the Postman collection's host variable in the local environment to the URL of the API Gateway or the Microservice you want to interact with.

10. When done, you can stop the services by running:

    docker compose down

This system is designed to be deployed on a Kubernetes cluster. The deployment files are located in the k8s directory.

To deploy the system on a local Kubernetes cluster, we use Minikube. To set up Minikube, follow these steps:

1. Install Minikube by following the instructions on the Minikube website.

2. Install Kubectl by following the instructions on the Kubernetes website.

3. Start Minikube by running the following command (you can adjust the memory and CPU according to your machine's specifications, but lower values may lead to issues or pods dying due to out of memory OOM errors):

   minikube start --driver=docker --memory=7000 --cpus=8

4. Add your stripe secret key to the stripe-secret.yaml file in the k8s/services/transactions directory.

5. Run our deployment script to deploy the system to the Minikube (Note: This script is used instead of the kubectl apply -f ./k8s/ -r command as it does some additional setup work to set up the Stripe CLI sandbox and dynamically update the K8s secret with the Stripe webhook secret after its deployment):

   ./bash/deploy.sh

6. Expose the API Gateway service to access it from outside the cluster:

   minikube service apigateway-service

7. Copy the API Gateway URL that will be opened in your browser. This URL will be used to access the API Gateway and, consequently, the entire system.

8. Download the Postman collection from the docs/postman directory and import it into Postman.

9. Update the Postman collection's host variable in the local environment to the URL of the API Gateway you previously copied.

10. Expose the Grafana service to access it in your browser:

minikube service grafana

11. Log in to Grafana using the default credentials:

    • Username: admin
    • Password: admin

12. When done, you can stop the Minikube cluster by running:

    minikube stop

This system is designed to be deployed on a Kubernetes cluster on DigitalOcean. Our CD pipeline is set up to automatically deploy the system to a DigitalOcean Kubernetes cluster whenever changes are pushed to the main branch, and what remains is to set up the DigitalOcean Kubernetes cluster and the necessary resources on the cloud.

Our system is designed to be tested using JUnit 5 with Spring Boot Test.

Testing was done primarily using Postman, and the Postman collection is available in the docs/postman directory. The collection includes tests for each microservice's API endpoints, and it can be used to test the entire system end-to-end.

Unit Tests and Integration Tests are yet to be implemented for the microservices, but the system is designed to be easily testable using JUnit 5. Each microservice should have its own test suite, and the tests can be run using Maven.

This project is a collaborative effort of 15 extraordinary engineers split into 3 teams of 5, each responsible for different microservices and components of the system.

See, additionally, the AUTHORS file for a list of the contributors to this project.

This team is responsible for the Users Microservice, API Gateway, CI/CD, Observability, and Kubernetes deployment, service discovery, and load balancing.

• Mahmoud Abou Eleneen (@mahmoudaboueleneen) - Scrum Master & Team A Leader
• Maya Khaled Saad (@mayaasaadd)
• Abdelrahman Abouelkheir (@abdelrahmanAbouelkheir)
• Marwan Amgad (@marwanamgad)
• Ahmed Sherif Said (@Ahmedsherif74)

This team is responsible for the Transactions and Notifications Microservices, as well as the Message Queues and the Caching for their microservices.

• Ibrahim Soltan (@Ibrahim-Soltan) - Team B Leader
• Hamza Gehad (@Hamza-gehad)
• Mohamed Ahmed Elfar (@mohamedahmedelfar)
• Khaled Magdy (@khaledmagdy-oss)
• Remas Osama (@RemasOsama)

This team is responsible for the Merchants and Search Microservices, as well as the Caching for their microservices.

• Ahmed Sherif (@AhmedSherif9) - Team C Leader
• Mohamed Ahmed Aly (@mohamed-ahmed121)
• Ahmed Hassaballah (@hassaballah17)
• Kirollos Naguib (@kirollosn)
• Youssef Khawaga (@khawagaa)

This project is licensed under the GNU General Public License v3.0. See the LICENSE file for more details.

• Workup - GitHub Repo
• Marwan Fouad - Creator of 'Namazone' Logo (@svjp7)
• Architecture of Massively Scalable Applications course, the German University in Cairo