Skip to content

Fix SCO-22193 in CICD #6509

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ryancooley merged 3 commits into from
Mar 27, 2024
Merged

Fix SCO-22193 in CICD #6509

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e751c3f
Print branch k8sDistribution
mavalosn Mar 27, 2024
5eb40de
Print branch k8sDistribution
mavalosn Mar 27, 2024
e11b23a
Fix SCO-22193
mavalosn Mar 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Fix SCO-22193
  • Loading branch information
@mavalosn
mavalosn committed Mar 27, 2024
commit e11b23af106e9c4b8aa39ebce62d44c8ed087747
296 changes: 146 additions & 150 deletions .github/workflows/deploy-pm4.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,6 @@ run-name: ${{ github.actor }} send deploy EKS 🚀
on:
pull_request:
types: [opened, reopened, synchronize, edited, closed]
push:
branches:
- CTT-51
#schedule:
# - cron: '30 2 * * *' # every day at midnight
workflow_dispatch:
Expand Down Expand Up @@ -44,150 +41,149 @@ jobs:
run: |
RESOLVED_IMAGE_TAG=${{ env.IMAGE_TAG }}
echo "IMAGE=${{ secrets.REGISTRY_HOST }}/processmaker/enterprise:$RESOLVED_IMAGE_TAG" >> $GITHUB_ENV
echo $K8S_BRANCH
#- name: Clone repo K8S
# run: |
# echo "IMAGE: ${{ env.IMAGE }}"
# git clone --depth 1 -b "$K8S_BRANCH" "https://$GITHUB_TOKEN@github.com/ProcessMaker/pm4-k8s-distribution.git" pm4-k8s-distribution
#- name: Generate image EKS
# run: |
# cd pm4-k8s-distribution/images
# branch=$CI_PACKAGE_BRANCH tag=${{env.IMAGE_TAG}} bash build.k8s-cicd.sh
# echo "VERSION=${{ env.IMAGE_TAG }}" >> $GITHUB_ENV
#- name: List Images
# run: |
# docker images
#- name: Run Trivy vulnerability scanner
# uses: aquasecurity/trivy-action@master
# with:
# image-ref: processmaker/enterprise:${{ env.VERSION }}
# format: 'table'
# exit-code: '0'
# ignore-unfixed: false
# vuln-type: 'os,library'
# scanners: 'vuln,secret'
# severity: 'MEDIUM,HIGH,CRITICAL'
# env:
# TRIVY_TIMEOUT: 30m
#- name: Login to Harbor
# uses: docker/login-action@v2
# with:
# registry: ${{ secrets.REGISTRY_HOST }}
# username: ${{ secrets.REGISTRY_USERNAME }}
# password: ${{ secrets.REGISTRY_PASSWORD }}
#- name: Push Enterprise Image to Harbor
# run: |
# docker tag processmaker/enterprise:${{env.IMAGE_TAG}} ${{ secrets.REGISTRY_HOST }}/processmaker/enterprise:${{env.IMAGE_TAG}}
# docker push ${{ secrets.REGISTRY_HOST }}/processmaker/enterprise:${{env.IMAGE_TAG}}
#deployEKS:
# name: build-deploy-EKS
# if: contains(github.event.pull_request.body, 'ci:deploy')
# needs: imageEKS
# runs-on: ${{ vars.RUNNER }}
# steps:
# - name: Clone private repository
# run: |
# git clone --depth 1 -b eng "https://$GITHUB_TOKEN@github.com/ProcessMaker/argocd.git" argocd
# - name: Install pm4-tools
# run: |
# git clone --depth 1 -b "$K8S_BRANCH" "https://$GITHUB_TOKEN@github.com/ProcessMaker/pm4-k8s-distribution.git" pm4-k8s-distribution
# cd pm4-k8s-distribution/images/pm4-tools
# composer install --no-interaction
# cd ..
# - name: Configure AWS credentials
# uses: aws-actions/configure-aws-credentials@v1
# with:
# aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID1 }}
# aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY1 }}
# aws-region: ${{ secrets.AWS_REGION }}
# - name: Set up kubectl
# run: |
# curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
# chmod +x kubectl
# sudo mv kubectl /usr/local/bin/
# - name: Authenticate with Amazon EKS
# run: aws eks update-kubeconfig --region us-east-1 --name pm4-eng
# - name: Deploy instance EKS
# run: |
# cd argocd
# deploy=$(echo -n ${{env.IMAGE_TAG}} | md5sum | head -c 10)
# current_datetime=$(echo -n ${{env.CURRENT_DATE}} | md5sum | head -c 10)
# echo "NAMESPACE : ci-$deploy-ns-pm4"
# helm repo add processmaker ${{ secrets.HELM_REPO }} --username ${{ secrets.HELM_USERNAME }} --password ${{ secrets.HELM_PASSWORD }} && helm repo update
# if ! kubectl get namespace/ci-$deploy-ns-pm4 ; then
# echo "Creating DB"
# sed -i "s/{{instance}}/ci-$deploy/" template-db.yaml
# kubectl apply -f template-db.yaml
# echo "Creating Deploy :: $deploy"
# sed -i "s/{{instance}}/ci-$deploy/" template-instance.yaml
# sed -i "s/{{image}}/${{env.IMAGE_TAG}}/" template-instance.yaml
# cat template-instance.yaml
# helm install --timeout 60m -f template-instance.yaml ci-$deploy processmaker/enterprise
# #Add cert
# sed -i "s/{{instance}}/ci-$deploy/" template-cert.yaml
# sed -i "s/{{tls-crt}}/${{ secrets.TLS_CRT }}/" template-cert.yaml
# sed -i "s/{{tls-key}}/${{ secrets.TLS_KEY }}/" template-cert.yaml
# kubectl apply -f template-cert.yaml
# else
# echo "Bouncing Instance ";
# sed -i "s/{{instance}}/ci-$deploy/g" template-bounce.yaml
# sed -i "s/{{current_datetime}}/$current_datetime/g" template-bounce.yaml
# helm upgrade --timeout 60m ci-$deploy processmaker/enterprise --version 2.1.0
# kubectl apply -f template-bounce.yaml
# fi
# export INSTANCE_URL=https://ci-$deploy$DOM_EKS
# echo "INSTANCE_URL=${INSTANCE_URL}" >> "$GITHUB_ENV"
# ../pm4-k8s-distribution/images/pm4-tools/pm wait-for-instance-ready
# - name: Comment Instance
# run: |
# echo "Instance URL: '${INSTANCE_URL}'"
# bash argocd/gh_comment.sh "$CI_PROJECT" "$pull_req_id"
#deleteEKS:
# name: Delete Instance
# if: github.event.action == 'closed'
# runs-on: self-hosted
# steps:
# - name: Delete instance EKS
# run: |
# deploy=$(echo -n ${{env.IMAGE_TAG}} | md5sum | head -c 10)
# if kubectl get namespace/ci-$deploy-ns-pm4 ; then
# echo "Deleting Instace :: ci-$deploy"
# helm delete ci-$deploy
# kubectl delete namespace ci-$deploy-ns-pm4
# #Drop database
# deploy_db="\`pm4_ci-$deploy\`"
# deploy_ai="\`pm4_ci-${deploy}_ai\`"
# mysql -u${{ secrets.USER_MYSQL_ENG }} -p${{ secrets.PASS_MYSQL_ENG }} -e "DROP DATABASE $deploy_db" -h ${{ secrets.RDS_ENG }}
# mysql -u${{ secrets.USER_MYSQL_ENG }} -p${{ secrets.PASS_MYSQL_ENG }} -e "DROP DATABASE $deploy_ai" -h ${{ secrets.RDS_ENG }}
# #Drop image Harbor
# curl -X DELETE -u ${{ secrets.REGISTRY_USERNAME }}:${{ secrets.REGISTRY_PASSWORD }} "https://${{ secrets.REGISTRY_HOST }}/api/v2.0/projects/processmaker/repositories/enterprise/artifacts/${{env.IMAGE_TAG}}"
# echo "The instance [https://ci-$deploy.engk8s.processmaker.net] was deleted!!"
# else
# echo "The pull request does not have an instance on K8s [https://ci-$deploy.engk8s.processmaker.net] not found!!"
# fi
#runPhpUnit:
# name: run-phpunit
# if: github.event.action != 'closed'
# needs: imageEKS
# runs-on: ${{ vars.RUNNER }}
# steps:
# - name: Export Params
# run: |
# echo "IMAGE=${{ secrets.REGISTRY_HOST }}/processmaker/enterprise:${{env.IMAGE_TAG}}" >> $GITHUB_ENV
# - name: Clone repo K8S
# run: |
# echo "IMAGE: ${{ env.IMAGE }}"
# git clone --depth 1 -b "$K8S_BRANCH" "https://$GITHUB_TOKEN@github.com/ProcessMaker/pm4-k8s-distribution.git" pm4-k8s-distribution
# - name: Login to Harbor
# uses: docker/login-action@v2
# with:
# registry: ${{ secrets.REGISTRY_HOST }}
# username: ${{ secrets.REGISTRY_USERNAME }}
# password: ${{ secrets.REGISTRY_PASSWORD }}
# - name: PHPUnits
# run: |
# cd pm4-k8s-distribution/images/pm4-tools
# docker pull $IMAGE
# docker-compose down -v
# docker-compose build phpunit
# docker-compose run phpunit
- name: Clone repo K8S
run: |
echo "IMAGE: ${{ env.IMAGE }}"
git clone --depth 1 -b "$K8S_BRANCH" "https://$GITHUB_TOKEN@github.com/ProcessMaker/pm4-k8s-distribution.git" pm4-k8s-distribution
- name: Generate image EKS
run: |
cd pm4-k8s-distribution/images
branch=$CI_PACKAGE_BRANCH tag=${{env.IMAGE_TAG}} bash build.k8s-cicd.sh
echo "VERSION=${{ env.IMAGE_TAG }}" >> $GITHUB_ENV
- name: List Images
run: |
docker images
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
with:
image-ref: processmaker/enterprise:${{ env.VERSION }}
format: 'table'
exit-code: '0'
ignore-unfixed: false
vuln-type: 'os,library'
scanners: 'vuln,secret'
severity: 'MEDIUM,HIGH,CRITICAL'
env:
TRIVY_TIMEOUT: 30m
- name: Login to Harbor
uses: docker/login-action@v2
with:
registry: ${{ secrets.REGISTRY_HOST }}
username: ${{ secrets.REGISTRY_USERNAME }}
password: ${{ secrets.REGISTRY_PASSWORD }}
- name: Push Enterprise Image to Harbor
run: |
docker tag processmaker/enterprise:${{env.IMAGE_TAG}} ${{ secrets.REGISTRY_HOST }}/processmaker/enterprise:${{env.IMAGE_TAG}}
docker push ${{ secrets.REGISTRY_HOST }}/processmaker/enterprise:${{env.IMAGE_TAG}}
deployEKS:
name: build-deploy-EKS
if: contains(github.event.pull_request.body, 'ci:deploy')
needs: imageEKS
runs-on: ${{ vars.RUNNER }}
steps:
- name: Clone private repository
run: |
git clone --depth 1 -b eng "https://$GITHUB_TOKEN@github.com/ProcessMaker/argocd.git" argocd
- name: Install pm4-tools
run: |
git clone --depth 1 -b "$K8S_BRANCH" "https://$GITHUB_TOKEN@github.com/ProcessMaker/pm4-k8s-distribution.git" pm4-k8s-distribution
cd pm4-k8s-distribution/images/pm4-tools
composer install --no-interaction
cd ..
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID1 }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY1 }}
aws-region: ${{ secrets.AWS_REGION }}
- name: Set up kubectl
run: |
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl
sudo mv kubectl /usr/local/bin/
- name: Authenticate with Amazon EKS
run: aws eks update-kubeconfig --region us-east-1 --name pm4-eng
- name: Deploy instance EKS
run: |
cd argocd
deploy=$(echo -n ${{env.IMAGE_TAG}} | md5sum | head -c 10)
current_datetime=$(echo -n ${{env.CURRENT_DATE}} | md5sum | head -c 10)
echo "NAMESPACE : ci-$deploy-ns-pm4"
helm repo add processmaker ${{ secrets.HELM_REPO }} --username ${{ secrets.HELM_USERNAME }} --password ${{ secrets.HELM_PASSWORD }} && helm repo update
if ! kubectl get namespace/ci-$deploy-ns-pm4 ; then
echo "Creating DB"
sed -i "s/{{instance}}/ci-$deploy/" template-db.yaml
kubectl apply -f template-db.yaml
echo "Creating Deploy :: $deploy"
sed -i "s/{{instance}}/ci-$deploy/" template-instance.yaml
sed -i "s/{{image}}/${{env.IMAGE_TAG}}/" template-instance.yaml
cat template-instance.yaml
helm install --timeout 60m -f template-instance.yaml ci-$deploy processmaker/enterprise
#Add cert
sed -i "s/{{instance}}/ci-$deploy/" template-cert.yaml
sed -i "s/{{tls-crt}}/${{ secrets.TLS_CRT }}/" template-cert.yaml
sed -i "s/{{tls-key}}/${{ secrets.TLS_KEY }}/" template-cert.yaml
kubectl apply -f template-cert.yaml
else
echo "Bouncing Instance ";
sed -i "s/{{instance}}/ci-$deploy/g" template-bounce.yaml
sed -i "s/{{current_datetime}}/$current_datetime/g" template-bounce.yaml
helm upgrade --timeout 60m ci-$deploy processmaker/enterprise --version 2.1.0
kubectl apply -f template-bounce.yaml
fi
export INSTANCE_URL=https://ci-$deploy$DOM_EKS
echo "INSTANCE_URL=${INSTANCE_URL}" >> "$GITHUB_ENV"
../pm4-k8s-distribution/images/pm4-tools/pm wait-for-instance-ready
- name: Comment Instance
run: |
echo "Instance URL: '${INSTANCE_URL}'"
bash argocd/gh_comment.sh "$CI_PROJECT" "$pull_req_id"
deleteEKS:
name: Delete Instance
if: github.event.action == 'closed'
runs-on: self-hosted
steps:
- name: Delete instance EKS
run: |
deploy=$(echo -n ${{env.IMAGE_TAG}} | md5sum | head -c 10)
if kubectl get namespace/ci-$deploy-ns-pm4 ; then
echo "Deleting Instace :: ci-$deploy"
helm delete ci-$deploy
kubectl delete namespace ci-$deploy-ns-pm4
#Drop database
deploy_db="\`pm4_ci-$deploy\`"
deploy_ai="\`pm4_ci-${deploy}_ai\`"
mysql -u${{ secrets.USER_MYSQL_ENG }} -p${{ secrets.PASS_MYSQL_ENG }} -e "DROP DATABASE $deploy_db" -h ${{ secrets.RDS_ENG }}
mysql -u${{ secrets.USER_MYSQL_ENG }} -p${{ secrets.PASS_MYSQL_ENG }} -e "DROP DATABASE $deploy_ai" -h ${{ secrets.RDS_ENG }}
#Drop image Harbor
curl -X DELETE -u ${{ secrets.REGISTRY_USERNAME }}:${{ secrets.REGISTRY_PASSWORD }} "https://${{ secrets.REGISTRY_HOST }}/api/v2.0/projects/processmaker/repositories/enterprise/artifacts/${{env.IMAGE_TAG}}"
echo "The instance [https://ci-$deploy.engk8s.processmaker.net] was deleted!!"
else
echo "The pull request does not have an instance on K8s [https://ci-$deploy.engk8s.processmaker.net] not found!!"
fi
runPhpUnit:
name: run-phpunit
if: github.event.action != 'closed'
needs: imageEKS
runs-on: ${{ vars.RUNNER }}
steps:
- name: Export Params
run: |
echo "IMAGE=${{ secrets.REGISTRY_HOST }}/processmaker/enterprise:${{env.IMAGE_TAG}}" >> $GITHUB_ENV
- name: Clone repo K8S
run: |
echo "IMAGE: ${{ env.IMAGE }}"
git clone --depth 1 -b "$K8S_BRANCH" "https://$GITHUB_TOKEN@github.com/ProcessMaker/pm4-k8s-distribution.git" pm4-k8s-distribution
- name: Login to Harbor
uses: docker/login-action@v2
with:
registry: ${{ secrets.REGISTRY_HOST }}
username: ${{ secrets.REGISTRY_USERNAME }}
password: ${{ secrets.REGISTRY_PASSWORD }}
- name: PHPUnits
run: |
cd pm4-k8s-distribution/images/pm4-tools
docker pull $IMAGE
docker-compose down -v
docker-compose build phpunit
docker-compose run phpunit