Infrastructure and CI/CD Overview

The infrastructure of DevLaunchers is designed to ensure scalability, efficiency, and reliability in hosting, testing, and deploying applications. By leveraging modern containerization, orchestration, and CI/CD practices, DevLaunchers maintains a robust platform for both development and production environments. This section provides a detailed breakdown of the infrastructure components, hosting setup, and CI/CD pipeline.

1. Hosting Environment

DevLaunchers leverages Azure Kubernetes Service (AKS) to host and manage its applications. The platform uses Kubernetes for container orchestration, providing a scalable and resilient infrastructure. Both staging and production environments run on the same Kubernetes cluster, but in different namespaces.

• Staging Environment with Serverless Optimization:

  • The staging environment is designed to optimize costs by leveraging Kubernetes-native serverless technology (Knative).

  • Knative enables the staging deployments to scale workloads to zero when not in use, minimizing resource consumption during idle periods.

  • The deployment is scaled up when there is a new request.

2. GitOps and Deployment Automation

DevLaunchers uses a GitOps workflow powered by FluxCD to handle deployments in both staging and production environments. This ensures that the infrastructure and application states are always consistent with the configuration defined in the Git repository. Flux deploys a few controllers in the flux-system to achieve this.

• FluxCD Workflow:

  • The image reflect controller scans Docker Hub for new docker images

  • When there is a new image, the image automation controller commits the new image version to the Git repository. For example, this commit updates the staging strapi image

  • The source controller monitors the GitHub repository for changes to manifests, configurations, or Docker image tags.

  • When changes are detected, the kustomize controller applies the new Kubernetes manifest to the relevant namespace.

  • This approach ensures declarative, version-controlled deployments with minimal manual intervention.

3. CI/CD Pipeline Overview

The CI is implemented using GitHub Actions to automate the testing and building processes. The CD pipeline is managed by FluxCD. The pipeline is broken into distinct stages to ensure code quality, efficiency, and reliability.

CI/CD Pipeline

Pipeline Architecture

• Continuous Integration (CI):

  • Runs automated tests, including linting, unit testing, and end-to-end testing.

  • Generates Docker images for both staging and production environments.

• Continuous Delivery (CD):

  • Deploys Docker images to AKS clusters using FluxCD, ensuring that new versions are seamlessly integrated.

Key Workflows:

The frontend and backend have similar workflows. In the following example, we will reference workflows for the backend.

Github Action Workflows

1. Test Workflow (test.yaml):

   • Triggered on each pull request (PR)

   • Runs automated tests on each PR to ensure code quality and prevent regressions.

   • Includes end-to-end tests using Playwright to validate application behavior.

2. Staging Image Workflow (staging-image.yaml):

   • Triggered when there is a new commit in the main branch

   • Builds and tags Docker images by the timestamp of the build (YearMonthDayHourMinute)

   • Push Docker images to Docker Hub.
3. Release Workflow (release.yaml):

   • Triggered when there is a new commit in the release branch

   • Handles semantic versioning using semantic-release.
   • Creates a new release tag

4. Production Image Workflow (production-image.yaml):

   • Triggered when there is a new release is published (by the release workflow)

   • Builds and tags Docker images by the release version

   • Push Docker images to Docker Hub

---

Triggering CI/CD:

We will use an example to demonstrate our CI/CD. The following happens when a PR is merged to the main branch:

1. The staging image workflow builds a new Docker image that is tagged in the format of <sha>-<timestamp>.
2. The image is pushed to DockerHub which triggers a webhook.

3. The webhook is sent to a dockerhub flux receiver. We configured the receiver to reconcile the ImageRepository resource.
4. The image reflector controller will discover a new image tag.

5. The ImagePolicy will extract the timestamp and pick the latest timestamp as the latest image.
6. The ImageUpdateAutomation writes the new tag back to the repository.
7. The repository has a webhook to notify the github receiver. The receiver reconciles the GitRepository resource.
8. The source controller pulls the latest Kubernetes manifests from the repository. It'll see the new image tag and update the deployment.

4. Monitoring and Logging

DevLaunchers uses a combination of tools to monitor application health, gather metrics, and alert the team in case of issues:

• Monitoring:

  • Prometheus: Used to collect metrics from the Strapi backend and Kubernetes clusters. The current setup generates application and system metrics but requires further development to set up dashboards and store metrics for analysis.

  • External Monitoring: The Uptime Robot platform monitors the production environment’s uptime by sending regular HTTP requests to the production endpoints.

• Alerting:

  • Alerts from Uptime Robot notify the team when the production website is down or experiencing performance issues. The backend service is also integrated into this monitoring system.

5. Storage and Data Management

The database for the DevLaunchers platform is hosted on a cloud cluster platform, providing scalable and highly available Postgres cluster. The staging and production use different databases but on the same cluster.