CI/CD

Building a Multi-Cloud CI/CD Pipeline with GitHub Actions

January 10, 2024•15 min read•Amol Tribhuwan

Building a Multi-Cloud CI/CD Pipeline with GitHub Actions

Multi-cloud strategies are becoming essential for modern enterprises. This guide shows you how to build a unified CI/CD pipeline that deploys to AWS, Azure, and GCP using GitHub Actions.

Why Multi-Cloud?

Benefits

Avoid vendor lock-in: Reduce dependency on a single provider
Cost optimization: Use the best-priced service for each workload
Regulatory compliance: Meet data residency requirements
High availability: Geographic redundancy across providers

Challenges

Increased complexity
Multiple authentication mechanisms
Different CLI tools and APIs
Varied deployment patterns

Architecture Overview

┌─────────────┐
│   GitHub    │
│ Repository  │
└──────┬──────┘
       │
       ▼
┌─────────────────┐
│ GitHub Actions  │
│    Workflow     │
└────┬───┬───┬────┘
     │   │   │
     ▼   ▼   ▼
   AWS Azure GCP

Setting Up GitHub Actions Workflow

The `.github/workflows/multi-cloud-deploy.yml`

name: Multi-Cloud Deployment

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

env:
  APP_NAME: my-app
  VERSION: ${{ github.sha }}

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
      
      - name: Build and push Docker image
        uses: docker/build-push-action@v4
        with:
          context: .
          push: true
          tags: |
            ghcr.io/${{ github.repository }}:${{ env.VERSION }}
            ghcr.io/${{ github.repository }}:latest
          cache-from: type=gha
          cache-to: type=gha,mode=max

  deploy-aws:
    needs: build
    runs-on: ubuntu-latest
    environment: production-aws
    steps:
      - uses: actions/checkout@v3
      
      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1
      
      - name: Deploy to ECS
        run: |
          aws ecs update-service \
            --cluster production \
            --service ${{ env.APP_NAME }} \
            --force-new-deployment
      
      - name: Wait for deployment
        run: |
          aws ecs wait services-stable \
            --cluster production \
            --services ${{ env.APP_NAME }}

  deploy-azure:
    needs: build
    runs-on: ubuntu-latest
    environment: production-azure
    steps:
      - uses: actions/checkout@v3
      
      - name: Azure Login
        uses: azure/login@v1
        with:
          creds: ${{ secrets.AZURE_CREDENTIALS }}
      
      - name: Deploy to Azure Container Instances
        uses: azure/aci-deploy@v1
        with:
          resource-group: production-rg
          dns-name-label: ${{ env.APP_NAME }}
          image: ghcr.io/${{ github.repository }}:${{ env.VERSION }}
          name: ${{ env.APP_NAME }}
          location: eastus

  deploy-gcp:
    needs: build
    runs-on: ubuntu-latest
    environment: production-gcp
    steps:
      - uses: actions/checkout@v3
      
      - name: Authenticate to Google Cloud
        uses: google-github-actions/auth@v1
        with:
          credentials_json: ${{ secrets.GCP_SA_KEY }}
      
      - name: Deploy to Cloud Run
        run: |
          gcloud run deploy ${{ env.APP_NAME }} \
            --image ghcr.io/${{ github.repository }}:${{ env.VERSION }} \
            --platform managed \
            --region us-central1 \
            --allow-unauthenticated

  smoke-tests:
    needs: [deploy-aws, deploy-azure, deploy-gcp]
    runs-on: ubuntu-latest
    steps:
      - name: Test AWS Endpoint
        run: curl -f https://aws.example.com/health || exit 1
      
      - name: Test Azure Endpoint
        run: curl -f https://azure.example.com/health || exit 1
      
      - name: Test GCP Endpoint
        run: curl -f https://gcp.example.com/health || exit 1

Authentication Setup

AWS

Create an IAM user with programmatic access:

aws iam create-user --user-name github-actions
aws iam attach-user-policy \
  --user-name github-actions \
  --policy-arn arn:aws:iam::aws:policy/AmazonECS_FullAccess
aws iam create-access-key --user-name github-actions

Add to GitHub Secrets:

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY

Azure

Create a service principal:

az ad sp create-for-rbac \
  --name "github-actions" \
  --role contributor \
  --scopes /subscriptions/{subscription-id} \
  --sdk-auth

Add entire JSON output to GitHub Secret: AZURE_CREDENTIALS

GCP

Create a service account:

gcloud iam service-accounts create github-actions \
  --display-name="GitHub Actions"

gcloud projects add-iam-policy-binding PROJECT_ID \
  --member="serviceAccount:github-actions@PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/run.admin"

gcloud iam service-accounts keys create key.json \
  --iam-account=github-actions@PROJECT_ID.iam.gserviceaccount.com

Add key.json contents to GitHub Secret: GCP_SA_KEY

Infrastructure as Code

Terraform for Multi-Cloud

# main.tf
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.0"
    }
    google = {
      source  = "hashicorp/google"
      version = "~> 5.0"
    }
  }
}

# AWS ECS Cluster
resource "aws_ecs_cluster" "main" {
  name = "production"
  
  setting {
    name  = "containerInsights"
    value = "enabled"
  }
}

# Azure Container Instance
resource "azurerm_container_group" "main" {
  name                = var.app_name
  location            = azurerm_resource_group.main.location
  resource_group_name = azurerm_resource_group.main.name
  os_type             = "Linux"

  container {
    name   = var.app_name
    image  = var.container_image
    cpu    = "1"
    memory = "1.5"

    ports {
      port     = 80
      protocol = "TCP"
    }
  }
}

# GCP Cloud Run Service
resource "google_cloud_run_service" "main" {
  name     = var.app_name
  location = "us-central1"

  template {
    spec {
      containers {
        image = var.container_image
      }
    }
  }
}

Monitoring & Observability

Unified Metrics Dashboard

Use Prometheus and Grafana to aggregate metrics:

# prometheus.yml
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'aws-app'
    static_configs:
      - targets: ['aws.example.com:9090']
    
  - job_name: 'azure-app'
    static_configs:
      - targets: ['azure.example.com:9090']
    
  - job_name: 'gcp-app'
    static_configs:
      - targets: ['gcp.example.com:9090']

Centralized Logging

Aggregate logs with the ELK stack:

# filebeat.yml
filebeat.inputs:
  - type: log
    enabled: true
    paths:
      - /var/log/aws/*.log
      - /var/log/azure/*.log
      - /var/log/gcp/*.log
    
output.elasticsearch:
  hosts: ["elk.example.com:9200"]
  index: "multi-cloud-%{+yyyy.MM.dd}"

Cost Optimization

Use Spot/Preemptible Instances

# AWS Spot
aws ecs update-service \
  --capacity-provider-strategy \
    capacityProvider=FARGATE_SPOT,weight=1

# GCP Preemptible
gcloud run deploy --execution-environment=gen2 \
  --min-instances=0 --max-instances=10

Resource Tagging

# Uniform tagging strategy
tags:
  Environment: production
  Project: multi-cloud-demo
  ManagedBy: terraform
  CostCenter: engineering

Disaster Recovery

Multi-Cloud Failover

# nginx.conf for Global Load Balancing
upstream backends {
  server aws.example.com:443 max_fails=3 fail_timeout=30s;
  server azure.example.com:443 backup;
  server gcp.example.com:443 backup;
}

Automated Health Checks

# GitHub Actions workflow
- name: Health Check & Failover
  run: |
    if ! curl -f https://aws.example.com/health; then
      echo "AWS failed, updating DNS to Azure"
      aws route53 change-resource-record-sets \
        --hosted-zone-id Z123 \
        --change-batch file://failover-to-azure.json
    fi

Best Practices

Environment Isolation: Use separate GitHub Environments
Secret Management: Rotate credentials every 90 days
Staged Rollouts: Deploy to dev → staging → production
Automated Rollback: Detect failures and rollback automatically
Cost Monitoring: Set up billing alerts on all platforms

Real-World Results

Implementation at a SaaS company:

Before:

Manual deployments (2-3 hours)
Single cloud provider (vendor lock-in)
98.5% uptime

After:

Automated deployments (15 minutes)
Multi-cloud redundancy
99.97% uptime
40% cost savings through optimal resource placement

Common Issues & Solutions

Problem: Authentication Failures

Solution: Use OIDC instead of static credentials

- uses: aws-actions/configure-aws-credentials@v2
  with:
    role-to-assume: arn:aws:iam::123456789012:role/GitHubActions
    aws-region: us-east-1

Problem: Slow Deployments

Solution: Parallel deployments with matrix strategy

strategy:
  matrix:
    cloud: [aws, azure, gcp]
  max-parallel: 3

Conclusion

Multi-cloud CI/CD pipelines offer resilience and flexibility but require careful planning. Key takeaways:

Use GitHub Actions for orchestration
Standardize on containers for portability
Implement comprehensive monitoring
Automate everything possible

Start small with one additional cloud provider, then expand as confidence grows.

Resources

Questions about multi-cloud deployments? Let's connect on LinkedIn!

#CI/CD#GitHub Actions#Multi-Cloud#AWS#Azure#GCP

Building a Multi-Cloud CI/CD Pipeline with GitHub Actions

Why Multi-Cloud?

Benefits

Challenges

Architecture Overview

Setting Up GitHub Actions Workflow

The .github/workflows/multi-cloud-deploy.yml

Authentication Setup

AWS

Azure

GCP

Infrastructure as Code

Terraform for Multi-Cloud

Monitoring & Observability

Unified Metrics Dashboard

Centralized Logging

Cost Optimization

Use Spot/Preemptible Instances

Resource Tagging

Disaster Recovery

Multi-Cloud Failover

Automated Health Checks

Best Practices

Real-World Results

Common Issues & Solutions

Problem: Authentication Failures

Problem: Slow Deployments

Conclusion

Resources

The `.github/workflows/multi-cloud-deploy.yml`