CI/CD Pipeline (GitHub Actions)

Set up and manage CI/CD pipelines using GitHub Actions. Covers workflow creation, testing, deployment, release automation, and debugging.

When to Use

Setting up automated testing on push/PR
Creating deployment pipelines (staging, production)
Automating releases with changelogs and tags
Debugging failing CI workflows
Setting up matrix builds for cross-platform testing
Managing secrets and environment variables in CI
Optimizing CI with caching and parallelism

Quick Start: Add CI to a Project

Node.js project

# .github/workflows/ci.yml
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]

jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
- run: npm ci
- run: npm test
- run: npm run lint

Python project

# .github/workflows/ci.yml
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]

jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: pip
- run: pip install -r requirements.txt
- run: pytest
- run: ruff check .

Go project

# .github/workflows/ci.yml
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]

jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: "1.22"
- run: go test ./...
- run: go vet ./...

Rust project

# .github/workflows/ci.yml
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]

jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- run: cargo test
- run: cargo clippy -- -D warnings

Common Patterns

Matrix builds (test across versions/OSes)

jobs:
  test:
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
        node-version: [18, 20, 22]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
      - run: npm ci
      - run: npm test

Conditional jobs

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm test

deploy:
needs: test
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: ./deploy.sh

Caching dependencies

# Node.js (automatic with setup-node)
- uses: actions/setup-node@v4
  with:
    node-version: 20
    cache: npm  # or yarn, pnpm
Generic caching

uses: actions/cache@v4
with:
path: |
~/.cache/pip
~/.cargo/registry
node_modules
key: ${{ runner.os }}-deps-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-deps-

Artifacts (save build outputs)

- uses: actions/upload-artifact@v4
  with:
    name: build-output
    path: dist/
    retention-days: 7
Download in another job

uses: actions/download-artifact@v4
with:
name: build-output
path: dist/

Run on schedule (cron)

on:
  schedule:
    - cron: "0 6 * * 1"  # Every Monday at 6 AM UTC
  workflow_dispatch:  # Also allow manual trigger

Deployment Workflows

Deploy to production on tag

name: Release
on:
push:
tags:
- "v*"
jobs:
release:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
- run: npm ci
- run: npm run build
- run: npm test
  # Create GitHub release
  - uses: softprops/action-gh-release@v2
    with:
      generate_release_notes: true
      files: |
        dist/*.js
        dist/*.css

Deploy to multiple environments

name: Deploy
on:
push:
branches: [main, staging]
jobs:
deploy:
runs-on: ubuntu-latest
environment: ${{ github.ref == 'refs/heads/main' && 'production' || 'staging' }}
steps:
- uses: actions/checkout@v4
- run: npm ci && npm run build
- run: |
if [ "${{ github.ref }}" = "refs/heads/main" ]; then
./deploy.sh production
else
./deploy.sh staging
fi
env:
DEPLOY_TOKEN: ${{ secrets.DEPLOY_TOKEN }}

Docker build and push

name: Docker
on:
push:
branches: [main]
tags: ["v*"]
jobs:
build:
runs-on: ubuntu-latest
permissions:
packages: write
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/build-push-action@v6
with:
push: true
tags: |
ghcr.io/${{ github.repository }}:latest
ghcr.io/${{ github.repository }}:${{ github.sha }}
cache-from: type=gha
cache-to: type=gha,mode=max

npm publish on release

name: Publish
on:
release:
types: [published]

jobs:
publish:
runs-on: ubuntu-latest
permissions:
id-token: write
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
registry-url: https://registry.npmjs.org
- run: npm ci
- run: npm test
- run: npm publish --provenance
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

Secrets Management

Set secrets via CLI

# Set a repository secret
gh secret set DEPLOY_TOKEN --body "my-secret-value"
Set from a file
gh secret set SSH_KEY < ~/.ssh/deploy_key
Set for a specific environment
gh secret set DB_PASSWORD --env production --body "p@ssw0rd"
List secrets
gh secret list
Delete a secret

gh secret delete OLD_SECRET

Use secrets in workflows

env:
  # Available to all steps in this job
  DATABASE_URL: ${{ secrets.DATABASE_URL }}
steps:

run: echo "Deploying..."
env:
Available to this step only
API_KEY: ${{ secrets.API_KEY }}

Environment protection rules

Set up via GitHub UI or API:

Required reviewers before deployment
Wait timers
Branch restrictions
Custom deployment branch policies

# View environments
gh api repos/{owner}/{repo}/environments | jq '.environments[].name'

Workflow Debugging

Re-run failed jobs

# List recent workflow runs
gh run list --limit 10
View a specific run
gh run view <run-id>
View failed job logs
gh run view <run-id> --log-failed
Re-run failed jobs only
gh run rerun <run-id> --failed
Re-run entire workflow
gh run rerun <run-id>

Debug with SSH (using tmate)

# Add this step before the failing step
- uses: mxschmitt/action-tmate@v3
  if: failure()
  with:
    limit-access-to-actor: true

Common failures and fixes

"Permission denied" on scripts

- run: chmod +x ./scripts/deploy.sh && ./scripts/deploy.sh

"Node modules not found"

# Make sure npm ci runs before npm test
- run: npm ci     # Install exact lockfile versions
- run: npm test   # Now node_modules exists

"Resource not accessible by integration"

# Add permissions block
permissions:
  contents: write
  packages: write
  pull-requests: write

Cache not restoring

# Check cache key matches - use hashFiles for lockfile
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
# NOT: key: ${{ runner.os }}-node-${{ hashFiles('package.json') }}

Workflow not triggering

Check: is the workflow file on the default branch?
Check: does the trigger event match? (
```
push
```
vs
```
pull_request
```
)
Check: is the branch filter correct?

# Manually trigger a workflow
gh workflow run ci.yml --ref main

Workflow Validation

Validate locally before pushing

# Check YAML syntax
python3 -c "import yaml; yaml.safe_load(open('.github/workflows/ci.yml'))" && echo "Valid"
Use actionlint (if installed)
actionlint .github/workflows/ci.yml
Or via Docker
docker run --rm -v "$(pwd):/repo" -w /repo rhysd/actionlint:latest

View workflow as graph

# List all workflows
gh workflow list
View workflow definition
gh workflow view ci.yml
Watch a running workflow

gh run watch

Advanced Patterns

Reusable workflows

# .github/workflows/reusable-test.yml
name: Reusable Test
on:
  workflow_call:
    inputs:
      node-version:
        required: false
        type: string
        default: "20"
    secrets:
      npm-token:
        required: false

jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: ${{ inputs.node-version }}
- run: npm ci
- run: npm test

# .github/workflows/ci.yml - caller
name: CI
on: [push, pull_request]
jobs:
  test:
    uses: ./.github/workflows/reusable-test.yml
    with:
      node-version: "20"

Concurrency (prevent duplicate runs)

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true  # Cancel previous runs for same branch

Path filters (only run for relevant changes)

on:
  push:
    paths:
      - "src/**"
      - "package.json"
      - "package-lock.json"
      - ".github/workflows/ci.yml"
    paths-ignore:
      - "docs/**"
      - "*.md"

Monorepo: only test changed packages

jobs:
  changes:
    runs-on: ubuntu-latest
    outputs:
      api: ${{ steps.filter.outputs.api }}
      web: ${{ steps.filter.outputs.web }}
    steps:
      - uses: actions/checkout@v4
      - uses: dorny/paths-filter@v3
        id: filter
        with:
          filters: |
            api:
              - 'packages/api/**'
            web:
              - 'packages/web/**'
test-api:
needs: changes
if: needs.changes.outputs.api == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: cd packages/api && npm ci && npm test

test-web:
needs: changes
if: needs.changes.outputs.web == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: cd packages/web && npm ci && npm test

Tips

Use
```
workflow_dispatch
```
on every workflow for manual triggering during debugging
Pin action versions to SHA for supply chain security:
```
uses: actions/checkout@b4ffde...
```
Use
```
continue-on-error: true
```
for non-critical steps (like linting)
Set
```
timeout-minutes
```
on jobs to prevent runaway builds (default is 360 minutes)

Use job outputs to pass data between jobs:

outputs: result: ${{ steps.step-id.outputs.value }}

For self-hosted runners:
```
runs-on: self-hosted
```
with labels for targeting specific machines

CI/CD Pipeline

AI Skill Market Insights

Be Part of the 0+ Developer Community

CI/CD Pipeline (GitHub Actions)

When to Use

Quick Start: Add CI to a Project

Node.js project

Python project

Go project

Rust project

Common Patterns

Matrix builds (test across versions/OSes)

Conditional jobs

Caching dependencies

Generic caching

Artifacts (save build outputs)

Download in another job

Run on schedule (cron)

Deployment Workflows

Deploy to production on tag

Deploy to multiple environments

Docker build and push

npm publish on release

Secrets Management

Set secrets via CLI

Set from a file

Set for a specific environment

List secrets

Delete a secret

Use secrets in workflows

Available to this step only

Environment protection rules

Workflow Debugging

Re-run failed jobs

View a specific run

View failed job logs

Re-run failed jobs only

Re-run entire workflow

Debug with SSH (using tmate)

Common failures and fixes

Workflow Validation

Validate locally before pushing

Use actionlint (if installed)

Or via Docker

View workflow as graph

View workflow definition

Watch a running workflow

Advanced Patterns

Reusable workflows

Concurrency (prevent duplicate runs)

Path filters (only run for relevant changes)

Monorepo: only test changed packages

Tips

Quick Start

Manual Installation

TEAR & SHARE

Tags

Channels

Learn

Compare

Company