GitHub Actions

This guide shows how to start a LoadFocus JMeter test from a GitHub Actions workflow, wait for it to finish and fail the workflow when a threshold is missed. It uses the LoadFocus API Client (loadfocus-api, installed from npm). Create and save the test in LoadFocus first, see How to run a new JMeter load test; the workflow starts that saved test by name.

Setup Steps

1. Store Credentials as GitHub Secrets

First, store your LoadFocus API credentials as GitHub repository secrets:

1. Go to your GitHub repository

2. Navigate to Settings > Secrets and variables > Actions

3. Add the following repository secrets:

   • LOADFOCUS_API_KEY: Your LoadFocus API key
   • LOADFOCUS_TEAM_ID: Your LoadFocus team ID

   Both are on the API Keys page of your LoadFocus account settings.

2. Create a GitHub Actions Workflow

Create a new file in your repository at .github/workflows/performance-test.yml:

name: Performance Tests

on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main ]
  # Optional: Run on a schedule
  schedule:
    - cron: '0 0 * * 1'  # Run at midnight every Monday

jobs:
  performance-test:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v3

    - name: Setup Node.js
      uses: actions/setup-node@v3
      with:
        node-version: '16'

    - name: Install LoadFocus JMeter API Client
      run: npm install -g @loadfocus/loadfocus-api-client

    - name: Configure LoadFocus API Client
      run: |
        loadfocus-api config set --apikey ${{ secrets.LOADFOCUS_API_KEY }} --teamid ${{ secrets.LOADFOCUS_TEAM_ID }}

    - name: Run Performance Tests
      run: |
        # --name is the exact name of a test you saved in LoadFocus
        loadfocus-api jmeter run-test \
          --name "My_JMeter_Test" \
          --thresholds "avgresponse<=200,errors==0,p95<=250" \
          --format json > performance_results.json

    - name: Upload Performance Test Results
      if: always()
      uses: actions/upload-artifact@v4
      with:
        name: performance-test-results
        path: performance_results.json

run-test starts the saved test, waits for it to finish and checks the thresholds. If any threshold is missed, or the run fails, the command exits with code 1 and the step fails. Thresholds are checked for every request label separately, so one slow request is enough to fail the run. errors is a count of failed requests, not a percentage. In the examples below, replace the test names with the exact name of your saved test.

The output file starts with a few progress lines before the JSON object. To get a clean JSON file, keep only the lines from the first line that is just {: sed -n '/^{$/,$p' performance_results.json > results.json.

3. Add Performance Testing to Your Deployment Workflow

To make deployment dependent on performance test results:

name: Build, Test, and Deploy

on:
  push:
    branches: [ main ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      # Your build steps...

  performance-test:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '16'

      - name: Install LoadFocus JMeter API Client
        run: npm install -g @loadfocus/loadfocus-api-client

      - name: Configure LoadFocus API Client
        run: |
          loadfocus-api config set --apikey ${{ secrets.LOADFOCUS_API_KEY }} --teamid ${{ secrets.LOADFOCUS_TEAM_ID }}

      - name: Run Performance Tests
        run: |
          loadfocus-api jmeter run-test \
            --name "My_JMeter_Test" \
            --thresholds "avgresponse<=200,errors==0,p95<=250" \
            --format json > performance_results.json

      - name: Upload Performance Test Results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: performance-test-results
          path: performance_results.json

  deploy:
    needs: performance-test
    runs-on: ubuntu-latest
    steps:
      # Your deployment steps...

Advanced Configuration

Matrix Testing for Multiple Environments

Run tests against multiple environments or configurations:

jobs:
  performance-test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        environment: [dev, staging, production]
        test-type: [api, frontend]

    steps:
      # Setup steps...

      - name: Run Performance Tests
        run: |
          TEST_NAME="${{ matrix.test-type }}_test_${{ matrix.environment }}"

          # Adjust thresholds based on environment
          if [ "${{ matrix.environment }}" == "production" ]; then
            THRESHOLDS="avgresponse<=150,errors==0,p95<=200"
          else
            THRESHOLDS="avgresponse<=300,errors==0,p95<=500"
          fi

          loadfocus-api jmeter run-test \
            --name "$TEST_NAME" \
            --thresholds "$THRESHOLDS" \
            --format json > "performance_results_${{ matrix.environment }}_${{ matrix.test-type }}.json"

Commenting Test Results on PRs

Add performance test results as a comment on pull requests:

- name: Comment PR
  uses: actions/github-script@v6
  if: always() && github.event_name == 'pull_request'
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    script: |
      const fs = require('fs');
      const raw = fs.readFileSync('performance_results.json', 'utf8');
      // the file starts with progress lines; the JSON begins at the first line that is only {
      const results = JSON.parse(raw.slice(raw.search(/^\{$/m)));

      let comment = '## Performance Test Results\n\n';
      comment += `**Overall Result:** ${results.overallResult}\n\n`;

      comment += '### Results by Label\n\n';
      for (const label of results.labels) {
        comment += `#### ${label.label}\n`;
        comment += `- **Result:** ${label.result}\n`;
        comment += `- **Samples:** ${label.metrics.samples}\n`;
        comment += `- **Avg Response:** ${label.metrics.avgresponse}ms\n`;
        comment += `- **Error Rate:** ${label.metrics.errors}\n\n`;
      }

      github.rest.issues.createComment({
        issue_number: context.issue.number,
        owner: context.repo.owner,
        repo: context.repo.repo,
        body: comment
      });

Reusable Workflow

Create a reusable workflow for performance testing:

.github/workflows/reusable-performance-test.yml

name: Reusable Performance Test

on:
  workflow_call:
    inputs:
      test-name:
        required: true
        type: string
      thresholds:
        required: false
        type: string
        default: "avgresponse<=200,errors==0,p95<=250"
      wait-timeout:
        required: false
        type: number
        default: 1800
    secrets:
      LOADFOCUS_API_KEY:
        required: true
      LOADFOCUS_TEAM_ID:
        required: true

jobs:
  performance-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '16'

      - name: Install LoadFocus JMeter API Client
        run: npm install -g @loadfocus/loadfocus-api-client

      - name: Configure LoadFocus API Client
        run: |
          loadfocus-api config set --apikey ${{ secrets.LOADFOCUS_API_KEY }} --teamid ${{ secrets.LOADFOCUS_TEAM_ID }}

      - name: Run Performance Tests
        run: |
          loadfocus-api jmeter run-test \
            --name "${{ inputs.test-name }}" \
            --thresholds "${{ inputs.thresholds }}" \
            --wait-timeout ${{ inputs.wait-timeout }} \
            --format json > performance_results.json

      - name: Upload Performance Test Results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: performance-test-results
          path: performance_results.json

Then call it from another workflow:

.github/workflows/main.yml

jobs:
  call-performance-test:
    uses: ./.github/workflows/reusable-performance-test.yml
    with:
      test-name: "API_Performance_Test"
      thresholds: "avgresponse<=150,errors==0"
    secrets:
      LOADFOCUS_API_KEY: ${{ secrets.LOADFOCUS_API_KEY }}
      LOADFOCUS_TEAM_ID: ${{ secrets.LOADFOCUS_TEAM_ID }}

Tips for GitHub Actions Integration

1. Caching: Cache npm dependencies to speed up workflow runs:

   - name: Cache Node modules
     uses: actions/cache@v3
     with:
       path: ~/.npm
       key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
       restore-keys: |
         ${{ runner.os }}-node-

2. Concurrency Control: Limit concurrent performance tests:

   concurrency:
     group: performance-test-${{ github.ref }}
     cancel-in-progress: false

3. Environment-specific Tests: Use GitHub environments for different test configurations:

   jobs:
     performance-test:
       runs-on: ubuntu-latest
       environment: staging
       # Environment-specific variables are available here

4. Conditional Testing: Only run performance tests when specific files change:

   jobs:
     performance-test:
       if: |
         contains(github.event.pull_request.labels.*.name, 'performance-test') ||
         github.event_name == 'schedule' ||
         contains(github.event.head_commit.message, '[perf-test]')

For more information, refer to the GitHub Actions documentation and the LoadFocus API Client documentation.