Azure DevOps

This guide shows how to start a LoadFocus JMeter test from an Azure DevOps pipeline, wait for it to finish and fail the pipeline when a threshold is missed. It uses the LoadFocus API Client (loadfocus-api, installed from npm). Create and save the test in LoadFocus first, see How to run a new JMeter load test; the pipeline starts that saved test by name, so --name must be the exact name of your saved test.

run-test starts the saved test, waits for it to finish and checks the thresholds. If any threshold is missed, or the run fails, the command exits with code 1 and the step fails. Thresholds are checked for every request label separately, and errors is a count of failed requests, not a percentage. In the examples below, replace My_JMeter_Test with the exact name of your saved test.

The output file starts with a few progress lines before the JSON object. To get a clean JSON file, keep only the lines from the first line that is just {: sed -n '/^{$/,$p' performance_results.json > results.json.

Setup Steps

1. Store Credentials in Azure Key Vault

For secure credential management, store your LoadFocus API credentials in Azure Key Vault:

  1. Create a Key Vault in Azure if you don't have one
  2. Add the following secrets:
    • loadfocus-api-key: Your LoadFocus API key
    • loadfocus-team-id: Your LoadFocus team ID
  3. Set up a service connection to access the Key Vault from your pipeline

2. Create an Azure Pipeline

Create a new file named azure-pipelines.yml in your repository:

trigger:
  - main
  - develop


pool:
  vmImage: 'ubuntu-latest'


variables:
  - group: loadfocus-variables  # Variable group containing Key Vault references


stages:
  - stage: Build
    jobs:
      - job: BuildAndTest
        steps:
          # Your existing build and test steps...


  - stage: PerformanceTest
    dependsOn: Build
    condition: succeeded()
    jobs:
      - job: RunPerformanceTests
        steps:
          - task: NodeTool@0
            inputs:
              versionSpec: '16.x'
            displayName: 'Install Node.js'


          - script: |
              npm install -g @loadfocus/loadfocus-api-client
            displayName: 'Install LoadFocus JMeter API Client'


          - script: |
              loadfocus-api config set --apikey $(LOADFOCUS_API_KEY) --teamid $(LOADFOCUS_TEAM_ID)
            displayName: 'Configure LoadFocus API Client'


          - script: |
              loadfocus-api jmeter run-test \
                --name "My_JMeter_Test" \
                --thresholds "avgresponse<=200,errors==0,p95<=250" \
                --format json > $(Build.ArtifactStagingDirectory)/performance_results.json
            displayName: 'Run Performance Tests'
            continueOnError: false


          - task: PublishBuildArtifacts@1
            inputs:
              pathtoPublish: '$(Build.ArtifactStagingDirectory)'
              artifactName: 'performance-test-results'
            displayName: 'Publish Performance Test Results'


  - stage: Deploy
    dependsOn: PerformanceTest
    condition: succeeded()
    jobs:
      - job: DeployApplication
        steps:
          # Your deployment steps...

3. Set Up Variable Group with Key Vault Integration

  1. Go to Pipelines > Library > Variable Groups
  2. Create a new Variable Group named "loadfocus-variables"
  3. Link it to your Azure Key Vault
  4. Add the following variables, linking them to your Key Vault secrets:
    • LOADFOCUS_API_KEY: Link to loadfocus-api-key secret
    • LOADFOCUS_TEAM_ID: Link to loadfocus-team-id secret

Advanced Configuration

Using YAML Templates

For reusable performance testing steps, create a template file performance-test-template.yml:

parameters:
  testName: 'My_JMeter_Test'
  thresholds: 'avgresponse<=200,errors==0,p95<=250'
  waitTimeout: 1800


steps:
- script: |
    loadfocus-api jmeter run-test \
      --name "${{ parameters.testName }}" \
      --thresholds "${{ parameters.thresholds }}" \
      --wait-timeout ${{ parameters.waitTimeout }} \
      --format json > $(Build.ArtifactStagingDirectory)/performance_results.json
  displayName: 'Run Performance Tests'
  continueOnError: false

Then in your main pipeline:

- template: performance-test-template.yml
  parameters:
    testName: 'My_JMeter_Test'
    thresholds: 'avgresponse<=150,errors==0,p95<=200'
    waitTimeout: 2400

Running Multiple Tests

To run multiple performance tests in sequence:

- script: |
    # Run API test
    loadfocus-api jmeter run-test \
      --name "API_Test" \
      --thresholds "avgresponse<=200,errors==0" \
      --format json > $(Build.ArtifactStagingDirectory)/api_test_results.json


    # Run UI test
    loadfocus-api jmeter run-test \
      --name "UI_Test" \
      --thresholds "avgresponse<=500,errors==0" \
      --format json > $(Build.ArtifactStagingDirectory)/ui_test_results.json
  displayName: 'Run Multiple Performance Tests'

Integration with Azure Test Plans

To integrate performance test results with Azure Test Plans:

- task: PublishTestResults@2
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: '$(Build.ArtifactStagingDirectory)/test-results.xml'
    testRunTitle: 'Performance Tests'
  displayName: 'Publish Test Results'
  condition: succeededOrFailed()

LoadFocus does not provide a JUnit converter, so you need to convert the JSON output to JUnit yourself with a script like the convert-to-junit.js below (strip the progress lines first, see the note at the top of this page):

- script: |
    # Run performance test
    loadfocus-api jmeter run-test \
      --name "My_JMeter_Test" \
      --thresholds "avgresponse<=200,errors==0" \
      --format json > $(Build.ArtifactStagingDirectory)/performance_results.json


    # Convert JSON to JUnit format (using a custom script)
    node convert-to-junit.js \
      $(Build.ArtifactStagingDirectory)/performance_results.json \
      $(Build.ArtifactStagingDirectory)/test-results.xml
  displayName: 'Run Performance Tests and Convert Results'

Tips for Azure DevOps Integration

  1. Parallel Jobs: If you have multiple performance tests, consider using parallel jobs. Your LoadFocus plan limits how many tests can run at the same time; above that limit the client reports "Maximum number of parallel tests exceeded for your plan.":

    jobs:
    - job: API_Performance_Test
      steps:
        # Run API performance test
    

    - job: UI_Performance_Test
      steps:
        # Run UI performance test

  2. Deployment Gates: Use performance test results as a deployment gate:

    - job: DeploymentGate
      steps:
      - script: |
          # Check if performance tests passed
          if grep -q '"overallResult": "FAILED"' $(Build.ArtifactStagingDirectory)/performance_results.json; then
            echo "##vso[task.logissue type=error]Performance tests failed"
            echo "##vso[task.complete result=Failed;]"
          fi

  3. Custom Dashboard: Create a custom dashboard to visualize performance test results over time.

  4. Notifications: Azure DevOps has no built-in email task for pipelines. Use the build-failure notifications in Project settings > Notifications, or add a task from an extension of your choice, to tell your team when the performance step fails.

For more information, refer to the Azure DevOps documentation and the LoadFocus API Client documentation.

Previous
Run JMeter Tests with the LoadFocus API Client
Next
CircleCI