Integrating JMeter API Client with CI/CD Pipelines

This guide explains how to run a saved LoadFocus JMeter test from CircleCI, Azure DevOps, GitHub Actions, GitLab CI/CD or Jenkins, so a performance test can pass or fail your deployment.

Table of Contents

• General Approach
• CircleCI Integration
• Azure DevOps Integration
• GitHub Actions Integration
• GitLab CI/CD Integration
• Jenkins Integration
• Best Practices
• Troubleshooting

General Approach

Regardless of the CI/CD platform, there are common steps to integrate the LoadFocus JMeter API Client:

1. Install the client: Install @loadfocus/loadfocus-api-client at the start of your workflow
2. Configure credentials: Save your API key and team ID from your platform's secret variables
3. Execute tests: Run a saved JMeter test by name, with thresholds
4. Process results: Read the output, or save it with --format json
5. Make decisions: run-test exits with code 1 when a threshold is missed or the run fails, so the pipeline step fails on its own

The three commands, with --name set to the exact name of a test you saved in LoadFocus:

npm install -g @loadfocus/loadfocus-api-client
loadfocus-api config set --apikey "$LOADFOCUS_API_KEY" --teamid "$LOADFOCUS_TEAM_ID"
loadfocus-api jmeter run-test --name "My_JMeter_Test" --thresholds "avgresponse<=200,errors==0"

Best Practices

When integrating LoadFocus JMeter API Client with your CI/CD pipeline, consider these best practices:

Security

• Never hardcode your API key or team ID in your pipeline configuration
• Use your CI/CD platform's secrets management system
• Restrict access to your performance test results to authorized personnel

Performance Test Configuration

• Use descriptive test names that include the branch/environment being tested
• Set appropriate thresholds based on your application's performance requirements
• Consider running different test profiles for different environments (e.g., lighter tests for dev, full tests for staging)

Pipeline Integration

• Run performance tests after functional tests have passed
• Consider running performance tests in parallel with other non-blocking tasks
• Set up notifications for performance test failures

Results Analysis

• Store performance test results as artifacts for historical comparison
• Consider implementing trend analysis to detect performance degradation over time
• Document baseline performance metrics for comparison

Troubleshooting

Common Issues

Test Execution Fails

• Verify your API key and team ID are correctly configured
• Check if you have reached your account's concurrent test limit
• Verify the test name exists in your LoadFocus account

Thresholds Not Working as Expected

• Ensure threshold syntax is correct: a comma-separated list of metric, operator and number, for example avgresponse<=200,errors==0. Metrics are avgresponse, errors, p95, p99, min, max and hitspersec (response times in milliseconds). Operators are <, <=, >, >=, == (or =) and !=
• In version 1.1.7 a threshold that cannot be read is skipped with a warning, and if none can be read the default avgresponse<=100 is used instead. Check the first lines of the output for "Invalid threshold"
• Check if the metrics you're evaluating are relevant to your test
• Verify that the test is generating enough load to produce meaningful results

Pipeline Times Out

• Adjust the timeout settings in your CI/CD configuration
• Use --wait-timeout <seconds> to set how long the client waits for the test to finish (default 1800). When it runs out, the command fails with exit code 1, but the test keeps running in LoadFocus
• --poll-interval <seconds> sets how often the status is checked (default 10)

For platform-specific integration details, see the dedicated guides for CircleCI, Azure DevOps, GitHub Actions, Jenkins, and GitLab CI/CD.