TL;DR

  • Artillery = Node.js load testing with YAML scenarios
  • Define virtual users, phases (ramp-up, sustained load), think time
  • Built-in: HTTP, WebSocket, Socket.io support
  • Plugins: Custom protocols, metrics, reporters
  • CLI-first design, perfect for CI/CD pipelines

Best for: Node.js teams, modern web apps, developers wanting code-as-config Skip if: Need GUI test building or extensive protocol support (use JMeter) Reading time: 12 minutes

Your API needs load testing before release. You’re a Node.js team — JMeter’s XML feels alien. You want tests versioned in Git, running in CI/CD alongside your code.

Artillery speaks your language. YAML configs, npm installation, CLI execution. Tests that read like documentation, run like code.

What is Artillery?

Artillery is a modern load testing toolkit built for developers. It uses YAML for scenario definition and runs on Node.js, making it natural for JavaScript/TypeScript teams.

Why Artillery:

  • YAML-based — readable, version-controlled scenarios
  • Developer-friendly — npm install, CLI execution
  • Protocol support — HTTP, WebSocket, Socket.io built-in
  • Extensible — JavaScript functions, custom plugins
  • CI/CD native — JSON output, exit codes, GitHub Actions ready

Installation

npm install -g artillery

Verify installation:

artillery version

Your First Load Test

Basic YAML Scenario

# load-test.yml
config:
  target: "https://api.example.com"
  phases:
    - duration: 60
      arrivalRate: 10
      name: "Warm up"

scenarios:
  - name: "Get users"
    flow:
      - get:
          url: "/users"

Run the Test

artillery run load-test.yml

Output

Summary report @ 14:30:00(+0000)
  Scenarios launched:  600
  Scenarios completed: 600
  Requests completed:  600
  Mean response time:  45.2 ms
  95th percentile:     89 ms
  99th percentile:     142 ms
  Codes:
    200: 600

Load Phases

Ramp-Up Pattern

config:
  target: "https://api.example.com"
  phases:
    # Start slow
    - duration: 30
      arrivalRate: 5
      name: "Warm up"

    # Ramp to peak
    - duration: 60
      arrivalRate: 5
      rampTo: 50
      name: "Ramp up"

    # Sustained load
    - duration: 120
      arrivalRate: 50
      name: "Sustained load"

    # Cool down
    - duration: 30
      arrivalRate: 50
      rampTo: 5
      name: "Cool down"

Fixed User Count

config:
  phases:
    - duration: 60
      arrivalCount: 100  # Exactly 100 users total

User Scenarios

Sequential Requests

scenarios:
  - name: "User journey"
    flow:
      - get:
          url: "/products"
      - think: 2  # Wait 2 seconds
      - get:
          url: "/products/1"
      - post:
          url: "/cart"
          json:
            productId: 1
            quantity: 2

Capture and Reuse Data

scenarios:
  - name: "Login and use token"
    flow:
      - post:
          url: "/auth/login"
          json:
            username: "test"
            password: "secret"
          capture:
            - json: "$.token"
              as: "authToken"

      - get:
          url: "/profile"
          headers:
            Authorization: "Bearer {{ authToken }}"

Random Data

config:
  variables:
    productIds:
      - 1
      - 2
      - 3
      - 4
      - 5

scenarios:
  - name: "Random product view"
    flow:
      - get:
          url: "/products/{{ $randomNumber(1, 100) }}"
      - get:
          url: "/products/{{ productIds }}"  # Random from list

Assertions

Response Validation

scenarios:
  - name: "API health check"
    flow:
      - get:
          url: "/health"
          expect:
            - statusCode: 200
            - contentType: application/json
            - hasProperty: status
            - equals:
                - "{{ status }}"
                - "healthy"

Custom JavaScript

config:
  processor: "./helpers.js"

scenarios:
  - flow:
      - get:
          url: "/users"
          afterResponse: "validateUsers"

// helpers.js
module.exports = {
  validateUsers: function(req, res, ctx, events, done) {
    if (res.body.length < 1) {
      events.emit('error', 'No users returned');
    }
    return done();
  }
};

Plugins

Metrics to Datadog

npm install artillery-plugin-statsd

config:
  plugins:
    statsd:
      host: "localhost"
      port: 8125
      prefix: "artillery."

Expect Plugin (Advanced Assertions)

npm install artillery-plugin-expect

config:
  plugins:
    expect: {}

scenarios:
  - flow:
      - get:
          url: "/users"
          expect:
            - statusCode: 200
            - hasHeader: "x-request-id"
            - matchesRegexp:
                - body
                - "users"

Environment Configuration

Using Variables

config:
  target: "{{ $processEnvironment.API_URL }}"
  variables:
    apiKey: "{{ $processEnvironment.API_KEY }}"

scenarios:
  - flow:
      - get:
          url: "/data"
          headers:
            X-API-Key: "{{ apiKey }}"

Multiple Environments

# test.yml
config:
  environments:
    dev:
      target: "https://dev.api.example.com"
    staging:
      target: "https://staging.api.example.com"
    prod:
      target: "https://api.example.com"

artillery run test.yml -e staging

Reports

JSON Report

artillery run load-test.yml --output results.json

HTML Report

artillery run load-test.yml --output results.json
artillery report results.json --output report.html

CI/CD Integration

GitHub Actions

name: Load Tests

on:
  schedule:
    - cron: '0 2 * * *'  # Daily at 2 AM

jobs:
  load-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install Artillery
        run: npm install -g artillery

      - name: Run load test
        run: artillery run tests/load-test.yml --output results.json
        env:
          API_URL: ${{ secrets.API_URL }}

      - name: Generate report
        run: artillery report results.json --output report.html

      - name: Upload results
        uses: actions/upload-artifact@v4
        with:
          name: load-test-results
          path: |
            results.json
            report.html

Exit Codes for CI

config:
  ensure:
    p95: 500      # 95th percentile < 500ms
    maxErrorRate: 1  # < 1% errors

Artillery exits with code 1 if thresholds are exceeded — failing the CI job.

Artillery with AI Assistance

AI tools can help write and optimize Artillery scenarios.

What AI does well:

  • Generate YAML scenarios from API specs
  • Create realistic user journeys
  • Suggest load phase configurations
  • Convert other formats to Artillery YAML

What still needs humans:

  • Determining realistic load levels
  • Interpreting results in business context
  • Infrastructure capacity decisions
  • Production vs test environment differences

FAQ

What is Artillery?

Artillery is a modern Node.js load testing tool. You write test scenarios in YAML, defining virtual users and their behavior. It supports HTTP, WebSocket, and Socket.io out of the box, with plugins for custom protocols. Tests run from CLI, output JSON, and integrate easily with CI/CD pipelines.

Artillery vs JMeter — which is better?

Artillery uses YAML configuration (developer-friendly, version-controllable). JMeter uses GUI (easier for non-programmers). Artillery is lighter, faster to set up, and natural for Node.js teams. JMeter has more protocol support, enterprise features, and a larger ecosystem. Choose Artillery for modern web apps, JMeter for complex enterprise needs.

Is Artillery free?

Artillery Core (the CLI tool) is free and open-source under MPL-2.0 license. It handles most load testing needs. Artillery Pro is a paid service adding cloud-based execution, advanced analytics, team collaboration, and support. Most teams start with Core and upgrade if they need cloud scale.

Can Artillery test WebSockets?

Yes, WebSocket support is built-in. Define connection, emit, and expect steps in YAML just like HTTP requests. Test real-time applications, chat systems, multiplayer games, and Socket.io backends. The same scenario can mix HTTP and WebSocket steps for realistic user journeys.

Official Resources

See Also