Headless Testing

What Is Headless Testing? #

A headless browser is a web browser that operates without a graphical user interface. It has the complete browser engine — HTML parser, CSS engine, JavaScript runtime, networking stack — but it does not render pixels to a screen. When you run tests in headless mode, the browser performs all the same operations as a visible browser, but without the overhead of painting to a display.

Headless testing became mainstream when Chrome introduced headless mode in 2017, replacing older headless-only tools like PhantomJS. Today, all major browsers support headless operation: Chrome, Firefox, Edge (via Chromium), and WebKit (via Playwright).

Why Use Headless Mode? #

Speed #

Without rendering pixels, headless browsers execute faster. The browser still calculates layouts and processes CSS, but it skips the final painting step. Speed improvements vary by test type but are typically 20-40% for UI-heavy tests.

CI/CD Environments #

Most CI/CD servers (GitHub Actions runners, Jenkins agents, Docker containers) do not have a graphical display. Headless mode lets tests run in these environments without installing a virtual display server (Xvfb).

Resource Efficiency #

Headless browsers use less memory and CPU because they do not maintain a visible window, render fonts to screen, or handle window management events.

Configuring Headless Mode #

Playwright #

// Playwright defaults to headless
const browser = await chromium.launch(); // Headless by default

// Explicitly set headed for debugging
const browser = await chromium.launch({ headless: false });

// playwright.config.ts
export default defineConfig({
  use: {
    headless: true, // Default for CI
  },
});

Cypress #

// cypress.config.js — Cypress runs headed in interactive mode, headless in CI
module.exports = defineConfig({
  e2e: {
    // Run headless from CLI:
    // npx cypress run (headless by default)
    // npx cypress run --headed (force headed)
    // npx cypress open (interactive/headed)
  }
});

Selenium #

ChromeOptions options = new ChromeOptions();
options.addArguments("--headless=new"); // Chrome headless mode
options.addArguments("--no-sandbox");
options.addArguments("--disable-dev-shm-usage");
options.addArguments("--window-size=1920,1080");

WebDriver driver = new ChromeDriver(options);

# Python
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--headless=new")
options.add_argument("--window-size=1920,1080")
driver = webdriver.Chrome(options=options)

When to Use Headed vs Headless #

Limitations and Pitfalls #

Rendering Differences #

Headless and headed modes can produce different visual output in edge cases:

• Font rendering may differ slightly
• Certain CSS animations may behave differently
• Viewport handling can vary
• Some GPU-accelerated features may not render in headless

Recommendation: Run visual regression tests in headed mode (or in a consistent Docker environment with Xvfb) to ensure screenshots match production.

Missing Browser Extensions #

Headless mode typically does not load browser extensions. If your application depends on extensions (password managers, accessibility tools), those interactions cannot be tested headless.

Download and Print Dialogs #

System-level dialogs (file downloads, print dialogs) behave differently or are unavailable in headless mode. Use framework-specific APIs to handle these:

// Playwright — handle downloads in headless
const [download] = await Promise.all([
  page.waitForEvent('download'),
  page.click('#download-button')
]);
const path = await download.path();

Debugging Difficulty #

When a test fails in headless mode, you cannot see what happened. Mitigations:

• Capture screenshots on failure
• Record video of test execution
• Use Playwright’s trace viewer for detailed analysis
• Switch to headed mode to reproduce the issue

// Playwright trace on failure
use: {
  trace: 'on-first-retry', // Capture trace only when retrying a failed test
  screenshot: 'only-on-failure',
  video: 'retain-on-failure',
}

Docker and Headless Testing #

# Dockerfile for headless testing
FROM mcr.microsoft.com/playwright:v1.40.0-focal

WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .

CMD ["npx", "playwright", "test"]

# docker-compose.yml
services:
  tests:
    build: .
    environment:
      - CI=true
    volumes:
      - ./test-results:/app/test-results

Exercises #

Exercise 1: Headless Configuration #

1. Configure a test suite to run in both headless and headed modes
2. Measure execution time difference between the two modes
3. Compare screenshots taken in headless vs headed mode
4. Identify any visual differences between the two modes

Exercise 2: CI/CD Pipeline #

1. Create a GitHub Actions workflow that runs tests headless
2. Configure screenshot and video capture on failure
3. Store artifacts for post-run debugging
4. Add a manual trigger option to run in headed mode (via Xvfb)

Exercise 3: Debugging Headless Failures #

1. Create a test that passes headed but fails headless (viewport-dependent)
2. Use Playwright trace viewer to diagnose the failure
3. Fix the test to work in both modes
4. Document the root cause and prevention strategy