Pairwise Testing with PICT

Yuri Kan

Pairwise Testing with PICT

Master pairwise testing with Microsoft PICT — generate efficient test suites that cover all parameter pairs while dramatically reducing the number of test

Quick Answer

Pairwise Testing with PICT covers essential QA skills — after this lesson you can understand why pairwise testing catches most combination defects with far fewer tests.

— Yuri Kan, Senior QA Lead

What You Will Learn

Understand why pairwise testing catches most combination defects with far fewer tests
Use Microsoft PICT tool to generate pairwise test suites
Define constraints and sub-models in PICT for realistic test combinations

Table of Contents

What Is Pairwise Testing?

Pairwise testing (also called all-pairs testing) is a combinatorial test design technique based on a key observation: most defects are caused by interactions between two parameters, not three or more simultaneously.

Instead of testing every possible combination (which grows exponentially), pairwise testing guarantees that every pair of parameter values appears in at least one test case.

The Combinatorial Explosion Problem

Consider testing a web application across:

OS: Windows, macOS, Linux (3)
Browser: Chrome, Firefox, Safari, Edge (4)
Language: EN, ES, RU (3)
Screen: Desktop, Tablet, Mobile (3)

Exhaustive testing: 3 x 4 x 3 x 3 = 108 combinations

Pairwise testing: approximately 12-15 test cases — covering every pair.

How Pairwise Works

flowchart TD A[List all parameters and values] --> B[Generate all possible pairs] B --> C[Create test cases that cover multiple pairs each] C --> D[Optimize: minimize total test cases] D --> E[Result: compact test suite]

For 3 parameters (A, B, C) with 2 values each (1, 2):

All pairs that need coverage:

A-B: (1,1), (1,2), (2,1), (2,2)
A-C: (1,1), (1,2), (2,1), (2,2)
B-C: (1,1), (1,2), (2,1), (2,2)

A pairwise suite with just 4 test cases:

Test	A	B	C
1	1	1	1
2	1	2	2
3	2	1	2
4	2	2	1

Every pair appears at least once. Exhaustive would need 2^3 = 8 tests.

Introduction to PICT

PICT (Pairwise Independent Combinatorial Testing) is Microsoft’s free, open-source command-line tool for generating pairwise test suites.

Installation:

# macOS
brew install pict

# Linux (build from source)
git clone https://github.com/microsoft/pict.git
cd pict && make

# Windows
# Download from GitHub releases

Basic usage:

Create a model file web-test.txt:

OS:       Windows, macOS, Linux
Browser:  Chrome, Firefox, Safari, Edge
Language: EN, ES, RU
Screen:   Desktop, Tablet, Mobile

Run PICT:

pict web-test.txt

Output (example — actual combinations may vary):

OS       Browser  Language  Screen
Windows  Chrome   EN        Desktop
macOS    Firefox  ES        Tablet
Linux    Safari   RU        Mobile
Windows  Edge     RU        Tablet
macOS    Chrome   RU        Desktop
Linux    Firefox  EN        Desktop
...

PICT generates approximately 12-16 test cases instead of 108.

Why Pairwise Is Effective

Research by Kuhn, Wallace, and Gallo (NIST) found:

67% of defects are triggered by a single parameter value
93% are triggered by pairs of values (2-way interactions)
98% are triggered by triples (3-way)
Nearly 100% by 4-way interactions

Pairwise covers the 93% level with a fraction of exhaustive testing effort.

Advanced PICT Features

Adding Constraints

Not all combinations are valid. PICT supports constraints to exclude impossible cases:

OS:       Windows, macOS, Linux, iOS, Android
Browser:  Chrome, Firefox, Safari, Edge

# Constraints
IF [OS] = "iOS"    THEN [Browser] <> "Edge";
IF [OS] = "iOS"    THEN [Browser] <> "Firefox";
IF [OS] = "Android" THEN [Browser] <> "Safari";
IF [OS] = "Android" THEN [Browser] <> "Edge";
IF [OS] = "Linux"  THEN [Browser] <> "Safari";

PICT will never generate a test case with iOS + Edge or Android + Safari.

Sub-Models for Higher-Order Coverage

Sometimes certain parameter groups need stronger coverage. PICT supports sub-models:

OS:       Windows, macOS, Linux
Browser:  Chrome, Firefox, Safari
Language: EN, ES, RU
Theme:    Light, Dark

# Default: pairwise (2-way)
# But OS-Browser-Language needs 3-way coverage
{ OS, Browser, Language } @ 3

This ensures every triple of OS-Browser-Language values appears in at least one test, while other parameter interactions remain at pairwise level.

Seeding Specific Combinations

Force specific important combinations to appear:

Create a seed file seeds.txt:

OS        Browser  Language  Theme
Windows   Chrome   EN        Light
macOS     Safari   ES        Dark

Run with seeding:

pict model.txt /e:seeds.txt

PICT will include these exact combinations and optimize around them.

Weighted Parameters

Give priority to common configurations:

OS:       Windows(7), macOS(2), Linux(1)
Browser:  Chrome(6), Firefox(2), Safari(1), Edge(1)

Higher weights make those values appear in more test cases, increasing coverage for common configurations.

Negative Testing with PICT

Add invalid values prefixed with ~:

Age:    18, 30, 65, ~-1, ~200
Email:  valid@test.com, ~invalid-email, ~""

PICT ensures negative values appear in test cases but only one negative value per test case (single fault assumption).

Real-World Example: API Testing Matrix

Method:      GET, POST, PUT, DELETE
Auth:        Bearer, API-Key, None
ContentType: application/json, multipart/form-data, text/plain
Status:      200, 400, 401, 404, 500

IF [Method] = "GET"  THEN [ContentType] <> "multipart/form-data";
IF [Auth] = "None"   THEN [Status] IN { 401 };
IF [Method] = "DELETE" THEN [ContentType] = "application/json";

Exercise: Generate a Pairwise Suite with PICT

Scenario: You’re testing a payment processing form with:

Card type: Visa, MasterCard, Amex
Currency: USD, EUR, GBP
Amount range: Small (<$10), Medium ($10-$100), Large (>$100)
3D Secure: Enabled, Disabled
Recurring: Yes, No

Constraints:

Amex does not support recurring payments
GBP is not available with 3D Secure Disabled

Tasks:

Write the PICT model file
Calculate how many exhaustive tests would be needed
Estimate the pairwise count
Add constraints and verify the output

Hint

Exhaustive: 3 x 3 x 3 x 2 x 2 = 108 tests. PICT will generate roughly 12-18 tests with constraints.

The model file should define parameters on separate lines and constraints with IF/THEN syntax.

Solution

PICT model file (payment-test.txt):

CardType:  Visa, MasterCard, Amex
Currency:  USD, EUR, GBP
Amount:    Small, Medium, Large
3DSecure:  Enabled, Disabled
Recurring: Yes, No

IF [CardType] = "Amex" THEN [Recurring] = "No";
IF [Currency] = "GBP"  THEN [3DSecure] = "Enabled";

Exhaustive: 3 x 3 x 3 x 2 x 2 = 108 tests (minus impossible combinations)

Pairwise with PICT: approximately 12-15 test cases

Run: pict payment-test.txt

Example output:

CardType     Currency  Amount  3DSecure  Recurring
Visa         USD       Small   Enabled   Yes
MasterCard   EUR       Medium  Disabled  No
Amex         GBP       Large   Enabled   No
Visa         EUR       Large   Disabled  Yes
MasterCard   GBP       Small   Enabled   Yes
Amex         USD       Medium  Enabled   No
Visa         GBP       Medium  Enabled   No
MasterCard   USD       Large   Enabled   No
Amex         EUR       Small   Enabled   No
Visa         USD       Large   Enabled   No
MasterCard   EUR       Small   Enabled   Yes
Visa         EUR       Small   Disabled  No

~12 tests instead of 108 — an 89% reduction while covering all pairs.

Pro Tips

Start with pairwise, increase order if needed. 2-way (pairwise) catches ~93% of defects. If your domain is high-risk, bump critical parameter groups to 3-way.
Use constraints liberally. Every impossible combination removed is a wasted test avoided. Review your model with developers to catch all constraints.
Combine with other techniques. Use pairwise for the parameter combinations, then apply BVA to select specific values within each parameter.
Automate the pipeline. Integrate PICT into your CI: model file → PICT generation → parameterized test execution.
Version control your PICT models. Treat model files as test artifacts — review them, store them alongside tests, update them when parameters change.

Knowledge Check

1. What does pairwise (all-pairs) testing guarantee?

2. A system has 4 parameters with 3 values each. How many tests does exhaustive testing require vs. pairwise?

3. What is a constraint in PICT?

Frequently Asked Questions

What is pairwise testing with pict?

Pairwise Testing with PICT is a key concept in Test Design Techniques. This lesson teaches you to understand why pairwise testing catches most combination defects with far fewer tests, providing practical skills you can apply immediately in your testing work.

How do I apply pairwise testing with pict in real projects?

Start by practicing the core techniques covered in this lesson. Specifically, you should use Microsoft PICT tool to generate pairwise test suites. Apply these skills in your current project to see immediate results.

Why is pairwise testing with pict important for QA engineers?

Pairwise Testing with PICT is a core skill that employers look for in QA professionals. It directly impacts test coverage, defect detection, and team efficiency. Mastering it strengthens your Test Design Techniques capabilities and makes you more effective at delivering quality software.

What should I know before learning pairwise testing with pict?

You should have a basic understanding of software testing fundamentals. Familiarity with pairwise testing will help, but the lesson includes review sections for key prerequisites.

How does pairwise testing with pict help my QA career?

Knowledge of pairwise testing with pict is frequently listed in QA job descriptions and interview questions. It demonstrates expertise in pairwise testing, PICT tool and shows you can contribute to quality assurance at a professional level. Senior roles especially value this competency.

Pairwise Testing with PICT

What You Will Learn

What Is Pairwise Testing? #

The Combinatorial Explosion Problem #

How Pairwise Works #

Introduction to PICT #

Why Pairwise Is Effective #

Advanced PICT Features #

Adding Constraints #

Sub-Models for Higher-Order Coverage #

Seeding Specific Combinations #

Weighted Parameters #

Negative Testing with PICT #

Real-World Example: API Testing Matrix #

Exercise: Generate a Pairwise Suite with PICT #

Pro Tips #

Knowledge Check

Frequently Asked Questions

What Is Pairwise Testing?

The Combinatorial Explosion Problem

How Pairwise Works

Introduction to PICT

Why Pairwise Is Effective

Advanced PICT Features

Adding Constraints

Sub-Models for Higher-Order Coverage

Seeding Specific Combinations

Weighted Parameters

Negative Testing with PICT

Real-World Example: API Testing Matrix

Exercise: Generate a Pairwise Suite with PICT

Pro Tips