Security Group Testing: Validating AWS Security Groups, Azure NSGs, and GCP Firewall Rules

Master security group testing across AWS, Azure, and GCP with Checkov, InSpec, and Terraform. Learn automated validation of firewall rules, CI/CD integration, and compliance testing strategies.

TL;DR
Use Checkov in CI/CD to catch misconfigurations before deployment—CKV_AWS_24 (no SSH from 0.0.0.0/0) is non-negotiable
Test at multiple layers: static analysis (Checkov), integration tests (InSpec/Terratest), and runtime validation (AWS Config)
Cross-cloud testing requires understanding platform differences: AWS SGs are stateful, GCP firewall rules can be stateful or stateless
Best for: Teams managing cloud workloads who need to prevent unauthorized network access Skip if: You’re using a managed service where security groups are abstracted away Read time: 13 minutes

Security group misconfigurations remain one of the top causes of cloud breaches. A single rule allowing SSH from 0.0.0.0/0 or an overly permissive egress rule can expose your entire infrastructure. This guide covers testing strategies that catch these issues before they reach production.

For broader context on network testing, see Network Configuration Testing and Terraform Testing Strategies.

AI-Assisted Approaches

AI tools excel at generating comprehensive security group tests and identifying rule conflicts.

Generating Checkov custom policies:

Write a custom Checkov policy in Python that validates:

1. All security groups must have a description (not the default)
2. Inbound rules cannot use CIDR 0.0.0.0/0 except for ports 80 and 443
3. Security groups tagged as "internal" cannot have any public-facing rules
4. All egress rules must specify explicit destination (no 0.0.0.0/0)

Include the policy class, check method, and supported resource types.
Show how to register and run it with Checkov.

Analyzing security group rule conflicts:

Analyze these AWS security group rules for potential issues:

SG-web:

- Inbound: 443 from 0.0.0.0/0
- Inbound: 80 from 0.0.0.0/0
- Outbound: all to 0.0.0.0/0

SG-app:

- Inbound: 8080 from SG-web
- Inbound: 22 from 10.0.0.0/8
- Outbound: 443 to 0.0.0.0/0
- Outbound: 5432 to SG-db

SG-db:

- Inbound: 5432 from SG-app
- Outbound: none specified

Identify: Security risks, unnecessary rules, missing rules for return traffic,
and recommendations for least-privilege configuration.

Creating InSpec profiles for compliance:

Create an InSpec profile for CIS AWS Benchmark security group controls:

1. 5.1: Ensure no security groups allow ingress from 0.0.0.0/0 to port 22
2. 5.2: Ensure no security groups allow ingress from 0.0.0.0/0 to port 3389
3. 5.3: Ensure the default security group restricts all traffic
4. 5.4: Ensure VPC flow logging is enabled

Include proper control metadata, impact scores, and remediation guidance.

When to Use Different Testing Approaches

Testing Strategy Decision Framework

Test Type	Tool	When to Run	What It Catches
Static analysis	Checkov, tfsec	Pre-commit, CI	Obvious misconfigs (SSH from 0.0.0.0/0)
Policy validation	OPA, Sentinel	Pre-apply	Custom org rules, naming conventions
Integration testing	InSpec, Terratest	Post-apply	Actual rule state vs expected
Drift detection	AWS Config, Cloud Custodian	Continuous	Manual changes, unauthorized rules
Penetration testing	Manual/automated scans	Periodic	Real-world exploitability

Critical Security Group Checks

Check ID	Description	Risk Level
CKV_AWS_24	No inbound SSH from 0.0.0.0/0	Critical
CKV_AWS_25	No inbound RDP from 0.0.0.0/0	Critical
CKV_AWS_260	No inbound from 0.0.0.0/0 to any port	High
CKV_AWS_277	No overly permissive egress	Medium
CKV_AZURE_10	NSG restricts SSH from internet	Critical
CKV_GCP_3	Firewall rule not world-open	High

AWS Security Group Testing

Checkov for Terraform

# Install Checkov
pip install checkov

# Scan Terraform files
checkov -d ./terraform --framework terraform

# Scan with specific checks
checkov -d ./terraform --check CKV_AWS_24,CKV_AWS_25,CKV_AWS_260

# Output to JUnit for CI
checkov -d ./terraform --output junitxml > checkov-results.xml

Custom Checkov Policy

# custom_checks/security_group_description.py
from checkov.terraform.checks.resource.base_resource_check import BaseResourceCheck
from checkov.common.models.enums import CheckResult, CheckCategories

class SecurityGroupHasDescription(BaseResourceCheck):
    def __init__(self):
        name = "Ensure security group has a meaningful description"
        id = "CKV_CUSTOM_SG_1"
        supported_resources = ['aws_security_group']
        categories = [CheckCategories.NETWORKING]
        super().__init__(name=name, id=id, categories=categories,
                        supported_resources=supported_resources)

    def scan_resource_conf(self, conf):
        description = conf.get('description', [''])[0]

        # Fail if no description or default description
        if not description or description == "Managed by Terraform":
            return CheckResult.FAILED

        # Fail if description is too short
        if len(description) < 20:
            return CheckResult.FAILED

        return CheckResult.PASSED

check = SecurityGroupHasDescription()

InSpec Tests for AWS Security Groups

# controls/aws_security_groups.rb

title "AWS Security Group Compliance"

control 'sg-no-public-ssh' do
  impact 1.0
  title 'Security groups must not allow SSH from the internet'
  desc 'SSH access from 0.0.0.0/0 exposes instances to brute force attacks'

  aws_security_groups.group_ids.each do |sg_id|
    describe aws_security_group(group_id: sg_id) do
      it { should_not allow_in(port: 22, ipv4_range: '0.0.0.0/0') }
      it { should_not allow_in(port: 22, ipv6_range: '::/0') }
    end
  end
end

control 'sg-no-public-rdp' do
  impact 1.0
  title 'Security groups must not allow RDP from the internet'

  aws_security_groups.group_ids.each do |sg_id|
    describe aws_security_group(group_id: sg_id) do
      it { should_not allow_in(port: 3389, ipv4_range: '0.0.0.0/0') }
    end
  end
end

control 'sg-no-unrestricted-egress' do
  impact 0.7
  title 'Security groups should have restricted egress rules'

  aws_security_groups.group_ids.each do |sg_id|
    describe aws_security_group(group_id: sg_id) do
      # Allow 443 to 0.0.0.0/0 is acceptable, but not all ports
      it { should_not allow_out(port: 0, ipv4_range: '0.0.0.0/0') }
    end
  end
end

control 'default-sg-restricted' do
  impact 1.0
  title 'Default security group should restrict all traffic'

  aws_vpcs.vpc_ids.each do |vpc_id|
    default_sg = aws_security_groups.where(vpc_id: vpc_id, group_name: 'default')

    default_sg.group_ids.each do |sg_id|
      describe aws_security_group(group_id: sg_id) do
        its('inbound_rules.count') { should eq 0 }
        its('outbound_rules.count') { should eq 0 }
      end
    end
  end
end

Terratest Security Group Validation

package test

import (
    "testing"
    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/service/ec2"
    "github.com/gruntwork-io/terratest/modules/terraform"
    "github.com/stretchr/testify/assert"
    "github.com/stretchr/testify/require"
)

func TestSecurityGroupRules(t *testing.T) {
    t.Parallel()

    terraformOptions := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
        TerraformDir: "../modules/security-groups",
    })

    defer terraform.Destroy(t, terraformOptions)
    terraform.InitAndApply(t, terraformOptions)

    webSGID := terraform.Output(t, terraformOptions, "web_sg_id")
    appSGID := terraform.Output(t, terraformOptions, "app_sg_id")
    dbSGID := terraform.Output(t, terraformOptions, "db_sg_id")

    // Test web tier SG
    t.Run("WebTierSG", func(t *testing.T) {
        webSG := getSecurityGroup(t, webSGID)

        // Should allow HTTPS from internet
        assert.True(t, hasInboundRule(webSG, 443, "0.0.0.0/0"),
            "Web SG should allow HTTPS from internet")

        // Should NOT allow SSH from internet
        assert.False(t, hasInboundRule(webSG, 22, "0.0.0.0/0"),
            "Web SG should not allow SSH from internet")
    })

    // Test database tier SG
    t.Run("DatabaseTierSG", func(t *testing.T) {
        dbSG := getSecurityGroup(t, dbSGID)

        // Should NOT have any 0.0.0.0/0 inbound rules
        for _, rule := range dbSG.IpPermissions {
            for _, ipRange := range rule.IpRanges {
                assert.NotEqual(t, "0.0.0.0/0", *ipRange.CidrIp,
                    "Database SG should not allow public access")
            }
        }

        // Should only allow from app tier
        assert.True(t, hasSecurityGroupSource(dbSG, 5432, appSGID),
            "Database should only accept connections from app tier")
    })
}

func hasInboundRule(sg *ec2.SecurityGroup, port int64, cidr string) bool {
    for _, rule := range sg.IpPermissions {
        if *rule.FromPort == port && *rule.ToPort == port {
            for _, ipRange := range rule.IpRanges {
                if *ipRange.CidrIp == cidr {
                    return true
                }
            }
        }
    }
    return false
}

Azure NSG Testing

Checkov for Azure NSGs

# Azure-specific checks
checkov -d ./terraform --check CKV_AZURE_9,CKV_AZURE_10,CKV_AZURE_12

InSpec for Azure NSGs

control 'nsg-no-public-ssh' do
  impact 1.0
  title 'NSGs must not allow SSH from the internet'

  azure_network_security_groups.ids.each do |nsg_id|
    describe azure_network_security_group(resource_id: nsg_id) do
      it { should_not allow_ssh_from_internet }
    end
  end
end

control 'nsg-no-public-rdp' do
  impact 1.0
  title 'NSGs must not allow RDP from the internet'

  azure_network_security_groups.ids.each do |nsg_id|
    describe azure_network_security_group(resource_id: nsg_id) do
      it { should_not allow_rdp_from_internet }
    end
  end
end

GCP Firewall Testing

Checkov for GCP Firewalls

# GCP-specific checks
checkov -d ./terraform --check CKV_GCP_2,CKV_GCP_3,CKV_GCP_88

InSpec for GCP Firewalls

control 'gcp-firewall-no-public-ssh' do
  impact 1.0
  title 'GCP firewalls must not allow SSH from the internet'

  google_compute_firewalls(project: gcp_project_id).firewall_names.each do |fw|
    describe google_compute_firewall(project: gcp_project_id, name: fw) do
      its('source_ranges') { should_not include '0.0.0.0/0' }
    end
  end
end

CI/CD Integration

GitHub Actions Workflow

name: Security Group Testing

on:
  pull_request:
    paths:

      - 'terraform/**/*.tf'

jobs:
  checkov-scan:
    runs-on: ubuntu-latest
    steps:

      - uses: actions/checkout@v4

      - name: Run Checkov
        uses: bridgecrewio/checkov-action@v12
        with:
          directory: terraform/
          framework: terraform
          check: CKV_AWS_24,CKV_AWS_25,CKV_AWS_260,CKV_AWS_277
          output_format: sarif
          output_file_path: checkov-results.sarif
          soft_fail: false

      - name: Upload SARIF
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: checkov-results.sarif

  inspec-compliance:
    runs-on: ubuntu-latest
    needs: checkov-scan
    if: github.event.pull_request.draft == false
    steps:

      - uses: actions/checkout@v4

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1

      - name: Run InSpec
        run: |
          curl https://omnitruck.chef.io/install.sh | sudo bash -s -- -P inspec
          inspec exec compliance/security-groups \
            -t aws:// \
            --reporter cli json:inspec-results.json

      - name: Upload InSpec results
        uses: actions/upload-artifact@v4
        with:
          name: inspec-results
          path: inspec-results.json

Measuring Success

Metric	Before Testing	After Testing	How to Track
Public SSH/RDP rules	Found in audits	0 in production	Checkov reports
Security incidents from SG	2-3/year	0	Incident response logs
Compliance violations	5-10/audit	0-1/audit	Audit findings
Time to detect misconfig	Days/weeks	Minutes	CI/CD pipeline metrics

Warning signs your testing isn’t working:

Checkov passing but InSpec failing (drift between plan and reality)
Manual security group changes bypassing CI/CD
Exception requests becoming routine
Default security groups still have rules

Conclusion

Effective security group testing requires defense in depth:

Static analysis (Checkov) catches obvious misconfigurations in code
Policy validation (OPA/Sentinel) enforces organizational standards
Integration testing (InSpec/Terratest) verifies actual deployed state
Continuous monitoring (AWS Config) detects drift and unauthorized changes

The key insight: security groups are too important to test only once. Implement continuous validation that catches issues at every stage—from development to production.