When Software Can Harm People
Most software bugs are annoying — a broken button, a slow page, a formatting error. But in some industries, software bugs can injure or kill people, cause financial fraud, or compromise national security.
In these regulated industries, governments and industry bodies mandate specific standards that software must comply with. Testing in these environments goes far beyond typical QA — it requires validation, formal documentation, audit trails, and regulatory approval before software can be used.
This lesson covers the major regulated industries, their standards, and what additional testing requirements they impose.
Healthcare and Medical Devices
Key Standards
| Standard | Scope | Issuing Body |
|---|---|---|
| FDA 21 CFR Part 11 | Electronic records and signatures | US FDA |
| IEC 62304 | Medical device software lifecycle | International |
| ISO 14971 | Risk management for medical devices | International |
| HIPAA | Health data privacy and security | US HHS |
| EU MDR | Medical device regulation | European Union |
Testing Requirements
Software Validation (FDA): The FDA requires that medical device software be “validated” — meaning there is documented evidence that the software consistently produces results meeting predetermined specifications.
Validation is not the same as testing. Validation includes:
- Installation Qualification (IQ) — is the software installed correctly?
- Operational Qualification (OQ) — does the software work as specified?
- Performance Qualification (PQ) — does the software perform correctly in the real environment?
Risk-Based Testing (ISO 14971): Every function must be assessed for risk to patient safety. Higher-risk functions require more thorough testing.
Audit Trails: Every action in a medical system must be logged — who accessed patient data, when, and what changes were made. These logs must be tamper-evident and retained for the system’s lifecycle.
Traceability: Full traceability from requirements to design to test cases to test results to defects. This is not optional — auditors will verify it.
Finance
Key Standards
| Standard | Scope | Issuing Body |
|---|---|---|
| PCI-DSS | Payment card data security | PCI Security Council |
| SOX (Sarbanes-Oxley) | Financial reporting integrity | US Congress |
| Basel III/IV | Banking risk management | Basel Committee |
| GDPR/CCPA | Data privacy (affects financial data) | EU/US States |
| MiFID II | Financial markets regulation | European Union |
Testing Requirements
PCI-DSS Compliance Testing: Any system that stores, processes, or transmits credit card data must comply with PCI-DSS. This requires:
- Regular vulnerability scanning (quarterly, by an Approved Scanning Vendor)
- Annual penetration testing
- Secure code review
- Network segmentation testing
- Access control verification
SOX Testing: SOX requires that financial systems maintain data integrity. Testing must verify:
- That financial calculations are accurate
- That data cannot be altered without authorization and audit trail
- That access controls prevent unauthorized modifications
- That backup and recovery procedures work correctly
Fraud Detection Testing: Financial systems must be tested for:
- Transaction monitoring accuracy
- False positive/negative rates in fraud detection
- System behavior under unusual patterns
Automotive
Key Standards
| Standard | Scope | Issuing Body |
|---|---|---|
| ISO 26262 | Functional safety for road vehicles | ISO |
| ASPICE | Automotive software process improvement | VDA |
| MISRA | Coding standards for automotive software | MISRA Consortium |
| SOTIF (ISO 21448) | Safety of intended functionality | ISO |
Testing Requirements
ASIL Levels (ISO 26262): Automotive Safety Integrity Levels range from A (lowest risk) to D (highest risk). The ASIL level determines the rigor of testing required:
| ASIL Level | Example | Testing Requirements |
|---|---|---|
| A | Interior lighting control | Basic functional testing |
| B | Headlight systems | Systematic testing, requirements coverage |
| C | Cruise control | Requirements-based testing, fault injection |
| D | Braking system, steering | Exhaustive testing, formal methods, independent verification |
Hardware-in-the-Loop (HIL) Testing: Automotive software is tested using simulated hardware environments before deploying to real vehicles. This allows testing of scenarios that would be dangerous in real conditions.
Model-Based Testing: Many automotive systems use model-based development (e.g., Simulink). Testing verifies both the model and the generated code.
Aviation
Key Standards
| Standard | Scope | Issuing Body |
|---|---|---|
| DO-178C | Airborne software | RTCA/EUROCAE |
| DO-254 | Airborne electronic hardware | RTCA |
| ARP 4754A | Development of civil aircraft | SAE |
Testing Requirements
Design Assurance Levels (DO-178C): Similar to ASIL in automotive, aviation uses DAL levels A through E:
| DAL | Failure Condition | Example |
|---|---|---|
| A | Catastrophic | Flight control software |
| B | Hazardous | Engine monitoring |
| C | Major | Navigation display |
| D | Minor | Cabin entertainment |
| E | No effect | Maintenance logging |
Structural Coverage (DO-178C): DAL A software requires Modified Condition/Decision Coverage (MC/DC) — the most rigorous form of code coverage analysis. Every condition in every decision must be shown to independently affect the outcome.
Independence: For DAL A and B, the verification activities must be performed independently from the development activities. The person who wrote the code cannot be the person who tests it.
Pharmaceutical
Key Standards
| Standard | Scope | Issuing Body |
|---|---|---|
| GAMP 5 | GxP computerized systems | ISPE |
| 21 CFR Part 11 | Electronic records | US FDA |
| EU Annex 11 | Computerized systems | EMA |
| CSV (Computer System Validation) | Validation approach | Industry practice |
GAMP 5 Software Categories
| Category | Type | Validation Effort | Example |
|---|---|---|---|
| 1 | Infrastructure | Minimal | Operating systems, databases |
| 3 | Non-configured products | Low | Off-the-shelf commercial software |
| 4 | Configured products | Medium | ERP systems, LIMS |
| 5 | Custom applications | High | Custom-developed lab systems |
Higher categories require more extensive validation, including documented evidence that the software performs its intended function correctly and consistently.
Common Requirements Across Regulated Industries
Despite different standards, regulated industries share common testing requirements:
1. Validation vs Verification
| Concept | Question It Answers | Example |
|---|---|---|
| Verification | “Did we build the product right?” | Does the code match the specification? |
| Validation | “Did we build the right product?” | Does the product meet the user’s actual needs? |
Both are required in regulated environments, and both must be documented.
2. Documentation Requirements
Regulated testing produces significantly more documentation than typical testing:
- Validation Plan
- Validation Protocol (test cases with expected results)
- Validation Report (actual results, deviations, conclusions)
- Traceability Matrix
- Risk Assessment
- Change Control Records
3. Change Control
Any change to validated software must go through a formal change control process:
- Change request documented
- Impact analysis performed (what tests need to be re-run?)
- Change approved by authorized personnel
- Change implemented and tested
- Regression testing performed
- Updated documentation reviewed and approved
4. Data Integrity (ALCOA+)
Regulated data must comply with ALCOA+ principles:
| Principle | Meaning |
|---|---|
| Attributable | Who generated the data? |
| Legible | Can the data be read? |
| Contemporaneous | Was it recorded when it happened? |
| Original | Is it the original record? |
| Accurate | Is it correct? |
| +Complete | Is all data included? |
| +Consistent | Are there contradictions? |
| +Enduring | Will it be available throughout the retention period? |
| +Available | Can it be accessed when needed? |
Compliance Testing Basics
Compliance testing verifies that the software meets regulatory requirements. It includes:
- Regulatory requirement mapping — Map each regulatory requirement to specific test cases
- Evidence collection — Document all test results as evidence for auditors
- Gap analysis — Identify areas where the software does not yet meet requirements
- Remediation testing — Re-test after fixes to verify compliance
- Periodic re-validation — Regulations may require periodic re-validation (e.g., annually)
Exercise: Identify Compliance Requirements
Scenario: You are the QA Lead for a healthcare startup building a telemedicine app. The app allows patients to:
- Video call with doctors
- View and download medical records (lab results, prescriptions)
- Pay for consultations via credit card
- Receive medication reminders via push notification
The app will launch in the US market first, then expand to the EU.
Tasks:
- Which regulatory standards apply to this app? List all that apply and explain why.
- For each standard, identify the top 3 testing requirements it imposes.
- Create a compliance testing checklist (minimum 15 items) organized by regulation.
- What would be the consequences of non-compliance for each standard?
Hint
Think about what data the app handles:
- Medical records → HIPAA, FDA (if classified as medical device)
- Credit card data → PCI-DSS
- EU expansion → GDPR
- Medication reminders → Could be classified as a medical device by FDA
Also consider:
- Video calls may store recordings → data retention requirements
- The app stores PHI (Protected Health Information) → encryption requirements
- EU launch requires GDPR compliance from day one
Solution
1. Applicable Regulatory Standards
| Standard | Why It Applies |
|---|---|
| HIPAA | App handles Protected Health Information (medical records, prescriptions) |
| FDA 21 CFR Part 11 | Electronic medical records and e-prescriptions require compliant electronic signatures and records |
| FDA SaMD guidance | The app may be classified as Software as a Medical Device if it aids clinical decisions |
| PCI-DSS | App processes credit card payments |
| GDPR | EU expansion means processing EU citizens’ personal and health data |
| CCPA | California users have specific data rights |
| ADA/Section 508 | Healthcare apps must be accessible |
2. Top 3 Testing Requirements per Standard
HIPAA:
- Encryption testing — PHI must be encrypted at rest and in transit
- Access control testing — role-based access, minimum necessary standard
- Audit trail testing — all access to PHI must be logged
PCI-DSS:
- Penetration testing — annual testing of payment processing components
- Vulnerability scanning — quarterly scans by approved vendor
- Data storage testing — verify credit card numbers are not stored unencrypted
GDPR:
- Consent management testing — verify consent collection, withdrawal, and record-keeping
- Data subject rights testing — right to access, deletion, portability
- Data breach notification — verify breach detection and 72-hour notification capability
3. Compliance Testing Checklist
HIPAA Compliance:
- PHI encrypted at rest (AES-256 or equivalent)
- PHI encrypted in transit (TLS 1.2+)
- Role-based access control enforced
- Audit trail captures all PHI access (who, when, what)
- Session timeout after inactivity
- Business Associate Agreements in place for third-party services
PCI-DSS Compliance:
- Credit card data never stored in plain text
- Payment processing uses tokenization
- Quarterly vulnerability scans pass
- Annual penetration test completed
- Network segmentation between payment and other systems
GDPR Compliance:
- Explicit consent obtained before processing health data
- Users can access all their personal data
- Users can request data deletion
- Data Processing Impact Assessment completed
- Privacy policy clearly describes data processing
Accessibility:
- WCAG 2.1 AA compliance verified
- Screen reader compatibility tested
4. Consequences of Non-Compliance
| Standard | Consequence |
|---|---|
| HIPAA | Fines up to $1.5M per violation category per year, criminal penalties |
| PCI-DSS | Fines $5K-100K/month, loss of ability to process credit cards |
| GDPR | Fines up to 4% of annual global turnover or EUR 20M |
| FDA | Warning letters, product recalls, injunctions, criminal prosecution |
Key Takeaways
- Regulated industries require testing that goes far beyond typical QA — including validation, formal documentation, and audit trails
- Key regulated industries include healthcare (FDA/HIPAA), finance (PCI-DSS/SOX), automotive (ISO 26262), aviation (DO-178C), and pharma (GAMP 5)
- All regulated industries share common requirements: traceability, change control, data integrity, and independent verification
- Non-compliance can result in severe consequences including fines, recalls, and criminal prosecution
- The level of testing rigor is proportional to the risk — higher risk to life or safety requires more thorough testing