Event-Driven Architecture Overview
In event-driven architecture (EDA), services communicate by producing and consuming events rather than making direct API calls. When something happens in Service A (a user places an order), it publishes an event. Other services react to that event independently.
This approach enables loose coupling, high scalability, and resilience — but it introduces testing challenges that do not exist in synchronous systems: eventual consistency, event ordering, duplicate delivery, and complex failure modes.
Event Sourcing
Instead of storing the current state of an entity, event sourcing stores the sequence of events that led to that state.
Traditional approach (state-based):
Order #123: status=shipped, total=99.99, items=[A, B]
Event sourcing approach:
Event 1: OrderCreated { orderId: 123, items: [A, B] }
Event 2: PaymentReceived { orderId: 123, amount: 99.99 }
Event 3: OrderShipped { orderId: 123, trackingNumber: "ABC123" }
The current state is derived by replaying all events in order.
Testing Event Sourcing
Test 1: State reconstruction Given a sequence of events, verify that the aggregate’s current state is correct.
test('order state after creation and payment', () => {
const events = [
{ type: 'OrderCreated', data: { orderId: '123', items: ['A', 'B'], total: 99.99 } },
{ type: 'PaymentReceived', data: { orderId: '123', amount: 99.99 } },
];
const order = replayEvents(new Order(), events);
expect(order.status).toBe('paid');
expect(order.total).toBe(99.99);
expect(order.items).toEqual(['A', 'B']);
});
Test 2: Event ordering matters Applying events in a different order should produce different states (or be rejected).
Test 3: Snapshot consistency If the system uses snapshots for performance, verify that state from snapshot + subsequent events equals state from replaying all events.
CQRS (Command Query Responsibility Segregation)
CQRS separates the write model (commands) from the read model (queries). Commands produce events that update the write store. A projector processes events to update read-optimized views.
Command → Write Model → Events → Projector → Read Model
Testing CQRS
Test the command side:
- Sending a valid command produces the expected events.
- Sending an invalid command is rejected with the correct error.
- Commands enforce business rules (cannot ship an unpaid order).
Test the read side (projections):
- After events are processed, the read model reflects the correct state.
- Projections handle out-of-order events gracefully.
- Projection rebuilds produce the same result as incremental updates.
Test the consistency gap: After a command succeeds, there is a delay before the read model updates. Test that:
- The system handles this gap gracefully (no stale data errors).
- The read model eventually catches up.
Saga Pattern
Sagas coordinate distributed transactions across services. There are two types:
Choreography-Based Saga
Each service listens for events and decides what to do next. No central coordinator.
OrderService: OrderCreated →
PaymentService: (listens) → PaymentProcessed →
InventoryService: (listens) → StockReserved →
ShippingService: (listens) → ShipmentCreated
Testing choreography:
- Trigger the initial event and verify the entire chain completes.
- Simulate a failure at each step and verify compensating events are triggered.
- Test timeout scenarios — what happens if a service does not respond?
Orchestration-Based Saga
A central orchestrator coordinates the saga steps.
SagaOrchestrator:
Step 1: Command → PaymentService → Success/Failure
Step 2: Command → InventoryService → Success/Failure
Step 3: Command → ShippingService → Success/Failure
On Failure: Execute compensating commands in reverse
Testing orchestration:
- Happy path: All steps succeed → saga completes.
- Failure at each step: Verify compensating actions execute correctly.
- Orchestrator crash/restart: Verify the saga resumes from where it left off.
Eventual Consistency Testing
The hardest part of testing event-driven systems is verifying that services eventually reach a consistent state.
Polling Pattern
async function waitForConsistency(checkFn, timeout = 10000, interval = 500) {
const start = Date.now();
while (Date.now() - start < timeout) {
const result = await checkFn();
if (result) return result;
await new Promise(resolve => setTimeout(resolve, interval));
}
throw new Error('Consistency timeout exceeded');
}
// Usage
test('order appears in read model after creation', async () => {
await commandBus.send(new CreateOrderCommand({ items: ['A'] }));
const order = await waitForConsistency(async () => {
const result = await readModel.getOrder('123');
return result?.status === 'created' ? result : null;
});
expect(order.items).toEqual(['A']);
});
Idempotency Testing
Verify that processing the same event twice produces the same result:
test('payment event is idempotent', async () => {
const event = { type: 'PaymentReceived', orderId: '123', amount: 99.99 };
await processEvent(event);
await processEvent(event); // duplicate delivery
const payments = await db.getPayments('123');
expect(payments.length).toBe(1); // not 2
expect(payments[0].amount).toBe(99.99);
});
Event Schema Evolution
As systems evolve, event schemas change. Consumers must handle both old and new event formats.
Test scenarios:
- Consumer receives an event with a new field it does not know about — does it ignore it gracefully?
- Consumer receives an event missing a field that was added later — does it use a default value?
- Consumer receives an event with a renamed field — does the system handle the migration?
Exercise: Event-Driven System Testing
Design and execute tests for an event-driven e-commerce system.
System Description
The system has four services communicating via events:
OrderService → PaymentService → InventoryService → NotificationService
Events:
OrderCreated— published when a customer places an orderPaymentProcessed— published when payment is confirmedPaymentFailed— published when payment is declinedStockReserved— published when inventory is reservedStockInsufficient— published when inventory is unavailableOrderCancelled— compensating event when a step failsRefundInitiated— compensating event when payment needs reversal
Task 1: Happy Path Test
Write a test that verifies the complete flow:
- Publish
OrderCreatedwith order details. - Wait for
PaymentProcessedto appear. - Wait for
StockReservedto appear. - Verify the order status in the read model is “confirmed”.
- Verify inventory was decremented.
Task 2: Compensation Tests
For each failure point, verify the saga rolls back correctly:
Scenario A: Payment fails
- Publish
OrderCreated. - PaymentService publishes
PaymentFailed. - Verify
OrderCancelledis published. - Verify order status is “cancelled” in the read model.
Scenario B: Insufficient stock
- Publish
OrderCreated. - PaymentService publishes
PaymentProcessed. - InventoryService publishes
StockInsufficient. - Verify
RefundInitiatedis published (compensating for payment). - Verify
OrderCancelledis published. - Verify order status is “cancelled” and payment is refunded.
Task 3: Idempotency Test
- Publish the same
OrderCreatedevent twice (simulating duplicate delivery). - Verify only one order is created in the database.
- Verify only one payment attempt is made.
Task 4: Event Ordering Test
- Publish
PaymentProcessedbeforeOrderCreated(out of order). - Verify the system handles this gracefully — either buffering the payment event or rejecting it.
Task 5: Eventual Consistency Verification
- Create an order via the command endpoint.
- Immediately query the read model — it may not be updated yet.
- Poll the read model until the order appears (with a timeout).
- Record the consistency delay.
- Repeat 10 times and calculate average consistency delay.
Deliverables
- Test code for each scenario with assertions.
- A timing report showing eventual consistency delays.
- Documentation of any race conditions or edge cases discovered.