EventForge
An event arrives twice. What should happen next?
Overview
An educational exploration of order processing with a transactional outbox, a message broker, and idempotent consumers. The diagram below is an illustrative model, not a deployed system.
The problem
Writing an order to a database and publishing its event are two independent operations. A failure between them can leave downstream services unaware of a committed order.
The architecture
Follow the event.
One system. More than a happy path.
One transaction writes the order and its outbox event. Either both commit, or neither does.
Commit → publish → process. The order and event share a transaction; the consumer applies the state change.
Read the architecture as text
PostgreSQL atomically stores the order and outbox record. The outbox relay publishes to Kafka, partitioned by order ID. A consumer group processes events; each handler stores a durable event ID with the status update in a single transaction. Duplicate IDs are acknowledged without a second update. Transient failures retry with bounded backoff; exhausted retries go to a DLQ. Kafka provides at-least-once delivery in this design, not global ordering or end-to-end exactly-once effects.
My contribution
The learning goals and architecture are defined here. Implementation, deployment, and individual completed milestones remain unverified; this is deliberately presented as work in progress.
Decisions & trade-offs
- Write the order and outbox row in one PostgreSQL transaction. A relay publishes committed outbox entries to Kafka.
- Use an order identifier as the partition key to preserve order within a partition. Consumer groups distribute partitions across workers.
- Record a processed event ID atomically with the business update. Retry transient failures; isolate exhausted retries in a dead-letter queue.
At-least-once delivery simplifies recovery but permits duplicates. Idempotency needs durable state, and the outbox introduces cleanup and relay operations. Kafka ordering applies within a partition, not globally.
What the evidence says
No production results or benchmark numbers are claimed. The interactive model explains intended behavior; it does not verify a running Go/Kafka implementation.
The takeaway
A delivery guarantee is a contract with operational consequences. Make duplicate handling, retries, observability, and recovery part of the design from the beginning.