I Was Blind, but Now I See. Event Sourcing and Observability

Event sourcing is an architectural pattern in which every change to an application’s state is recorded as a sequence of events (Fowler, 2005). Event sourcing is a companion architecture to domain-driven design, with events related to aggregate root entities within the domain. In domain-driven design, we model the domain by creating a ubiquitous language that captures its core concepts. Those objects are subject to certain actions, and it’s natural to think of these actions as events that have occurred. As an example, if you have an aggregate root “Order”, when the order is created, you may emit an event “order.created”. When the credit for the order is approved, you may have “order.approved”. When the order is shipped, “order.shipped” would be emitted and so on. What you end up with is an append-only log of all the things that are happening in the system.

flowchart LR
    subgraph Command Side
        A[User Action] --> B[Application]
        B --> C[Publish Event]
    end

    subgraph Event Store
        C --> D[(Append-Only Log)]
        D --> E[order.created]
        D --> F[order.approved]
        D --> G[order.shipped]
        D --> H[order.delivered]
    end

    subgraph Read Side / Projections
        D --> I[Event Bus / Broker]
        I --> J[Read Model / Snapshot]
        I --> K[Activity Feed]
        I --> L[Process Mining]
        I --> M[Analytics]
    end

In practice, these events are typically stored in an event store, which is a specialized database or a dedicated table in a relational database designed to persist events in sequence. Each event includes details such as a timestamp, type, payload, and a reference to the aggregate it affects. When a change occurs, the application code publishes a new event to the event store within a transaction. Other parts of the system can react to these events by subscribing to them, for example, using a message broker or an event bus to notify interested components that something has happened. This allows for decoupled processing and integration with other services.

This append-only log is useful in many ways. One of the most common uses I have for those events is troubleshooting. In non-event-sourced systems, the states are updated in line with the record in the database. Want to know if the order was approved before it was shipped? You have to assume that the code enforced that variant (you know what they say about assuming), or check the application logs, which may or may not have the details you’re looking for. With event sourcing, you can examine all the events related to an entity to see how it changed over time. Similar to how git log helps understand how code changes over time, the events related to an aggregate root provide similar clarity.

Not only can you see the history of an entity, but you can also get a snapshot of an object’s state at a specific time. These snapshots can be created routinely to reduce the need to constantly replay millions of events, or on an as-needed basis. This is done by replaying the events up to a certain time and reducing them to obtain the aggregate root’s state at that time. This state could be projected onto a data store to make quick querying more feasible in an event-sourced system (querying the raw event store isn’t very practical). The ability to retrieve an entity’s state at a specific time can be useful for debugging. It is also useful when facilitating object versioning and rollback functionality. Rolling back an entity is simply reducing the events to the point in time you want to revert to, giving you a time-machine experience for your entities.

In addition to debugging, events can provide insights into who is using an application and how they use it. The most basic insights can be gained by exposing events in an activity feed that shows who did what in the application’s user interface (e.g., “Jane just approved Order ABC”).

Events can also be a source of high-quality data for process mining. Process mining aims to discover, monitor, and improve real business processes by extracting knowledge from event logs available in information systems (e.g., ERP, CRM). Rather than rely on “assumed” workflows, process mining uses actual data to reconstruct what is really happening. Process mining sits at the intersection of data science and process science and typically relies on data science techniques to gather data. (What is Process Mining?, 2021) With applications built on event sourcing, you would rely less on ETL techniques, as you would have access to high-fidelity data to support process analysis.

To give a concrete example, consider an e-commerce company using event sourcing for its order fulfillment process. Every time an order is placed, approved, packaged, shipped, and delivered, a corresponding event is appended to the event store. When the company wants to analyze its order delivery process, process mining tools can directly access these events to map out the actual flows and timelines of orders. This approach can reveal steps that often cause delays, such as repeated packaging attempts or shipping bottlenecks. By having this granular, reliable event data at hand, the company can identify unexpected inefficiencies and focus its efforts on the areas most in need of improvement, leading to faster and more consistent order deliveries.

There are some challenges working with events. One challenge is the high volume of data generated by events. Each aggregate root instance can generate 10s or hundreds of events (depending on how you model the actions and how many of those actions occur during its lifetime). The volume of events can grow quickly depending on your data model. To mitigate these events, there should be sparse updates with payloads that contain only the data relevant to the update. Instead of storing the entity’s current state in each event update, store only the minimum data needed for that update. Additionally, the event should be projected using data stores optimized for reads to avoid frequent reads from the event store. The event store itself should be purpose-built to optimize for sequential reading and unstructured data (No SQL databases are great candidates for event stores)

Another challenge is that having events serve as the canonical source of all updates to an entity makes complying with the GDPR’s “right to be forgotten” requirements more difficult. Unlike in a fully normalized database, where a reference to a user may be removed upon request, user information may be spread across many events and, conceptually, contradict the idea of an append-only event log. To mitigate this, each event row could be encrypted with a key specific to that row. If a request is to be removed, the key used to decrypt it will be disposed of. This still violates the event log’s immutability. Another approach is to isolate personally identifiable information (PII) from the event log and enrich the data with the personal information on demand.

In summary, event sourcing provides deeper visibility, traceability, and data-driven insights for modern applications. By treating every state change as a durable event, you can not only reconstruct and audit the lifecycle of any entity but also empower advanced analytics, process optimization, and user-facing transparency. While event sourcing introduces certain technical and compliance challenges, these can be addressed with thoughtful design and supporting technologies. Ultimately, event sourcing enables systems that are not only robust and resilient but also observable and ready to adapt to evolving business needs.