Phase 4: Architecture & Scaling

Event sourcing & CQRS patterns

Advanced ~3 min read
Think of it this way A friendly analogy. Read this if the technical version feels dense. Show Hide

Imagine playing a super fun board game, like Monopoly. Instead of just knowing who has how much money or what properties they own right now, what if we wrote down every single thing that ever happened in the game? Not just "Sarah has $1000," but "Sarah started with $1500," then "Sarah landed on Park Place and paid rent," then "Sarah passed Go and collected $200." This isn't just about the current picture; it's like having a detailed storybook of the whole game from start to finish. This way, if you ever wonder, "How did Sarah lose all her money?" you can just read the story and see every step.

So, instead of erasing "Sarah has $1000" and writing "$900" when she pays rent, we add "Sarah paid $100 rent to Tom" to our game history. These history entries are like little, unchanging facts. To know Sarah's money right now, you simply start from the beginning of the history book and add up or subtract everything related to her. It’s like hitting rewind and play to see exactly what happened. This helps us understand complex games, fix mistakes, or even see what the game looked like five turns ago!

Now, consider two ways we interact with our game. When we're playing – rolling dice, buying properties, paying rent – these actions change the game, and we call them "commands." When we're just looking at the game – like asking, "Who owns the most railroads?" or "How many houses are on Boardwalk?" – these questions just get information, and we call them "queries." It's helpful to have one super-organized system for handling commands (our rulebook and playing pieces), and a completely separate, super-fast system just for answering queries (like a special scorecard that quickly shows who owns what). This way, the part that changes things isn't slowed down by people just wanting to look.

By keeping a perfect historical record and separating how we make changes from how we get information, we can build really smart and flexible systems. Imagine building a fancy scoreboard that updates instantly with cool graphs, without slowing down players making their moves. Or imagine going back in time to see exactly how a game unfolded, step-by-step. This means you can create programs that are super good at remembering details, understanding complex timelines, and providing different kinds of information to different people, all at the same time.

Event Sourcing isn't about storing the current state of an application, but rather the sequence of events that led to that state. Instead of updating a row in a database (e.g., SET balance = new_balance), you persist an immutable event record like MoneyDeposited(accountId, amount, timestamp). The application's current state is then derived by replaying these events in order. This provides an unparalleled audit log, allows for "time travel" (reconstructing state at any past point), and makes temporal queries straightforward. It forms the backbone of highly reliable and auditable systems, naturally integrating with event-driven architectures where these events are also published for other services to consume.

CQRS, or Command Query Responsibility Segregation, is a pattern that separates the concerns of modifying data (commands) from reading data (queries). Traditionally, a single data model and database schema serve both. With CQRS, you'll have distinct "write models" designed for handling commands and "read models" optimized for specific query needs. For instance, your write model might be a rich domain model focused on business rules, while your read models could be highly denormalized PostgreSQL tables, Elasticsearch indices, or even materialized views tailored for specific UI displays or reporting requirements. This separation allows independent scaling, optimization, and evolution of read and write sides.

When combined, Event Sourcing and CQRS create a powerful, robust architecture. Commands flow into the write model, which validates business rules and then emits events (Event Sourcing). These events are persisted as the system's single source of truth and then asynchronously processed to update one or more read models (CQRS). This loose coupling means read models can be eventually consistent, tailored for performance, and even discarded and rebuilt from the event store if requirements change. While introducing complexity like eventual consistency and managing event schema evolution, the benefits in terms of auditability, scalability, and flexibility for complex domains often outweigh the trade-offs for advanced backend systems.

Key Takeaways

  • Event Sourcing: Store every state-changing action as an immutable event; derive current state by replaying events.
  • CQRS: Separate data models for commands (writes) and queries (reads) to optimize each responsibility.
  • Synergy: Commands generate events (ES); events update highly optimized, often denormalized, read models (CQRS).
  • Benefits: Enhanced auditability, temporal querying, improved read scalability, and flexible query performance.
  • Trade-offs: Introduces eventual consistency and increased architectural complexity.

Code Example

python
class AccountCreated:
    def __init__(self, account_id, initial_balance):
        self.account_id = account_id
        self.initial_balance = initial_balance

class MoneyDeposited:
    def __init__(self, account_id, amount):
        self.account_id = account_id
        self.amount = amount

event_store = []
def create_account(account_id, initial_balance): event_store.append(AccountCreated(account_id, initial_balance))
def deposit_money(account_id, amount): event_store.append(MoneyDeposited(account_id, amount))

current_balances_read_model = {}
def apply_event_to_read_model(event):
    if isinstance(event, AccountCreated):
        current_balances_read_model[event.account_id] = event.initial_balance
    elif isinstance(event, MoneyDeposited):
        current_balances_read_model[event.account_id] += event.amount

create_account("acc123", 100)
deposit_money("acc123", 50)

for event in event_store:
    apply_event_to_read_model(event)

# Query the read model (e.g., current_balances_read_model["acc123"] would be 150)

How this code works

This code demonstrates fundamental concepts of Event Sourcing and CQRS by managing bank account balances through a sequence of events. Instead of directly modifying account balances, every change is recorded as an immutable event. The system then rebuilds a query-optimized view from these events. AccountCreated and MoneyDeposited are simple event classes, representing facts that have occurred. When commands like create_account or deposit_money are invoked, they don't alter application state directly. Instead, they append new events to the event_store, which serves as the definitive, ordered log of everything that has ever happened in the system.

The current_balances_read_model is a separate data structure designed for efficient querying, detached from the event store. The apply_event_to_read_model function defines how each event type modifies this read model's state, for example, setting an initial_balance or adding an amount. A key insight is the final for loop: it rebuilds the entire current_balances_read_model by replaying all events from the event_store from the beginning. A subtle point for beginners is that the current_balances_read_model isn't updated by create_account or deposit_money directly; it's updated only by processing replayed events, ensuring the event_store remains the single source of truth.