6 Guidelines For Implementing EDA

Implementing an Event-Driven Architecture (EDA) is a complex process that requires careful consideration and planning. Just like with any software development project, it’s difficult to predict everything. Even the most experienced developers can make mistakes along the way, like the implementation pitfalls we discussed in one of our previous blogs.

Anticipating these issues and recognising them as soon as they surface is crucial to handling them effectively and avoiding technical debt. However, since event hubs and nearly everything on them are made of immutable data, you will need a different approach than the tried-and-true mutable solutions.

That's why we're sharing six more tips in this blog post to help you avoid common mistakes when implementing an Event-Driven Architecture. From schema definition to naming policies, we'll cover a range of topics to provide you with the handholds and inspiration you need to tackle problems down the line. Let's dive in!

1. Use versioning for topics

As you're defining your topic's domain and naming conventions, it's important to plan for the future. One way to do this is to add a version concept to your topics from the start. If you end up needing to make a breaking change to the data contract on the topic, you'll need to create a new topic version for it. If you don't, you will need to have tight control over all the consumers as they will all have to be prepared for the new data contract.

When you create a new version of the topic, it's a good idea to provide your consumers with a grace period to switch to the new topic. You can facilitate this process by setting up a stream between the old and the new versions of the data contract. If you put this stream in a separate piece of logic, you can hand this over to any consumers that have not switched by the end of the grace period.

By using these streams, and handing them over at the end of a grace period, you place the technical debt and maintenance cost of not adhering to the evolving data contracts in the consuming applications' responsibility. This ensures that your event-driven architecture stays scalable and adaptable in the long run.

2. Simplify data transfers between topics

Moving data between topics can be daunting at first, because it’s quite different from more traditional tasks like copying and viewing a database table. However, it's a common operation that you'll eventually encounter no matter what, so it's important to have a clear way to handle it.

Ideally, you should provide an easy way to move data between topics, and practise it with everyone in the team. This will allow you to keep everything aligned with any changing needs and guidelines, instead of being left with legacy data that nobody dares to touch. A bit of custom code or tools like Benthos can be used to solve this problem. Internal topics are ideal practice targets, because they might be used in your applications but they are not real cross-domain data products. By making everyone in the team perform these actions, you will avoid having a single “event hub guru” who is responsible for everything.

3. Use schemas

We’ve noticed that organisations often start without using schemas when working with data. While schemaless data may seem more convenient initially, or for rare use cases with just a single consumer, it can become problematic as the organization’s data scales.

Using schemas is like writing tests for code: it provides structure and support for evolving software. Schemas serve as a safeguard against mistakes, and they create implicit information about what a piece of data is meant to communicate. This information is crucial for both producers and consumers of data.

Investing time in setting up schemas might seem like an unnecessary additional step at the beginning, but it can save you a lot of trouble in the long run. It doesn't have to be fancy, but it's important to have a clear definition of the data structure and format that will be used across the organisation. With schemas, you'll have better control over your data and ensure that it's accurate and reliable.

4. Create events without a consumer

When developing new solutions, it's a good idea to create events even if they don't have a consumer yet. As you're modelling and implementing a specific solution that owns a part of a domain, it's often just a minor effort to create business events. By creating events upfront, you'll save yourself a lot of time and effort down the line when you actually need them.

Creating events without a consumer also provides added value. It allows you to create new features and solutions quickly and effortlessly. This practice will become increasingly valuable over time, as you'll have a growing pool of events that can be used in various parts of your system. This will give you much more agility, and enable you to respond quickly to changing business needs - which is what Event-Driven Architecture is all about.

5. Consider the level of detail for events

At Cymo, we always aim for what we call business events. We think that every event should be a cornerstone occurrence in a business process. By only including something that is worthwhile enough to register as part of a process, we can usually find the right level of detail for each event.

If you go into too much detail, you might end up with events that describe individual field changes, like "HouseNumberChanged" or "PostalCodeChanged". While technically correct, these events are often meaningless on their own and require the consumer to piece them together. It's like looking at something from way too close - you lose the bigger picture.

Going up a level to "AddressChanged" can be more useful, but still leaves some room for interpretation by the consumer. A better approach is to create an event that directly expresses the business context, like "CustomerMoved". This event captures everything the consumer needs to know about the change and avoids any ambiguity.

It's a balancing act to get the zoom level right, and it's easy to go too far in either direction. But it's crucial to keep context in mind and create events that are meaningful to the business. When in doubt, it's better to have too many events than too little. It's easy to ignore events, but it's hard to add new ones later on.

6. Consume your own events

As you dive into the rabbithole of building event-driven solutions, you'll face the question of how far to take your events. One extreme is to put events at the core of everything, with your application's state or database becoming an ephemeral read model that can be recreated on the fly. However, this can be a daunting task in a brownfield solution, so it may not be feasible from the start.

Alternatively, you could keep the application's database as the master and just publish events as they happen. While these approaches are acceptable from an engineering standpoint, they carry an added risk for your events. If you mutate the database to fix a problem or change it without publishing the corresponding event, every downstream consumer will be unaware of the change, leading to inconsistencies across domains.

One solution is to put transactions around mutations, but this has a performance and maintenance cost, we’d recommend only using this for synchronous logic. For everything else, consider “eating your own dog food”: consume your own events to mutate your database, or validate that it's consistent with what you have in your state model every now and then. This way, you'll catch any inconsistencies before they snowball into larger problems down the line.

Conclusion

In conclusion, Event-Driven Architecture is a powerful approach to software development that can bring many benefits to your organisation when done right. Versioning, schemas, and simplified data transfer work together to streamline the flow of information in your system. By thinking about your events in terms of the business processes they represent, and by creating events even if they don't have a consumer yet, you will also create a more agile and responsive system. Finally, considering the "zoom level" of your events and consuming your own events ensures consistency and the success of your event-driven project.