In this post, the author will give an overview of how to write event-driven distributed systems in Golang, with gRPC, NATS JetStream, and CockroachDB. As a Golang developer and a solutions architect, my favorite technologies include gRPC, NATS ecosystem, and CockroachDB. I will use these technologies to demonstrate a simple demo of event-driven architecture to solve some practical challenges of building distributed systems in general, particularly Microservices based distributed systems. This post will also give a brief idea about Event Sourcing and CQRS.
Practical challenges of building distributed systems and microservices
Although people who have never worked on distributed systems from an engineering perspective have been aggressively claiming and marketing that Microservices architecture is all about containers and Kubernetes, in reality, Microservices architecture is all about functional decomposition. Although we may use containers and Kubernetes for infrastructure, the fundamental idea of Microservices architecture is functional decomposition for building highly-scalable systems. Because we decompose larger systems into several autonomous services, it may bring new kinds of complexities, especially this may make it hard to make transactions that span in multiple microservices and querying data where data is scattered in multiple databases owned by individual microservices.
Challenges for managing distributed transactions and querying data from decentralized data stores
Because we broke up a monolithic system into several autonomous services, the data is scattered amongst several databases owned by individual microservices. This creates a lot of complexity in your applications and architecture. For example, a business transaction may span multiple microservices. Let’s say you build an e-commerce system with a Microservices architecture were placing an order would be initially handled by OrderService — a microservice, then payment processing might be done by another service — a PaymentService, and so on. And another challenge is querying data from multiple databases. You can easily perform join queries from a single database with a monolithic database. Because the monolithic database is moved into several databases as part of the decomposition of functional components, you can’t simply execute join queries, thus you must get data from multiple databases. Here you don’t have any centralized database.
