Skip to main content

EVENT DRIVEN MICROSERVICES

EVENT BASED MICROSERVICES - Event Sourcing

In a Microservice Architecture, especially with Database per Microservice, the Microservices need to exchange data. For resilient, highly scalable, and fault-tolerant systems, they should communicate asynchronously by exchanging Events. In such a case, you may want to have Atomic operations, e.g., update the Database and send the message. If you have SQL databases and want to have distributed transactions for a high volume of data, you cannot use the two-phase locking (2PL) as it does not scale. If you use NoSQL Databases and want to have a distributed transaction, you cannot use 2PL as many NoSQL databases do not support two-phase locking. In such scenarios, use Event based Architecture with Event Sourcing. In traditional databases, the Business Entity with the current “state” is directly stored. In Event Sourcing, any state-changing event or other significant events are stored instead of the entities. It means the modifications of a Business Entity is saved as a series of immutable events. The State of a Business entity is deducted by reprocessing all the Events of that Business entity at a given time. Because data is stored as a series of events rather than via direct updates to data stores, various services can replay events from the event store to compute the appropriate state of their respective data stores.

Pros

  • Provide atomicity to highly scalable systems.
  • Automatic history of the entities, including time travel functionality.
  • Loosely coupled and event-driven Microservices.

Cons

  • Reading entities from the Event store becomes challenging and usually need an additional data store (CQRS pattern)
  • The overall complexity of the system increases and usually need Domain-Driven Design.
  • The system needs to handle duplicate events (idempotent) or missing events.
  • Migrating the Schema of events becomes challenging.

When to use Event Sourcing

  • Highly scalable transactional systems with SQL Databases.
  • Transactional systems with NoSQL Databases.
  • Highly scalable and resilient Microservice Architecture.
Typical Message Driven or Event-Driven systems (e-commerce, booking, and reservation systems).

When not to use Event Sourcing

  • Lowly scalable transactional systems with SQL Databases.
  • In simple Microservice Architecture where Microservices can exchange data synchronously (e.g., via API).

Enabling Technology Examples


Event Store: EventStoreDB, Apache Kafka, Confluent Cloud, AWS Kinesis, Azure Event Hub, GCP Pub/Sub, Azure Cosmos DB, MongoDB, Cassandra, Amazon DynamoDB.
Frameworks: Lagom, Akka, Spring, akkatecture, Axon, Eventuate

Comments

Post a Comment

Popular posts from this blog

Introduction to Customer Segmentation in Python !!!

Introduction to Customer Segmentation in Python In this tutorial, you're going to learn how to implement customer segmentation using RFM(Recency, Frequency, Monetary) analysis from scratch in Python. In the Retail sector, the various chain of hypermarkets generating an exceptionally large amount of data. This data is generated on a daily basis across the stores. This extensive database of customers transactions needs to analyze for designing profitable strategies. All customers have different-different kind of needs. With the increase in customer base and transaction, it is not easy to understand the requirement of each customer. Identifying potential customers can improve the marketing campaign, which ultimately increases the sales. Segmentation can play a better role in grouping those customers into various segments. In this tutorial, you will cover the following topics: What is Customer Segmentation? Need of Customer Segmentation Types of Segmentation Customer
Post a Comment
Read more

Let's Understand Ten Machine Learning Algorithms

Ten Machine Learning Algorithms to Learn Machine Learning Practitioners have different personalities. While some of them are “I am an expert in X and X can train on any type of data”, where X = some algorithm, some others are “Right tool for the right job people”. A lot of them also subscribe to “Jack of all trades. Master of one” strategy, where they have one area of deep expertise and know slightly about different fields of Machine Learning. That said, no one can deny the fact that as practicing Data Scientists, we will have to know basics of some common machine learning algorithms, which would help us engage with a new-domain problem we come across. This is a whirlwind tour of common machine learning algorithms and quick resources about them which can help you get started on them. 1. Principal Component Analysis(PCA)/SVD PCA is an unsupervised method to understand global properties of a dataset consisting of vectors. Covariance Matrix of data points is analyzed here to un
Post a Comment
Read more

Tapping Into the “Long Tail” of Big Data

Variety, not volume or velocity, drives big-data investments !!! Gartner defines big data as the three Vs: high-volume, high-velocity, high-variety information assets. While all three Vs are growing, variety is becoming the single biggest driver of big-data investments, as seen in the results of a recent survey by New Vantage Partners. This trend will continue to grow as firms seek to integrate more sources and focus on the “long tail” of big data. From schema-free JSON to nested types in other databases (relational and NoSQL), to non-flat data (Avro, Parquet, XML), data formats are multiplying and connectors are becoming crucial. In 2017, analytics platforms will be evaluated based on their ability to provide live direct connectivity to these disparate sources. Tapping Into the “Long Tail” of Big Data When asked about drivers of Big Data success, 69% of corporate executives named greater data variety as the most important factor, followed by volume (25%), with velocity (6%)
Post a Comment
Read more