Crashes, Races, and Brakes: Message Broker Stream Pipeline Problems and How to Solve Them

Asynchronous integration of applications via message brokers is often presented as a reliable and fault-tolerant solution. However, in reality, a Kafka cluster (or another broker) is not a Swiss watch. It also experiences one or multiple nodes failures or network problems that can lead to a race condition during partition leader re-election. And if one of the consumers in the group fails, the performance of the streaming system will degrade due to rebalancing. 

In this practical workshop, I will discuss these and other issues that can arise during streaming event processing, as well as possible solutions. 

The workshop will involve group work on case studies (Hotel, Factory, SaaS solution).