Today we will take a closer look at the Ethereum blockchain and discuss its most important structures, namely transactions, blocks and state. I assume that you are familiar with the basics of the blockchain technology, if not, I suggest that you read a few of my earlier posts on blocks, transactions and mining. This will … Continue reading Basics of the Ethereum blockchain
If you have followed my blog for some time, you might know that it started with a few posts on the bitcoin blockchain - about its foundations in elliptic curve cryptography, blocks, mining and transactions. The bitcoin blockchain has been established in 2009, and since then, a lot has happened in the blockchain world. Maybe … Continue reading The Ethereum blockchain, smart contracts and token
In the previous post, we have seen how iterators and generators can be used in Python to implement coroutines. With this approach, a coroutine is simply a function that contains a yield statement somewhere. This is nice, but makes the code hard to read, as the function signature does not immediately give you a hint … Continue reading Asynchronous I/O with Python part III – native coroutines and the event loop
As explained in my previous post, historically coroutines in Python have evolved from iterators and generators, and understanding generators is still vital to understanding native coroutines. In this post, we take a short tour through iterators in Python and how generators have traditionally been implemented. Iterables and iterators In Python (and in other programming languages), … Continue reading Asynchronous I/O with Python part II – iterators and generators
Though not really new, a programming model commonly known as asynchronous I/O has been attracting a lot of attention over the last couple of years and even influenced the development of languages like Java, Go or Kotlin. In this and the next few posts, we will take a closer look at this model and how … Continue reading Asynchronous I/O with Python part I – the basics
In the past few posts, we have discussed approaches to implement at-least-once processing on the consumer side, i.e. mechanisms that make sure that every record in the partition is only processed once. Today, we will look at a similar problem on the producer side - how can we make sure that every record is written … Continue reading Learning Kafka with Python – retries and idempotent writes
In the previous posts, we have already used the Python client to implement Kafka consumers. Today, we will take a closer look at the components that make up a consumer and discuss their inner workings and how they communicate with the Kafka cluster. High level overview of the consumer Our discussion will be based on … Continue reading Learning Kafka with Python – a deep dive into consumers and rebalancing
Very often, either the source or the target of a Kafka based message queue is a classical relational database. Consuming data and using it to update a database table sounds straightforward, but poses a few challenges around reliability and delivery semantics. In this post, we look into two options to realize such an architecture. The … Continue reading Learning Kafka with Python – implementing a database sink
We now understand how Kafka producers add data to partitions. So let us move on and take a look at consumers - how they operate, how they are configured and how different levels of reliability and delivery guarantees can be achieved. Consumer groups In the previous post on producers, we have seen that the interaction … Continue reading Learning Kafka with Python – consuming data
As proud owners of a brand new Kafka installation, we are now ready to explore how applications interact with Kafka. Today, we will look at producers and understand how they write data to Kafka. Basic design considerations At first glance, writing data to Kafka sounds easy - connect to a Kafka broker and submit a … Continue reading Learning Kafka with Python – producing data
