The message broker Kafka, which has been further developed under the umbrella of the Apache Software Foundation (ASF), is now available in version 3.0.0. Confluent and the community have contributed numerous new functions and bug fixes for the current major release of the open source project – including improvements to KRaft, the self-managed metadata quorum that will be integrated into Kafka in the future and will replace Apache ZooKeeper. In addition, developers in version 3.0.0 have to be prepared for breaking changes in the APIs.
KRaft is not quite ready for production yet
With the release of version 2.8 in the spring, the Kafka team had already given a first impression of how the message broker can be used in the future without ZooKeeper. Since then, a number of improvements have been made to the KRaft metadata and APIs. Among other things, it is now possible to reassign partitions and KRaft offers support for
In addition, KRaft controllers and KRaft brokers from version 3.0 can take snapshots for the metadata topic partition
__cluster_metadata create, replicate and load. The Kafka cluster uses this topic to store and replicate metadata information such as the broker configuration or topic partition assignments. Given still known gaps The team does not yet recommend KRaft for use in production.
The list of functions and components marked as out of date (deprecated) includes Java 8, Scala 2.12 and Version 1 of MirrorMaker. While a new LTS release of the programming language has just appeared with Java 17, Kafka users have to say goodbye to Java 8. As of Kafka 4.0 at the latest, there will be no support for the outdated Java version. The same applies to Scala 2.12. While MirrorMaker2 (MM2) can now be used to save offset data records on any cluster in order to save the Kafka source cluster as a pure read-only instance, the development team is sending MirrorMaker v1 to its old age. All further development efforts and new functions are reserved for MM2.
Cleaning up Kafka Streams
As part of the for Kafka Streams in accordance with the Kafka Improvement Proposal KIP-740 proposed major renovation of the class
TaskId some methods and all internal fields are considered deprecated. The getters
partition() replace this
partition-Fields. The proposal made KIP-744 also provides to separate the implementation from the public API of some classes. The interfaces
StreamsMetadata newly introduced, the previous classes with the same name are no longer applicable.
Provide a complete overview of all new features and more details the Confluent blog post announcing the new version, the introductory video with Tim Berglund as well the release notes for Apache Kafka 3.0.0.