Apache Kafka 2.5.0 includes improvements for Kafka Streams

Share
  • April 17, 2020
Technology

Another new release from Apache Kafka is here. On April 16, 2020 version 2.5.0 was unveiled by the Apache Software Foundation. Included in the new update are additional security enhancements, new metrics, and plenty of improvements.

The distributed streaming platform has come a long way since it was developed by LinkedIn and open sourced to the Apache Software Foundation back in 2011. In nine years, it has transformed the way we move and process data.

Let’s see what’s improved in 2.5.0 and all the newest Kafka Improvement Proposals.

SEE ALSO: “The Open Source Way has proven itself as the leading way to develop software solutions”

2.5.0 changes

Work on KIP-500 includes removing direct access to ZooKeeper from the admin tools. This update continues to work towards deprecating ZooKeeper and expands the non-ZK functionality of dynamic configs.

A new change to Kafka Connect (KIP-558)  tracks the set of actively used topics by connectors. This improvement also inherits the security characteristics of similar functionalities. KIP-558 is enabled by default.

The Kafka Streams client library sees three new KIPs. KIP-150 adds Cogroup to the DSL, which will help clean up syntax. KIP-523 adds KStream#toTable to the Streams DSL. Lastly, KIP-535 will allow state stores to serve stale reads during rebalance so users can see if replicas are falling behind the primary.

Improvements

This version mainly focuses on making foundational and operational improvements.

  • KIP-447: Producer scalability for exactly once semantics (EOS): This simplifies semantics and makes it easier to create EOS applications with a large number of partitions.
  • KIP-360: Improve reliability of idempotent/transactional producer: This change fixes a commonly encountered issue regarding producer state retention on the broker and helps avoid a potential UnknownProducerId error.
  • KIP-515: Enable ZK client to use the new TLS supported authentication: Apache Kafka now ships with the latest version of ZooKeeper 3.5.7 with TLS support in ZooKeeper 3.5.
  • KIP-511: Collect and Expose Client’s Name and Version in the Brokers
  • KIP-559: Make the Kafka Protocol Friendlier with L7 Proxies
  • KIP-541: Create a fetch.max.bytes configuration for the broker: The newly added configuration creates a new maximum size of fetch requests and a new default value of 55 megabytes.

Dropped support and deprecations

Before you upgrade, be aware that with this version, Apache Kafka has dropped support for Scala 2.11. Now, it only supports Scala 2.12 and 2.13.

The default SSL protocol has been changed to TLS 1.2.

SEE ALSO: Turning binary thinking into better software development

Upgrading

View the release notes for 2.5.0 for a full rundown of everything new and improved, as well as all the latest bug fixes and newly updated documentation changes.

In addition to this, familiarize yourself with the upgrade documentation and follow the instructions for your current version. Documentation for Apache Kafka 2.5.0 is available here.

Watch the informative video from Tim Berglund and learn all about ten new KIPs in Core Kafka, Kafka Connect, and Kafka Streams.

The post Apache Kafka 2.5.0 includes improvements for Kafka Streams appeared first on JAXenter.

Source : JAXenter

You may also like...