Apache Flink 1.10.0 adds native Kubernetes integration as beta version

The latest version of Apache Flink, v1.10.0, has arrived. Apache Flink is a framework and distributed processing engine for stateful computations. It supports unbounded streams with a defined start, as well as bounded streams with a defined start and end.

SEE ALSO: Apache Kafka 2.4 lets consumers fetch from closest replica

In Apache Flink 1.10, over 1,200 issues were implemented by more than 200 contributors. Let’s see what this means for Flink users.

Native Kubernetes integration

Version 1.10 has the first phase of active Kubernetes integration on board, which adds initial support for session clusters. In order to allocate new pods on-demand, Flink’s ResourceManager K8sResMngr natively communicates with Kubernetes. This is similar to how Flink’s integration with Yarn and Mesos works.

PyFlink

PyFlink was first introduced as preview in Apache Flink 1.9, and its goal is to add full Python support to Flink. In v1.10, it has received additional features. For one thing, it is now possible to install PyFlink, still in “a very initial version,” via pip: pip install apache-flink.

But that’s not all: Python User-Defined Functions (UDF) in the TableAPI/SQL are now supported, and support for UDTF/UDAF is planned.

Since Python 2 reached its end of life last month, it is no longer supported. Python 3.5 is now the minimum requirement.

Other changes

There are several other updates in Apache Flink 1.10 as well. For example, managed memory can now account for memory usage of RocksDBStateBackend, the configuration of RocksDB has been simplified, and v1.10 can be compiled and run on Java 11.

SEE ALSO: Apache Tika – “Data-driven analytics are at the heart of modern applications”

See the blog post and the release notes for further details.

The post Apache Flink 1.10.0 adds native Kubernetes integration as beta version appeared first on JAXenter.