HStreamDB v0.9 released: Partition model extension, support for integration with external systems

HStreamDB 最新版本 v0.9 现已正式发布！

v0.9 The main highlights are the following updates：

• stream Partition model extension,Allows users to directly access data at specified locations on the partition;
• 新增 HStreamDB 's internal data integration framework HStream IO;
• Cluster switching is based on SWIM Member discovery and failure detection mechanisms;
• Brand new stream processing engine;
• 升级了 Java 和 Go 客户端,并新增了 Python 客户端.

Stream Partition model extension

v0.9 An extension to the previous partition model,Allows direct user manipulation and access stream Internal partition,从而可以对 stream Fine-grained control over data distribution and partition scaling.HStreamDB 采用的是 key-range-based 分区机制,stream All the partitions below jointly divide the whole key space,Each partition belongs to a contiguous subspace(key range).若 record 所带 partitionKey The hash value of falls in some subspace,那么这条 record will be stored in the corresponding partition.

具体地,v0.9 The partitioning model for , adds the following capabilities:

• 在创建 stream When configuring the initial number of partitions
• 通过​​partitionKey​​ 将写入的 record 分发到相应的 stream 的分区
• Read the data of the specified partition directly from any location
• 查看 stream The included partitions correspond to each partition key range

在之后的版本中,We will support splitting and merging pairs by partition stream 进行动态伸缩.

HStream IO 发布

HStream IO 是 v0.9 An internal data integration framework is included,包含 source connectors、sink connectors、IO runtime 等组件,它能够实现 HStreamDB 和多种外部系统的互联互通,Facilitate the efficient flow of data throughout the enterprise data stack and real-time value release.

v0.9 提供了以下的 connectors,Incremental synchronization between multiple databases can be supported.

Source connectors:

• ​ ​source-mysql​​
• ​ ​source-postgresql​​
• ​ ​source-sqlserver​​

Sink connectors:

• ​ ​sink-mysql​​
• ​ ​sink-postgresql​​

用户可以通过对应的 SQL commands 创建和管理 IO task,具体可参考​ ​文档​​了解 HStream IO 的功能和使用.

New stream processing engine

v0.9 The stream processing engine is reimplemented based on iterative and differential computation principles,Significantly improved throughput,and reduced latency.此外,The new engine also supports multiplexing Join 语句、子查询(sub-queries)and the more general materialized view(materialized view).

This feature is still under development,It is an experimental feature,用户可以参考​ ​ SQL 指南​​ 进行试用.

基于 Gossip 的 HServer 集群

v0.9 对 HServer The cluster implementation has been refactored,The new implementation mainly adopts gossip style The clustering mechanism and based on SWIM 的故障检测机制,Supersedes the previous version based on ZooKeeper 的实现.The new implementation will improve the scalability of the cluster,And reduce reliance on external systems.

Advertised Listeners

Deployment and use in production can involve complex network setups.例如,If the server cluster is hosted internally,It needs to be an externally visible one IP The address lets clients connect to the cluster,Especially when encountering use docker Or cloud hosting, etc,make the environment more complicated.

To ensure that clients from different network environments can interact with the cluster,HStreamDB 0.9 支持配置 advertised listerners.在配置了 advertised listerners 后,The server can send requests based on the port on which the client is sent,Return corresponding addresses for different clients.

统一的 HStream CLI

为了使 CLI More unified and simple,We have put the old one HStream SQL Shell and some other node management features migrated to the new one HStream CLI.HStream CLI Launching interactively is currently supported SQL Shell、send cluster bootstrap Functions such as requesting and checking server node status.Users can pass the reference ​ ​CLI 文档​​Learn how to use it.

基于 Grafana 的监控

v0.9 新增了通过 Prometheus 和 Grafana 对 HStreamDB Cluster monitoring support,HStreamDB 内部的 Metrics 将通过 exporter 存储到 Prometheus 并最终展示在 Grafana 面板上.Please refer to the specific deployment and usage process​ ​文档​​.

支持用 Helm 在 K8s 上进行部署

v0.9 提供了 HStreamDB 的 Helm Chart,现在可通过 Helm 在 K8s上快速部署 HStreamDB 集群,For more detailed usage steps, please refer to​ ​文档​​.

Client version upgrades and improvements

Java 客户端 v0.9.0、Go 客户端 v0.2.0、Python 客户端 v0.2.0 均已发布,提供对 HStreamDB 0.9 的支持.详情请见：

• Java 客户端 v0.9.0：​ ​https://github.com/hstreamdb/hstreamdb-java/releases/tag/v0.9.0​​
• Go 客户端 v0.2.0：​ ​https://github.com/hstreamdb/hstreamdb-go/releases/tag/v0.2.0​​
• Python 客户端 v0.2.0：​ ​https://github.com/hstreamdb/hstreamdb-py/releases/tag/v0.2.0​​

版权声明： 本文为 EMQ 原创,转载请注明出处.

原文链接：​ ​https://hstream.io/zh/blog/hstreamdb-v-0-9-release-notes​​