OpenMLDB v0.5.0 has been officially released recently, and the performance and functions have been significantly optimized, updated and upgraded.

Three core upgrades bring comprehensive optimization of performance, cost and flexibility:

• Online performance improved by orders of magnitude: Introducing pre-aggregation technology to optimize long-window real-time processing efficiency

• Low-cost landing option: online engine introduces external memory-based storage engine

• Extended usage scenario flexibility: support for user-defined function (UDF) development

Upgrade 1: Pre-aggregation technology to optimize real-time processing efficiency for long windows

• Application scenario: In a business scenario that contains millions of records in a time window (such as a long-term window spanning several years), the feature generation method based on pure calculation will bring high latency, andMay contain a lot of double counting.

• Version upgrade: OpenMLDB v0.5.0 introduces a new pre-aggregation technology. When data arrives, data-driven calculation is performed, and pre-aggregation tables are updated in real time, which greatly saves the workload of real-time calculation.10x performance boost.

Upgrade 2: Online engine introduces external memory-based storage engine

• Application scenarios: Scenarios that are more sensitive to cost, but can tolerate a certain performance degradation (OpenMLDB's online engine uses a self-developed high-performance memory time series data storage engine by default. Although the memory-based storage engine bringsExtreme access performance, but when the amount of data is large, memory will bring significant cost overhead).

• Version upgrade: OpenMLDB v0.5.0 introduces an external memory-based storage engine as an additional option. Using an HDD/SSD-based storage engine can reduce the overall usage cost of OpenMLDB by more than 75%.

Upgrade 3: Support user-defined function (UDF) development

• Application scenarios: very complex application scenarios (Although OpenMLDB provides extended SQL for feature development, for very complex scenarios, there may still be insufficient expressive ability, resulting in the failure of user scenarios to go online).

• Version upgrade: OpenMLDB v0.5.0 opens the function of C/C++-based user-defined function (UDF) and supports dynamic registration.Users' complex scenarios can be easily implemented through UDF extensions, breaking through the limitations of the original SQL expression capabilities.

Advance | Test Report

The first version of the official performance benchmark report of OpenMLDB will be released soon, which will give you a detailed understanding of the expected performance of OpenMLDB in different scenarios, so stay tuned~

Announcement | OpenMLDB Kafka Connector

The development of the OpenMLDB Kafka Connector has been completed. The principle and operation guide will also be released in the near future. Welcome to pay attention~

Related Links:

OpenMLDB official website: OpenMLDB - full-stack solution for production-level feature development

OpenMLDB GitHub link: GitHub - 4paradigm/OpenMLDB: OpenMLDB is an open-source machine learning database that provides a feature platform enabling consistent features for trainingand inference.

OpenMLDB v 0.5.0: Release v0.5.0 · 4paradigm/OpenMLDB · GitHub