OpenMLDB + Jupyter Notebook: Quickly Build Machine Learning Applications

1. Introduction

OpenMLDB is an open source machine learning database that provides a consistent production-grade feature platform both online and offline.Jupyter Notebook provides functions such as data calculation, code development, document editing, code execution, and result display based on browser web pages. It is one of the most popular and easy-to-use development environments.This article introduces the deep integration of OpenMLDB and Notebook, making full use of the powerful features of OpenMLDB and the convenience of Notebook to create a fast and easy-to-use machine learning development environment.

2. Integrated SQL Magic Functions

The first integration point between Notebook and OpenMLDB is the SQL magic function.Magic functions are an extension of Notebook. By registering magic functions, users can directly execute commands in Notebook cells without writing complex Python code, and can customize the output style.OpenMLDB provides a standard SQL magic function. Users can directly write and run SQL statements supported by OpenMLDB on Notebook, and the corresponding statement will be submitted to OpenMLDB for execution, and the results will be previewed and returned in Notebook.

2.1 Registering OpenMLDB SQL Magic Functions In order to support OpenMLDB magic functions in Notebook, you first need to register in the following way:

import openmldb db = openmldb.dbapi.connect('demo_db','0.0.0.0:2181','/openmldb') openmldb.sql_magic.register(db)

2.2 Execute a single-line SQL statement

Developers can use the prompt % to execute a single-line SQL statement, as shown in the following figure.
[image]

2.3 Execute multi-line SQL statements

Developers can also use the prompt %% to write multi-line SQL statements, as shown in the following figure.
[Picture]
Note that it is currently not supported to execute multiple SQL statements in a Notebook cell at the same time, and they need to be executed separately in different cells.

2.4 Magic Function Support Function

The SQL magic functions provided by OpenMLDB can execute all supported SQL syntaxes, including using OpenMLDB's unique offline mode to asynchronously submit complex SQL statements that need to process big data to the offline execution engine for execution, as shown in the following figure.
[Picture]
For details on how to use OpenMLDB magic functions, please refer to the introduction to Magic functions in the official OpenMLDB documentation https://openmldb.ai/docs/zh/main/quickstart/python_sdk.html#notebook-magic-function .

3. Notebook integration with OpenMLDB Python SDK

The second feature of the OpenMLDB and Notebook integration is the integration with the OpenMLDB Python SDK.Notebook supports Python to run the kernel, so various Python libraries can be imported through import. OpenMLDB provides a full-featured Python SDK for calling within Notebook.OpenMLDB not only provides a DBAPI based on the Python PEP249 standard, but also supports the mainstream SQLAlchemy interface in the Python industry. You can connect to an existing OpenMLDB cluster with just one line of code.

3.1 Using OpenMLDB DBAPI

It is very simple to use the DBAPI interface. For the cluster version, you only need to specify the ZooKeeper address and node path to connect. After the connection is successful, there will be corresponding log information; for the stand-alone version, you can use the IP and port as parameters to connect to the stand-alone version of OpenMLDB,As follows.

import openmldb.dbapi db = openmldb.dbapi.connect('demo_db','0.0.0.0:2181','/openmldb')

After the connection is successful, you can call the DBAPI interface of the OpenMLDB Python SDK in Notebook for development. For details, see https://openmldb.ai/docs/zh/main/quickstart/python_sdk.html#openmldb-dbapi.

3.2 Using OpenMLDB SQLAlchemy

If you want to use SQLAlchemy, it is also very simple. For the cluster version, you can complete the connection by specifying the URI of OpenMLDB through the SQLAlchemy library; you can also use the IP and port as parameters to connect to the stand-alone version of the OpenMLDB database, as shown below.

import sqlalchemy as db engine = db.create_engine('openmldb://demo_db?zk=127.0.0.1:2181&zkPath=/openmldb') connection = engine.connect()

After the connection is successful, you can develop through the SQLAlchemy interface of the OpenMLDB Python SDK. For details, see https://openmldb.ai/docs/zh/main/quickstart/python_sdk.html#openmldb-sqlalchemy.

4. References

• OpenMLDB Python SDK: https://openmldb.ai/docs/en/main/quickstart/python_sdk.html
• OpenMLDB usage documentation: https://openmldb.ai/docs/en/main/index.html