当我们使用keras训练好模型之后,The next step is to deploy services,采用flask直接加载keras的h5模型,Service of concurrent performance is low.If the pursuit of high concurrency performance,就可以采用Nginx+gunicorn+gevent的方式来启动服务,Service though this way can achieve very high concurrency performance,但却存在一个问题,也就是gunicornWay is initiated the process of multiple,Each process will load a model,Cause the service starts the memory a lot,The other direct loadingh5模型,The whole process as memory in use process will also gradually increase,这就是pythonThe memory leak problem.I a project before deployment to a production environment,使用了两天,Memory has reached a dozenG,Therefore have to use every time to restart services this strategy.因此采用tensorflow/serving的方式来部署服务.
This article USES the environment：
python 3.8.13
TensorFlow 2.50

1 keras的h5模型转换为pb格式

Here is a verification code recognition model transformation,Verification code identification usingcrnn+ctx的形式训练,The model file ash5格式.

使用如下代码将h5模型转换为tensorflow/serving需要的pb格式文件：

from keras import models
import keras.backend as K
from tensorflow.python.framework import graph_util

import tensorflow._api.v2.compat.v1 as tf
tf.disable_eager_execution()

''' tf.saved_model.builder.SavedModelBuilder 这种模型转换的模型可以提供给TensorFlow Serving使用 多一个Variable文件 '''

def keras_model_to_tfs(model, export_path):
    signature = tf.saved_model.signature_def_utils.predict_signature_def(
        inputs={
    'input_x': model.input}, 
        outputs={
    'output_y': model.output}
    )
    builder = tf.saved_model.builder.SavedModelBuilder(export_path)
    legacy_init_op = tf.group(tf.tables_initializer(), name='legacy_init_op')
    builder.add_meta_graph_and_variables(
        sess=K.get_session(),
        tags=[tf.saved_model.tag_constants.SERVING],
        signature_def_map={
    
            tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: signature,
        },
        legacy_init_op=legacy_init_op)
    builder.save()
    print('转换完成！')


if __name__=='__main__':
    network=models.load_model('./model/hebei.h5')
    export_path='pb2'
    keras_model_to_tfs(network,export_path)

运行如上代码,得到转换好的pb文件.

重复以上过程,To convert a model,新建如下目录yzmModels,The directory tree structure for：

hebei目录下的100001Directory of the files for the transformation of the first model;
chongqing目录下的100001Directory of the files for the transformation of the second model;

models.config的文件内容如下：

model_config_list:{
    
	config:{
    
      name:"hebei",
      base_path:"/models/multiModel/hebei",
      model_platform:"tensorflow"
    },
    config:{
    
      name:"chongqing",
      base_path:"/models/multiModel/chongqing",
      model_platform:"tensorflow"
    }
}

2 使用docker启动tensorflow/serving服务

将yzmModelsThis directory is uploaded to the server,如下所示：

拉取一个tensorflow/serving的docker镜像

docker pull tensorflow/serving:2.5.1

接着启动docker服务：

docker run -p 9501:8501 --mount type=bind,source=/root/hhc_etax_login/yzm/yzmModels,target=/models/multiModel -t tensorflow/serving:2.5.1 --model_config_file=/models/multiModel/models.config &

9501Mainly on the port,8501For in a container port,容器中的端口8501This cannot be changed.source=/root/hhc_etax_login/yzm/yzmModelsThis directory is given priority to on the model file directory.启动成功后,如下所示：

Service after open is good,We have to test theTensorFlow/servingWhether the service operation success,Container service corresponding to the host port for9501,So using the following commands to test：

curl http://localhost:9501/v1/models/hebei

返回结果如下：

{
    
 "model_version_status": [
  {
    
   "version": "100001",
   "state": "AVAILABLE",
   "status": {
    
    "error_code": "OK",
    "error_message": ""
   }
  }
 ]
}

使用python代码来测试hebei这个模型,TensorFlow/servingTransmit the data service is in the form of matrix,代码如下：

import numpy as np 
import requests
import json



if __name__=='__main__':
    #(1, 32, 77, 3)
    #img_array代表图像数据,Is introduced to model the input data
    img_array=np.random.randn(1,32,77,3)
    #An array data into list format
    img_list=img_array.tolist()
    json_data={
    
        'instances':img_list
    }

    url='http://localhost:9501/v1/models/hebei:predict'
    res=requests.post(url=url,json=json_data)
    if res.status_code==200:
        #To deal with the predictions of a model
        predictions=json.loads(res.text).get('predictions')
        if predictions:
            y_pred=np.array(predictions)
            print(y_pred.shape)

如上所示,因为对于hebeiModel of the shape of the input image data format as(1,32,77,3),In the code constructs a this data,In the use of models to predict captcha images,This place was introduced into matrix of the real image data can be.运行结果如下：