当前位置：网站首页>AI+Medical: Using Neural Networks for Medical Image Recognition and Analysis

AI+Medical: Using Neural Networks for Medical Image Recognition and Analysis

2022-08-09 23:14:00 【51CTO】

AI+医疗：Medical image recognition analysis using neural networks _计算机视觉

作者：韩信子@ ShowMeAI
计算机视觉实战系列： https://www.showmeai.tech/tutorials/46
Industry famous enterprise application series： https://www.showmeai.tech/tutorials/63
本文地址： https://www.showmeai.tech/article-detail/298
声明：版权所有,转载请联系平台与作者并注明出处
收藏 ShowMeAI查看更多精彩内容

深度学习+医疗科技

In recent years, the rapid development of artificial intelligence technology has been applied to various vertical fields,For example, applying deep learning to various medical diagnoses,The effect is significant and even surpasses that of human experts in some aspects.典型的 CV The latest technology has been applied to the classification of Alzheimer's disease、肺癌检测、In medical imaging tasks such as retinal disease detection.

AI+医疗：Medical image recognition analysis using neural networks _计算机视觉_02

图像分割

Image segmentation is the process of dividing images into different groups according to their content,It locates objects and boundaries in an image.Semantic segmentation is pixel-level recognition,Our typical applications in many fields,The technical support behind it is image segmentation algorithm,比如：医学影像、Unmanned driving area detection、背景虚化等.

AI+医疗：Medical image recognition analysis using neural networks _计算机视觉_03

AI+医疗：Medical image recognition analysis using neural networks _神经网络_04

The basics of deep learning covered in this article,and detailed knowledge of computer vision,推荐大家阅读 ShowMeAI的教程专栏：
深度学习教程：吴恩达专项课程 · 全套笔记解读
深度学习与计算机视觉教程：斯坦福CS231n · 全套笔记解读

Semantic segmentation canonical network U-Net

U-Net is a convolutional network architecture,用于快速、Precisely segment biomedical images.

A comparison of the principles and advantages and disadvantages of various algorithms for semantic segmentation（包括U-Net）, ShowMeAI in past articles 深度学习与CV教程(14) | 图像分割 (FCN,SegNet,U-Net,PSPNet,DeepLab,RefineNet) There are detailed explanations.

U-Net 的结构如下图所示：

AI+医疗：Medical image recognition analysis using neural networks _图像分割_05
在 U-Net 中,Like all other convolutional neural networks,It consists of layers like convolution and max pooling.

U-Net 简单地将编码器的特征图拼接至每个阶段解码器的上采样特征图,从而形成一个梯形结构.该网络非常类似于 Ladder Network 类型的架构.
通过跳跃 拼接 连接的架构,在每个阶段都允许解码器学习在编码器池化中丢失的相关特征.
上采样采用转置卷积.

使用 U-Net Perform lung image segmentation

AI+医疗：Medical image recognition analysis using neural networks _计算机视觉_06

The dataset we use here is Montgomery County X Radiology datasets. The dataset consists of various types of lungs X Radiographic images and each X Image composition of the segmented images of the left and right lungs of the ray.You can also pass directlyShowMeAIThe Baidu network disk link to download this dataset.

实战数据集下载（百度网盘）：公众号『ShowMeAI研究中心』回复『实战』,或者点击这里获取本文 [10] Lung medical image recognition and analysis using neural networks 『masked montgomery county x-ray set Pulmonary medical imaging dataset』

ShowMeAI官方GitHub： https://github.com/ShowMeAI-Hub

AI+医疗：Medical image recognition analysis using neural networks _图像识别_07

① Tool library import&环境设置

First import the tool library we used this time.

# 导入工具库
import os
import numpy as np
import cv2
from glob import glob
from sklearn.model_selection import train_test_split
import tensorflow as tf
from tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.metrics import Recall, Precision

     1.
2.
3.
4.
5.
6.
7.
8.
9.
10.

② 数据读取

Next we finish the data reading part,The content read here includes images and masks（mask,That is, a label of the same size as the image）.We will adjust the dimension size,in order to be able to act U-Net 的输入.

# 读取X射线图像
def imageread(path,width=512,height=512):
    x = cv2.imread(path, cv2.IMREAD_COLOR)
    x = cv2.resize(x, (width, height))
    x = x/255.0
    x = x.astype(np.float32)
    return x

# Read label mask
def maskread(path_l, path_r,width=512,height=512):
    x_l = cv2.imread(path_l, cv2.IMREAD_GRAYSCALE)
    x_r = cv2.imread(path_r, cv2.IMREAD_GRAYSCALE)
    x = x_l + x_r
    x = cv2.resize(x, (width, height))
    x = x/np.max(x)
    x = x > 0.5
    x = x.astype(np.float32)
    x = np.expand_dims(x, axis=-1)
    return x

     1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.

③ 数据切分

We want to effectively evaluate the performance of the model,So next we do data partitioning,We divide all data into training set、验证集和测试集.具体代码如下：

"""加载与切分数据"""
def load_data(path, split=0.1):
    images = sorted(glob(os.path.join(path, "CXR_png", "*.png")))
    masks_l = sorted(glob(os.path.join(path, "ManualMask", "leftMask", "*.png")))
    masks_r = sorted(glob(os.path.join(path, "ManualMask", "rightMask", "*.png")))
    split_size = int(len(images) * split) # 9:1的比例切分
    train_x, val_x = train_test_split(images, test_size=split_size, random_state=42)
    train_y_l, val_y_l = train_test_split(masks_l, test_size=split_size, random_state=42)
    train_y_r, val_y_r = train_test_split(masks_r, test_size=split_size, random_state=42)
    train_x, test_x = train_test_split(train_x, test_size=split_size, random_state=42)
    train_y_l, test_y_l = train_test_split(train_y_l, test_size=split_size, random_state=42)
    train_y_r, test_y_r = train_test_split(train_y_r, test_size=split_size, random_state=42)

    return (train_x, train_y_l, train_y_r), (val_x, val_y_l, val_y_r), (test_x, test_y_l, test_y_r)

     1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.

④ TensorFlow IO准备

我们会使用到 TensorFlow Train and estimate,我们用 TensorFlow 读取 numpy array 格式的数据,转为 TensorFlow 的 tensor 形式,And build convenience to batch Morphology read and trained dataset 格式.

# tensor格式转换
def tf_parse(x, y_l, y_r):
    def _parse(x, y_l, y_r):
        x = x.decode()
        y_l = y_l.decode()
        y_r = y_r.decode()
        x = imageread(x)
        y = maskread(y_l, y_r)
        return x, y
    x, y = tf.numpy_function(_parse, [x, y_l, y_r], [tf.float32, tf.float32])
    x.set_shape([512, 512, 3])
    y.set_shape([512, 512, 1])
    return x, y

# 构建tensorflow dataset
def tf_dataset(X, Y_l, Y_r, batch=8):
    dataset = tf.data.Dataset.from_tensor_slices((X, Y_l, Y_r))
    dataset = dataset.shuffle(buffer_size=200)
    dataset = dataset.map(tf_parse)
    dataset = dataset.batch(batch)
    dataset = dataset.prefetch(4)
    return dataset

     1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.

⑤ U-Net 网络构建

下面我们构建 U-Net 网络.

from tensorflow.keras.layers import Conv2D, BatchNormalization, Activation, MaxPool2D, Conv2DTranspose, Concatenate, Input
from tensorflow.keras.models import Model

# A convolutional block structure
def conv_block(input, num_filters):
    x = Conv2D(num_filters, 3, padding="same")(input)
    x = BatchNormalization()(x)
    x = Activation("relu")(x)

    x = Conv2D(num_filters, 3, padding="same")(x)
    x = BatchNormalization()(x)
    x = Activation("relu")(x)

    return x

# 编码器模块
def encoder_block(input, num_filters):
    x = conv_block(input, num_filters)
    p = MaxPool2D((2, 2))(x)
    return x, p

# 解码器模块
def decoder_block(input, skip_features, num_filters):
    x = Conv2DTranspose(num_filters, (2, 2), strides=2, padding="same")(input)
    x = Concatenate()([x, skip_features])
    x = conv_block(x, num_filters)
    return x

# 完整的U-Net
def build_unet(input_shape):
    inputs = Input(input_shape)
    
    # 编码器部分
    s1, p1 = encoder_block(inputs, 64)
    s2, p2 = encoder_block(p1, 128)
    s3, p3 = encoder_block(p2, 256)
    s4, p4 = encoder_block(p3, 512)

    b1 = conv_block(p4, 1024)
    
    # 解码器部分
    d1 = decoder_block(b1, s4, 512)
    d2 = decoder_block(d1, s3, 256)
    d3 = decoder_block(d2, s2, 128)
    d4 = decoder_block(d3, s1, 64)
    
    # 输出
    outputs = Conv2D(1, 1, padding="same", activation="sigmoid")(d4)

    model = Model(inputs, outputs, name="U-Net")
    return model

     1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.

⑥ Evaluation criteria and loss functions

We target semantic segmentation scenarios,Write evaluation criteria IoU 的计算方式,并构建 Dice Loss Loss function for more targeted training and learning under semantic segmentation of medical scenes.

关于IoU、mIoUOther evaluation criteria can be viewed ShowMeAI的文章 深度学习与CV教程(14) | 图像分割 (FCN,SegNet,U-Net,PSPNet,DeepLab,RefineNet) 做更多了解.

关于Dice LossThe interpretation of the loss function is as follows：

Dice 系数

根据 Lee Raymond Dice 命名,是一种集合相似度度量函数,通常用于计算两个样本的相似度(值范围为 $[0, 1]$ )：

s = \frac{2|X \cap Y|}{|X|+|Y|}

$|X \cap Y|$ 表示 $X$ 和 $Y$ 之间的交集; $|X|$ 和 $|Y|$ 分别表示 $X$ 和 $Y$ 的元素个数.其中,分子中的系数 $2$ ,是因为分母存在重复计算 $X$ 和 $Y$ 之间的共同元素的原因.

针对,For the semantic segmentation problem, $X$ Standard answer for segmented images GT, $Y$ Predict labels for segmented images Pred.

Dice Coefficient difference function（Dice loss）

s =1- \frac{2|X \cap Y|}{|X|+|Y|}

The code implementation of the evaluation criteria and loss function is as follows：

# IoU计算
def iou(y_true, y_pred):
    def f(y_true, y_pred):
        intersection = (y_true * y_pred).sum()
        union = y_true.sum() + y_pred.sum() - intersection
        x = (intersection + 1e-15) / (union + 1e-15)
        x = x.astype(np.float32)
        return x
    return tf.numpy_function(f, [y_true, y_pred], tf.float32)

# Dice Loss定义
smooth = 1e-15
def dice_coef(y_true, y_pred):
    y_true = tf.keras.layers.Flatten()(y_true)
    y_pred = tf.keras.layers.Flatten()(y_pred)
    intersection = tf.reduce_sum(y_true * y_pred)
    return (2. * intersection + smooth) / (tf.reduce_sum(y_true) + tf.reduce_sum(y_pred) + smooth)

def dice_loss(y_true, y_pred):
    return 1.0 - dice_coef(y_true, y_pred)

     1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.

⑦ Hyperparameter setting and model compilation

Next before starting model training,Let's hammer out some hyperparameters first,如下：

批次大型 batch size = 2
学习率 learning rate= 1e-5
迭代轮次 epoch = 30

我们使用 Adam 优化器进行训练,The evaluation metrics used include Dice 系数、IoU、召回率和精度.

# 超参数
batch_size = 2
lr = 1e-5
epochs = 30
model_path = "models/model.h5"

# 读取数据
dataset_path = './NLM-MontgomeryCXRSet/MontgomerySet'
(train_x, train_y_l, train_y_r), (val_x, val_y_l, val_y_r), (test_x, test_y_l, test_y_r) = load_data(dataset_path)

# 训练集与验证集
train_dataset = tf_dataset(train_x, train_y_l, train_y_r, batch=batch_size)
val_dataset = tf_dataset(val_x, val_y_l, val_y_r, batch=batch_size)

# 构建模型
model = build_unet((512, 512, 3))
# 评估准则
metrics = [dice_coef, iou, Recall(), Precision()]
# 编译模型
model.compile(loss=dice_loss, optimizer=Adam(lr), metrics=metrics)

     1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.

可以使用model.summaryView model structure information and parameter quantities：

结果如下图所示（部分内容截图,All model information is longer）：

AI+医疗：Medical image recognition analysis using neural networks _神经网络_08

⑧ 回调函数&模型训练

We set the model storage related settings in the callback function,学习率调整策略等,Then train on the dataset.

# 回调函数
callbacks = [
        ModelCheckpoint(model_path, verbose=1, save_best_only=True),
        ReduceLROnPlateau(monitor='val_loss', factor=0.1, patience=5, min_lr=1e-8, verbose=1)
        ]

# 模型训练
history = model.fit(
        train_dataset,
        epochs=epochs,
        validation_data=val_dataset,
        callbacks=callbacks
    )

     1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.

The intermediate information of the training part is shown in the figure below.

AI+医疗：Medical image recognition analysis using neural networks _U-Net_09
while training the model over 30 个 epoch 后,保存的模型（The validation loss is 0.10216）The relevant evaluation index results are as follows：

dice coef：0.9148
iou：0.8441
recall：0.9865
precision：0.9781
val_loss：0.1022
val_dice_coef: 0.9002
val_iou：0.8198
val_recall：0.9629
val_precision：0.9577

⑨ Model loading and new data estimation

We can reload the model we just saved into memory,And make an estimate on the unseen test data set,代码如下：

# 重新载入模型
from tensorflow.keras.utils import CustomObjectScope
with CustomObjectScope({'iou': iou, 'dice_coef': dice_coef, 'dice_loss': dice_loss}):
  model = tf.keras.models.load_model("/content/model.h5")


# 测试集预估
from tqdm import tqdm
import matplotlib.pyplot as plt
ct=0

# 遍历测试集
for x, y_l, y_r in tqdm(zip(test_x, test_y_l, test_y_r), total=len(test_x)):
    """ Extracing the image name. """
    image_name = x.split("/")[-1]

    # Read the test image set
    ori_x = cv2.imread(x, cv2.IMREAD_COLOR)
    ori_x = cv2.resize(ori_x, (512, 512))
    x = ori_x/255.0
    x = x.astype(np.float32)
    x = np.expand_dims(x, axis=0)

    # 读取标签信息
    ori_y_l = cv2.imread(y_l, cv2.IMREAD_GRAYSCALE)
    ori_y_r = cv2.imread(y_r, cv2.IMREAD_GRAYSCALE)
    ori_y = ori_y_l + ori_y_r
    ori_y = cv2.resize(ori_y, (512, 512))
    ori_y = np.expand_dims(ori_y, axis=-1)  # (512, 512, 1)
    ori_y = np.concatenate([ori_y, ori_y, ori_y], axis=-1)  # (512, 512, 3)

    # 预估
    y_pred = model.predict(x)[0] > 0.5
    y_pred = y_pred.astype(np.int32)
    #plt.imshow(y_pred)

    # Store the estimated resultsmask
    save_image_path = "./"+str(ct)+".png"
    ct+=1
    y_pred = np.concatenate([y_pred, y_pred, y_pred], axis=-1)
    sep_line = np.ones((512, 10, 3)) * 255
    cat_image = np.concatenate([ori_x, sep_line, ori_y, sep_line, y_pred*255], axis=1)
    cv2.imwrite(save_image_path, cat_image)

     1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.

Partial results visualization：

下面为2The original images of the test samples、original mask（标准答案）and the combined image of the prediction mask：

The input image for the test case（左侧）、Original mask label（中间）、预测掩码（右侧）

AI+医疗：Medical image recognition analysis using neural networks _计算机视觉_10

参考资料

实战数据集下载（百度网盘）：公众号『ShowMeAI研究中心』回复『实战』,或者点击这里获取本文 [10] Lung medical image recognition and analysis using neural networks 『masked montgomery county x-ray set Pulmonary medical imaging dataset』
ShowMeAI官方GitHub： https://github.com/ShowMeAI-Hub
深度学习教程：吴恩达专项课程 · 全套笔记解读： https://www.showmeai.tech/tutorials/35
深度学习与计算机视觉教程：斯坦福CS231n · 全套笔记解读： https://www.showmeai.tech/tutorials/37
深度学习与CV教程(14) | 图像分割 (FCN,SegNet,U-Net,PSPNet,DeepLab,RefineNet)： https://www.showmeai.tech/article-detail/273