(10) Sequence and deserialization of image data

欢迎访问个人网络日志知行空间

1.Pythoninput and output streams

Python的io模块提供了PythonThe main functions in processing various types of input and output,PythonThe main ones dealt with are3种类型的io,分别是
text I/O,binary I/O和raw I/O.All three types of concrete objects are文件对象,有时也被称为流, 类文件对象.Various stream objects handle different file types.例如,往binaryWrite to the stream objectstrAn object of type will report a type errorTypeError,同样往textWrite to the stream objectbinaryType of content will also report an error.

textStream objects are mainly dealt withstrType file read and write,When the file is stored in bytes,通过textStream objects can make file codecs and cross-platform support such as newlines insensitive.最简单的创建textThe way to stream objects is throughopen函数,如

import io
f = open('text.txt', 'r', encoding='utf8')
# A stream of bytes in memory can be passed throughio.StringIO来创建
f = io.StringIO("some text")

binary I/OUsed to handle bytes-like objects,并返回一个字节对象bytes object,Python中的bytes类和str差不多,Just add the quote before the definitionb,如bytes_data = b'data',bytes对象只支持ASCII码字符,Any binary code exceeds127的字符在bytesobject must start with 转移序列来表示.类字节对象bytes-like object是支持缓存协议Buffer ProtocolAnd can output oneCAn object of the language contiguous cache,binary I/OThe file is not automatically encoded, decoded and converted to newlines.binary I/OWhen working with non-text data and or when you want to manually control the processing of text data.创建binary I/OThe easiest way to object is：

f = open("myfile.jpg", "rb")
# 从内存中读取
f = io.BytesIO(b"some initial binary data: \x00\x01")

2.png文件格式简介

png图片格式是Portable Network Graphics（P ortable N etwork G raphics,PNG）It is a bitmap graphics format that supports lossless compression,支援索引、灰度、RGB三种颜色方案以及Alpha通道等特性.PNG的开发目标是改善并取代GIF作为适合网络传输的格式而不需专利许可,PNG于1997年3月作为知识性RFC 2083发布,于2004年作为ISO/IEC标准发布2.

png文件中,You can see it after reading byte by byte,开头的8A bytecode is always a hexadecimal number89 50 4E 47 0D 0A 1A 0A,这正是pngThe signature of the format file,是用来标识png格式的,So just modify the suffix name of the file,The software is still able to know what the picture ispng格式的.下面是通过python的builtin.open函数按bytes对象读取pngThe output section you get after the file,

p = "sample.png"
with open(p,'rb') as f:
    img_data = f.read()
    print(img_data)
# b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x0...'

可以看到前8个字节分别是\x89PNG\r\n\x1a\n,十六进制数0x50,0x4e,0x47,0x0d,oxoa分别对应的ASCIIcode is exactlyP,N,G,CR,LF3.

pngThere are two types of data blocks defined in ,一种是PNGFile must contain、Read and write software must also support key blocks（critical chunk）;另一种叫做辅助块（ancillary chunks）,PNG允许软件忽略它不认识的附加块.

关键数据块中有4个标准数据块：

• File header data blockIHDR（header chunk）：包含有图像基本信息,such as width,作为第一个数据块出现并只出现一次,Its length is fixed13个字节：

  描述	长度
  图片宽度	4字节
  图片高度	4字节
  图像深度	1字节
  颜色类型	1字节
  压缩方法	1字节
  过滤方式	1字节
  扫描方式	1字节

  As in the open image aboveIHDR\x00\x00\x00\xb6\x00\x00\x00\x86It can be seen that the image width and height are \x00\x00\x00\xb6即182和\x00\x00\x00\x86即134.

• Palette data blockPLTE（palette chunk）：必须放在图像数据块之前.

• 图像数据块IDAT（image data chunk）：Stores actual image data.PNG数据允许包含多个连续的图像数据块.

• 图像结束数据IEND（image trailer chunk）：at the end of the file,表示PNG数据流结束.

辅助数据块：常见的有：

3.PILThe library parses image data

PILThe library parses image data4

• PIL.Image.open(fp, mode='r', formats=None)

fpThe parameter supports image file paths,如string/Path/文件对象.对于普通的png文件,可以通过PIL.Image.open(filename)function to read directly,

from PIL import Image
p = "sample.png"
# 1.read from data path
img = Image.open(p)
# 获取图像的exif信息
print(img.getexif())

# 2.read from file object
with open(p) as f:
     img_b = io.BytesIO(f.read())
     img = Image.open(img_b)

• PIL.Image.frombuffer(mode, size, data, decoder_name='raw', *args)

Creates an image based on a byte buffer,值得注意的是frombufferParse pixel data only,So only from pixel data（同png中IDAT部分）restore the image from the byte cache,不能直接使用png中读取的数据,Because it contains auxiliary data, etc.可以解析Image.tobytes或numpy.tobytes(order='c'))Generate the result to construct the image. 同样还有Image.frombytes函数.

import numpy as np

with open(p) as f:
     img_b = io.BytesIO(f.read())
     img = Image.open(img_b)

# 1.from Image.tobytes
img_bytes = img.tobytes()
img = Image.frombuffer('RGB', (182, 134), img_bytes)

# 2.from numpy.tobytes
fb = io.BytesIO()
fb.write(img_bytes)
img_bytes_buffer = fb.getbuffer()
img_bytes = np.frombuffer(img_bytes_buffer,dtype=np.uint8).reshape(182, 134, -1)
img_bytes = img_bytes.tobytes(order='C')img = Image.frombuffer('RGB', (182, 134), img_bytes)

4.opencv解析图像

同样,使用OpenCV读取图像,可通过imread,也可以使用decodeMethods are parsed from memory data,譬如png格式的文件,可以先通过builtin.open读取成字节流,再使用np.frombuffer(bytes, dtype=uint8)Used after constructing the cached data of the imageimdecodemethod to obtain image data.

from PIL import Image
p = "sample.png"
with open(p, 'rb') as f:
     img_bytes_data = f.read()
     img_buffer = np.frombuffer(img_bytes_data, dtype=np.uint8)
     img = cv2.imdecode(img_buffer, cv2.IMREAD_COLOR)
     cv2.cvtColor(img, cv2.COLOR_RGB2BGR, img)
     plt.imshow(img)

Of course this is just an example,正常cv2.imdecodeCertainly not used in this way,通常与cv2.imencode结合使用,For the transfer of image data.

5.labelme标注文件中的img_data数据解析

import base64
import cv2
import numpy as np
from PIL import Image
from matplotlib import pyplot as plt

json_path = "sample.json"
data = json.load(open(json_path))
data = data["imageData"]
bytes_data = base64.b64decode(data)
# 1.use Image to reconstruct image
bd = io.BytesIO(bytes_data)
img = Image.open(bd)

# 2.use cv2.imdecode to reconstruct image
img_data = np.frombuffer(bytes_data, dtype=np.uint8)
img = cv2.imdecode(img_data, cv2.IMREAD_COLOR)
cv2.cvtColor(img, cv2.COLOR_RGB2BGR, img)
plt.imshow(img)

Image data is saved tojson文件中时,为了可视化的方便,Typically used for image database64进行了编码,因此解析json中的img_datadata to be used firstbase64Decode the bytecode of the image data,再使用numpy.frombuffer（bytes, dtype=uint8）方法创建cv2.imdecodeThe memory block data that can be parsed,cv2.imdecodeThe full image data is parsed instead of just pixel data,Include information such as the width and height of the image and the creation date,So no additional information is needed to restore the image.base64模块支持Base16/Base32/Base64/Base85编解码方法5,Its role is mainly to encode binary data into printableASCII码,以Base16算法为例,Base16Encoding will slice the bytes into 4个一组,So this encoding will be used later2个字符,The data will become the original2倍.类推,Base32Encoding will slice the bytes into 5 个一组,每 5 bytes can be reassembled into 8个字符.

欢迎访问个人网络日志知行空间

参考资料

• 1.https://docs.python.org/3/library/io.html
• 2.https://zh.wikipedia.org/zh-hk/PNG#%E6%96%87%E4%BB%B6%E7%BB%93%E6%9E%84
• 3.https://tool.oschina.net/commons?type=4
• 4.https://pillow.readthedocs.io/en/stable/reference/Image.html#PIL.Image.frombuffer
• 5.https://docs.python.org/3/library/base64.html