使用ImageFolderDataset读取图片在进行PIL转化的时候出现报错

经验分享 Tech Blogs 数据加载及处理-Data Loading&Processing
1

1 系统环境

硬件环境(Ascend/GPU/CPU): Ascend/GPU/CPU
MindSpore版本: mindspore=2.0.0
执行模式（PyNative/ Graph）:不限
Python版本: Python=3.7.5
操作系统平台: 不限

2 报错信息

2.1 问题描述

定义了一个transforms的操作集合，通过ImageFolderDataset读取图片数据，transforms_list里面包含转ToPIL以及Resize操作，运行之后，会出现如下报错。显示图片的维度信息不对。

2.2 脚本信息

import mindspore.dataset as ds
import mindspore.dataset.vision as vision
import nupy as np
from mindspore.dataset.transforms import Compose
from PIL import Image

image path = "/data2/shenwei/Minddata/test/mindspore/tests/ut/data/dataset/testPK/data"

transforms_List = Compose([vision.ToPIL(),vision.Resize((200, 200))])

# pipeline mode
dataset = ds.ImageFoderDataset(image_path, shuffle=False)
dataset = dataset.map(operations=transforms_list, input_coumns="image")
for item in dataset.create_dict_iterator():
    print(item[″image"].shape)
    print(item[″image"].dtype)
    break

# eager mode
img = np.fromfile("/data2/shenwei/Minddata/test/mindspore/tests/ut/data/dataset/testPK/data/class1/Θ.jpg",np.uint8)
img = transforms_list(img)[0]
assert isinstance(img, Image.Image) print(type(img))

2.3 问题描述

Traceback (most recentcall ast):
  File "tset_numpy.py", line 15, in<module>
    for item in dataset.create_dict .iterator():
  File"/home/shenwei/.conda/envs/㏕ib/python3.7/s ite-packages/mindspore/dataset/engine/iterators.py",line 152, in _next_ 
    data = sef._get_next( )
  File "/home/shenwei/.conda/envs/lib/python3.7/s ite-packages/mindspore/dataset/engine/iterators.py", line 277, in _get_next
    raise err
  File"/home/shenwei/.conda/envs/lib/python3.7/site-packages/mindspore/dataset/engine/iterators.py", line 260, in _get_next 
-Python Call stack: 
map operation: [PyFunc] failed. Thecorresponding data file is: /data2/shenwei/Minddata/test/mindspore/tests/ut/data/dataset/testpK/data/class1/2.jpg. Error description: valueÈrror: Traceback (most recentcall last): 
  File'"/home/shenwei/.conda/envs/shenweb/python3.7/s ite-packages/mindspore/dataset/transforms/py_transforms_util.py", line 63, in copose 
    args = transform(*args ) 
  File "/home/shenwei/.conda/envs/lib/python3.7/s ite-packages/mindspore/dataset/transforms/transforms.py", Line 133, in _call_ 
    return self._execute py( img) 
  File"/home/shenwei/.conda/envs/lib/python3.7/s ite-packages/mindspore/dataset/vision/transforms.py"，line 4739, in _execute_py 
    return util.to_pil( img) 
  File "/home/shenwei/ conda/envs/lib/python3.7/s ite-packages/mindspore/dataset/vis ion/py_transforms_util.py", line 178, in to_pil 
    raise ValueError("Thedimens ion of input image should be 2or 3. Got {}.".format(img.ndim)) 
ValueError: The dimens ion ofinput image should be 2 or 3. Got 1 

- Dataset Pipeline Error Message: 
ERROR Execute user Python code failed. check 'pvthon Call Stack' above
- C++ Call Stack: (For framework developers.
mindspore/ccsrc/minddata/dataset/engine/datasetops/map op/map iob.h(57)复制

3 根因分析

代码有两个错:(还有一些字符输入错误的暂不管)

1.ImageFolderDataset 读取目录下的图像文件后输入的就是一个图像, 所以第一步应该是做Decode

做完Decode后可以调用vision.ToPIL(), 或者直接调用Decode的时候指定参数, vision.Decode(to_pil=True)

2.eager mode下输入的是一个numpy, 是可以直接用vision.ToPIL()的

但是np.fromfile读取图像的rank是1 同样会报错, 所以要用Image.open后转numpy才行

4 解决方案

import mindspore.dataset as ds  
import mindspore.dataset.vision as vision  
import numpy as np  
from mindspore.dataset.transforms import Compose  
from PIL import Image  
image_path="C:\\temp\\pic"  
transforms_list = Compose([#vision.Decode(),  
                            #vision.ToPIL(),  
                            vision.Decode(to_pil=True),  
                            vision.Resize((200, 200))])  
#transforms_list = Compose([vision.ToPIL(),vision.Resize((200, 200))])  
# pipeline mode  
dataset = ds.ImageFolderDataset(image_path, shuffle=False)  
dataset = dataset.map(operations=transforms_list, input_columns="image")  
for item in dataset.create_dict_iterator():  
    print(item["image"].shape)  
    print(item["image"].dtype)  
    break  
    
# eager mode  
transforms_list1 = Compose([vision.ToPIL(),vision.Resize((200, 200))])  
#img = np.fromfile("C:\\temp\\pic\\01\\000000000064.jpg",np.uint8)  
img = Image.open("C:\\temp\\pic\\01\\000000000064.jpg")  
img = np.array(img)  
img = transforms_list1(img)[0]  
assert isinstance(img, Image.Image)  
print(type(img))

cke_12103.png