增强任何外部库中的tf.data.Dataset元素在我的情况下为相册

如何解决增强任何外部库中的tf.data.Dataset元素在我的情况下为相册

我正在创建一个映射函数，该函数会扩充数据集中的每个图像，然后由tf.numpy_function包装以创建一个tensorflow op。现在这个tensorflow操作被传递给tf.data.Dataset.map函数

由于代码有时运行并在其他时间显示错误，我遇到了奇怪的行为。


transformations = Compose([
            Rotate(limit=40),RandomBrightness(limit=0.1),JpegCompression(quality_lower=85,quality_upper=100,p=0.5),HueSaturationValue(hue_shift_limit=20,sat_shift_limit=30,val_shift_limit=20,RandomContrast(limit=0.2,HorizontalFlip(),])

def aug(image):
    aug_img = transformations(image=image)['image']
    aug_img = tf.image.convert_image_dtype(aug_img,'float32')
    return aug_img


def tf_augment(image,label):
    aug_img = tf.numpy_function(func = aug,inp =[image],Tout =tf.float32)
    return aug_img,label


augmented_ds = data.batch(10).map(tf_augment,num_parallel_calls=AUTOTUNE)
it = iter(augmented_ds)
batch = next(it)
images,labels = batch
for image,label in zip(images,labels):
    show_image(image,label)

tf.data.Dataset对象data包含image,label个元组。我已经对data进行了预处理，以使每个图像都具有相同的大小，这样做是为了确保data可以被批处理。 aug是一种增强功能，可从image

中提取data

因此，在我的情况下，aug是映射功能。现在，在将映射应用于aug之前，必须将data转换为tensorflow op。

tf_augment是将传递给data.map方法的tensorflow操作

为了向量化映射，我想先对data进行批处理，然后再将映射应用于它。因此，我先使用data.batch.map，然后再应用map。

现在代码的最后一块

augmented_ds = data.batch(10).map(tf_augment,label)

有时会运行，而在其他时候会抛出错误。

它引发的错误是

UnkNownError                              Traceback (most recent call last)
~\anaconda3\envs\tf23\lib\site-packages\tensorflow\python\eager\context.py in execution_mode(mode)
   2101       ctx.executor = executor_new
-> 2102       yield
   2103     finally:

~\anaconda3\envs\tf23\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py in _next_internal(self)
    754         # handles execute on the same device as where the resource is placed.
--> 755         ret = gen_dataset_ops.iterator_get_next(
    756             self._iterator_resource,~\anaconda3\envs\tf23\lib\site-packages\tensorflow\python\ops\gen_dataset_ops.py in iterator_get_next(iterator,output_types,output_shapes,name)
   2609     except _core._NotOkStatusException as e:
-> 2610       _ops.raise_from_not_ok_status(e,name)
   2611     except _core._FallbackException:

~\anaconda3\envs\tf23\lib\site-packages\tensorflow\python\framework\ops.py in raise_from_not_ok_status(e,name)
   6842   # pylint: disable=protected-access
-> 6843   six.raise_from(core._status_to_exception(e.code,message),None)
   6844   # pylint: enable=protected-access

~\anaconda3\envs\tf23\lib\site-packages\six.py in raise_from(value,from_value)

UnkNownError: error: OpenCV(4.4.0) C:\Users\appveyor\AppData\Local\Temp\1\pip-req-build-k8sx3e60\opencv\modules\imgproc\src\imgwarp.cpp:2594: error: (-215:Assertion Failed) src.cols > 0 && src.rows > 0 in function 'cv::warpAffine'

Traceback (most recent call last):

  File "C:\Users\aksha\anaconda3\envs\tf23\lib\site-packages\tensorflow\python\ops\script_ops.py",line 244,in __call__
    ret = func(*args)

  File "C:\Users\aksha\anaconda3\envs\tf23\lib\site-packages\tensorflow\python\autograph\impl\api.py",line 302,in wrapper
    return func(*args,**kwargs)

  File "<ipython-input-68-80185d06bd35>",line 2,in aug
    aug_img = transformations(image=image)['image']

  File "C:\Users\aksha\anaconda3\envs\tf23\lib\site-packages\albumentations\core\composition.py",line 176,in __call__
    data = t(force_apply=force_apply,**data)

  File "C:\Users\aksha\anaconda3\envs\tf23\lib\site-packages\albumentations\core\transforms_interface.py",line 87,in __call__
    return self.apply_with_params(params,**kwargs)

  File "C:\Users\aksha\anaconda3\envs\tf23\lib\site-packages\albumentations\core\transforms_interface.py",line 100,in apply_with_params
    res[key] = target_function(arg,**dict(params,**target_dependencies))

  File "C:\Users\aksha\anaconda3\envs\tf23\lib\site-packages\albumentations\augmentations\transforms.py",line 526,in apply
    return F.rotate(img,angle,interpolation,self.border_mode,self.value)

  File "C:\Users\aksha\anaconda3\envs\tf23\lib\site-packages\albumentations\augmentations\functional.py",line 70,in wrapped_function
    result = func(img,*args,**kwargs)

  File "C:\Users\aksha\anaconda3\envs\tf23\lib\site-packages\albumentations\augmentations\functional.py",line 202,in rotate
    return warp_fn(img)

  File "C:\Users\aksha\anaconda3\envs\tf23\lib\site-packages\albumentations\augmentations\functional.py",line 188,in __process_fn
    img = process_fn(img,**kwargs)

cv2.error: OpenCV(4.4.0) C:\Users\appveyor\AppData\Local\Temp\1\pip-req-build-k8sx3e60\opencv\modules\imgproc\src\imgwarp.cpp:2594: error: (-215:Assertion Failed) src.cols > 0 && src.rows > 0 in function 'cv::warpAffine'



     [[{{node PyFunc}}]] [Op:IteratorGetNext]

During handling of the above exception,another exception occurred:

UnkNownError                              Traceback (most recent call last)
<ipython-input-73-82392f6b5110> in <module>
      1 augmented_ds = resized_ds.batch(10).map(tf_augment,num_parallel_calls=AUTOTUNE)
      2 it = iter(augmented_ds)
----> 3 batch = next(it)
      4 images,labels = batch
      5 images.shape

~\anaconda3\envs\tf23\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py in __next__(self)
    734 
    735   def __next__(self):  # For Python 3 compatibility
--> 736     return self.next()
    737 
    738   def _next_internal(self):

~\anaconda3\envs\tf23\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py in next(self)
    770   def next(self):
    771     try:
--> 772       return self._next_internal()
    773     except errors.OutOfRangeError:
    774       raise stopiteration

~\anaconda3\envs\tf23\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py in _next_internal(self)
    762         return self._element_spec._from_compatible_tensor_list(ret)  # pylint: disable=protected-access
    763       except AttributeError:
--> 764         return structure.from_compatible_tensor_list(self._element_spec,ret)
    765 
    766   @property

~\anaconda3\envs\tf23\lib\contextlib.py in __exit__(self,type,value,traceback)
    129                 value = type()
    130             try:
--> 131                 self.gen.throw(type,traceback)
    132             except stopiteration as exc:
    133                 # Suppress stopiteration *unless* it's the same exception that

~\anaconda3\envs\tf23\lib\site-packages\tensorflow\python\eager\context.py in execution_mode(mode)
   2103     finally:
   2104       ctx.executor = executor_old
-> 2105       executor_new.wait()
   2106 
   2107 

~\anaconda3\envs\tf23\lib\site-packages\tensorflow\python\eager\executor.py in wait(self)
     65   def wait(self):
     66     """Waits for ops dispatched in this executor to finish."""
---> 67     pywrap_tfe.TFE_ExecutorWaitForAllPendingNodes(self._handle)
     68 
     69   def clear_error(self):

UnkNownError: error: OpenCV(4.4.0) C:\Users\appveyor\AppData\Local\Temp\1\pip-req-build-k8sx3e60\opencv\modules\imgproc\src\imgwarp.cpp:2594: error: (-215:Assertion Failed) src.cols > 0 && src.rows > 0 in function 'cv::warpAffine'

Traceback (most recent call last):

  File "C:\Users\aksha\anaconda3\envs\tf23\lib\site-packages\tensorflow\python\ops\script_ops.py",**kwargs)

cv2.error: OpenCV(4.4.0) C:\Users\appveyor\AppData\Local\Temp\1\pip-req-build-k8sx3e60\opencv\modules\imgproc\src\imgwarp.cpp:2594: error: (-215:Assertion Failed) src.cols > 0 && src.rows > 0 in function 'cv::warpAffine'



     [[{{node PyFunc}}]]

我还将链接附加到colab笔记本上。请复制它。 colab

解决方法

您可以在映射后尝试使用批处理。对我有用

ds_alb = resized_ds.map(partial(process_data,img_size=120),num_parallel_calls=AUTOTUNE,deterministic=False).batch(32)

增强任何外部库中的tf.data.Dataset元素在我的情况下为相册

如何解决增强任何外部库中的tf.data.Dataset元素在我的情况下为相册

解决方法

相关推荐