如何解决在具有透明度的 3D 图像上叠加 2D 热图
我有一个 3D python Image 及其 2D 特征向量。我用 imshow() 显示了每一个,如这里的代码所示,我可以清楚地看到它。我现在想要的是将 2D 特征向量作为热图叠加在其 3D 图像之上。我尝试添加它们,但引发了维度问题,我通过添加第三维扩展了 2D 特征向量,但叠加的图像混乱了。维度在这里:- 特征 >> (32,96),图片 >> (32,96,3)
img_feature = np.uint8(feature22 + img_raw_np_resize)
ax0.imshow(img,extent =extent)
ax1.imshow(feature,alpha = 0.75,interpolation = 'gaussian',cmap = plt.cm.jet,extent =extent)
ax2.imshow(img_feature,extent =extent)
extent 是指使所有这些都以相同的大小显示,并且已经定义了。 这是我通过上述代码尝试的结果
我正在寻找的显示器如下......
这是执行此操作的伪代码(我正在尝试生成最少的代码来共享,但这需要时间-因此我共享此代码以防有助于找出问题:-
def save_feature_to_img_2(self):
feature = (
get_feature()
) # get feature of size -- > torch.Size([1,256,32,96])
feature2 = features[
:,:,:
] # I pick one from the 256 feature maps - [1,1,96]
feature2 = (
feature2.data.numpy()
) # convert it to numpy array for later fusion with image array - [1,96]
features2 = features2.view(
features2.shape[1],features2.shape[2]
) # reshape into 2D feature map [32,96]
img_raw1 = Image.open(self.img_path) # read raw image size [128,64,3]
img_raw_np = np.array(img_raw1) # imge to numpy array
newsize = (h,w) # shape to re-size image to same size a feature
img_raw_resize = img_raw1.resize(newsize)
img_raw_np_resize = np.array(img_raw_resize) # size is Now [32,3]
# use sigmod to normalize feature to [0,1]
feature2 = 1.0 / (1 + np.exp(-1 * feature2))
# display setup
dx,dy = 0.05,0.05
y = np.arange(-8,8,dx) # Y - axis range
x = np.arange(-4.0,4.0,dy) # X -axis range
X,Y = np.meshgrid(x,y) # meshgrid to enclose my display into
extent = (
np.min(x),np.max(x),np.min(y),np.max(y),) # extent - the X and Y range - just to make unifrom display
feature2 = (255.5 * feature2 / np.amax(feature2)).astype(
np.uint8
) # put feature into [0-255] range for colour image
# to display img,feature and img_feature
fig = plt.figure()
ax0 = fig.add_subplot(131,title="Image")
ax1 = fig.add_subplot(132,title="Heatmap")
ax2 = fig.add_subplot(133,title="overlayed")
# hereunder I used this code share in the answer to fuse
alpha = 0.5
img_heatmap = (
feature2[:,None].astype(np.float64) * alpha
+ img_raw_np_resize * (1 - alpha)
).astype(np.uint8)
ax0.imshow(img_raw1,alpha=1.0,interpolation="gaussian",cmap=plt.cm.jet)
ax1.imshow(feature2,cmap=plt.cm.jet)
ax2.imshow(
img_heatmap,alpha=0.7,cmap=plt.cm.jet
)
cv2.imwrite("./img_heatmap.jpg",img_heatmap)
这是我得到的新显示器
我使用以下内容来融合图像和特征......
alpha = 0.5
img_feature = ((plt.cm.jet(feature2)[:,:3] * 255) * alpha + (1-alpha)*img_raw_np_resize).astype(np.uint8) and displaying it with ax2.imshow(img_feature,alpha = 0.7,cmap = plt.cm.jet)
解决方法
变体 1
如果您只想显示图像的加热区域,则必须乘以热图而不是相加。
适合您的公式为 img_feature = (img * (feature[:,:,None].astype(np.float64) / np.amax(feature))).astype(np.uint8)
。
完整示例代码(带有我自己的图像和自动生成的示例热图):
import requests,PIL.Image,io,numpy as np,matplotlib.pyplot as plt
# load some image
img = np.array(PIL.Image.open(io.BytesIO(requests.get('https://i.stack.imgur.com/vPlCG.jpg').content)))
# load or compute some features
h,w,_ = img.shape
mg = np.mgrid[:h,:w]
feature = mg[0].astype(np.float64) * mg[1].astype(np.float64)
feature = (255.5 * feature / np.amax(feature)).astype(np.uint8)
# compute heated image
img_feature = (img * (feature[:,None].astype(np.float64) / np.amax(feature))).astype(np.uint8)
# show images
fig,(ax0,ax1,ax2) = plt.subplots(1,3)
ax0.imshow(img)
ax1.imshow(feature,alpha = 1.,interpolation = 'gaussian',cmap = plt.cm.jet)
ax2.imshow(img_feature,cmap = plt.cm.jet)
plt.show()
输出:
变体 2
如果您只想使加热的区域变亮(变白)并使未加热的区域变暗(变暗),您只需通过公式 alpha = 0.5; img_feature = (feature[:,None].astype(np.float64) * alpha + img * (1 - alpha)).astype(np.uint8)
进行 alpha 混合。
import requests,:w]
feature = mg[0].astype(np.float64) * mg[1].astype(np.float64)
feature = (255.5 * feature / np.amax(feature)).astype(np.uint8)
# compute heated image
alpha = 0.5; img_feature = (feature[:,None].astype(np.float64) * alpha + img * (1 - alpha)).astype(np.uint8)
# show images
fig,cmap = plt.cm.jet)
plt.show()
输出:
变体 3
与 Variant-2 相同(带有 alpha 混合),但使用基于 matplotlib.pyplot.cm.jet
配色方案的 RGB 特征阵列代替黑白特征阵列。
正如您从代码中看到的,您可以在表达式 plt.cm.jet(feature)
中使用 any coloring scheme 而不是 plt.cm.jet
颜色。
import requests,:w]
feature = mg[0].astype(np.float64) * mg[1].astype(np.float64)
feature = (255.5 * feature / np.amax(feature)).astype(np.uint8)
# compute heated image
alpha = 0.5; img_feature = ((plt.cm.jet(feature)[:,:3] * 255) * alpha + img * (1 - alpha)).astype(np.uint8)
# show images
fig,axs = plt.subplots(2,2)
axs[0,0].imshow(img)
axs[0,1].imshow(feature,cmap = plt.cm.jet)
axs[1,0].imshow(img_feature,1].remove()
plt.show()
输出:
附注。我只是注意到 Matplotlib 在绘制热图时做了标准化 (feature - min) / (max - min)
因此我决定在叠加公式中做同样的事情,最终公式变成这样:
alpha = 0.5; img_feature = ((
plt.cm.jet(
(feature - np.amin(feature)).astype(np.float32)
/ (np.amax(feature) - np.amin(feature)).astype(np.float32)
)[:,:3] * 255
) * alpha + img * (1 - alpha)).astype(np.uint8)
使用上面is here(或here和for your images)公式的示例代码。使用上面is here(和for your images)公式的结果图像。最终覆盖您的图像:
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。